Diffusion pipeline runtime matching

Set diffusionPipeline on an entry in a serving runtime's supportedModelFormats to limit the entry to the diffusion pipelines the runtime can serve. OME compares it with the pipeline recorded on the BaseModel or ClusterBaseModel, and the entry matches only models that pass. The entry's modelArchitecture already checks the pipeline class, and diffusionPipeline can also require components, such as a transformer from the diffusers library. The check is pass or fail: it decides which runtimes OME can pick, and adds nothing to their score. The catalog's diffusion runtimes, srt-qwen-image and srt-qwen-image-edit, leave it unset and match on the class through modelArchitecture.

The matching rule¶

An entry's diffusionPipeline has the same fields as the model's: className, the components scheduler, textEncoder, tokenizer, transformer and vae, and additionalComponents, keyed by the component's name in model_index.json. Each component has a library and a type. Every field is optional, so set only what the runtime depends on. Each field you set adds a requirement:

Values are compared as exact, case-sensitive strings. An entry without diffusionPipeline passes for every model. DiffusionPipelineSpec in the API reference describes each field.

Declare a pipeline in a runtime¶

The rest of the entry must match the model too:

• Set modelArchitecture to the pipeline class. OME records the class as the model's architecture, and an entry needs an architecture when the model has one.
• Set modelFormat and modelFramework to diffusers, each with the model's diffusers version. A dev or pre-release version, such as 0.34.0.dev0, matches only the same version, whatever the entry's operator. See Model version matching.

This ClusterServingRuntime serves QwenImagePipeline models at diffusers version 0.34.0.dev0 whose transformer comes from the diffusers library:

apiVersion: ome.io/v1beta1
kind: ClusterServingRuntime
metadata:
  name: srt-qwen-image-pipeline
spec:
  supportedModelFormats:
    - modelFormat:
        name: diffusers
        version: "0.34.0.dev0"
      modelFramework:
        name: diffusers
        version: "0.34.0.dev0"
      modelArchitecture: QwenImagePipeline
      diffusionPipeline:
        className: QwenImagePipeline
        transformer:
          library: diffusers
      autoSelect: true
      priority: 2
  engineConfig:
    runner:
      name: ome-container
      image: sglang:0.5.6-diffusion-cuda129-amd64
      ports:
        - containerPort: 8080
          name: http1
          protocol: TCP
      command:
        - sglang
        - serve
        - --host
        - "0.0.0.0"
        - --port
        - "8080"
        - --model-path
        - $(MODEL_PATH)
        - --served-model-name
        - Qwen/Qwen-Image
      resources:
        requests:
          cpu: "32"
          memory: 100Gi
          nvidia.com/gpu: 1
        limits:
          cpu: "32"
          memory: 100Gi
          nvidia.com/gpu: 1

kubectl apply -f runtime.yaml

clusterservingruntime.ome.io/srt-qwen-image-pipeline created

priority: 2 makes OME prefer this runtime to srt-qwen-image when both match a model. See Runtime selection scoring. A runtime that serves several pipelines has an entry for each, as srt-qwen-image-edit does for QwenImageEditPlusPipeline and QwenImageEditPipeline.

Adding diffusionPipeline to a runtime that OME already picks for pvc:// models stops it from matching them, because they have no recorded pipeline. Set spec.diffusionPipeline on those models first.

What OME records for the model¶

When the model agent downloads a diffusion model, it reads model_index.json instead of config.json, and OME fills in these fields:

OME fills in only empty fields, so a diffusionPipeline that you set stays as written, even one that lists only some components.

The keys of model_index.json become these diffusionPipeline fields:

A component's value lists its library and then its type, as in ["diffusers", "AutoencoderKL"]. For example, OME reads this model_index.json:

{
  "_class_name": "StableDiffusionPipeline",
  "_diffusers_version": "0.24.0",
  "scheduler": ["diffusers", "EulerDiscreteScheduler"],
  "text_encoder": ["transformers", "CLIPTextModel"],
  "tokenizer": ["transformers", "CLIPTokenizer"],
  "unet": ["diffusers", "UNet2DConditionModel"],
  "vae": ["diffusers", "AutoencoderKL"],
  "safety_checker": ["diffusers", "StableDiffusionSafetyChecker"]
}

and records this pipeline:

diffusionPipeline:
  className: StableDiffusionPipeline
  scheduler:
    library: diffusers
    type: EulerDiscreteScheduler
  textEncoder:
    library: transformers
    type: CLIPTextModel
  tokenizer:
    library: transformers
    type: CLIPTokenizer
  transformer:
    library: diffusers
    type: UNet2DConditionModel
  vae:
    library: diffusers
    type: AutoencoderKL
  additionalComponents:
    safety_checker:
      library: diffusers
      type: StableDiffusionSafetyChecker

unet becomes transformer, and safety_checker, which has no field of its own, goes under additionalComponents.

Models without a recorded pipeline¶

OME records no pipeline for a pvc:// model: its metadata Job fills in the architecture, format and framework, but not diffusionPipeline. Other models get none of these fields if they're vendor:// models, if the agent can't read their files, or if they have the annotation ome.oracle.com/skip-config-parsing: "true", so set them yourself.

To check a model, print its architecture and pipeline. For qwen-image-pvc, a ClusterBaseModel served from a PVC at diffusers version 0.34.0.dev0, the pipeline is empty:

kubectl get clusterbasemodel qwen-image-pvc \
  -o 'custom-columns=ARCHITECTURE:.spec.modelArchitecture,PIPELINE:.spec.diffusionPipeline'

ARCHITECTURE        PIPELINE
QwenImagePipeline   <none>

For a BaseModel, use basemodel and add -n with its namespace.

To match an entry that sets diffusionPipeline, set one on the model yourself, with at least the fields that the entry requires. This patch makes qwen-image-pvc match srt-qwen-image-pipeline:

kubectl patch clusterbasemodel qwen-image-pvc --type merge \
  -p '{"spec":{"diffusionPipeline":{"className":"QwenImagePipeline","transformer":{"library":"diffusers"}}}}'

clusterbasemodel.ome.io/qwen-image-pvc patched

OME doesn't check InferenceServices again when a model changes. Apply a rejected InferenceService again, or change an existing one, as Step 5: Verify recovery shows.

Where a rejection shows up¶

An entry that fails the pipeline check, or sets no modelArchitecture, gets one of these reasons. The pipeline check names only its first mismatch, so fix it and check again.

When no runtime matches qwen-image-pvc, OME lists the example runtime among the runtimes it excluded:

srt-qwen-image-pipeline (model format 'mt:diffusers:0.34.0.dev0:QwenImagePipeline:diffusers:0.34.0.dev0' not in supported formats: diffusion pipeline required by runtime but not specified in model)

Related pages¶

• Troubleshoot runtime selection: find out why OME can't choose a runtime for a model.
• Serving runtimes: how a runtime declares the models it supports.
• Runtime selection scoring: how OME ranks the runtimes that match.
• Model version matching: how OME compares format and framework versions.
• Reference a runtime explicitly: name a runtime on an InferenceService instead of letting OME choose one.
• kubectl ome runtime: see why a runtime was skipped.