OME API

ome.io/v1beta1

AcceleratorClass

Standard object metadata, such as name and labels.

The desired state of the AcceleratorClass.

The observed state of the AcceleratorClass.

ome.io/v1beta1

AcceleratorQuota

Standard object metadata, such as name and labels.

The desired state of the AcceleratorQuota.

The observed state of the AcceleratorQuota.

ome.io/v1beta1

AutoscalerPolicy

Standard object metadata, such as name and labels.

The desired state of the AutoscalerPolicy.

The observed state of the AutoscalerPolicy.

ome.io/v1beta1

BaseModel

Standard object metadata, such as name and labels.

The desired state of the BaseModel.

The observed state of the BaseModel.

ome.io/v1beta1

BenchmarkJob

Standard object metadata, such as name and labels.

The desired state of the BenchmarkJob.

The observed state of the BenchmarkJob.

ome.io/v1beta1

ClusterBaseModel

Standard object metadata, such as name and labels.

The desired state of the ClusterBaseModel.

The observed state of the ClusterBaseModel.

ome.io/v1beta1

ClusterServingRuntime

Standard object metadata, such as name and labels.

The desired state of the ClusterServingRuntime.

The observed state of the ClusterServingRuntime.

ome.io/v1beta1

FineTunedWeight

Standard object metadata, such as name and labels.

The desired state of the FineTunedWeight.

The observed state of the FineTunedWeight.

ome.io/v1beta1

InferenceReplica

Standard object metadata, such as name and labels.

The desired state of the InferenceReplica.

The observed state of the InferenceReplica.

ome.io/v1beta1

InferenceService

Standard object metadata, such as name and labels.

The desired state of the InferenceService.

The observed state of the InferenceService.

ome.io/v1beta1

RolloutPolicy

Standard object metadata, such as name and labels.

The desired state of the RolloutPolicy.

The observed state of the RolloutPolicy.

ome.io/v1beta1

ServingRuntime

Standard object metadata, such as name and labels.

The desired state of the ServingRuntime.

The observed state of the ServingRuntime.

ome.io/v1beta1

TrafficMap

Standard object metadata, such as name and labels.

The desired state of the TrafficMap.

The observed state of the TrafficMap.

ome.io/v1beta1

WorkloadCluster

Standard object metadata, such as name and labels.

The desired state of the WorkloadCluster.

The observed state of the WorkloadCluster.

ResourceName is the Kubernetes resource this allowance counts, such as nvidia.com/gpu or google.com/tpu. It becomes the resource entry in the materialized ClusterQueue's resourceGroup. Which names are budgetable is operator config; the API accepts any qualified resource name.

Maximum length: 253. Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*(/[A-Za-z0-9]([-A-Za-z0-9_.]*[A-Za-z0-9])?)?$.

ResourceFlavor names an existing Kueue ResourceFlavor. OME references flavors and does not own node labeling; a flavor missing on a target cluster makes that cluster's ClusterQueue inactive, so the budget is not materialized there.

Maximum length: 253. Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$.

Policy overrides the node's default split for this flavor only, so a leaf can be Explicit on a reserved flavor and Proportional on a fungible one. Leaves only.

Nominal is the allowance for this flavor, in the plane's own terms: the fleet-wide total where the tree is authored, this cluster's resolved share on a projected copy. It becomes the ClusterQueue's nominalQuota.

BorrowingLimit is how far above its share this leaf may burst by borrowing idle sibling capacity, passed through to Kueue. Unset, this leaf does not borrow: nominal is its ceiling, and the materialized queue carries an explicit limit of zero to say so, because the enforcement backend reads an absent limit as unbounded.

Borrowing is cluster-local, so setting it turns nominal from a ceiling into a floor plus overage, and the relaxed ceiling is reported in status.

WARNING: overage is not currently reclaimable. This plane sets no preemption policy, so a borrower holds what it took until its own work finishes -- a leaf that bursts can leave a sibling waiting for the share that sibling was guaranteed. Leaves only.

LendingLimit is how much of this leaf's share siblings may borrow while it is idle, passed through to Kueue. It MUST NOT exceed the share Kueue sees. Leaves only.

PerCluster is the admin-authored split, required when the effective policy is Explicit and rejected otherwise. Entries MUST sum to Nominal. It is absent on a projected copy, which carries one already-resolved share rather than a fleet split.

ResourceName is the resource these numbers describe.

ResourceFlavor is the flavor these numbers describe.

Nominal is the resolved allowance for this plane: the fleet total where the tree is authored, this cluster's share on a projection.

Admitted is the quantity of this flavor currently admitted against the materialized queues, including anything borrowed.

Reserved is the quantity held by workloads carrying a quota reservation, admitted or not. It is never below Admitted, and the gap between them is work that owns the chips but has not started — a tenant sitting at its ceiling with Admitted low reads as idle unless this is read too.

Borrowed is the admitted quantity above the nominal share, taken from idle siblings and reclaimable on contention.

PerCluster breaks the numbers above down by cluster. Populated where the tree is authored; empty on a projected copy, which knows only itself.

Memory capacity in GB

Compute capability (NVIDIA) or equivalent

Level Zero version (for Intel accelerators)

Clock speeds

Memory bandwidth

Features supported by this accelerator

Performance metrics

FlavorUID identifies the observed ResourceFlavor object.

NodeLabels is the flavor's node selector at observation time.

FlavorSetHash identifies all flavor definitions and configured resource names used by the attribution rule, including more specific selectors.

Complete means every observed node advertising this resource was unambiguously attributed and every flavor had a verifiable identity.

ResourceName is the resource these numbers describe.

ResourceFlavor is the flavor these numbers describe.

Allocatable is the currently schedulable quantity: the flavor's resource summed over Ready, uncordoned nodes matching the flavor's node labels. Nodes that are NotReady or cordoned remain physical inventory but contribute no schedulable capacity.

HighWaterMark is the greatest Allocatable observed for this flavor. Budget checks compare against it rather than the instantaneous value, so capacity that shrinks — a drain, a cordon, a device-plugin restart — never marks a node Degraded. It is lowered only once the observed value stays below it by more than the configured hysteresis band.

ObservedAt is when Allocatable was last sampled.

PerCluster breaks capacity down by cluster on a management plane. Empty on a workload cluster, whose root reports only its own capacity.

Attribution identifies the hardware mapping used for this observation. Omission cannot establish whether an observed zero is a complete sample.

Vendor of the accelerator (nvidia, amd, intel, etc.)

Family of the accelerator (ampere, hopper, cdna2, etc.)

Model name (a100, h100, mi250x, etc.)

Discovery patterns to identify nodes with this accelerator

Capabilities of this accelerator class

Resources exposed by this accelerator

Integration with external systems

Cost information for optimization decisions

Nodes that have this accelerator

Total number of accelerators in the cluster

Available accelerators (not allocated)

Last update time

Conditions represent the latest available observations

AvailableNodes is the number of nodes that have this accelerator available

Cluster is the WorkloadCluster's name.

Nominal is the share assigned to this cluster.

Admitted is the quantity admitted on this cluster, including borrowed.

Reserved is the quantity reserved on this cluster, admitted or not.

Borrowed is the admitted quantity above this cluster's share.

Cluster is the WorkloadCluster's name.

ClusterUID identifies the registration through which the report was read. Omission cannot establish that a same-name cluster owns this sample.

ReportUID identifies the member's root AcceleratorQuota.

ReportResourceVersion identifies the source object read for this sample.

ReportAvailable means the most recent collection verified the source object and registration. It does not imply that ObservedAt is fresh.

Allocatable is that cluster's schedulable quantity of the flavor.

HighWaterMark is the greatest Allocatable observed on this cluster.

ObservedAt is when this cluster's value was last sampled. A stale stamp distinguishes "capacity dropped" from "the cluster stopped reporting".

Attribution is the member's hardware mapping at observation time.

Cluster is the WorkloadCluster's name.

Maximum length: 253.

Nominal is that cluster's share, and becomes the nominalQuota of the ClusterQueue materialized there.

MinMemory in GB

MaxMemory in GB (useful for cost control)

MinComputePerformanceTFLOPS in TFLOPS

MinArchitectureVersion Compute capability (NVIDIA) or equivalent

RequiredFeatures that must be present

ExcludedClasses lists AcceleratorClasses to avoid

ArchitectureFamilies limits selection to specific families. Examples: ["nvidia-hopper", "nvidia-ampere"].

PreferredPrecisions lists numeric precisions in order of preference. Examples: ["fp8", "fp16", "fp32"].

Cost per hour in dollars

Cost per million tokens (for usage-based pricing)

Spot instance pricing if available

Cost tier for simplified selection (low, medium, high)

NodeSelector to identify nodes with this accelerator

Affinity for more complex node selection

PCIVendorID for device discovery (e.g., "10de" for NVIDIA)

DeviceIDs list of PCI device IDs

KueueResourceFlavor name to sync with

VolcanoGPUType for Volcano integration

Average latency in milliseconds

Maximum latency in milliseconds

MinMemoryPerBillionParams specifies memory required per billion parameters. OME doesn't read it.

Minimum: 0.

TensorParallelismOverride overrides the default tensor parallelism settings

RuntimeArgsOverride provides accelerator-specific runtime arguments

EnvironmentOverride provides accelerator-specific environment variables

FP32 performance in TFLOPS

FP16 performance in TFLOPS

INT8 performance in TOPS

INT4 performance in TOPS

Latency metrics

Cluster is the WorkloadCluster's name.

AppliedGeneration is the source generation this cluster's projection last carried. Behind metadata.generation means the update has not landed.

AppliedTime is when the projection was last applied successfully.

MaterializedGeneration is the source generation this cluster has actually materialized, read back from the projection's status.sourceGeneration. Both it and AppliedGeneration count source revisions, so the pair is comparable: equal means caught up, lower means the remote controller has not yet rendered what was projected.

Message explains why this cluster is behind.

Policy is the default split for every budget on this node that does not override it.

Frozen reports that output is held at its last-good state because a computed invariant is violated. While frozen the controller writes no new projections or Kueue objects for this node, and deletes none; only explicit deletion of the CR reaps them.

FrozenAt is when the freeze began.

Reason is the condition reason that caused the freeze, retained so the cause survives a later condition update.

LastAppliedGeneration is the spec generation the frozen output was built from, which is the state a reader is currently looking at.

LastAppliedTime is when that output was written.

Name is the parent AcceleratorQuota's metadata.name.

Maximum length: 253. Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$.

Role declares whether this node groups other nodes or is a leaf carrying the budget. A node is one or the other, never both. Changing it is admitted only under the drained-leaf / childless-grouping guard, because the change deletes the underlying Kueue object.

ParentRef is the edge to this node's parent. Every node sets it except the single reserved root. It is mutable: re-parenting updates the materialized ClusterQueue's cohort in place, so the queue, its LocalQueues, and its admitted workloads all survive.

PriorityTier names the WorkloadPriorityClass stamped on this leaf's workloads. It is a default, not a partition: a workload declaring its own priority keeps it, and the leaf's ClusterQueue admits mixed priorities. OME references the class and does not create it. Capped at AcceleratorQuotaMaxNameLength because it is stamped into a label value.

Maximum length: 63. Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$.

Distribution is this leaf's default split policy, overridable per budget entry. Unset means the operator-level default from the quota config applies; absent that too, the node reports Degraded rather than guessing.

Budgets is the per-flavor allowance, one entry per ResourceFlavor.

On a leaf this is the budget: the fleet-wide allowance where the tree is authored, that cluster's resolved share on a projected copy. On a Cohort node it is an authoring guardrail only — the parent number that parent >= sum(children) is checked against. A grouping materializes as a Kueue Cohort with no resourceGroups, so a Cohort budget is never quota anyone can consume.

Capped at the number of entries one Kueue resourceGroup accepts.

Maximum items: 16.

ObservedGeneration is the metadata.generation the most recent status flush reflects.

Parent echoes the resolved parent's name, so a node's edge is readable from status alone once the controller has confirmed it resolves.

SourceGeneration is the source CR's generation this node's materialization reflects, echoed from the projection annotation once the local controller has materialized it. Set only on a projected copy: it is what lets the management plane compare remote progress against what it projected, which object-local generations cannot express.

Budgets reports, per resource and flavor, the resolved allowance and the usage rolled up from the materialized ClusterQueues.

Capacity is the observed accelerator capacity behind this tree, reported on the root node only. On a management plane it is the fleet view broken down per cluster; on a workload cluster it is that cluster's own derived capacity, which is the only thing the local controller writes on its root.

Clusters reports per-cluster projection state on a management plane: which generation each workload cluster has, and why one is behind. Empty on a workload cluster, which projects nothing.

Materialization records what this node last wrote successfully and whether that output is currently frozen. A frozen node keeps serving its last-good materialization; nothing is deleted while an invariant is violated.

Conditions reports Ready, Degraded, and Materialized.

AcceleratorClasses lists the names of AcceleratorClasses this runtime supports

MinMemory specifies minimum GPU memory required in GB

Minimum: 0.

MinComputePerformanceTFLOPS specifies minimum compute capability in TFLOPS

Minimum: 0.

MinArchitectureVersion Compute capability (NVIDIA) or equivalent

RequiredFeatures lists hardware features that must be present. Examples: ["tensor-cores", "fp8", "nvlink"].

PreferredPrecisions lists numeric precisions in order of preference. Examples: ["fp8", "fp16", "fp32"].

Name of the resource (e.g., nvidia.com/gpu)

Quantity per accelerator

Default: "1".

Divisible indicates if the resource can be subdivided

AcceleratorClass that was selected

Reason explains why this accelerator was selected

NodeSelector that was applied to pods

ResourceRequests that were applied to pods

AcceleratorClass explicitly selects a specific AcceleratorClass. Takes precedence over other selectors.

Constraints defines requirements that accelerators must meet

Policy defines the selection policy when multiple accelerators match

Allowed values: BestFit, Cheapest, MostCapable, FirstAvailable.

Name identifies the metric in status and events; unique within Metrics.

Query is a PromQL expression, templated with the rollout's revision context before execution. Available variables: {{.Namespace}}, {{.ISVCName}}, {{.Component}}, {{.CanaryService}}, {{.StableService}}, {{.CanaryRevision}}, {{.StableRevision}}. A result with multiple series is reduced to the worst series for Operator before comparison.

Operator is the comparison applied as "result <Operator> Threshold" to decide whether this metric passes.

Allowed values: LT, LTE, GT, GTE.

Threshold is the bound the query result is compared against. Parsed as a float (e.g. "0.05", "200", "1.5"); admission rejects a non-numeric value.

Name matches the AnalysisMetric.Name this result is for.

Value is the query result at the last sample, formatted for display (e.g. "0.012"). Empty when the sample was inconclusive.

Threshold echoes the bound that was applied, so the result is self-describing in status.

Operator echoes the comparison that was applied.

Passed is whether this metric satisfied its condition at the last sample.

Message carries the reason when a metric did not pass or could not be evaluated (e.g. "no data", a query error).

Time is when this result was recorded.

ProviderRef names a logical metric provider bound per cluster in the operator's metricProviders configuration (shared with AutoscalerPolicy triggers), which supplies the server address, auth secret, and default headers. In a RolloutPolicy body this is the ONLY admitted spelling — a raw cluster-local URL inside a fleet-portable object defeats portability and is an SSRF surface. An unbound name is a deterministic config error: the rollout parks at run open rather than starting a roll whose gate cannot sample.

ServerAddress is the base URL of the Prometheus HTTP API (e.g. http://prometheus.monitoring.svc:9090). Optional: when empty (and no ProviderRef is set), the operator-configured default source (canaryAnalysis.bundledPrometheusAddress) is used; if that is also unset the analysis has no source and its samples read inconclusive. Admission does not check the source — an unreachable or missing source surfaces controller-side as an inconclusive sample, not a rejection.

AuthRef points at a Secret key (in the InferenceService's namespace) holding a bearer token sent as "Authorization: Bearer <token>". Omit for an unauthenticated Prometheus (the bundled one). TLS/mTLS is not supported.

Headers are extra HTTP headers sent on every Prometheus query — e.g. "X-Scope-OrgID" for a multi-tenant Cortex/Thanos/Mimir front-end that scopes queries by tenant. Without the tenant header such a backend returns no data, which the gate reads as a permanent stall. Plaintext (not secret); use AuthRef for the bearer token.

Name of the AutoscalerPolicy in the InferenceService's namespace.

ObservedGeneration is the policy generation the rendered block came from.

PortableDigest is the policy's canonical spec digest — equal across clusters iff the policy specs match.

ResolvedDigest is a digest of the rendered ComponentAutoscaler plus the effective bounds and bound provider endpoint. It differs per cluster by design; its job is per-home provenance ("did my policy edit land here?").

Name of an AutoscalerPolicy in the InferenceService's namespace.

Minimum length: 1.

Kind of the referenced policy. "ClusterAutoscalerPolicy" is a reserved shape: admission rejects it until the cluster-scoped twin ships.

Allowed values: AutoscalerPolicy, ClusterAutoscalerPolicy.

Default: AutoscalerPolicy.

Enforcement tier. v1 admits only "Default"; "Required" is a reserved shape rejected at admission.

Allowed values: Default, Required.

Default: Default.

Class selects the template. External and None are inline-only: "someone else scales this component" is a per-service statement, not a reusable behavior.

Allowed values: HPA, KEDA.

Keda holds the parameterized KEDA template. Required when Class=KEDA.

HPA holds HorizontalPodAutoscaler configuration, verbatim (no templating). Optional when Class=HPA; if absent the rendered block gets the default CPU=80% metric downstream, exactly like an inline block with a nil HPA field.

ObservedGeneration is the spec generation this status reflects.

PortableDigest is a digest of the spec after API defaulting, in a canonical encoding. Equal across clusters iff the specs are semantically identical; the multi-cluster preflight compares it against the control plane's copy.

AttachedComponents counts the components in this namespace that currently reference the policy.

APIVersion of the resource (e.g. "gateway.envoyproxy.io/v1alpha1").

Kind of the resource (e.g. "BackendTrafficPolicy").

Name of the resource, in the same namespace as the InferenceService.

ModelType defines the architecture family of the model (e.g., "bert", "gpt2", "llama"). This value typically corresponds to the "model_type" field in a Hugging Face model's config.json. It is used to identify the transformer architecture and inform runtime selection and tokenizer behavior.

ModelFramework specifies the underlying framework used by the model, such as "ONNX", "TensorFlow", "PyTorch", "Transformer", or "TensorRTLLM". This value helps determine the appropriate runtime for model serving.

ModelArchitecture specifies the concrete model implementation or head, such as "LlamaForCausalLM", "GemmaForCausalLM", or "MixtralForCausalLM". This is often derived from the "architectures" field in Hugging Face config.json.

Quantization defines the quantization scheme applied to the model weights, such as "fp8", "fbgemm_fp8", "int4", "nvfp4", or "mxfp4". This influences runtime compatibility and performance.

ModelParameterSize indicates the total number of parameters in the model, expressed in human-readable form such as "7B", "13B", or "175B". This can be used for scheduling or runtime selection.

ModelCapabilities of the model, e.g., "TEXT_GENERATION", "TEXT_SUMMARIZATION", "TEXT_EMBEDDINGS"

API capabilities supported by the model, e.g., "OPENAI_V1_CHAT_COMPLETIONS"

Configuration of the model, stored as generic JSON for flexibility.

Storage configuration for the model

Since v1.3. Distribution selects how the model's weights are laid out across the cluster. PerNode, the default, puts a full copy on each selected node. Sharded is for loading the model through an external cache that holds it in chunks across the cluster. Sharded is in development: OME has no way to configure such a cache yet, so a Sharded model can't be served. pvc:// models can't use Sharded.

ModelExtension is the common extension of the model

The fields of ModelExtensionSpec appear directly in this object.

ServingMode of the model, e.g., ["On-demand", "Dedicated"]

MaxTokens is the maximum number of tokens that can be processed by the model

DiffusionPipeline captures pipeline-specific metadata for diffusion models (from model_index.json).

Additional metadata for the model

HuggingFaceSecretReference is a reference to a Kubernetes Secret containing the Hugging Face API key. The referenced Secret must reside in the same namespace as the BenchmarkJob.

Endpoint is the reference to the inference service to benchmark.

ServiceMetadata records metadata about the backend model server or service being benchmarked. This includes details such as server engine, version, and GPU configuration for filtering experiments.

Task specifies the task to benchmark, pattern: <input-modality>-to-<output-modality> (e.g., "text-to-text", "image-to-text").

Allowed values: text-to-text, image-to-text, text-to-embeddings, image-to-embeddings.

TrafficScenarios contains a list of traffic scenarios to simulate during the benchmark. If not provided, defaults will be assigned via genai-bench.

NumConcurrency defines a list of concurrency levels to test during the benchmark. If not provided, defaults will be assigned via genai-bench.

MaxTimePerIteration specifies the maximum time (in minutes) for a single iteration. Each iteration runs for a specific combination of TrafficScenarios and NumConcurrency.

MaxRequestsPerIteration specifies the maximum number of requests for a single iteration. Each iteration runs for a specific combination of TrafficScenarios and NumConcurrency.

AdditionalRequestParams contains additional request parameters as a map.

Dataset is the dataset used for benchmarking. It is optional and only required for tasks other than "text-to-<output-modality>".

OutputLocation specifies where the benchmark results will be stored (e.g., object storage).

ResultFolderName specifies the name of the folder that stores the benchmark result. A default name will be assigned if not specified.

PodOverride customizes the benchmark pod. OME merges it into the pod it builds by default, so fields you leave unset keep their defaults.

State represents the current state of the benchmark job: "Pending", "Running", "Completed", "Failed".

Allowed values: Pending, Running, Completed, Failed.

StartTime is the timestamp for when the benchmark job started.

CompletionTime is the timestamp for when the benchmark job completed, either successfully or unsuccessfully.

LastReconcileTime is the timestamp for the last time the job was reconciled by the controller.

FailureMessage contains any error messages if the benchmark job failed.

Details provide additional information or metadata about the benchmark job.

Reason names the gate that gave up.

Allowed values: CapacityTimeout, AnalysisStalled, StableRevisionMissing.

Time is when the canary parked.

TargetID identifies the canary group's pinned Component target set. A new target set re-arms the canary even when the externally routed Component's revision did not change. It is scoped to the canary group rather than the whole rollout run, so unrelated groups cannot reset canary progress.

CanaryRevisionHash is the revision hash being rolled out.

StableRevisionHash is the revision hash that was serving as stable when the canary began — the rollback target. It is preserved across mid-canary retargets so a rollback always returns to the pre-canary stable revision, never to a partially-rolled intermediate. Cleared when the canary completes (the canary revision becomes the new stable).

CurrentStep is the zero-based index into spec.rollout.groups[i].canary.steps; equal to the number of steps once the canary has completed.

StepEnteredTime is when CurrentStep was entered. Auto promotion measures Pause.Duration from this timestamp.

ObservedTrafficWeight is the external traffic weight currently programmed for the canary revision (mirrors the active step's TrafficWeight once applied).

PromotedThrough is the ome.io/rollout-promote value the most recent step advance applied. It is stamped in the same status write as the advance — the durable record of the promotion, since metadata (the annotation) and status cannot be written atomically. While set, a lingering annotation with this value is inert (already applied; removal is retried best-effort). While the rollout is in progress it clears once the annotation is observed removed, re-arming manual promotion for later steps.

PreStepHold holds the canary's traffic at its currently-programmed weight after a repin whose clamped step would raise exposure: a repin may only hold or tighten, so the raise waits for an explicit ome.io/rollout-promote (value = the canary revision hash). Capacity may still converge to the clamped step — capacity ahead of traffic is the supported warm-up pattern; only traffic and step advance hold.

RolledBackRevisionHash is set when a rollback (ome.io/rollout-rollback) abandons a canary: it records the primary Component's rejected revision hash. While set, the group is held on its stable revisions and the rejected target set is NOT retried — even after the annotation is cleared. The rollout re-arms only when a different target set appears.

AnalysisFailedChecks counts failing analysis samples in the CURRENT step (reset to zero on each advance). Auto-rollback fires when it reaches Analysis.FailureLimit. Set only under Promotion=Analysis.

LastEvaluationTime is when the analysis metrics were last sampled. The controller throttles sampling to at most once per Analysis.Interval using this timestamp, so a burst of unrelated reconciles cannot over-count failures. Set only under Promotion=Analysis.

LastConclusiveEvaluationTime is when analysis last produced a conclusive sample (pass or fail — i.e. Prometheus answered). The stall timeout is measured from here: a long run of inconclusive samples escalates to Failed. Set only under Promotion=Analysis.

MetricResults is the most recent per-metric evaluation, for observability (kubectl get isvc shows why a step held or rolled back). Set only under Promotion=Analysis.

CapacityWaitSince is when the current step's capacity gate became unmet. The ready timeout is measured from here and never from the step's soak anchor, so a long soak cannot spend the capacity budget and a capacity dip cannot restart the soak. Cleared once the step's capacity is met.

Failed records that the canary is parked at CurrentStep: the capacity gate stayed unmet past the ready timeout, analysis stayed inconclusive past the stall timeout, or a rollback found no stable revision to return to. While set the step machine does not run, the phase reads Failed and the stable revision keeps serving. A re-arm toward a new target clears any park. A rollback request clears a capacity-timeout or analysis park and drains the rejected revision; a park for a missing stable revision refuses the request and removes it, since the revert it asks for is the one that could not run.

InventoryPending prevents unknown standing resources from becoming free migration capacity. It is cleared only by an identified member inventory.

HomeInputsPending holds movement when current full-policy inputs cannot be verified, while retaining the last accepted desired floors.

RaceCandidate authorizes cleanup of a Single race copy after another candidate wins. A retained winner does not carry this permission.

ReplacementStartedAt persists the start of an authorized bounded probe. The configured replacement timeout is evaluated against this instant.

Matched distinguishes a desired full-policy home from an outgoing home, including an affinity that matches every registration without terms.

CurrentHome retains the full policy authorized at a home through draining.

DesiredHome is present for every intended full-policy home.

MatchingTerms contains zero-based indexes into the source's affinity terms.

Weight is the effective static allocation weight, when applicable.

Minimum: 0.

ComponentMatches records, under the ComponentAffinity policy, how each component other than the primary one matched this cluster. MatchingTerms and Weight hold the primary component's match. The accepted plan keeps them so that member floors never follow unaccepted label changes.

Maximum items: 2.

Capacity is the accepted hardware sample, when capacity determines weight.

Minimum: 0.

Minimum: 0.

Minimum: 0.

DrainRequested withdraws this home from routing before physical removal.

Policies reports the portable digest of each policy as observed on this home. Every entry must match the control plane's copy; a mismatch is distribution skew.

Components maps each policy-consuming component to its per-home resolved digest ("did my policy edit land here?").

Ready is true when every policy-consuming component on this home reports specSource=policy with a resolved digest.

Allocation is the persisted floor and identity of this member.

AppliedPlanID identifies the allocation acknowledged by this member.

ObservationKnown distinguishes a current observation from retained evidence.

Since v1.3. ObservationFailingSince is when the control plane's reads of this home started failing. It is unset while reads succeed. While the configured observation grace lasts from this time, the home keeps the ready count from its last successful read.

Cluster is the WorkloadCluster name.

Phase is the candidate's state.

Endpoint is this candidate's own externally-addressable URL, mirrored from its derived InferenceService's status.url once admitted AND addressable. In Single mode only the winner carries one and the top-level PlacementStatus.Endpoint is authoritative; in All/Split mode every serving home carries its own, and this list is the source of truth an external LB/DNS consumes to route across homes. Empty until the home is addressable.

AdmittedReplicas is how many replicas this home's Kueue has admitted. In Split it also drives global accounting: the control plane sums it across homes to decide whether the desired count is met.

ReadyReplicas is how many of this home's replicas are serving traffic. It is the live count basis and health gate for routing across homes.

Autoscaling mirrors this home's AutoscalerPolicy state so digest skew between homes is visible from the source InferenceService. Set only when the source references at least one policy.

Rollout mirrors this home's rollout-run provenance (run identity plus per-group plan digests, never the plan bodies) so plan drift between homes is visible from the source InferenceService. Set only when the source's rollout groups reference a RolloutPolicy.

Source is where the group's progression came from (Inline or Policy).

PolicyName names the RolloutPolicy when Source is Policy.

PortableDigest is the pinned progression body's portable digest.

Outcome is how the run ended.

Digest folds the closed run's per-group portable digests into one order-sensitive comparable value.

ActiveRunID is the home's open rollout run, empty when none is open.

ActiveGroups is the active run's per-group provenance in pinned-plan order: which source each group pinned and under which digest.

LastRun is the bounded record of the home's most recently closed run.

Weight is permitted only for static Split. If any term has an explicit weight, matching term weights add and omitted weights contribute one. Omission must be preserved to distinguish unweighted overlapping terms. Weights are replica ratios, so they have no upper bound.

Minimum: 1.

MatchExpressions selects the WorkloadCluster's actual labels.

Maximum items: 64.

MatchFields selects the real metadata.name using In or NotIn. Multiple names are allowed; a label named metadata.name does not affect this match.

Maximum items: 64.

Key is a label key, or metadata.name for a field requirement.

Minimum length: 1.

Operator follows Kubernetes node-selector semantics for labels.

Allowed values: In, NotIn, Exists, DoesNotExist, Gt, Lt.

Values is nonempty for In/NotIn, empty for Exists/DoesNotExist, and a single integer for Gt/Lt.

Maximum items: 64.

Allowed values: engine, decoder, router.

MatchingTerms contains zero-based indexes into the component's terms.

Weight is the component's effective static weight, when applicable.

Minimum: 0.

Class selects the autoscaler implementation.

Allowed values: HPA, KEDA, External, None.

Keda holds KEDA-specific configuration. Required when Class == "KEDA".

HPA holds HorizontalPodAutoscaler-specific configuration. Optional when Class == "HPA"; if absent the controller emits a default CPU=80% HPA.

Class echoes the resolved autoscaler class as picked by the Component's deployment-mode resolver. One of HPA | KEDA | External | None. Mirrors the input that drove dispatch so the operator can confirm the inheritance chain landed on the expected class.

ManagedBy reports who owns the underlying scaler object. Distinguished from Class because none and external are dispatch-equivalent (both => delete any OME-managed HPA / SO); the surface-level distinction lives here.

• "ome": OME emitted the HPA / ScaledObject; conditions and replica counters below mirror its live status.
• "external": operator-owned scaler; OME publishes scaleTargetRef and stays out. Conditions empty.
• "none": no OME-managed scaler is active. Conditions empty; counters zero.

Allowed values: ome, external, none.

SpecSource reports which layer of the ISVC -> policy -> runtime -> legacy -> default inheritance chain produced the resolved Autoscaler block. One of "isvc" | "policy" | "runtime" | "legacy" | "default". Mirrors autoscaler.SpecSource so the operator can debug which layer is contributing the live config.

Policy identifies the AutoscalerPolicy that rendered the live block. Set only when SpecSource is "policy".

ShadowedPolicyRef reports a policy ref outranked by an inline block (SpecSource "isvc" with spec.<component>.autoscalerPolicyRef set). Its wouldRenderDigest is the in-cluster preview of what the policy would produce if the inline block were removed.

CurrentReplicas is mirrored from the HPA / ScaledObject status. Zero when ManagedBy != "ome" or when the scaler has not yet reported.

DesiredReplicas is mirrored from the HPA / ScaledObject status. Zero when ManagedBy != "ome" or when the scaler has not yet reported.

LastScaleTime mirrors the HPA / ScaledObject's last scale event. Nil when ManagedBy != "ome", when the scaler has not yet reported, or when no scaling has occurred since the scaler was created.

Conditions mirrors the HPA's or ScaledObject's conditions verbatim (HPA: ScalingActive / AbleToScale / ScalingLimited; ScaledObject: Ready / Active / Fallback / Paused). Empty when ManagedBy != "ome" — operator-owned scalers are surfaced via scaleTargetRef instead.

Engine selects clusters for the engine and, in Split, weights its shares. It is required when the service declares an engine.

Minimum items: 1. Maximum items: 64.

Decoder selects clusters for the decoder and, in Split, weights its shares over the clusters that hold engine replicas. It is required when the service declares a decoder.

Minimum items: 1. Maximum items: 64.

Router selects clusters for the router. In Split, it divides router.minReplicas across the clusters that hold engine replicas; without it, each such cluster runs the full router policy.

Minimum items: 1. Maximum items: 64.

Minimum number of replicas, defaults to 1 but can be set to 0 to enable scale-to-zero.

Maximum number of replicas for autoscaling.

TimeoutSeconds specifies the number of seconds to wait before timing out a request to the component.

Labels that will be added to the component pod. More info: http://kubernetes.io/docs/user-guide/labels

Annotations that will be added to the component pod. More info: http://kubernetes.io/docs/user-guide/annotations

Since v1.3. ServicePortAppProtocols maps generated Service port names to their Kubernetes appProtocol values. Declared ports use their container port name.

MinAvailable specifies how many component pods must still be available after the eviction

MaxUnavailable specifies how many component pods can be unavailable

The deployment strategy to use to replace existing pods with new ones. Only applicable for raw deployment mode.

Since v1.3. Lifecycle groups OMENative-specific lifecycle policies for this Component. Applies only when the Component resolves to deploymentMode OMENative (spec.deploymentMode or the per-Component ome.io/deploymentMode annotation); ignored otherwise. A field left unset here inherits the ServingRuntime's value for the Component; the controller fills what neither sets from cluster configuration and fixed fallbacks and never writes them back. The status counterpart is status.components.<component>.lifecycle.

Since v1.3. Autoscaler configures the per-Component autoscaler dispatch and the underlying KEDA / HPA configuration. Takes priority over the legacy ome.io/autoscalerClass annotation for this Component, and is the only way to configure per-Component autoscaling: scale metrics live here (Autoscaler.HPA.Metrics), alongside MinReplicas / MaxReplicas above. Alpha. The API may change without notice.

Since v1.3. AutoscalerPolicyRef names a same-namespace AutoscalerPolicy whose template renders this Component's autoscaler. An inline Autoscaler block above always outranks the ref; the two may coexist, which is the documented preview/rollback mechanism. Alpha. The API may change without notice.

URL holds the primary url for this component. It generally has the form http[s]://{name}.{namespace}.{cluster-level-suffix}

REST endpoint of the component if available.

Addressable endpoint for the InferenceService

SelectedAccelerator shows which AcceleratorClass was selected

Since v1.3. Lifecycle reports OMENative-managed lifecycle state for this Component when the Component resolves to deploymentMode OMENative (spec.deploymentMode or the per-Component ome.io/deploymentMode annotation). Counterpart of the LifecycleSpec sub-block on ComponentExtensionSpec. Nil otherwise.

Since v1.3. RolloutPhase is the canary step machine's state for this Component when a spec.rollout canary group governs it. One of Stable, Canarying, Pending, Paused, Promoting, RollingBack, RolledBack, Failed. Written on the Component the step machine drives (the unit's entrypoint) and never cleared: it reads Stable between runs, a completed canary included. Empty for Components no canary group governs (blueGreen/rollingUpdate groups report under status.rolloutCoordination) and on deployment modes without the rollout contract — e.g. RawDeployment.

Allowed values: Stable, Canarying, Pending, Paused, Promoting, RollingBack, RolledBack, Failed.

LatestReadyRevision is the per-revision Service name (<isvc>-<component>-rev-<hash>) fronting the most recent revision whose pods reached Ready. Equal to LatestRolledoutRevision once the rollout completes; set ahead of it during in-flight rollouts.

Since v1.3. LatestRolledoutRevision is the per-revision Service name (<isvc>-<component>-rev-<hash>) of the most recent revision that fully owns this Component's traffic (i.e., the rollout has completed). Drives the consumer-side HTTPRoute backendRef.

Since v1.3. PreviousRolledoutRevision is the per-revision Service name (<isvc>-<component>-rev-<hash>) of the revision that fully owned this Component's traffic immediately before LatestRolledoutRevision; recorded when LatestRolledoutRevision advances. Retained for diagnosis and traffic reference during partial rollbacks.

Since v1.3. Traffic reports per-revision traffic weights for this Component. Each entry corresponds to one revision currently receiving traffic. The HTTPRoute builder reads this to emit weighted backendRefs.

Since v1.3. Canary tracks the canary step machine for the unit this Component belongs to — the router alone, or engine+decoder together. It is written on the unit's entrypoint Component (the router, or the engine) and is absent on a secondary, so a reader never sees two copies of one run. Absent until the unit's first canary arms; afterwards it carries the unit's most recent run, a finished one included (see CanaryStatus).

Units advance independently, which is why the state cannot live in the single InferenceServiceStatus.Canary: two runs would overwrite each other's step counter and revision pair. That field is retained as an alias for the entrypoint unit's run so existing readers keep working.

Since v1.3. Autoscaler reports the per-Component autoscaler state — resolved Class / ManagedBy / SpecSource and (when ManagedBy == "ome") live CurrentReplicas / DesiredReplicas / LastScaleTime / Conditions mirrored from the underlying HPA or ScaledObject. Populated by the ISVC status writer for RawDeployment- and OMENative-managed Components. See ComponentAutoscalerStatus for the full field semantics.

Since v1.3. ScaleTargetRef is the canonical scale target for this Component. For OMENative-managed Components this points at the InferenceReplica's /scale subresource; for RawDeployment-managed Components it points at the underlying Deployment. Published so external scalers (when Autoscaler.ManagedBy == "external") know the GroupKind they should target. Populated whenever the Component has a defined scale target — independent of whether an OME-managed HPA / SO is active.

RevisionName is the per-revision Service name for this target.

Percent is the percentage of traffic the consumer should route to RevisionName. Sum across all entries for one Component is 100.

Minimum: 0. Maximum: 100.

Tag is an optional short identifier for the target (e.g. "latest", "prev"). Cosmetic; not used by the consumer.

LatestRevision is true when this entry corresponds to the LatestRolledoutRevision for the Component.

PairingProtocol is the engine↔decoder wire-compatibility token the target's revision was minted under (spec.rollout.pairingProtocol at mint time). Consumers MUST only pair engine and decoder targets whose PairingProtocol values are equal; an empty value pairs with anything.

Type selects the hash source.

Allowed values: Header, Cookie, SourceIP.

Headers to hash on. Required when Type=Header (one or more). Translators that support multi-header hashing concatenate the headers before hashing; when the active translator supports only single-header hashing, multi-header specs are rejected at admission. Forbidden when Type is not Header.

Cookie configuration. Required when Type=Cookie. Forbidden when Type is not Cookie.

This spec provides a full PodSpec for the decoder component. Allows complete customization of the Kubernetes Pod configuration including containers, volumes, security contexts, affinity rules, and other pod settings.

The fields of PodSpec appear directly in this object.

ComponentExtensionSpec defines deployment configuration like min/max replicas, scaling metrics, etc. Controls scaling behavior and resource allocation for the decoder component.

The fields of ComponentExtensionSpec appear directly in this object.

Runner container override for customizing the main container. This is essentially a container spec that can override the default container. Defines the main decoder container configuration, including image, resource requests/limits, environment variables, and command.

Leader node configuration (only used for multi-node deployment). Defines the pod and container spec for the leader node that coordinates distributed token generation in multi-node deployments.

Worker nodes configuration (only used for multi-node deployment). Defines the pod and container spec for worker nodes that perform distributed token generation tasks as directed by the leader.

Since v1.3. TopologyKey is the node-label key (e.g. an NVLink/RDMA fabric-domain label) used to co-locate all pods of one multi-node (leader+worker) gang into a single network / NVLink topology domain. When set on a multi-node component, OME auto-generates the per-instance worker→leader podAffinity that anchors every worker to its gang's leader on a node sharing this label value — so users need not hand-write that affinity. Only meaningful for multi-node components; ignored for single-pod (no leader to anchor to). Unset means no auto-generated gang affinity.

Since v1.3. TopologySpread spreads this component's instances across fault domains so losing one domain (rack / fabric slice / cube) cannot take out every instance at once. Rendered as a standard topologySpreadConstraint (maxSkew 1) on each instance's anchor pod (the leader, or the sole pod of a single-pod instance); workers follow their leader, so gangs stay whole. Required renders as DoNotSchedule — balanced spreading enforced by any scheduler that keeps PodTopologySpread's Filter, including the OME gang scheduler's packing profile. Preferred renders as ScheduleAnyway — advisory only, and currently without effect under the OME gang scheduler (its profile disables the spreading Score and pre-narrows gang candidates to one domain). Not part of the revision hash: changing it shapes future placements without rolling a healthy fleet. Unset keeps pure bin-packing.

Allowed values: Preferred, Required.

Since v1.3. TopologySpreadKey is the node-label key naming the fault domains TopologySpread spreads across. Defaults to TopologyKey — right for fabrics where the co-location domain IS the failure zone (an NVLink rack). Set it to a coarser label when the failure zone contains many gang domains: e.g. co-locate by the TPU partition label while spreading across cloud.google.com/gce-topology-subblock. Single-pod components have no co-location key, so spreading them requires setting this explicitly. Ignored unless TopologySpread is set.

AcceleratorOverride allows overriding the global accelerator selection for this component

Library providing the component implementation, e.g., "diffusers" or "transformers".

Type is the fully qualified class name for the component, e.g., "FlowMatchEulerDiscreteScheduler".

ClassName is the pipeline implementation, e.g., "StableDiffusionXLPipeline" or "QwenImagePipeline".

Scheduler component used by the pipeline.

TextEncoder component used by the pipeline.

Tokenizer component used by the pipeline.

Transformer (UNet/DiT) component used by the pipeline.

VAE component used by the pipeline.

AdditionalComponents captures any other pipeline parts keyed by their model_index.json entry.

URL represents the endpoint URL for the inference service.

Pattern: ^(http|https)://.

APIFormat specifies the type of API, such as "openai" or "oci-cohere".

Allowed values: openai, oci-cohere, cohere.

ModelName specifies the name of the model being served at the endpoint. Useful for endpoints that require model-specific configuration. For instance, for openai API, this is a required field in the payload

Type selects where to read the endpoint override from.

Allowed values: Header, Metadata.

Headers to read the endpoint reference from. Required when Type=Header. Each header value must be "<host>:<port>" or "<ip>:<port>".

InferenceService holds a Kubernetes reference to an internal inference service.

Endpoint holds the details of a direct endpoint for an external inference service, including URL and API details.

This spec provides a full PodSpec for the engine component. Allows complete customization of the Kubernetes Pod configuration including containers, volumes, security contexts, affinity rules, and other pod settings.

The fields of PodSpec appear directly in this object.

ComponentExtensionSpec defines deployment configuration like min/max replicas, scaling metrics, etc. Controls scaling behavior and resource allocation for the engine component.

The fields of ComponentExtensionSpec appear directly in this object.

Runner container override for customizing the engine container. This is essentially a container spec that can override the default container. Defines the main model runner container configuration, including image, resource requests/limits, environment variables, and command.

Leader node configuration (only used for multi-node deployment). Defines the pod and container spec for the leader node that coordinates distributed inference in multi-node deployments.

Worker nodes configuration (only used for multi-node deployment). Defines the pod and container spec for worker nodes that perform distributed processing tasks as directed by the leader.

Since v1.3. TopologyKey is the node-label key (e.g. an NVLink/RDMA fabric-domain label) used to co-locate all pods of one multi-node (leader+worker) gang into a single network / NVLink topology domain. When set on a multi-node component, OME auto-generates the per-instance worker→leader podAffinity that anchors every worker to its gang's leader on a node sharing this label value — so users need not hand-write that affinity. Only meaningful for multi-node components; ignored for single-pod (no leader to anchor to). Unset means no auto-generated gang affinity.

Since v1.3. TopologySpread spreads this component's instances across fault domains so losing one domain (rack / fabric slice / cube) cannot take out every instance at once. Rendered as a standard topologySpreadConstraint (maxSkew 1) on each instance's anchor pod (the leader, or the sole pod of a single-pod instance); workers follow their leader, so gangs stay whole. Required renders as DoNotSchedule — balanced spreading enforced by any scheduler that keeps PodTopologySpread's Filter, including the OME gang scheduler's packing profile. Preferred renders as ScheduleAnyway — advisory only, and currently without effect under the OME gang scheduler (its profile disables the spreading Score and pre-narrows gang candidates to one domain). Not part of the revision hash: changing it shapes future placements without rolling a healthy fleet. Unset keeps pure bin-packing.

Allowed values: Preferred, Required.

Since v1.3. TopologySpreadKey is the node-label key naming the fault domains TopologySpread spreads across. Defaults to TopologyKey — right for fabrics where the co-location domain IS the failure zone (an NVLink rack). Set it to a coarser label when the failure zone contains many gang domains: e.g. co-locate by the TPU partition label while spreading across cloud.google.com/gce-topology-subblock. Single-pod components have no co-location key, so spreading them requires setting this explicitly. Ignored unless TopologySpread is set.

AcceleratorOverride allows overriding the global accelerator selection for this component

Name of component to which the failure relates (usually Pod name)

High level class of failure

Detailed error message

Internal Revision/ID of model, tied to specific Spec contents

Time failure occurred or was discovered

Exit status from the last termination of the container

FailureThreshold is the number of consecutive failed metric evaluations after which KEDA serves the fallback (it trips on the threshold+1-th consecutive failure).

Minimum: 1.

Replicas is the fallback replica count.

Reference to the base model that this weight is fine-tuned from

ModelType of the fine-tuned weight, e.g., "Distillation", "Adapter", "Tfew"

HyperParameters used for fine-tuning, stored as generic JSON for flexibility

ModelExtension is the common extension of the model

The fields of ModelExtensionSpec appear directly in this object.

Configuration of the fine-tuned weight, stored as generic JSON for flexibility

Storage configuration for the fine-tuned weight

TrainingJobRef references the training job that produced this weight

Steps is the ordered progression of capacity + service-level traffic. The final step must reach 100% traffic for the rollout to complete.

Maximum items: 20.

Prometheus is the metrics source (connection) shared by every analysis step in this canary: where to query and how to authenticate. When omitted (or when its serverAddress is empty), the operator-configured default source (canaryAnalysis.bundledPrometheusAddress) is used; if that is also unset the analysis has no source and its samples read inconclusive. It is only the source — WHAT to check is each step's analysis — and it gates nothing by itself.

ScaleDownDelaySeconds is the wait between shifting traffic off the old revision and scaling its pods down, to drain in-flight requests.

ReadyTimeout bounds how long a step's capacity gate may stay unsatisfied before the canary is marked Failed (stable keeps serving — no traffic has shifted). Unset falls back to the operator-configured default (the rollout block of the operator's ConfigMap); the ome.io/rollout-ready-timeout annotation still overrides both.

MaxSurge is the maximum pods (fraction or absolute) above desired during the roll. Default 25%.

MaxUnavailable is the maximum pods (fraction or absolute) that may be not-Ready at once during the roll. Default 25% (matches the Kubernetes Deployment convention; a default of 0 would deadlock single-replica rolls).

Metrics are passed through verbatim to autoscalingv2.HorizontalPodAutoscalerSpec.Metrics. Supports Resource, ContainerResource, Pods, Object, and External metric sources. If empty, the controller defaults to a single CPU=80% Resource metric.

Behavior controls scaling stabilization windows and per-direction policies.

Name is the cookie name.

Minimum length: 1.

TTLSeconds, when greater than zero, makes the gateway auto-issue the cookie if absent on the request. When zero or unset, the client is responsible for sending the cookie.

Minimum: 0.

Name is the header name (case-insensitive per HTTP).

Minimum length: 1.

Name of the secret containing the Hugging Face API key. The secret must reside in the same namespace as the BenchmarkJob.

GracePeriodSeconds is accepted so manifests that set it keep validating and has no effect on the roll: a drained pod is deleted as soon as it leaves rotation, and its own terminationGracePeriodSeconds and preStop hooks cover the in-flight work it still owes.

Minimum: 0.

MarkNotReadyDuringLifecycle, when true, flips ome.io/serving=False on the pod before the in-place mutation so EndpointSlice drains traffic first. Unset inherits the ServingRuntime's value, then true.

Partition holds back updates: the Partition lowest-indexed Instances not yet on the target revision, skipping any already updating to it, keep their current revision. When set it takes precedence over lifecycle.updateStrategy.rollingUpdate.partition; nil defers to it.

Minimum: 0.

MaxUnavailable is reserved; setting it has no effect. Disruption during a rollout is paced by lifecycle.updateStrategy.rollingUpdate.maxUnavailable.

RollbackToRevision, when set, names a ControllerRevision the replica rolls every Instance back to: that revision's stored pod template becomes the update target while UpdateRevision keeps reporting the spec's own revision. The InferenceService controller sets it during a canary rollback; empty in steady state. If the named revision does not exist, the replica rolls to its spec. A revision this replica does not control is ignored and reported with a Warning event.

PlacementExecution is allocation authority projected from a derived service. Its generation must be observed before the control plane uses member evidence. Controller-only.

PlacementReplicaLimit reserves the largest replica count this component may request while placement pauses growth. Only a raised placement floor can increase this limit; autoscaler requests remain in Replicas. Controller-only.

Minimum: 1.

ParentRef names the InferenceService that projects this replica. The InferenceService controller sets it at create time and it is immutable thereafter; a standalone replica omits it. Pod, Service and ControllerRevision names derive from the parent name when set and from the replica's own name otherwise. Controller-only.

Component is the role this pod set fills: engine | decoder | router. Immutable; moving a workload between roles requires recreating the InferenceReplica.

ModelRef names the BaseModel (in the replica's namespace) or ClusterBaseModel this replica serves. With it set the controller renders the pods from the model and the runtime: the runtime named by RuntimeRef, or the runtime selected for the model when RuntimeRef is absent. Fine-tuned weights and overlays render as they do on an InferenceService. Exclusive with Runners. User field.

RuntimeRef names the ServingRuntime (in the replica's namespace) or ClusterServingRuntime whose piece for Component this replica renders. The runtime must declare that piece. Without ModelRef the piece renders as-is, with no model mounted. The runtime's pod spec places the pods (node selector, affinity, resources); a second accelerator pool is a second runtime. A pin (autoSync=false, revision) is not honored: the live runtime renders, and the validating webhook rejects a pinned reference. Exclusive with Runners. User field.

Replicas is the desired Instance count; nil or 0 runs one Instance. Whatever targets the scale subresource (HPA, KEDA or an external scaler) writes this field. On a projected replica the InferenceService controller writes the component's minReplicas (at least 1) at create, and thereafter while no autoscaler owns the count or the stored count is nil or 0.

Minimum: 0.

MinReadySeconds is the minimum time a newly Ready pod must stay Ready before it counts as Available. The workload engine paces rollout drains and promotions on Available pods and counts only Available pods in availableReplicas; 0 means Available as soon as Ready. On a projected replica the InferenceService controller copies the component's effective lifecycle.minReadySeconds: the InferenceService's value, else the runtime's, else the operator's deploy default.

Minimum: 0.

TopologyKey is the gang co-location node-label key for this Component. When set on a multi-node Component, the InferenceReplica controller auto-generates the per-Instance worker→leader podAffinity that co-locates every worker onto its gang's leader on a node sharing this label value. Nil means no auto-generated gang affinity. On a projected replica the value is the effective InferenceService or runtime component setting.

TopologySpread is the spreading policy for this Component. The InferenceReplica controller renders it as a topologySpreadConstraint on each Instance's anchor pod; nil keeps pure bin-packing. On a projected replica the value is the effective InferenceService or runtime component setting.

Allowed values: Preferred, Required.

TopologySpreadKey is the fault-domain node-label key TopologySpread spreads across; nil defaults to TopologyKey. On a projected replica the value is the effective InferenceService or runtime component setting.

PairingProtocol is the engine/decoder wire-compatibility token. It is folded into the revision hash (a change mints a new revision) and stamped as the ome.io/pairing-protocol label on rendered pods; nil pairs with anything. Only engine and decoder replicas pair on it; a change on any replica still mints a new revision. On a projected replica the InferenceService controller copies spec.rollout.pairingProtocol onto the engine and decoder replicas and never onto the router, which must not re-roll on a protocol change.

Maximum length: 63. Pattern: ^[A-Za-z0-9]([-A-Za-z0-9_.]*[A-Za-z0-9])?$.

Runners is the fully-rendered set of pod templates per Instance. A single-pod Instance has one Runner named "default" with Size=1; a multi-node Instance has "leader" (Size=1) and "worker" (Size=N). Required unless ModelRef or RuntimeRef is set, in which case it must be absent: the controller renders the runners from the refs and never stores them. The controller treats each Runner.Template as opaque input. User field.

Lifecycle holds the OMENative lifecycle policies (RestartPolicy, UpdateStrategy, ReadyPolicy, InstanceReadyTimeout, MigrationPolicy). An unset policy takes the fixed fallback its LifecycleSpec field names; an unset or zero instanceReadyTimeout falls back to the operator's lifecycle.instanceReadyTimeout on every replica, and with neither set operations open with no deadline. The operator's deploy defaults are applied only to a projected replica, before the copy. lifecycle.minReadySeconds is not read on a replica and is rejected on a standalone one; set spec.minReadySeconds. On a projected replica this is the component's effective lifecycle: the InferenceService's value, else the runtime's, with unset fields filled from the operator's deploy defaults.

Pacing is rollout control written onto this replica: the canary partition, the rollback target and a reserved disruption budget. The InferenceService controller writes it on the replicas it orchestrates; on a standalone replica its owner sets it. Nil means the replica rolls independently.

Autoscaler is the autoscaler configuration downstream scalers (HPA / KEDA / External) target. Only the InferenceService controller creates a scaler from it, so on a standalone replica the HPA and KEDA classes are rejected and an External scaler targets the scale subresource directly. On a projected replica the InferenceService controller writes the resolved per-Component autoscaler, replacing the whole block, whenever that autoscaler resolves; while an autoscaler policy fails to render, the stored block is kept as the last known good one.

Paused, when true, stops the InferenceReplica controller from initiating fleet changes: Update, Create, and Migration operations are neither started nor advanced. The component's RestartPolicy keeps repairing existing Instances at their current revision unless PauseMode is Freeze. In-flight operations resume on unpause.

PauseMode selects how much of the lifecycle Paused suspends. Only meaningful while Paused is true.

• "Recover" (default when empty): fleet changes are suspended; the RestartPolicy continues to repair existing Instances at the revision they already run.
• "Freeze": every lifecycle operation is suspended, including Instance repair. Only kubelet-level container restarts and deliberate replica reduction (scale-down) remain active. Status stays truthful in every depth: a Ready Instance whose pods are all gone (and whose RestartPolicy will not repair it) is demoted to Pending without running any recovery operation.

Allowed values: Recover, Freeze.

RevisionHistoryLimit caps how many non-live ControllerRevisions the InferenceReplica controller retains. Live revisions (CurrentRevision / UpdateRevision and every per-Instance running/target revision) are never deleted regardless of the limit. Nil falls back to the operator-level lifecycle.revisionHistoryLimit config; when that is also absent, no revisions are pruned. On a projected replica the InferenceService controller copies the parent's ome.io/revision-history-limit annotation.

Minimum: 1.

ObservedGeneration is the InferenceReplica.metadata.generation the most recent status flush reflects.

PlacementObservedGeneration identifies the generation reconciled with placement rollout and replica-growth guards. The control plane requires this acknowledgement before sharing surge allowance with the member.

Minimum: 1.

Replicas is the total Instance count (any phase).

ReadyReplicas is the number of Instances whose every pod is Ready.

ServingReplicas is the number of Instances whose every pod is both ContainersReady AND has serving=True on the controller-managed readiness gate (i.e., actually in the load balancer rotation). MaxUnavailable budgets are computed against this count, not ReadyReplicas, so "missing from traffic" reflects reality during in-place updates.

AvailableReplicas is the number of Instances whose desired pod count is Available: in rotation on the Component's headless Service and, when spec.minReadySeconds is set, Ready for at least that long.

UpdatedReplicas is the number of Instances on UpdateRevision.

UpdatedReadyReplicas is the number of Instances on UpdateRevision that are fully Ready. UpdatedReadyReplicas / Replicas drives MaxUnavailable rollout pacing.

CurrentRevision names the ControllerRevision currently serving traffic. Set by the InferenceReplica controller.

UpdateRevision names the ControllerRevision being rolled out.

CollisionCount is the salt folded into ControllerRevision hash input after a name collision is observed, so retries produce a different revision name. Same pattern as StatefulSet.

LabelSelector identifies pods of this InferenceReplica for the HPA scale subresource. Uses the same selector formula as the Component's pod labels, so HPAs targeting either the ISVC or the InferenceReplica resolve to the same pod set.

InstanceStatuses is the DenseV1 representation of the per-Instance status, one row per Instance. Absent when instanceStatusEncoding is ColumnarV2, in which case instanceStatusColumns carries the same rows.

InstanceStatusEncoding names the representation that carries the per-Instance rows. Absent means DenseV1 (instanceStatuses). The only accepted value is ColumnarV2, which requires instanceStatusColumns and forbids instanceStatuses.

InstanceStatusColumns is the ColumnarV2 representation of the per-Instance rows. Present exactly when instanceStatusEncoding is ColumnarV2; it decodes to the same rows, in the same order, that instanceStatuses would carry.

RetryBlocks records per-target-revision retry authority for update attempts. Revision-scoped: one block per failed target revision, shared by every Instance attempting it — allocating a new Instance cannot evade the block.

Migrations is the authoritative set of migration records for this InferenceReplica — one entry per request UUID. Manual entries are resumable processes born Accepted; Auto entries are born-terminal Relocated records. Executors select on non-terminal phase only — terminal entries are records, never work. Non-terminal entries are never trimmed; terminal entries are pruned once older than the migration-capacity rate window, so the list stays bounded. Full history remains in the audit ledger and events.

Conditions reports InferenceReplica-level conditions. Defined types include Available, Progressing, FailedScale, FailedUpdate, GangSchedulingUnavailable, OperationStuck.

Traffic reports per-revision traffic weights for this InferenceReplica. Each entry corresponds to one revision currently receiving traffic. The ISVC controller mirrors this onto ISVC.status.components[c].traffic[] for the HTTPRoute builder.

CoordinationGroupRef back-references the RolloutCoordinationGroup this replica is currently bound into (if any). Lets operators find the owning coord group from the child without parsing the ISVC spec.

RolloutHold records the most recent reason a per-Instance Update was denied for this Component — cross-Component coordination (Ratio, Sequential), a surge/unavailability budget, or a same-target retry hold (RetryBlock, Held). Nil while the Component is progressing or has no rollout in flight. See RolloutHold for the field semantics.

Name specifies the name of the inference service to benchmark.

Namespace specifies the Kubernetes namespace where the inference service is deployed. Cross-namespace references are allowed but require appropriate RBAC permissions.

Since v1.3. DeploymentMode selects the dispatch backend that drives every Component on this InferenceService. When set, it propagates to the Engine, Decoder, and Router resolver at read-time without mutating per-Component annotations — kubectl get -o yaml stays clean. A per-Component ome.io/deploymentMode annotation still wins as an explicit override (the escape hatch is preserved for backward compat and mixed-mode experiments).

Valid values:

• "OMENative" — Native lifecycle-managed pods (also handles multi-node serving when Leader/Worker are present)
• "RawDeployment" — Kubernetes Deployment/HPA-backed dispatch

MultiNode (LeaderWorkerSet-backed multi-node serving) is selected via the ome.io/deploymentMode annotation or the operator-level default, never via this field. PDDisaggregated is a shape descriptor derived from engine+decoder pairing, and VirtualDeployment is legacy; neither is accepted here.

Allowed values: OMENative, RawDeployment.

Engine defines the serving engine spec. This provides detailed container and pod specifications for model serving. It allows defining the model runner (container spec), as well as complete pod specifications including init containers, sidecar containers, and other pod-level configurations. Engine can also be configured for multi-node deployments using leader and worker specifications.

Decoder defines the decoder spec. This is specifically used for PD (Prefill-Decode) disaggregated serving deployments. Similar to Engine in structure, it allows for container and pod specifications, but is only utilized when implementing the disaggregated serving pattern to separate the prefill and decode phases of inference.

Model defines the model to be used for inference, referencing either a BaseModel or a custom model. This allows models to be managed independently of the serving configuration.

Runtime defines the serving runtime environment that will be used to execute the model. It is an inference service spec template that determines how the service should be deployed. Runtime is optional - if not defined, the operator will automatically select the best runtime based on the model's size, architecture, format, quantization, and framework.

Router defines the router spec

ReplicaRefs names, per role, the standalone InferenceReplicas this service fronts instead of rendering the role itself. The service creates the role's stable Service (<name>-engine, -decoder, -router) and the route in front of their pods and writes nothing on the replicas: their specs, rollouts and scaling stay with their owner. A service references either all of its roles this way or none, so spec.engine, spec.decoder and spec.router are unset with it, as are spec.model and spec.runtime (the replicas carry their own); engine is required. Fixed at create. A replica that needs a peer's address (an engine its decoder, a router its engine and decoder) is configured by its owner with the stable Service names, which exist once the service does.

AcceleratorSelector specifies accelerator selection preferences

Since v1.3. Traffic configures the load-balancing policy applied to all HTTPRoutes OME emits for this InferenceService. See the TrafficSpec doc for details.

Since v1.3. Rollout is the single home for rollout configuration on this InferenceService: an ordered list of rollout groups, each polymorphic in its progression (canary | blueGreen | rollingUpdate). Sequencing is the group list order; a group's Components roll together.

Since v1.3. ScalingPolicy coordinates replica counts across Components when more than one autoscaler would otherwise drift (e.g., engine + decoder in a PD-disaggregated service that requires 1:1 sharding). When nil, every Component autoscales independently. Alpha. The API may change without notice.

Since v1.3. Placement declares how the multi-cluster control plane selects workload clusters and allocates replicas. An absent block declares no placement intent. Derived member services carry allocation authority in metadata.

Since v1.3. Routing configures cross-cluster traffic distribution for this InferenceService. Unset fields inherit the operator-level routing configuration. Only consulted on the control-plane cluster; ignored in single-cluster deployments. Alpha; the API may change without notice.

Since v1.3. TTLSecondsAfterCreation limits how long this InferenceService lives. Once that many seconds have passed since its creationTimestamp, the controller deletes it, and garbage collection removes everything it owns. Unset means it lives until deleted. The deadline is computed from creationTimestamp on every reconcile, so changing or clearing the field moves or cancels it. Placement member copies ignore it; the source InferenceService's TTL governs them.

Minimum: 1.

Conditions for the InferenceService:

• EngineReady: engine readiness condition.
• DecoderReady: decoder readiness condition.
• RouterReady: router readiness condition.
• IngressReady: ingress resource readiness.
• Ready: component aggregate on workload clusters, or placement serving readiness on a multi-cluster control-plane source.

The fields of Status appear directly in this object.

Addressable endpoint for the InferenceService

URL holds the url that will distribute traffic over the provided traffic targets. It generally has the form http[s]://{route-name}.{route-namespace}.{cluster-level-suffix}

Since v1.3. Addresses lists every reachable endpoint for this service — one per gateway its route attaches to, plus the cluster-local address. Each entry's Name carries the operator-declared class (see IngressGatewaySpec.Class): "internal", "external", "cluster-local", or any config-defined value. Derived from the same builder that produces the HTTPRoutes, so it never drifts from the actual routing. status.url is the primary gateway entry and status.address is the "cluster-local" entry — both projections of this list.

Statuses for the components of the InferenceService

Model related statuses

Since v1.3. MigrationHistory is a rolling window of OMENative migration requests processed against this InferenceService. Bounded to the most recent entries; older entries are dropped (and optionally archived to a controller-owned ConfigMap).

Since v1.3. MountedOverlays lists overlays the controller attached to the pod spec. Skipped overlays (disabled, NotReady, NotFound) surface via the OverlaysReady condition.

Since v1.3. Traffic reflects the resolved backend traffic policy and the emitted policy resource. Populated only when traffic intent is declared via spec.traffic or any ome.io/* traffic annotation; otherwise nil so older clients see nothing.

Since v1.3. Canary mirrors the canary run of the unit that owns the ISVC entrypoint (the router's when it has one, else the engine's); the per-unit copy on ComponentStatusSpec.Canary is authoritative. Absent until a canary first arms; afterwards it carries that unit's most recent run, a finished one included (see CanaryStatus).

Since v1.3. Placement reports multi-cluster placement of this ISVC, set by the control-plane fan-out controller. Empty on workload clusters and on single-cluster deployments.

PinnedRevisionName is the ControllerRevision (OME ns) driving pod specs. Empty when autoSync is true.

LastRuntimeSyncToken is the ome.io/runtime-sync annotation value at last pin advance; the controller advances the pin when the live annotation differs from this.

Since v1.3. Rollout is the run-model surface: the pinned active-run plan the executors consume, the previous run's bounded record, and the always-current per-group source resolution (including shadowed-policy previews).

Since v1.3. RolloutCoordination reports per-group cross-Component coordination state for the InferenceService. Populated only when spec.rollout declares blueGreen/rollingUpdate groups (canary groups report under status.canary instead).

ID is the idempotency key for this operation. Retries with the same ID are a no-op.

Type identifies the operation kind: Create, Update, Restart, Migrate, Delete.

Allowed values: Create, Update, Restart, Migrate, Delete.

Step is the fine-grained resume point within the operation (e.g., Drain, DeletePods, WaitZero, Recreate, WaitReady).

StartedAt is when this operation began.

LastProgressAt is when the operation last advanced. Used for stall detection.

Deadline is the hard timeout for the current Step. Zero (null) means the step clock is parked (e.g. while the instance's pods are admission-gated); optional + nullable so the parked null is accepted by the apiserver.

RetryCount is the per-step escalation counter.

TargetRevision is the ControllerRevision the operation is converging toward.

Reason is a short cause string for the operation. Set once, when the operation starts, by whichever state machine owns it (the restart trigger, the revision-roll cause): it names WHY the operation exists and must survive for the operation's lifetime.

Waiting is a short token naming an EXTERNAL condition currently holding the operation back — e.g. "QuotaExceeded" when admission refuses the operation's pods for lack of quota. Distinct from Reason, which names why the operation exists: Waiting comes and goes while the operation runs, so overloading Reason would erase the operation's cause. Empty means nothing external is blocking. A token only: the blocking authority's own message is volatile (an exceeded-quota message moves as other workloads come and go) and belongs in the Event, not in status.

CapacityRefusedAt is when admission last refused one of the operation's pod creates for lack of quota. The refusal is an apiserver answer to a call the create pass made, so it cannot be read off the cluster on a later pass: it is recorded here, and the hold pass converts it into the Waiting token and the deadline park on every pass it stands. Cleared by a create that completes with no refusal. Empty means the last create was not refused for capacity.

Strategy is the update strategy the operation runs under, pinned when the operation opens. UpdateStrategy is not part of the revision payload, so editing it retargets nothing: an attempt keeps the mechanism it started with — a patch already under way is not re-dispatched as a recreate, a surge already spent is not unwound — and the edit reaches the Instance at its next admitted attempt. Set only when Type=Update.

SurgeIndex is the Instance index allocated for a surge replacement. Set only when Type=Migrate.

FromNode is the node the migrating Instance is being moved off. Set only when Type=Migrate.

HintTargetNodes is the caller-supplied preferred-node list for a migration. Set only when Type=Migrate.

RequestUUID is the migration request UUID. Set only when Type=Migrate.

Members is the index set of every Instance that has a row.

Minimum length: 1. Pattern: ^(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?(,(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?)*$.

RowOrder is the dense row order when it is not the ascending Members order: every member exactly once, in the order the DenseV1 list would carry. Absent means ascending Members order.

Minimum items: 1. Minimum per item: 0.

Phases groups Instances by lifecycle phase. Every member appears in exactly one group; there is no default phase.

Minimum items: 1.

RunningRevisions groups Instances by nonempty running revision. A member absent from every group has an empty running revision.

Minimum items: 1.

TargetRevisions groups Instances by nonempty target revision. A member absent from every group has an empty target revision.

Minimum items: 1.

Incarnations groups Instances by nonzero incarnation. A member absent from every group has incarnation 0.

Minimum items: 1.

PodCounts groups Instances by positive podCount. A member absent from every group has podCount 0.

Minimum items: 1.

ServingPodCounts groups Instances by positive servingPodCount. A member absent from every group has servingPodCount 0.

Minimum items: 1.

AvailablePodCounts groups Instances by positive availablePodCount. A member absent from every group has availablePodCount 0.

Minimum items: 1.

Admitted is the index set of Instances whose admitted is true. Absent means no Instance is admitted.

Minimum length: 1. Pattern: ^(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?(,(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?)*$.

ActiveOrdinalOne is the index set of Instances whose activeOrdinal is 1. Absent means every activeOrdinal is 0.

Minimum length: 1. Pattern: ^(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?(,(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?)*$.

Entries carries the per-Instance records that are not grouped: conditions, readySince, operation, and lastFailure. At most one entry per member, in ascending index order, each carrying at least one of those records. A member without an entry has none of them.

Minimum items: 1.

Value is the shared positive count.

Minimum: 1.

Indexes is the canonical index set of the Instances with this value.

Minimum length: 1. Pattern: ^(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?(,(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?)*$.

Index is the Instance this entry belongs to; it is a member.

Minimum: 0.

Conditions is the Instance's conditions.

Minimum items: 1.

ReadySince is the Instance's readySince.

Operation is the Instance's in-flight operation record.

LastFailure is the Instance's preserved failure diagnostics.

Announced is the Instance's record of once-only messages already delivered.

Minimum items: 1.

Value is the shared nonzero incarnation.

Indexes is the canonical index set of the Instances with this value.

Minimum length: 1. Pattern: ^(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?(,(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?)*$.

Value is the shared lifecycle phase.

Indexes is the canonical index set of the Instances with this value.

Minimum length: 1. Pattern: ^(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?(,(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?)*$.

Value is the shared revision name; the empty revision is never grouped.

Minimum length: 1.

Indexes is the canonical index set of the Instances with this value.

Minimum length: 1. Pattern: ^(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?(,(0|[1-9][0-9]*)(-(0|[1-9][0-9]*))?)*$.

PodName is the name of the pod that failed or was escalated.

ContainerName is the container the diagnostics were read from. Empty when the signal is pod-level (e.g. a pod-phase Failed with no per-container terminated state available).

Reason is the kubelet reason for the termination or wedge — a terminated-state reason (OOMKilled, Error, ContainerCannotRun) or a terminal waiting-state reason (CrashLoopBackOff, ImagePullBackOff, CreateContainerError). Falls back to "PodFailed" when only the pod phase is available.

ExitCode is the container's exit code when the diagnostics came from a terminated state. Nil when the pod was stuck in a waiting state (no process ever ran to produce an exit code).

Message is the kubelet-supplied detail message, if any.

Time is when OMENative recorded this termination (reconcile time, not the kubelet transition time, which isn't always exposed).

Triggers are passed through verbatim to kedav1.ScaledObjectSpec.Triggers. The full KEDA trigger surface (prometheus, cron, external, kafka, ...) is supported.

Minimum items: 1.

Advanced ScaledObject configuration — horizontalPodAutoscalerConfig, restoreToOriginalReplicaCount.

PollingInterval is the interval, in seconds, at which KEDA checks each trigger. Defaults to the KEDA-side default when unset.

CooldownPeriod is the time, in seconds, KEDA waits before scaling down after the last trigger fires. Defaults to the KEDA-side default when unset.

IdleReplicaCount enables scaling to a fixed count when no triggers fire (commonly 0 for scale-to-zero). Must be strictly less than the effective MinReplicaCount.

Fallback policy applied when the metric source becomes unavailable.

Minimum items: 1.

Advanced ScaledObject configuration, verbatim.

Type is the KEDA trigger type ("prometheus" in v1). Admission requires ProviderRef for network-endpoint trigger types.

Minimum length: 1.

ProviderRef binds the trigger's endpoint by logical provider name.

MetricType is forwarded to the rendered trigger.

Allowed values: AverageValue, Value.

QueryReturnsDesiredReplicas declares that the trigger's query already computes the desired replica count. When true, admission requires MetricType=AverageValue: the HPA's Value math is ceil((metric/threshold) x readyPods), which breaks desired-count queries.

Metadata is the KEDA trigger metadata. Admission rejects the keys "serverAddress" and "authModes" (the provider binding owns them), and requires "ignoreNullValues" to be explicit on prometheus triggers.

SecretRef is the Secret holding the kubeconfig.

Key is the Secret data key holding the kubeconfig. Defaults to "kubeconfig".

Default: kubeconfig.

Pod specification for the leader node. This overrides the main PodSpec when specified. Allows customization of the Kubernetes Pod configuration specifically for the leader node.

The fields of PodSpec appear directly in this object.

Runner container override for customizing the main container. This is essentially a container spec that can override the default container. Provides fine-grained control over the container that executes the leader node's coordination logic.

RestartPolicy controls what OMENative does when a managed pod fails. Distinct from the inlined corev1.PodSpec.RestartPolicy: that one tells kubelet whether to restart a container in-place; this one tells the OMENative controller whether to recreate the whole Instance. Unset inherits the ServingRuntime's value; when neither sets it, multi-pod Instances use RecreateInstanceOnPodRestart and single-pod Instances use None.

UpdateStrategy controls how OMENative rolls template changes across the Component's Instances.

ReadyPolicy controls how Instance-level readiness is aggregated from the underlying pods. None is accepted only for single-pod Instances, where it is equivalent to AllPodReady; admission rejects None on multi-pod OMENative Components because per-pod readiness reporting is not supported. Unset inherits the ServingRuntime's value; when neither sets it, multi-pod Instances use AllPodReady and single-pod Instances use None.

InstanceReadyTimeout is OMENative's upper bound for waiting on a newly-created Instance to become Ready (during migration, rollout, or initial creation).

MinReadySeconds is the minimum time a newly Ready pod must stay Ready before OMENative treats it as Available. A pod is Available when its PodReady condition is True and that condition's lastTransitionTime plus MinReadySeconds is not after the current time, evaluated per pod (the Deployment.spec.minReadySeconds rule). Ready remains the health signal; Available is the pacing signal: a rollout drains or promotes an Instance only once its new pods are Available, so the maxSurge / maxUnavailable budgets stay held for the whole window, and availableReplicas / availablePodCount count only Available pods. Unset or 0 means Available as soon as Ready. A value authored on the InferenceService wins; an unset value inherits the ServingRuntime's; when neither sets it, the cluster default from the inferenceservice-config ConfigMap applies.

Minimum: 0.

MigrationPolicy controls whether and how OMENative honors a migration request annotation for this Component.

ObservedGeneration is the InferenceService.metadata.generation reflected by this block.

CurrentRevision names the ControllerRevision currently serving traffic.

UpdateRevision names the ControllerRevision being rolled out.

CollisionCount is the salt folded into ControllerRevision hash input after a name collision is observed, so retries produce a different revision name. Same pattern as StatefulSet.

LabelSelector identifies pods of this Component for HPA-style consumers.

Replicas is the total Instance count (any phase).

ReadyReplicas is the number of Instances whose every pod is Ready.

ServingReplicas is the number of Instances whose every pod is BOTH ContainersReady AND has serving=True on the controller-managed readiness gate — i.e., the pod is actually in the load balancer's rotation. This is the count operators care about for traffic availability, and the count MaxUnavailable budgets gate against.

Diverges from ReadyReplicas during in-place updates and any other scenario where the controller flips the serving gate to False while containers remain technically Ready.

AvailableReplicas is the number of Instances whose desired pod count is Available: in rotation on the Component's headless Service and, when lifecycle.minReadySeconds is set, Ready for at least that long.

UpdatedReplicas is the number of Instances on UpdateRevision.

UpdatedReadyReplicas is the number of Instances on UpdateRevision and fully Ready. UpdatedReadyReplicas / Replicas drives maxUnavailable rollout pacing.

Conditions reports Component-level conditions. Defined types include Available, Progressing, FailedScale, FailedUpdate, GangSchedulingUnavailable, OperationStuck.

RolloutHold reports the most recent reason this Component's rollout did not advance, mirrored verbatim from the owning InferenceReplica's status. Nil while the Component is progressing or has no rollout in flight.

Tolerance is the maximum percentage drift from the starting ratio. A roll step that would exceed it is paused until the pools rebalance. An explicit 0 means zero drift. When omitted, the operator-configured default (the coordination block of the operator's ConfigMap) applies; if the operator configures no default either, the group rolls with no drift bound.

Minimum: 0. Maximum: 100.

Minimum length: 1.

At is the time the event was recorded.

Message describes the observation.

ID is the request UUID supplied by the caller.

Component identifies which Component was migrated.

Instance is the original Instance index that was migrated away from.

ReplacementInstance is the Instance index allocated for the surge replacement. Nil when the migration was rejected before allocation. Distinct from index 0, which is a valid replacement index after sparse migration.

Mode is the migration mechanism. Surge is the only defined mode.

Phase is the current lifecycle phase.

RequestedAt is the time the request annotation was first observed.

StartedAt is the time the controller began executing the migration.

CompletedAt is the time the migration reached a terminal phase.

Reason is the caller-supplied reason (e.g., "fragmentation").

RequestedBy identifies the caller that wrote the request annotation.

Events records observations made during the migration.

OutcomeReason is a short machine-readable reason describing why the migration ended in its terminal phase (e.g., "InstanceReadyTimeout").

Mode gates migration: Auto enables both deadline-disposition relocation and explicit migration requests; Never disables both. Unset inherits the ServingRuntime's value, then Auto.

RequestUUID uniquely identifies the migration request. Manual: supplied by the requester. Auto: generated by the controller.

Trigger records who initiated the migration.

SourceInstance is the Instance index being migrated away from.

SurgeInstance is the Instance index allocated for the surge replacement. Manual only; nil until allocated. Distinct from index 0, which is a valid surge index.

AllocatedAt is when the surge Instance index was allocated — the start of execution. Nil while the entry is queued (Accepted but not yet picked up by the executor). The capacity gate counts execution from this stamp, never queued intent.

FromNode is the node the source is being moved off. Auto: the excluded node.

HintTargetNodes are requester-supplied preferred placement targets for the surge replacement. Rendered as preferred (soft) node affinity; the scheduler may pick other nodes.

Phase is the migration lifecycle phase. See MigrationPhase for the Manual chain, the Auto born-terminal rule, and the terminal set.

Attempt is the relocation attempt ordinal. Auto: N of autoMigrate.maxAttempts.

Reason is the requester-supplied reason (Manual) or the disposition branch that produced the entry (Auto).

Message describes the current blocker (non-terminal) or the terminal outcome.

StartedAt is when the entry was created.

Deadline is when a non-terminal entry expires. Manual: accept time plus InstanceReadyTimeout.

CompletedAt is when the entry reached a terminal phase.

Succeeded reports the post-hoc outcome. Auto: stamped true when the relocated Instance next reaches Ready.

Backend identifies the configured cache provider.

SourceUri is the model source URI being cached.

How many copies of the model failed to load recently.

Default: 0.

How many copies of the model are loaded.

DisplayName is the user-friendly name of the model

Whether the model is enabled or not

Vendor of the model, e.g., "NVIDIA", "Meta", "HuggingFace"

CompartmentID is the compartment ID of the model

Name of the format in which the model is stored, e.g., "ONNX", "TensorFlow SavedModel", "PyTorch", "SafeTensors"

Version of the model format. Runtime selection uses it to check that a runtime supports the model. It can be "major", "major.minor" or "major.minor.patch".

Operator for the selector with supported values: "Equal", "GreaterThan" This is used to select the serving runtime based on the modelFormat version

Default: Equal.

Weight of the model format in the runtime selector, used to prioritize modelFormat

Default: 1.

Name of the library in which the model is stored, e.g., "ONNXRuntime", "TensorFlow", "PyTorch", "Transformer", "TensorRTLLM"

Version of the library. Runtime selection uses it to check that a runtime supports the model. It can be "major", "major.minor" or "major.minor.patch".

Operator for the selector with supported values: "Equal", "GreaterThan" This is used to select the serving runtime based on the modelFramework version

Default: Equal.

Weight of the framework in the runtime selector, used to prioritize modelFramework

Default: 1.

Default: ClusterBaseModel.

Default: ome.io.

Name of the model being referenced. Identifies the specific model to be used for inference.

Kind of the model being referenced. Specifies the Kubernetes resource kind of the referenced model.

Default: ClusterBaseModel.

APIGroup of the resource being referenced. Specifies the Kubernetes API group of the referenced model.

Default: ome.io.

FineTunedWeights names the FineTunedWeight resources to apply to the base model.

Since v1.3. Overlays declares additional BaseModels attached to every serving pod alongside the primary. Each overlay is mounted (PVC/PerNode) or env-addressable (Sharded) at /opt/ml/model-overlays/<name> and exposed via OVERLAY_<UPPERCASED_NAME>_MODEL_PATH. Runner selects at request time; OME does not detect failures or auto-swap. Disabled or NotReady overlays are silently omitted.

High level state string: Pending, Standby, Loading, Loaded, FailedToLoad

Default: Pending.

Default: "".

Minimum model parameter count supported by this runtime, given as a string with an optional suffix M (million), B (billion), or T (trillion), e.g. "1B", "7B", "70B", "1T". It is compared against the model's parameter size during runtime auto-selection. A value not in the <number>[M|B|T] form is treated as 0.

Maximum model parameter count supported by this runtime, given as a string with an optional suffix M (million), B (billion), or T (trillion), e.g. "1B", "7B", "70B", "1T". It is compared against the model's parameter size during runtime auto-selection. A value not in the <number>[M|B|T] form is treated as 0.

Whether the pods serving the model reflect the current spec or are in transition. Empty when the InferenceService is "lean" (no spec.model is declared) and no model-loading lifecycle applies; the controller leaves this field unset in that case. Populated as "InProgress" → "UpToDate" (or "BlockedByFailedLoad"/"InvalidSpec") by the controller once a model is referenced.

The model's load state in the pods that serve it.

Details of last failure, when load of target model is failed or blocked.

How many copies of the model are loaded, and how many failed.

LifeCycle is the model's lifecycle stage, such as Deprecated, Experiment, Public or Internal. The API server doesn't check the value, and OME doesn't set it.

Status of the model weight

ObservedGeneration is the .metadata.generation the controller last reconciled against. If this lags behind .metadata.generation, the controller has not yet observed the latest spec edit.

LastReconcileTime is the wall-clock timestamp of the controller's most recent reconcile pass for this object. Liveness signal — if it falls behind by more than the reconcile period, the controller is wedged or not scheduled. Distinct from Condition.lastTransitionTime, which only advances on actual status flips.

Conditions describe cluster-wide model readiness. OME sets them only for pvc:// models, which report SourceReachable and Ready. Other models report per-node state through NodesReady and NodesFailed.

Since v1.3. Cache identifies the external cache behind Sharded distribution, which is in development. OME doesn't set it yet.

MountPath is empty for Sharded overlays (fetched at runtime).