kubectl ome wait Unreleased: coming in v1.3

kubectl ome wait blocks until an InferenceService reports the state you name with --for, then prints one report. It exits 0 when it saw the state, 2 when the wait ended without it, and 1 on an error, so a script can branch on the result. It only reads, and gives up after --timeout: 1 minute by default, at most 24 hours.

That default suits a state that should already hold. Model loads, rollouts and migrations take longer, so set --timeout for them, such as --timeout=30m.

kubectl ome wait INFERENCESERVICE --for=PREDICATE [flags]

Predicates¶

--for takes one of these predicates:

The migration, replica and held-revision predicates read a component's InferenceReplica (IR), which OME writes only for components that run on OMENative. They see only the replicas the service owns, so on a service that fronts pools through spec.replicaRefs they never match: see Which replicas the predicates read.

Each predicate takes only the flags in its row, and refuses others with an error. The first check runs at once, so a state that already holds matches immediately. After that, condition=Ready and rollout watch the InferenceService, and the other predicates check every 5 seconds. Every wait ends with one of these outcomes:

NotFound, Deleted and Replaced end the wait right away, so create the InferenceService before you wait on it.

Status that's missing or malformed never matches: the report's validity row shows Unavailable or Invalid, and the wait goes on. The migration, replica and held-revision predicates also wait while an InferenceReplica lags its own spec, and the last two while it lags the InferenceService's. The replica and held-revision predicates follow one InferenceReplica, the one you name or the one they find at the first check. If it's replaced or deleted, the wait can't match and runs until the timeout. kubectl ome instance list shows what's wrong with an InferenceReplica.

The CLI treats a status timestamp later than your machine's clock as invalid, so when that clock runs behind the cluster's, a match waits until it catches up.

Which replicas the predicates read¶

The migration, replica and held-revision predicates keep only the InferenceReplicas that claim the InferenceService: spec.parentRef names it, and the service is the replica's controller owner reference. OME writes both on every InferenceReplica it creates for a service, so those appear. A standalone replica — an InferenceReplica you create directly — claims no service, whatever its name.

An InferenceService whose spec.replicaRefs fronts standalone replicas writes nothing on them, so they don't claim it, and it has no InferenceReplicas of its own. On such a service, these predicates can never match, even while the fronted pools serve. Every check reports the same reason, and the wait runs to TimedOut, exit 2:

To wait on a fronting service, use condition=Ready, which reads the InferenceService itself. rollout reads the service too, but a fronting service can't set spec.rollout, so that wait reports NotConfigured and never matches.

You can't wait on a replica instead: the argument names an InferenceService, which the first read GETs, so naming a standalone replica ends NotFound, exit 2. To follow a pool replica's own counts, read the replica itself. kubectl get inferencereplica shows them as DESIRED, CURRENT and READY, and -w prints a row on every change:

kubectl get inferencereplica pool-a -n prod -w

NAME     COMPONENT   DESIRED   CURRENT   READY   AVAILABLE   AGE
pool-a   engine      4         4         3       3           2d

In a script, block on a count with kubectl wait inferencereplica/pool-a -n prod --for=jsonpath='{.status.readyReplicas}'=4 --timeout=30m. kubectl ome get ir shows which replicas a service owns: a standalone replica's PARENT column shows -.

condition=Ready¶

kubectl ome wait INFERENCESERVICE --for=condition=Ready[=True|False|Unknown] [--timeout=DURATION]

Waits for the status of the InferenceService's Ready condition: True when you give none. condition=Ready=Unknown needs a Ready condition with that status. While there's none, the report shows NotRecorded, and the wait goes on. Ready conditions that disagree never match.

rollout¶

kubectl ome wait INFERENCESERVICE --for=rollout=stable|failed|rolled-back [--timeout=DURATION]

Waits for the rollout state that kubectl ome rollout status shows as REPORTED: stable waits for Succeeded, failed for Failed and rolled-back for RolledBack. Any other state keeps the wait going. So rollout=stable runs until its timeout after a rollout fails, and a canary waiting at a gate keeps it waiting. A service without rollouts reports NotConfigured, which never matches: wait for condition=Ready instead.

EpochUnverifiable and AnalysisInconclusive show as Rollout issue rows but don't block a match. EpochUnverifiable shows for any service with rollout groups.

migration=terminal¶

kubectl ome wait INFERENCESERVICE --for=migration=terminal --request-id=UUID [--timeout=DURATION]

Waits for the end of one Instance migration, named by the request ID that the alpha kubectl ome migration start prints. It matches when the request reaches Completed, Failed or Relocated, and all three exit 0. To tell them apart, read the Migration outcome row, or content.migration.migrationOutcome in JSON.

A request queues behind earlier migrations of the same component, including those from Alfred, which is alpha, and behind an update's surge Instance in flight on that component. It fails at its deadline, by default 30 minutes after OME accepts it. So give --timeout at least that, such as --timeout=30m, and more when a rollout may pause the migration.

replicas=ready¶

kubectl ome wait INFERENCESERVICE --for=replicas=ready --component=engine|decoder|router --replicas=N [--timeout=DURATION]

Waits until the component's InferenceReplica reports exactly N ready replicas in status.readyReplicas. N can be 0.

replicas=current¶

kubectl ome wait INFERENCESERVICE --for=replicas=current --component=engine|decoder|router --replicas=N [--ir-name=NAME --ir-uid=UID] [--timeout=DURATION]

Waits until the component's InferenceReplica asks for and reports exactly N Instances, in spec.replicas and status.replicas. N must be at least 1.

To wait on the InferenceReplica that a scale request acted on, pass --ir-name and --ir-uid. Take them from the target.name and target.uid that the alpha kubectl ome scale prints with -o json.

runtime-sync=acknowledged¶

kubectl ome wait INFERENCESERVICE --for=runtime-sync=acknowledged --request-id=UUID [--timeout=DURATION]

Waits until OME acknowledges a sync that the alpha kubectl ome runtime sync requested. Pass the request ID that command prints. OME acknowledges the sync when it repins the service to the runtime's current spec: it copies the request's token from the ome.io/runtime-sync annotation to status.lastRuntimeSyncToken.

The wait matches once the token is acknowledged, the service still has a managed pin, with autoSync: false and no revision, and there's no RuntimeDrifted condition. A service placed by multi-cluster placement, which is alpha, never matches.

If the pin already matches the runtime's current spec, OME doesn't repin, and Token state stays Pending until the timeout. A newer sync request replaces your token: Token state shows Superseded, and the wait can't match.

held-revision=unheld¶

kubectl ome wait INFERENCESERVICE --for=held-revision=unheld --component=engine|decoder|router --revision=REVISION --ir-name=NAME --ir-uid=UID [--timeout=DURATION]

Waits until a revision that OME held after repeated failures is no longer held on an InferenceReplica. Take the flag values from the result of the alpha kubectl ome instance release-held. Pass its target.name and target.uid as --ir-name and --ir-uid. For --revision, pass the full revision name, <service>-<component>-<hash>: add the prefix to the result's revisionHash, or copy the name from kubectl ome instance retry-blocks.

The wait matches once the revision's retry block is no longer Held and the ome.io/release-held-revision annotation is gone.

What a match means¶

Exit code 0 means a check saw the requested state on the InferenceService the wait is bound to. It isn't a receipt for a command you ran. To wait for one request, use a predicate that takes its ID: migration=terminal or runtime-sync=acknowledged.

After a spec change, the replica counts can include Instances of the old revision.

Flags¶

Plus the standard kubeconfig flags, such as -n and --context.

Output fields¶

The command prints one report when the wait ends, as a table with the columns FIELD and VALUE. The table cuts long labels and values with ..., and -o wide adds rows. Output formats describes the formats.

-o json and -o yaml print every value in full, in a report with apiVersion: cli.ome.io/v1alpha1 and kind: WaitReport. Its content holds:

• requested, outcome, reason and evidence;
• observed, the Ready condition, with status, validity, generationFreshness and inspection;
• one object for the other predicates: rollout, migration, readyReplicas, scale, runtimeSync or heldRevision;
• elapsedMilliseconds, counts, sourceMethod and pollingFallback.

These rows appear in every report, unless the row says otherwise:

The Request ID, Target ready, Target replicas and Revision rows show your flag values, as does Component outside migration=terminal. The caveat rows, Expansion, Count authority, Read scope, Cancellation, and Attribution outside migration=terminal, print fixed notes. What a match means explains the caveats.

condition=Ready fields¶

rollout fields¶

Reported rollout, Rollout state, Rollout epoch and Coordination Ready show REPORTED, STATE, EPOCH and COORDINATION from rollout status.

-o wide adds counts of what the CLI inspected, then Cancellation.

migration=terminal fields¶

-o wide adds Inspected IR sources and Inspected records, counts of what the CLI read, then Cancellation.

replicas=ready fields¶

replicas=current fields¶

Spec replicas, Current replicas and Ready replicas show <unavailable>, and Status encoding is empty, when the snapshot isn't valid.

runtime-sync=acknowledged fields¶

-o wide adds Generation freshness, Inspected conditions, Read scope and Cancellation.

held-revision=unheld fields¶

-o wide adds InferenceReplica UID, Status encoding, Inspected sources / blocks, Read scope and Cancellation.

Examples¶

Wait for chat in prod to be ready, allowing 30 minutes for the model to load:

kubectl ome wait chat --for=condition=Ready --timeout=30m -n prod

FIELD                   VALUE
Service                 prod/chat
Requested               Ready=True
Outcome                 Matched
Observed Ready          True
Condition validity      Valid
Reason                  ConditionMatched
Evidence                Reported
Generation freshness    Unverifiable
Freshness caveat        Reported condition; not current-spec convergence
Condition inspection    Complete (1/1)
Source method           InitialGET
Polling fallback        false
Elapsed milliseconds    0
GET / WATCH / polls     1 / 0 / 0
Events / observations   0 / 1

The same wait as JSON:

kubectl ome wait chat --for=condition=Ready --timeout=30m -n prod -o json

{
  "apiVersion": "cli.ome.io/v1alpha1",
  "kind": "WaitReport",
  "metadata": {
    "namespace": "prod",
    "name": "chat"
  },
  "collectedAt": "2026-09-28T14:05:12Z",
  "sources": [],
  "content": {
    "requested": "Ready=True",
    "outcome": "Matched",
    "reason": "ConditionMatched",
    "observed": {
      "status": "True",
      "validity": "Valid",
      "generationFreshness": "Unverifiable",
      "inspection": {
        "state": "Complete",
        "total": 1,
        "inspected": 1,
        "warnings": []
      }
    },
    "evidence": "Reported",
    "elapsedMilliseconds": 0,
    "counts": {
      "gets": 1,
      "watches": 0,
      "polls": 0,
      "events": 0,
      "observations": 1
    },
    "sourceMethod": "InitialGET",
    "pollingFallback": false
  },
  "warnings": []
}

In a script, keep the report and branch on the exit code. Here the wait timed out after 2 minutes:

kubectl ome wait chat --for=condition=Ready --timeout=2m -n prod -o json > wait.json
echo "exit code $?"
jq -r '.content.outcome + " " + .content.reason' wait.json

error: RequestedConditionUnmet
exit code 2
TimedOut ConditionNotMatched

To see why, run kubectl ome status or follow Troubleshoot an InferenceService.

Wait for the rollout of chat to succeed:

kubectl ome wait chat --for=rollout=stable --timeout=2m -n prod

FIELD                   VALUE
Service                 prod/chat
Requested               Rollout=Stable
Outcome                 Matched
Reported rollout        Succeeded
Rollout state           Unknown
Rollout validity        Valid
Reason                  RolloutMatched
Evidence                Reported
Rollout epoch           Unverifiable
Coordination Ready      NotApplicable
Freshness caveat        Reported rollout; not current-spec convergence
Attribution caveat      Same-object state; not action attribution
Rollout inspection      Complete
Expansion               JSON/YAML retain complete safe report values
Source method           InitialGET
Polling fallback        false
Elapsed milliseconds    0
GET / WATCH / polls     1 / 0 / 0
Events / observations   0 / 1
Rollout issue           EpochUnverifiable

Rollout state shows Unknown, with the issue EpochUnverifiable, because the CLI can't tie the reported state to the current spec.

Wait for a migration to end, with the request ID that kubectl ome migration start printed:

kubectl ome wait chat --for=migration=terminal --request-id=12345678-1234-4234-8234-123456789abc --timeout=30m -n prod

FIELD                  VALUE
Service                prod/chat
Requested              Migration=Terminal
Request ID             12345678-1234-4234-8234-123456789abc
Outcome                Matched
Migration phase        Completed
Migration outcome      Completed
Component              engine
Migration validity     Valid
Reason                 MigrationMatched
Evidence               Reported
Attribution            Exact request ID in live IR status
Source method          InitialGET
Polling fallback       false
Elapsed milliseconds   0
GET / WATCH / polls    1 / 0 / 0

Wait for the engine of chat to have exactly two ready replicas, with the rows -o wide adds:

kubectl ome wait chat --for=replicas=ready --component=engine --replicas=2 -n prod -o wide

FIELD                  VALUE
Service                prod/chat
Requested              Replicas=Ready
Component              engine
Target ready           2
Outcome                Matched
Observed ready         2
Count validity         Valid
Reason                 ReplicaReadyMatched
Evidence               Reported
Interpretation         Exact at observed IR snapshot
Source method          InitialGET
Polling fallback       false
Elapsed milliseconds   0
GET / WATCH / polls    1 / 0 / 0
Count authority        IR status; not /scale or an action receipt
Readiness caveat       Ready count; not serving or availability
Cancellation           Cooperative requests; plugins may ignore it

Wait for the engine of chat to ask for and report four Instances:

kubectl ome wait chat --for=replicas=current --component=engine --replicas=4 -n prod

FIELD                  VALUE
Service                prod/chat
Requested              Replicas=Current
Component              engine
Target replicas        4
Outcome                Matched
Spec replicas          4
Current replicas       4
Ready replicas         3
Status encoding        DenseV1
Validity               Valid
Freshness              Current
Reason                 ReplicaScaleMatched
Evidence               Reported
Interpretation         Exact desired and current count at IR snapshot
Count caveat           Current = logical Instances any phase; Ready separate
Attribution            Exact IR snapshot; not action attribution
Source method          InitialGET
GET / WATCH / polls    1 / 0 / 0
Elapsed milliseconds   0

Wait for OME to acknowledge a runtime sync, with the request ID that kubectl ome runtime sync printed:

kubectl ome wait chat --for=runtime-sync=acknowledged --request-id=123e4567-e89b-42d3-a456-426614174000 -n prod

FIELD                  VALUE
Service                prod/chat
Requested              RuntimeSync=Acknowledged
Request ID             123e4567-e89b-42d3-a456-426614174000
Outcome                Matched
Token state            Acknowledged
Drift state            Clear
Managed pin            Managed
Placement              Direct
Validity               Valid
Reason                 RuntimeSyncObserved
Evidence               Reported
Interpretation         Token acknowledged in bound parent snapshot
Sync caveat            Reported token; not live-runtime convergence
Attribution            Same-object state; not action attribution
Source method          InitialGET
Polling fallback       false
Elapsed milliseconds   0
GET / WATCH / polls    1 / 0 / 0

Wait for a held revision to be released, with the values from the result of kubectl ome instance release-held:

kubectl ome wait chat --for=held-revision=unheld \
  --component=engine --revision=chat-engine-7f9c4d2b \
  --ir-name=chat-engine --ir-uid=3f6d2c9e-8b41-4d7a-9c0e-5a1b7e2f4d68 \
  --timeout=2m -n prod

FIELD                  VALUE
Service                prod/chat
Requested              HeldRevision=Unheld
Component              engine
Revision               chat-engine-7f9c4d2b
InferenceReplica       chat-engine
Outcome                Matched
Target state           Absent
Mailbox                Absent
Validity               Valid
Reason                 Unheld
Evidence               Reported
Interpretation         Exact revision no longer Held on original IR
Attribution            Unverifiable; not action attribution
State caveat           Pruning/re-Held races; no request receipt
Source method          InitialGET
GET / WATCH / polls    1 / 0 / 0
Elapsed milliseconds   0

Exit codes¶

Errors print to stderr as error: <message>. After exit code 2, the message is error: RequestedConditionUnmet. A second Ctrl-C stops the command at once.

Error messages¶

Related guides¶

• Release a held revision
• Request a transient scale
• Request an instance migration
• Let Alfred migrate Instances
• Troubleshoot an InferenceService