kubectl ome runtime Unreleased: coming in v1.3

kubectl ome runtime shows how OME picks the serving runtime for a model or an InferenceService, and what that runtime is made of. explain runs OME's runtime selection and says why each runtime qualifies or doesn't. effective shows the runtime an InferenceService uses and the revision it's pinned to, and history lists the revisions kept for that runtime. tree shows a runtime's inheritance chain, the runtimes that inherit from it, and the InferenceServices that name them. The alpha sync action asks the controller to move a pinned InferenceService to its runtime's current spec.

kubectl ome runtime explain (--model NAME | --isvc NAME) [flags]
kubectl ome runtime effective INFERENCESERVICE [flags]
kubectl ome runtime tree RUNTIME [flags]
kubectl ome runtime history INFERENCESERVICE [flags]
kubectl ome runtime sync INFERENCESERVICE [flags]

effective, history and sync take the name of one InferenceService, and so does explain --isvc. The CLI looks for it in the namespace you give with -n, then in the namespace of your kubeconfig context, then in default. explain --model looks in that namespace for a BaseModel with the name, then for a ClusterBaseModel. tree takes the name of a runtime: a ServingRuntime in that namespace, or a ClusterServingRuntime.

effective, history and sync read the InferenceService, its model, the runtime it names with the runtimes that runtime inherits from, and runtime revisions in the OME namespace. When the InferenceService names no runtime, effective and history also list the ServingRuntimes in its namespace and the ClusterServingRuntimes, and select one the way OME does. effective reads only the revisions named in spec.runtime.revision and status.pinnedRevisionName, while history and sync list every revision of the runtime. sync also lists the InferenceService's InferenceReplicas when a component runs on OMENative. explain reads the model, or the InferenceService and its model, and lists the ServingRuntimes in the namespace and the ClusterServingRuntimes. --with-effective adds what effective reads. tree lists runtimes and InferenceServices, and reads no revisions. The kubectl-ome overview lists the RBAC each command needs, and Guarded actions has the rule sync adds.

effective and history print a table, and tree prints a tree. For all three, -o wide prints more detail, and -o json and -o yaml print the whole report, with apiVersion: cli.ome.io/v1alpha1 and the kind RuntimeEffectiveReport, RuntimeHistoryReport or RuntimeTreeReport. sync prints its result, an ActionResult, as a table, or in full with -o json and -o yaml. Its -o wide prints the same table. On a terminal narrower than a table, the CLI wraps long cells, or prints each row as a list of fields when even the headers don't fit. Output to a file or pipe keeps one line per row. explain has no -o flag: it prints one table, which it always wraps to fit 80 columns, or a narrower terminal.

explain¶

kubectl ome runtime explain (--model NAME | --isvc NAME) [flags]

explain runs the runtime selection that OME runs for an InferenceService that names no runtime, and prints a row for every ServingRuntime in the namespace and every ClusterServingRuntime. With --model, it matches the runtimes against a BaseModel or ClusterBaseModel. With --isvc, it matches them against the InferenceService's model, and also checks the accelerator class and the deployment modes the InferenceService asks for. It takes the model's name from spec.model.name, and looks the model up the same way as --model, whatever spec.model.kind says.

When the InferenceService names a runtime in spec.runtime, OME doesn't run selection for it, as When you name a runtime describes. explain still prints the table, and writes Note: spec.runtime is explicit; automatic selection below is hypothetical. to stderr.

Flags¶

Plus the standard kubeconfig flags, such as -n and --context.

Give exactly one of --model and --isvc, or the command fails with exactly one of --model or --isvc is required. --with-effective without --isvc fails with --with-effective requires --isvc. --ome-namespace matters only with --with-effective, which reads runtime revisions there.

Output fields¶

Yes rows come first, in the order OME ranks them, so the first Yes row is the runtime OME would select. No rows follow, ServingRuntimes first, each group by name.

WEIGHT looks only at the first matching entry, and leaves out weights of 0 or less. The score OME ranks by takes the runtime's best entry, and counts a weight of 0 as 10 or 5. So WEIGHT equals the score only when the first matching entry gives the score and sets no weight of 0 or less. The command doesn't show the score itself.

A No row shows the first check the runtime fails, in this order:

REASON wraps to fit the table in 80 columns, and breaks inside a word when it has to.

If the InferenceService or the model doesn't exist, the command fails with the API server's error, such as clusterbasemodels.ome.io "llama-3-2-3b-instruct" not found when the name matches neither a BaseModel nor a ClusterBaseModel. An InferenceService without spec.model fails with InferenceService "<name>" has no spec.model; pass --model instead. explain reads at most 1,000 runtimes, the namespace's ServingRuntimes and the ClusterServingRuntimes together. With more, it fails with runtime candidate snapshot exceeds the CLI collection limit. When there are no runtimes at all, it prints no table, writes No serving runtimes found in the selected namespace or at cluster scope. to stderr, and exits 0.

Effective context¶

With --isvc, --with-effective adds a second table after a blank line and the heading Effective context (separate observation; selector verdict unchanged):. The CLI collects it separately from the selection above, and it doesn't change the verdicts. It starts with these rows:

The rows that effective prints follow. When the InferenceService names no runtime, the context selects one from at most 1,000 runtimes in all. With more, the Live view and INHERITANCE are Unavailable. If the CLI can't collect the context, it prints only two rows, Service EVIDENCE Unavailable and Service CAVEAT Selector verdict above is unchanged, and the command still exits 0. If you interrupt the command, it prints no context and fails with the error instead. A read that times out doesn't fail it: the view that needs the read is Unavailable.

When there are no runtimes at all, --with-effective writes No serving runtimes found for selector. to stderr instead, and still prints the context.

Examples¶

List the runtimes OME can choose for the BaseModel llama-3-2-3b-instruct in team-a:

kubectl ome runtime explain --model llama-3-2-3b-instruct -n team-a

RUNTIME        SCOPE        COMPATIBLE   PRIORITY   WEIGHT   REASON
team-llama     Namespaced   Yes          1          2        -
vllm-llama     Cluster      Yes          2          6        -
srt-llama-1b   Cluster      No           -          -        model size 3.21B is
                                                             outside supported
                                                             range [500M, 2B]
vllm-base      Cluster      No           -          -        supports the model
                                                             but has no
                                                             supportedModelForma
                                                             ts[].
                                                             autoSelect=true
                                                             entry, so automatic
                                                             selection skips it
                                                             (pin it explicitly
                                                             via
                                                             spec.runtime.name
                                                             instead)
vllm-legacy    Cluster      No           -          -        runtime is disabled

OME would select the ServingRuntime team-llama, although its WEIGHT is lower, because every ServingRuntime in the namespace ranks above every ClusterServingRuntime. team-llama's entry has weights of 1 and a priority of 1, so its WEIGHT is (1 + 1) × 1 = 2, and vllm-llama's is (2 + 1) × 2 = 6. srt-llama-1b serves only models from 500M to 2B parameters. vllm-base, the runtime vllm-llama inherits from, matches the model, but none of its entries sets autoSelect: true, so OME uses it only for an InferenceService that names it. vllm-legacy is disabled.

Explain the selection for the InferenceService llama-chat, which serves the same model:

kubectl ome runtime explain --isvc llama-chat -n team-a

It prints the same table, since llama-chat asks for no accelerator class or deployment mode. llama-chat names the ClusterServingRuntime vllm-llama in spec.runtime, so OME never runs this selection for it, and the command writes Note: spec.runtime is explicit; automatic selection below is hypothetical. to stderr.

Add the effective context. The command prints the same table, then a blank line and this:

kubectl ome runtime explain --isvc llama-chat -n team-a --with-effective

Effective context (separate observation; selector verdict unchanged):
SCOPE     FIELD           VALUE
Service   SELECTION       Explicit
Service   SOURCE          CSR/vllm-llama
Source    INHERITANCE     Observed
Source    ROOT-FIRST      CSR/vllm-base
Source    ROOT-FIRST      CSR/vllm-llama
Service   CAVEAT          Independent snapshot; not rollout convergence
Live      STATE           Available
Live      RUNTIME         CSR/vllm-llama
Live      HASH            3c18f4a0
Live      ENGINE          RawDeployment (Default)
Active    STATE           Available
Active    RUNTIME         CSR/vllm-llama
Active    REVISION        cr-vllm-llama-1baf272d
Active    HASH            1baf272d
Active    ENGINE          RawDeployment (Default)
Service   PIN             ManagedPin/Resolved
Service   SYNC            Absent
Service   STATUS          Current
Service   DRIFT           ReportedTrue/RevisionMismatch
Service   LIVE-RELATION   Different

llama-chat names vllm-llama, which inherits from vllm-base. The rest is what effective shows for llama-chat.

effective¶

kubectl ome runtime effective INFERENCESERVICE [flags]

effective shows the runtime behind an InferenceService in two views, which Live and pinned runtimes explains:

• The Live view is the runtime as it's defined now: the one spec.runtime names, or the one OME selects for the model, merged with every runtime it inherits from.
• The Active view is the runtime spec the InferenceService is meant to run: the revision it's pinned to, or the Live view when it isn't pinned.

The report also shows the pin, the runtime sync request and the RuntimeDrifted condition. It shows hashes of the specs, never the specs themselves. When spec.runtime names no runtime, the CLI selects one the way explain does, from at most 1,000 runtimes in all. With more, the Live view is Unavailable.

Flags¶

Plus the standard kubeconfig flags, such as -n and --context.

The CLI reads the runtime revisions in --ome-namespace.

Output fields¶

The table has one row for each value: Live rows first, then Active rows, then Service rows. STATE always shows, and the other rows show only when they have a value.

REASON is one of:

The component rows show each component the InferenceService has. The modes are RawDeployment, OMENative, VirtualDeployment and MultiNode (deprecated; move to OMENative). The source says where the mode comes from, in the order OME checks them, as How OME resolves the mode describes:

The pin mode comes from spec.runtime, as How pinning works describes:

The pin state is one of:

SYNC compares the ome.io/runtime-sync annotation with status.lastRuntimeSyncToken, where the controller records the last token it used, as Roll forward to the latest runtime describes:

STATUS compares status.observedGeneration with metadata.generation:

STATUS compares two numbers and says nothing more: it doesn't tell you whether the controller has acted on the current spec. The controller doesn't record the InferenceService's generation in status.observedGeneration. It copies the observedGeneration of a component's workload, such as its Deployment, which counts that workload's own changes, and components on OMENative don't set it at all. So Current means only that the copied number equals the generation, Stale and Invalid that it's lower or higher, and Unobserved that nothing has set it, which is usual when every component runs on OMENative. sync doesn't rely on it: its preview shows the freshness as Unverifiable.

DRIFT shows the InferenceService's RuntimeDrifted condition. OME sets the condition only to True, and removes it when the reason no longer applies. The state is one of:

The cause is the condition's reason: RevisionMismatch, RevisionMissing, SourceRuntimeMissing or RuntimeMismatch, or Other for an empty or unknown reason.

LIVE-RELATION compares the full hashes of the Live and Active specs. It's Equal when they're the same, Different when they differ, Ambiguous when they differ but their eight-character short forms, which HASH shows, are the same, and Unknown when either view has no hash.

The ISSUE rows show these codes:

A revision code shows the revision in parentheses, as in RevisionNotFound(<revision>), and effective reports it only for the revisions that the InferenceService uses, requests in spec.runtime.revision or reports in status.pinnedRevisionName. history adds two codes of its own.

The table keeps each value within 54 characters. A long runtime name or namespace, revision name or issue keeps its start and end with ... between them, followed by # and eight hex digits that tell it apart from similar names. Other long values keep their first 51 characters and end in ....

-o wide prints one row for each view and component, under VIEW, STATE, REASON, RUNTIME, REVISION, HASH, COMPONENT, MODE, MODE-SOURCE, PIN, PIN-STATE, SYNC, STATUS, DRIFT, LIVE-RELATION and ISSUES. It shows the runtime's full kind and name, and repeats the Service fields on each row. -o json and -o yaml add the runtime selection and the inheritance chain, which the tables leave out.

effective fails with InferenceService name "<name>" is invalid: ... for a name that isn't valid, with get InferenceService: ... when it can't read the InferenceService, and with collect runtime evidence: ... when it can't collect the rest.

Examples¶

Show the runtime behind llama-chat:

kubectl ome runtime effective llama-chat -n team-a

SCOPE     FIELD           VALUE
Live      STATE           Available
Live      RUNTIME         CSR/vllm-llama
Live      HASH            3c18f4a0
Live      ENGINE          RawDeployment (Default)
Active    STATE           Available
Active    RUNTIME         CSR/vllm-llama
Active    REVISION        cr-vllm-llama-1baf272d
Active    HASH            1baf272d
Active    ENGINE          RawDeployment (Default)
Service   PIN             ManagedPin/Resolved
Service   SYNC            Absent
Service   STATUS          Current
Service   DRIFT           ReportedTrue/RevisionMismatch
Service   LIVE-RELATION   Different

llama-chat names the ClusterServingRuntime vllm-llama with autoSync: false and no revision, so OME manages its pin, and the pinned revision cr-vllm-llama-1baf272d resolves. vllm-llama changed after OME pinned it: the Live view hashes to 3c18f4a0, so LIVE-RELATION is Different, and OME reports RuntimeDrifted True with the reason RevisionMismatch. Nothing has set ome.io/runtime-sync yet. The engine is a RawDeployment, since nothing sets its mode. sync can move llama-chat to the live spec.

Show the same report with a row for each view and component:

kubectl ome runtime effective llama-chat -n team-a -o wide

VIEW     STATE       REASON   RUNTIME                            REVISION                 HASH       COMPONENT   MODE            MODE-SOURCE   PIN          PIN-STATE   SYNC     STATUS    DRIFT                           LIVE-RELATION   ISSUES
Live     Available   -        ClusterServingRuntime/vllm-llama   -                        3c18f4a0   engine      RawDeployment   Default       ManagedPin   Resolved    Absent   Current   ReportedTrue/RevisionMismatch   Different       -
Active   Available   -        ClusterServingRuntime/vllm-llama   cr-vllm-llama-1baf272d   1baf272d   engine      RawDeployment   Default       ManagedPin   Resolved    Absent   Current   ReportedTrue/RevisionMismatch   Different       -

llama-chat has only an engine, so each view takes one row.

tree¶

kubectl ome runtime tree RUNTIME [flags]

tree shows how a runtime fits into runtime inheritance: the chain of runtimes it inherits from, the runtimes that inherit from it, and the InferenceServices that name any of them in spec.runtime. It lists runtimes and InferenceServices, and prints no specs, status, labels, annotations or resourceVersions.

Without --kind, the CLI looks for a ServingRuntime with the name in the namespace and a ClusterServingRuntime with the name. When both exist, it fails with runtime "<name>" is ambiguous; pass --kind ServingRuntime or --kind ClusterServingRuntime: runtime target is ambiguous: "<name>" matched 2 runtimes. For a ServingRuntime, it lists the ClusterServingRuntimes, and the ServingRuntimes and InferenceServices in its namespace. For a ClusterServingRuntime, it lists the ServingRuntimes and InferenceServices in every namespace, since any of them can use it. Each list holds at most 1,000 objects, read in two pages of 500. A list whose snapshot expires between pages starts over once, as history describes, and its Collection: line counts only the pages of the list it kept.

Flags¶

Plus the standard kubeconfig flags, such as -n and --context.

Output fields¶

The tree starts with RUNTIME TREE and a Target: line, and prints its lines in contexts:

Under each Head: line, the chain prints root first: the root runtime at the left, then on each line the runtime that inherits from the one above it, after `-- and indented four more spaces. The target has [selected] after it. The InferenceServices that name the head follow it, one level deeper, after |-- , or `-- for the last one. Inside a Namespaced context, the names leave out the namespace.

The CLI attributes an InferenceService to a runtime by its spec.runtime. With kind: ServingRuntime, it looks only for a ServingRuntime in the InferenceService's namespace. With kind: ClusterServingRuntime, it looks for the ClusterServingRuntime, then for a ServingRuntime in the namespace. With no kind, it looks for a ServingRuntime in the namespace, then for the ClusterServingRuntime.

With --show-unattributed-users, the tree adds Unattributed users (not attributed to runtime tree): before the Snapshot: line, with the listed InferenceServices whose runtime the CLI can't tell from their spec. Each shows as [not attributed] InferenceService/<namespace>/<name>, then its state and reason:

When there are none, the block shows none observed.

tree needs complete runtime lists. When a runtime list is truncated or fails, it fails with runtime tree requires complete runtime evidence: <kind> collection is <unavailable or truncated> (pages=<n> items=<n>); restore list access or narrow the runtime scope, then retry. When the InferenceService list is incomplete or fails, it prints the tree with Snapshot: Partial and warnings. It fails with collect InferenceServices: ... only when it runs out of time or you interrupt it.

The default output keeps each line within 80 columns. It moves the end of a long Issue: or Collection: line to an indented line below, and shortens a long name, adding a short fingerprint so that different names stay distinct. -o wide prints full names, except in the unattributed block. -o json and -o yaml print the RuntimeTreeReport.

tree also fails with runtime name "<name>" is invalid: ... for a name that isn't valid, with unsupported runtime kind "<kind>" (supported: ServingRuntime, ClusterServingRuntime) for another --kind, and with project runtime inheritance tree: runtime target was not found: "<name>" when the runtime doesn't exist.

Examples¶

Show the tree around the ClusterServingRuntime vllm-llama:

kubectl ome runtime tree vllm-llama -n team-a

RUNTIME TREE
Target: ClusterServingRuntime/vllm-llama
Context: Cluster (resolution: Complete)
Head: ClusterServingRuntime/vllm-llama
ClusterServingRuntime/vllm-base
`-- ClusterServingRuntime/vllm-llama [selected]
    |-- InferenceService/team-a/llama-chat
    `-- InferenceService/team-b/llama-batch
Context: Namespaced/team-a (resolution: Complete)
Head: ServingRuntime/team-llama
ClusterServingRuntime/vllm-base
`-- ClusterServingRuntime/vllm-llama [selected]
    `-- ServingRuntime/team-llama
        `-- InferenceService/llama-dev
Snapshot: Complete
Collection: ClusterServingRuntime scope=Cluster status=Complete pages=1 items=4
Collection: ServingRuntime scope=AllNamespaces status=Complete pages=1 items=1
Collection: InferenceService scope=AllNamespaces status=Complete pages=1 items=4

vllm-llama inherits from vllm-base, so each chain starts there. In the Cluster context, llama-chat in team-a and llama-batch in team-b name vllm-llama. The ServingRuntime team-llama inherits from vllm-llama, so it heads a path in team-a, where llama-dev names it. vllm-llama is a ClusterServingRuntime, so the CLI listed ServingRuntimes and InferenceServices in every namespace. The fourth InferenceService, llama-auto in team-b, names no runtime, so the tree leaves it out.

Show the InferenceServices the CLI can't attribute as well. The command prints the same tree, with these lines before Snapshot: Complete:

kubectl ome runtime tree vllm-llama -n team-a --show-unattributed-users

Unattributed users (not attributed to runtime tree):
  [not attributed] InferenceService/team-b/llama-auto
  state=Unresolved reason=AutomaticSelection

Show the tree around the ServingRuntime team-llama:

kubectl ome runtime tree team-llama -n team-a

RUNTIME TREE
Target: ServingRuntime/team-a/team-llama
Context: Namespaced/team-a (resolution: Complete)
Head: ServingRuntime/team-llama
ClusterServingRuntime/vllm-base
`-- ClusterServingRuntime/vllm-llama
    `-- ServingRuntime/team-llama [selected]
        `-- InferenceService/llama-dev
Snapshot: Complete
Collection: ClusterServingRuntime scope=Cluster status=Complete pages=1 items=4
Collection: ServingRuntime scope=Namespace/team-a
  status=Complete pages=1 items=1
Collection: InferenceService scope=Namespace/team-a
  status=Complete pages=1 items=2

For a ServingRuntime, the CLI lists only in its namespace. llama-chat names vllm-llama, not team-llama, so this tree leaves it out. The two longer Collection: lines continue on indented lines to stay within 80 columns.

history¶

kubectl ome runtime history INFERENCESERVICE [flags]

history lists the revisions of the runtime behind an InferenceService: the ControllerRevisions labeled ome.io/runtime-of=<runtime name> in the OME namespace, newest first. It finds the runtime the way effective does. OME deletes unused revisions beyond the newest ones, as Garbage collection describes, so the list shows what OME kept. The CLI lists at most 1,000 revisions, in two pages of 500.

A paginated list reads one snapshot of the API server's state, and the server can expire that snapshot between pages, refusing the next request with HTTP 410. The CLI then discards the pages it has and lists again from the start, once, so the report never mixes two snapshots. With the restart, it sends at most four list requests in all. When the second snapshot expires too, the list fails, and the report shows a partial or unavailable window instead of the command failing.

Runtimes with the same name share the label, so revisions of a same-named runtime in another namespace or scope show up too, and fail the check with RevisionSourceMismatch.

Flags¶

Plus the standard kubeconfig flags, such as -n and --context.

Output fields¶

The table keeps each line within 80 columns:

ASKED counts failed requests and the pages of a snapshot the CLI discarded, so it can exceed SEEN. In a window that's still C, the gap means the read recovered an expired snapshot: C/B/2/4 says the CLI read a page, lost the snapshot, and read the complete list again in two pages, so the discarded page and the refused request put ASKED two past SEEN. When the restarted list's snapshot expires too, or a request fails for another reason, OBS is P with the pages the CLI kept, or U when it kept none, and BOUND is I.

The command's help also lists N for OBS, which history never prints, since it always asks for the list. When the CLI lists no revisions, as when it can't find a runtime name, the table has one row with the window, - in the next five columns, and the issues.

Use -o wide for the name to put in spec.runtime.revision. It prints OBSERVATION, COMPLETENESS, PAGES, REVISION, CREATED, HASH, ROLES, SOURCE, CONSISTENCY, RELATION, REVISION-ISSUES and REPORT-ISSUES, with full names, times and hashes, and the issue codes that effective uses. PAGES repeats the window's last two counts, as SEEN/ASKED. It adds two report issues: HistoryUnavailable when the list failed, and HistoryTruncated when it holds more than the CLI read. -o json and -o yaml print the RuntimeHistoryReport.

history fails the same ways as effective.

Examples¶

List the revisions of the runtime behind llama-chat:

kubectl ome runtime history llama-chat -n team-a

WINDOW    REVISION         CREATED           ROLES   CHECK   LIVE    ISSUES
C/B/1/1   cr...#1c63d586   26-09-20T14:30Z   H       OK      MATCH   R0/G0
C/B/1/1   cr...#fd54e47d   26-09-01T08:00Z   ARH     OK      DIFF    R0/G0

The CLI read one page with one request, and the list is complete within what OME keeps. The newer revision holds the live spec. The older one is the revision llama-chat is pinned to: the Active view, and the one status.pinnedRevisionName reports. Both pass the check. The names are too long for the column, so the table shows digests of them.

Show the full names and hashes:

kubectl ome runtime history llama-chat -n team-a -o wide

OBSERVATION   COMPLETENESS       PAGES   REVISION                 CREATED                HASH       ROLES                     SOURCE                             CONSISTENCY   RELATION          REVISION-ISSUES   REPORT-ISSUES
Complete      RetentionBounded   1/1     cr-vllm-llama-3c18f4a0   2026-09-20T14:30:00Z   3c18f4a0   History                   ClusterServingRuntime/vllm-llama   Consistent    MatchesLive       -                 -
Complete      RetentionBounded   1/1     cr-vllm-llama-1baf272d   2026-09-01T08:00:00Z   1baf272d   Active,Reported,History   ClusterServingRuntime/vllm-llama   Consistent    DiffersFromLive   -                 -

cr-vllm-llama-3c18f4a0 holds the live spec, with the hash 3c18f4a0 that effective shows for the Live view.

sync¶

kubectl ome runtime sync INFERENCESERVICE [flags]

sync rolls a pinned InferenceService forward to its runtime's live spec. It sets the ome.io/runtime-sync annotation to cli-runtime-sync- followed by a new request ID, a random UUID. When the controller uses the token, it pins the InferenceService to a revision that holds the merged live spec, and records the token in status.lastRuntimeSyncToken, as Roll forward to the latest runtime describes. It records the token only when it moves the pin. If the live spec matches the pinned revision again before the controller gets to the token, the controller clears the drift, doesn't record the token, and SYNC stays Pending. The next change to the live spec then moves the pin without a new request. The token asks for the live spec at the time the controller uses it: the annotation doesn't lock the spec that the preview showed.

Nothing removes the annotation. Once the controller records the token, the annotation and status.lastRuntimeSyncToken hold the same value, and the next sync replaces it. While a token waits, sync refuses, and OME moves the pin as soon as the live spec differs from the pinned revision. To withdraw a token that the controller hasn't recorded, remove the annotation:

kubectl annotate inferenceservice llama-chat -n team-a ome.io/runtime-sync-

inferenceservice.ome.io/llama-chat annotated

SYNC then shows StatusOnly or Absent, and the pin stays where it is.

Unlike kubectl annotate, sync checks first that rolling forward is safe. It works only on an InferenceService that effective shows as ManagedPin/Resolved, with DRIFT ReportedTrue/RevisionMismatch and LIVE-RELATION Different. Refusals lists every check. kubectl ome runtime sync shows the command in use.

Flags¶

Plus the standard kubeconfig flags, such as -n and --context.

How the action runs¶

sync changes the InferenceService in a guarded way:

1. The CLI checks the flags and the name, reads what the action needs, and refuses if the action isn't safe now. A refusal changes nothing and exits 1.
2. It prints a preview of the exact change to stderr.
3. It asks Confirm this exact action? [y/N] on stderr, and goes ahead only on y or yes. --yes confirms without the prompt, and without a terminal you must pass it.
4. It reads the InferenceService, the runtimes, the revisions and the InferenceReplicas again. If anything it checked has changed, or it can't read it all again, it fails with runtime sync safety snapshot changed; refresh runtime effective and retry explicitly and exits 3.
5. It sends a JSON Patch that tests the UID and resourceVersion from the preview, and any earlier ome.io/runtime-sync value, before it sets the annotation. If the InferenceService changed after the CLI read it, the API server rejects the patch, and the CLI fails with guarded annotation patch rejected; refresh runtime effective and retry explicitly and exits 3.
6. It prints the result, an ActionResult, to stdout.

--dry-run client runs every check, prints the preview, asks for confirmation and reads everything again, then stops without sending the patch. --dry-run server sends the same patch with dryRun=All, so the API server runs its checks and stores nothing.

The command has 45 seconds in all, including the time the prompt waits for your answer, and each API request has at most 10 seconds, or less if you set a shorter --request-timeout. When the time runs out before the patch, the command fails with context deadline exceeded and exits 1. Without --yes, it fails with action not confirmed; noninteractive input requires --yes when you answer anything but y or yes, press Ctrl-D, or run it without a terminal. Ctrl-C at the prompt fails with context canceled. A declined or interrupted prompt sends no patch.

The preview starts with ALPHA runtime sync preview (acceptance is not convergence). It lists the context, the namespaces, and the InferenceService with its UID, resourceVersion, generation and reported generation. It then lists the pinned revision, each runtime in the chain, root first, with its UID, resourceVersion and generation, and the revision's UID and resourceVersion. It ends with the current and target hashes, the revision the CLI expects OME to pin, the pause state, the annotation and the new token, and warnings about what the action doesn't do. Some values need a note:

• Freshness is always Unverifiable (advisory global status), because sync doesn't rely on STATUS.
• A ClusterServingRuntime shows as ClusterServingRuntime//<name>, with an empty namespace between the slashes.
• Target evidence is Existing when a revision with the target hash exists, and then Target UID and Target RV follow. It's NotFound when the controller has to create the revision.
• Old sync token is the SYNC state from effective: Absent, StatusOnly or Acknowledged.
• Pause state is Paused, Freeze or Not recognized, from the ome.io/rollout-paused annotation. A pause doesn't make sync refuse, and stays in effect for the rollout that follows.

The ActionResult table shows:

The table cuts each value to 56 characters: a longer value keeps its first 53 characters and ends in .... So the accepted message prints as API accepted annotation request; consumption and conv..., and the follow-up command always prints cut. -o json and -o yaml print every value in full. -o wide prints the same table as the default.

applied means the API server stored the annotation, not that the controller used it. Run the follow-up command, or kubectl ome wait with the request ID, to see what the controller did.

When the CLI can't tell whether the patch went through, it says the outcome is unknown. A server error, a timeout or a network error fails with runtime sync request outcome unknown; check runtime effective; do not replay. A response that's too large or doesn't match the request fails with runtime sync response is unbound or oversized; outcome unknown, check runtime effective; do not replay, or runtime sync response is unbound; outcome unknown, check runtime effective; do not replay. If the CLI can't write the result, it fails with write runtime sync result failed; request outcome may be unknown, check runtime effective; do not replay. Check effective before you run sync again: a second sync refuses while an earlier token waits for the controller. Other API errors, such as a denied request, fail with required Kubernetes API request failed; check access and connectivity, and so does a failure to read the InferenceService at the start, even when it doesn't exist. The CLI never retries on its own. Each of these errors exits 1.

Guarded actions describes the contract every mutating kubectl-ome command follows. Like the other actions, sync refuses, and changes nothing, in these cases:

Refusals¶

Besides the refusals every action shares, sync refuses in these cases:

sync goes ahead only when all of these hold:

• spec.runtime names the runtime with a valid name, sets autoSync: false and no revision, and sets kind and apiGroup, if at all, to ServingRuntime or ClusterServingRuntime and ome.io.
• PIN is ManagedPin/Resolved, and the Active view is the revision in status.pinnedRevisionName.
• DRIFT is ReportedTrue/RevisionMismatch, and LIVE-RELATION is Different.
• SYNC isn't Pending, so no earlier token waits for the controller.
• The runtime isn't disabled after the merge. sync checks the Live view, the runtime merged with every runtime it inherits from, not each runtime in the chain alone. So the disabled profiles a runtime inherits from don't make it refuse: the runtime's disabled: false overrides their disabled: true. A runtime that inherits disabled: true without setting disabled: false is disabled after the merge, and sync refuses.
• No runtime in the chain is being deleted, and the chain has at most five runtimes, none of them twice.
• The runtime has at most 32 revisions, all consistent with what OME writes and none disabled, and at most one of them has the target hash.

Because runtimes with the same name share their revisions' label, a same-named runtime in another namespace or scope that has revisions makes sync refuse.

Examples¶

Run every check and see the preview, without sending the patch. --yes skips the prompt:

kubectl ome runtime sync llama-chat -n team-a --dry-run client --yes

The preview goes to stderr:

ALPHA runtime sync preview (acceptance is not convergence)
FIELD             VALUE
Action            runtime sync
Context           prod-us-east
Workload NS       team-a
OME NS            ome
Target            InferenceService/llama-chat
UID               7d3f9b2e-5c1a-4e8f-9a6b-2f4c8d1e0a37
ResourceVersion   4711
Dry-run           client
Generation        3
Reported gen      3
Freshness         Unverifiable (advisory global status)
Pin               cr-vllm-llama-1baf272d
Pin source        ClusterServingRuntime//vllm-llama
Source            ClusterServingRuntime//vllm-base
Source UID        a41c7e9d-2b5f-4c83-9e6a-1f8d3b7c05e2
Source RV         187
Source gen        1
Source            ClusterServingRuntime//vllm-llama
Source UID        3b8e1f6a-9c2d-4a7e-8f15-6d0b4c9a2e81
Source RV         212
Source gen        2
Pin UID           c8f4a2e6-1d9b-4b7a-8e3c-5a0f7d2b9e64
Pin RV            101
Current hash      1baf272d
Target hash       3c18f4a0
Predicted pin     cr-vllm-llama-3c18f4a0
Target evidence   Existing
Old sync token    Absent
Target UID        0e6b9d3a-4f2c-4e1b-b7a8-3c5d9f1e8a20
Target RV         102
Pause state       Not recognized
Request UUID      ddf745b1-4bf6-4de0-9933-16e97855fb55
Set annotation    ome.io/runtime-sync
New token         cli-runtime-sync-ddf745b1-4bf6-4de0-9933-16e97855fb55
API acceptance is not convergence or serving-revision readiness.
Requests latest merged live runtime when controller consumes the token.
Previewed content is not locked by this annotation.
Parent CAS is not an atomic transaction across runtimes, revisions and IRs.
Global observed generation is advisory; freshness is Unverifiable.
Existing pause/freeze and lifecycle/rollout policies remain in effect.

The result goes to stdout:

FIELD           VALUE
action          runtime sync
target          InferenceService/team-a/llama-chat
dry-run         client
accepted        No
applied         No
request-id      ddf745b1-4bf6-4de0-9933-16e97855fb55
revision-hash   3c18f4a0
message         Validated locally; no patch sent.
follow-up       kubectl ome runtime effective llama-chat -n team-a --...
hint            Use -o json or -o yaml for full values.

The CLI checked the chain from vllm-base to vllm-llama, and expects OME to pin cr-vllm-llama-3c18f4a0, a revision that already exists with the target hash, as history shows. The client dry run sent nothing, so accepted and applied are No, and the next run gets a new request ID.

Roll llama-chat forward, confirming at the prompt:

kubectl ome runtime sync llama-chat -n team-a

After you answer y, the CLI reads everything again and sends the patch. The result shows accepted and applied as Yes, a new request-id, and the message API accepted annotation request; consumption and conv....

Have the API server check the patch without storing it:

kubectl ome runtime sync llama-chat -n team-a --dry-run server --yes

The result shows accepted as Yes, applied as No, and the message API dry-run accepted; no changes persisted.

Wait until the controller has used the token, passing the request ID from the result:

kubectl ome wait llama-chat -n team-a --for=runtime-sync=acknowledged --request-id=5b2e7c1a-9d4f-4e8b-a6c3-0f1d2e3b4a59

wait checks every five seconds. It exits 0 once the annotation and status.lastRuntimeSyncToken both hold the token, the InferenceService has no RuntimeDrifted condition, and its pin is still managed. It exits 2 if that doesn't happen within 60 seconds, or the --timeout you set, or if the InferenceService is missing, deleted or replaced, and 1 on an error. The new pods can still be rolling out then.

Exit codes¶

No runtime subcommand returns 2. Errors print to stderr as error: <message>.

Related guides¶

• Reference a runtime explicitly
• Troubleshoot an InferenceService
• Troubleshoot runtime selection