kubectl ome get Unreleased: coming in v1.3

kubectl ome get lists OME resources, or shows one by name, with columns picked for each kind. An InferenceService row shows its model, runtime and readiness, and an InferenceReplica row shows its Instance counts. The models and runtimes views list base models and serving runtimes of both scopes in one table.

kubectl ome get RESOURCE [NAME] [flags]

Give one RESOURCE from Resources, in any letter case, and at most one NAME. Unlike kubectl get, get has no TYPE/NAME form and takes no .ome.io suffix. Required RBAC lists the permissions it needs.

Resources¶

kubectl get knows the short names irep and wlc, but kubectl ome get doesn't: use ir and wc, or the full names.

Merged views¶

models and runtimes list the namespaced kind first, then every object of the cluster-scoped kind. The SCOPE column says which kind each row is, and -l filters both. When either list fails, get prints only the error. If you can't list the cluster-scoped kind, use basemodels or servingruntimes.

With a NAME, get shows the namespaced object, or the cluster-scoped one when there's none. When a BaseModel and a ClusterBaseModel share a name, an InferenceService in the BaseModel's namespace gets the BaseModel too, even when it asks for the ClusterBaseModel. For an InferenceService's runtime, OME looks for the ClusterServingRuntime first, unless the InferenceService sets spec.runtime.kind: ServingRuntime. So get runtimes NAME can show a different runtime from the one an InferenceService runs, which kubectl ome status -o wide names.

Flags¶

Plus the standard kubeconfig flags, such as -n and --context.

A NAME can't be combined with -A or -l. For a cluster-scoped resource, get ignores -n, and ignores -A with a warning on stderr. -o wide adds the Wide columns in Output fields, where a resource has any. -o json and -o yaml print Kubernetes objects, not a report. With a NAME, that's the object; otherwise it's a List with apiVersion: v1, whose items carry their own apiVersion and kind.

Output fields¶

Every table starts with NAME and, by default, ends with AGE, the time since the object was created, such as 3h. With -A, namespaced resources get a NAMESPACE column before NAME, which shows - for the cluster-scoped rows of a merged view. A text cell shows - when its field isn't set, and rows marked Wide are the columns that -o wide adds.

Some columns recur:

The RolloutPolicy, AutoscalerPolicy and TrafficMap tables hide stale values: a condition from an older generation shows as Unknown, and its reason as -. The policy tables also show - for their digest and counts until the status is current. The other tables show the last status as it is.

inferenceservices¶

kubectl ome status shows the full readiness of one InferenceService.

inferencereplicas¶

OME writes an InferenceReplica for each OMENative component of an InferenceService, and you can also create one directly, with no parent InferenceService: a standalone replica. The view lists both. An InferenceReplica holds its component's replicas, each one an Instance: one pod, or a leader and its workers. kubectl ome instance lists the Instances themselves.

models, basemodels and clusterbasemodels¶

ARCH, PARAMS and FORMAT show - until the spec sets them or OME reads them from the model's files, as What OME learns from the model explains.

runtimes, servingruntimes and clusterservingruntimes¶

finetunedweights¶

acceleratorclasses¶

acceleratorquotas¶

The table has a row for each budget. kubectl ome quota shows the whole quota tree.

When the status isn't current or has no budgets, the table adds a Declared row for each budget in the spec, with - for ADMITTED, BORROWED and RESERVED. That lets you compare what you declared with what the quota manager last reported.

benchmarkjobs¶

The columns are NAME, STATE and AGE. STATE is status.state: Pending, Running, Completed or Failed, as Status describes.

rolloutpolicies¶

Alpha. Conditions and status lists the condition reasons.

autoscalerpolicies¶

Alpha. Conditions and status lists the condition reasons.

workloadclusters¶

Alpha. A WorkloadCluster registers a cluster for multi-cluster serving, which is still in development. kubectl ome cluster shows more of a WorkloadCluster's status.

trafficmaps¶

Alpha. A TrafficMap is the routing table of a multi-cluster InferenceService, and multi-cluster routing is still in development. Conditions lists the condition reasons.

Examples¶

The objects in these examples have no creation time, so AGE shows -.

List the InferenceServices in team-a:

kubectl ome get isvc -n team-a

NAME     MODEL           RUNTIME     READY     URL   AGE
a-isvc   llama-3-3-70b   srt-llama   Unknown   -     -
b-isvc   llama-3-3-70b   srt-llama   Unknown   -     -

Neither InferenceService has a Ready condition or a URL yet.

List the BaseModels in team-a and every ClusterBaseModel:

kubectl ome get models -n team-a

NAME            SCOPE        ARCH               PARAMS   FORMAT   STATE   AGE
ns-model        Namespaced   LlamaForCausalLM   -        -        -       -
cluster-model   Cluster      LlamaForCausalLM   -        -        -       -

These models set only an architecture, so the other columns show -.

List the InferenceReplicas in demo:

kubectl ome get ir -n demo

NAME          COMPONENT   PARENT   DESIRED   CURRENT   READY   LIFECYCLE    AGE
chat-engine   engine      chat     4         3         2       Ready=True   -
pool-a        engine      -        2         2         2       Ready=True   -

The engine of chat should run 4 Instances. It has 3, of which 2 are ready, and its Ready condition is True. pool-a is a standalone replica, created with no parent InferenceService, so its PARENT shows -.

List the accelerator quotas:

kubectl ome get aq

NAME     ROLE           PARENT   RESOURCE         FLAVOR   NOMINAL   ADMITTED   SOURCE     STATUS-FRESHNESS   READY   DEGRADED   AGE
team-a   ClusterQueue   root     nvidia.com/gpu   h100     8         3          Reported   Current            True    False      -

team-a has a share of 8 h100 GPUs, and 3 are admitted. The status describes the current spec, so the row shows the reported budget.

List the traffic maps in team-a, with the wide columns:

kubectl ome get tm -n team-a -o wide

NAME       MODE    TARGETS   ROUTABLE   PUBLISHED   AGE   SERVICE    ACTIVE   HEALTHY   OVERRIDE   OVERRIDE-REASON   REASON     GATEWAY                                               PUBLISHER-FRESHNESS
checkout   Split   2         True       True        -     checkout   1        1         False      NoOverrides       Routable   gateway.networking.k8s.io/HTTPRoute:team-a/checkout   Current

Of the map's two entries, one has a weight above 0 and one is healthy. The InferenceService has no drains, and the publisher has published this generation of the map.

When nothing matches, get prints a message to stderr and exits 0:

kubectl ome get workloadclusters

No workloadclusters found.

With -o json or -o yaml, get prints an empty list instead:

kubectl ome get isvc -n team-a -o yaml

apiVersion: v1
items: []
kind: List
metadata: {}

Errors¶

An API error names the operation, the resource and the object or namespace, then the reason.

Exit codes¶

Errors print to stderr as error: <message>.

Related guides¶

• Drain a workload cluster, which uses the alpha multi-cluster routing
• Troubleshoot an InferenceService