OME
ome-projects/ome
  • v1.2.2
  • 516
  • 102
↑ ↓ to move ↵ to open esc to close See all results

kubectl-ome overview and install Unreleased: coming in v1.3

Install the kubectl-ome plugin, and look up its commands, global flags, output formats, exit codes and the RBAC it needs.

kubectl-ome is the OME command-line tool, a kubectl plugin that you run as kubectl ome. Most commands read your OME resources and explain what they find. status shows why an InferenceService is or isn't ready, and runtime explain shows which runtimes match a model. A few alpha actions, such as rollout pause, change a service. Each runs the checks in Guarded actions. The plugin doesn't install OME: see Install OME.

Install

kubectl-ome is new in v1.3. Until v1.3.0 is released, build it from source. From v1.3.0, you can also install a release with krew. kubectl runs the kubectl-ome binary on your PATH when you type kubectl ome.

Build from source

You need git, make and Go 1.26 or later.

Shell
git clone --depth 1 https://github.com/ome-projects/ome.git
cd ome
make kubectl-ome

This builds bin/kubectl-ome from the latest commit of main. To build a release instead, add its tag to the clone, such as --branch v1.3.0. Put the binary on your PATH:

Shell
export PATH="$PWD/bin:$PATH"

To keep the plugin after you close the shell, copy bin/kubectl-ome to a directory on your PATH.

Install with krew

From v1.3.0, releases ship the plugin for Linux, macOS and Windows, on amd64 and arm64, with a krew manifest. Install it with krew:

Shell
kubectl krew install --manifest-url=https://github.com/ome-projects/ome/releases/download/v1.3.0/ome.yaml

krew downloads the archive for your platform and installs kubectl-ome. For another release, replace v1.3.0 in the URL with its tag.

Check the install

Check that kubectl finds the plugin, and which version of OME the cluster is set to run:

Shell
kubectl ome version

Once v1.3.0 is released, with the plugin and OME both from it, the output is:

Output
COMPONENT   VERSION
Client      v1.3.0 (commit 9c41e7a2d85b3f60e1a7c4d29b8f305e6d7a1c42)
Operator    v1.3.0

Client is the version and commit of the plugin. A build from the shallow clone of main above shows the commit's short hash as its version. Operator is the version of OME that the manager Deployment declares, in the namespace ome unless you pass --ome-namespace. When the CLI can't find it, Operator is unknown with the reason, and the command still exits 0. Without the RBAC below, the reason is Forbidden. kubectl ome version explains both values.

Commands

Command What it does
accelerator Shows the accelerator class or policy that the engine and decoder of an InferenceService ask for, and the resource requests their pods start from.
admin Checks that the cluster serves the APIs OME needs, and reads the Instance moves that Alfred, OME's alpha GPU cluster caretaker, recommends.
autoscale Shows each component's autoscaling state, and which layer supplies its autoscaler.
cluster Alpha. Shows the WorkloadClusters in the cluster of your current context, and whether OME reports each as connected.
get Lists OME resources, including merged views of BaseModels with ClusterBaseModels and ServingRuntimes with ClusterServingRuntimes.
instance Lists and inspects the OMENative Instances of an InferenceService. The alpha release-held action releases a held revision.
logs Streams logs from the pods behind an InferenceService.
migration Shows the running and past OMENative migrations of an InferenceService. The alpha start action requests one.
placement Inspects the alpha multi-cluster placement of an InferenceService.
quota Shows the AcceleratorQuota tree (since v1.3) with its reported budgets, and validates the topology.
rollout Shows rollout progress, the pinned plan and history, and validates rollout settings. The alpha actions pause, resume, promote, roll back and repin a rollout.
runtime Explains which runtimes match a model, and shows a service's runtime, revisions and inheritance tree. The alpha sync action moves a pinned service to the runtime's current spec.
scale Alpha. Requests a transient replica count for one OMENative component.
status Shows why an InferenceService is or isn't ready, in one report: its conditions, pods, model, runtime, rollout, autoscaling, placement, traffic and warning events.
traffic Shows an InferenceService's traffic routes, weights and canary split, and checks them against your settings. The alpha drain and undrain actions take a cluster out of multi-cluster routing and back.
version Prints the kubectl-ome and OME operator versions.
wait Waits until an InferenceService reaches a condition, rollout outcome, migration result, replica state or runtime sync, or a held revision is released.

Global flags

All commands take kubectl's standard connection flags, which pick the cluster, the identity and the namespace:

Flag What it does
-n, --namespace The namespace to use. The default is your kubeconfig context's namespace, or default.
--context, --kubeconfig, --cluster, --user The kubeconfig context, file, cluster or user to use.
-s, --server, --token The API server's address and port, and a bearer token for it.
--certificate-authority, --client-certificate, --client-key, --tls-server-name The TLS files, and the server name to check the certificate against.
--insecure-skip-tls-verify Don't check the server's certificate. The connection is then insecure.
--as, --as-uid, --as-group, --as-user-extra Impersonate a user, a UID, groups or user extras. You can repeat --as-group and --as-user-extra.
--request-timeout How long to wait for each API request, as in 10s. The default, 0, sets no limit, but the guarded actions and some other commands set their own.
--cache-dir, --disable-compression The cache directory, .kube/cache in your home directory by default, and a switch that turns off response compression.

Commands that read objects in the OME namespace also take --ome-namespace, which defaults to ome. -h or --help prints any command's help.

Output formats

-o or --output sets the format. Most commands take these formats:

Format What it prints
table The default. On a narrow terminal, long cells wrap, or rows print as lists of fields. A file or pipe gets one line per row.
wide More detail, on commands that have it. The command's page says what it adds.
json, yaml The whole report, with apiVersion: cli.ome.io/v1alpha1 and a kind such as RolloutStatusReport. The action commands print an ActionResult with the same apiVersion.

These commands take fewer formats, or print something else:

Command Formats
autoscale explain, instance list, instance retry-blocks, migration status All but wide.
logs, runtime explain, version No -o flag.
get All four, but json and yaml print the object you name, or a List of objects, instead of a report.

In scripts, read -o json or -o yaml and the exit code, not the table layout.

Evidence levels

Many reports say how the CLI got their values:

Evidence Meaning
Declared Read from the spec or annotations of the resource.
Reported Read from the resource's status.
Observed Read from the API server by this command.
Computed Worked out by the CLI from other values.
Unavailable The CLI couldn't get the value.

instance, autoscale and migration also use levels of their own, which their pages explain.

Exit codes

kubectl-ome exits with one of these codes:

Code Meaning Returned by
0 Success Any command that finishes.
1 General error Any command, for an invalid argument or flag, a failed API request, a refusal, an action you didn't confirm, Ctrl-C, or an action whose outcome is unknown.
2 Assertion unmet rollout validate, quota validate and admin doctor when their check fails, and wait when it ends without a match.
3 Mutation conflict An action whose target or another input changed after the CLI read it, and some migration start refusals. The action didn't apply. Guarded actions lists the cases.

version, admin and some other read commands show a failed read in their output, and exit 0. A command that exits 2 prints its report first, so a script can read the -o json output and still branch on the exit code. An action that exits 1 might have applied, so check the target before you run it again, as When the outcome is unknown shows. Errors print to stderr as error: <message>.

Required RBAC

kubectl-ome runs as the identity in your kubeconfig. The read commands need the rules in this ClusterRole, and a few need an extra rule. The actions also need patch, as Guarded actions lists.

kubectl-ome-reader.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kubectl-ome-reader
rules:
  - apiGroups: ["ome.io"]
    resources: ["*"]
    verbs: ["get", "list"]
  - apiGroups: [""]
    resources: ["pods", "events"]
    verbs: ["list"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get"]
  - apiGroups: ["apps"]
    resources: ["controllerrevisions"]
    verbs: ["get", "list"]
Rule What needs it
get and list on every ome.io resource Most commands. * also covers subresources, including the inferencereplicas/scale that autoscale status --live-scale reads.
list on pods and events status, logs, instance status and migration start list the pods of an InferenceService. status and instance status list warning events.
get on pods/log logs streams the pods' logs.
get on deployments version and admin doctor read the ome-controller-manager Deployment in the OME namespace.
get and list on controllerrevisions status, runtime effective, runtime history and the other commands that read a service's runtime revisions in the OME namespace, and migration start, in the service's namespace.

Bind it with a ClusterRoleBinding, since the commands read cluster-scoped resources and objects in the OME namespace:

Shell
kubectl apply -f kubectl-ome-reader.yaml
kubectl create clusterrolebinding kubectl-ome-reader --clusterrole=kubectl-ome-reader --group=ome-operators
Output
clusterrole.rbac.authorization.k8s.io/kubectl-ome-reader created
clusterrolebinding.rbac.authorization.k8s.io/kubectl-ome-reader created

Replace ome-operators with your group, or bind a user with --user or a service account with --serviceaccount=NAMESPACE:NAME. To check a permission, ask the API server: kubectl auth can-i list inferencereplicas.ome.io -n prod prints yes or no.

Extra rules

These commands need one more rule, in the namespace the table names:

Command Rule Namespace Without it
migration start get on configmaps The service's The preview warns that it can't read the audit ConfigMap, <name>-ome-migration-audit, and the action goes on. With --request-id, the command fails.
migration history get on configmaps The service's The report leaves out the audit record and lists an AuditUnavailable issue.
admin recommendations get on configmaps Alfred's: --alfred-namespace, the OME namespace by default The report shows the state Forbidden.
autoscale status --live-scaler get on horizontalpodautoscalers in autoscaling, or on scaledobjects in keda.sh, whichever scales the component The service's SCALER-EVIDENCE shows Forbidden.
wait --for condition=... and wait --for rollout=... watch on inferenceservices in ome.io The service's The command polls the InferenceService instead of watching it.

Grant get on configmaps with a Role in each namespace where people use these commands. Bound with a ClusterRoleBinding, the rule reads every ConfigMap in the cluster. This Role grants the extra rules in the namespace prod:

kubectl-ome-extras.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: kubectl-ome-extras
  namespace: prod
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get"]
  - apiGroups: ["autoscaling"]
    resources: ["horizontalpodautoscalers"]
    verbs: ["get"]
  - apiGroups: ["keda.sh"]
    resources: ["scaledobjects"]
    verbs: ["get"]
  - apiGroups: ["ome.io"]
    resources: ["inferenceservices"]
    verbs: ["watch"]
Shell
kubectl apply -f kubectl-ome-extras.yaml
kubectl create rolebinding kubectl-ome-extras --role=kubectl-ome-extras --group=ome-operators -n prod
Output
role.rbac.authorization.k8s.io/kubectl-ome-extras created
rolebinding.rbac.authorization.k8s.io/kubectl-ome-extras created

The view role

Instead of the roles above, you can bind the built-in view ClusterRole with a ClusterRoleBinding. The ome-resources Helm chart installs a ClusterRole, ome-supplemental-viewer, that Kubernetes aggregates into view. It grants get, list and watch on every ome.io resource and on nodes. With it, view grants every rule above except get on KEDA ScaledObjects, but it also grants read access to every ConfigMap in the cluster.