OMENative update strategies Unreleased: coming in v1.3

When the pod template of an OMENative component changes, OME records a new revision and moves each of the component's Instances onto it. The component's lifecycle.updateStrategy sets how OME replaces an Instance's pods, and how many Instances move at once. By default, the new pods come up and serve before the old ones drain, even for a leader and its workers.

A new image in the InferenceService changes the pod template. So can an edit to its serving runtime, unless the InferenceService pins the runtime. A RawDeployment component uses deploymentStrategy instead, and rollout groups in spec.rollout coordinate several components.

The four strategies¶

Set the type in the component's lifecycle.updateStrategy, for example spec.engine.lifecycle.updateStrategy.type. UpdateStrategy lists every field.

A multi-pod Instance always moves as a whole. A change to its worker pods alone rolls the leader too, and the in-place strategies recreate its pods.

This version of the InferenceService from Opt in to OMENative runs two engine Instances and updates them in place when it can:

apiVersion: ome.io/v1beta1
kind: InferenceService
metadata:
  name: llama-3-2-3b-instruct
  namespace: llama-demo
spec:
  deploymentMode: OMENative
  model:
    name: llama-3-2-3b-instruct
  runtime:
    name: srt-llama-3-2-3b-instruct
  engine:
    minReplicas: 2
    maxReplicas: 2
    lifecycle:
      updateStrategy:
        type: InPlaceIfPossible

Apply the file:

kubectl apply -f isvc.yaml

inferenceservice.ome.io/llama-3-2-3b-instruct configured

Changing the strategy doesn't start a rollout. Instance updates that start after the change use it.

SurgeThenDrain¶

For a single-pod Instance, the new pod takes the Instance's other ordinal: llama-3-2-3b-instruct-engine-0-default-1 replaces llama-3-2-3b-instruct-engine-0-default-0. If the new pod fails, the old one keeps serving while OME tries the update again. For a leader and its workers, OME starts a whole replacement Instance at a new index, so the indices can have gaps. If the replacement fails, OME deletes it, records a GangSurgeAbandoned warning event, and the old Instance keeps serving. For what a further change does to a surge that's still in flight, see Changing the pod template mid-surge.

When OME drains the old pods, their preStop hook and terminationGracePeriodSeconds cover the requests they're still serving. The update isn't finished, and the Instance's surge slot isn't free, until the drained pods are gone, so a long grace period can make the next Instance wait. Pods that no update owns hold slots of their own: see Extra pods hold surge slots.

While the extra pods can't be scheduled, the old pods keep serving. After unschedulableGracePeriod, 15 minutes with the chart, OME fails the update and tries it again later.

In-place updates¶

When only the images of regular containers change, InPlaceIfPossible and InPlaceOnly update a single-pod Instance in place. OME takes the pod out of rotation, sets the new images, and puts the pod back once its restarted containers are ready. The pod keeps its name, its node and its emptyDir volumes. For any other change, such as an environment variable, a resource request or an init container's image, InPlaceIfPossible recreates the pods.

A label that you add to the pod template reaches the pod only when OME recreates it. With inPlaceUpdateStrategy.markNotReadyDuringLifecycle: false, the pod stays in rotation while its containers restart, so requests to it can fail. See InPlaceUpdateStrategy.

InPlaceOnly refuses any other change. The Instance stays on its old revision, the rollout stalls, and OME records an InPlaceUpdateNotPossible warning event on the InferenceService. For example, after you change an environment variable of an engine that runs InPlaceOnly, list the refusals:

kubectl get events -n llama-demo --field-selector reason=InPlaceUpdateNotPossible \
  -o custom-columns=KIND:.involvedObject.kind,NAME:.involvedObject.name,MESSAGE:.message

KIND               NAME                    MESSAGE
InferenceService   llama-3-2-3b-instruct   OMENative component=engine instance=0 rejected update: InPlaceOnly strategy but diff exceeds container images

To go ahead, change the type, for example to InPlaceIfPossible, or undo the change.

Pacing with rollingUpdate¶

lifecycle.updateStrategy.rollingUpdate limits how many Instances move at once, and can hold some on the old revision:

spec:
  engine:
    lifecycle:
      updateStrategy:
        type: InPlaceIfPossible
        rollingUpdate:
          partition: 1
          maxUnavailable: 1

A budget is a whole number, or a percentage of the component's Instances rounded up: 25% of two Instances is one. When nothing sets the budget that the strategy reads, every Instance can move at once, and with RecreatePod that takes the whole component offline. A manifest install sets no default budgets, so set them in the InferenceService or the runtime. In a rollout group, the group's limits apply too, and a canary's current step replaces your partition. See RollingUpdate.

With the settings above and two Instances, the next change to the pod template moves one Instance and holds the other. Once the moved Instance is ready, the component's InferenceReplica, which OME writes from the InferenceService, reports Staged in its Ready condition:

kubectl get inferencereplica llama-3-2-3b-instruct-engine -n llama-demo \
  -o jsonpath='{.status.conditions[?(@.type=="Ready")].message}'

Staged at partition 1: 1/2 Instances on llama-3-2-3b-instruct-engine-7b4c2e19, 1 held on the prior revision

To finish the rollout, set partition to 0 or remove it. See Conditions.

Where the defaults come from¶

OME fills in each field of a component's update strategy from the first of these that sets it:

1. The component in the InferenceService, such as spec.engine.lifecycle.updateStrategy.
2. The runtime's engineConfig, decoderConfig or routerConfig, such as spec.engineConfig.lifecycle.updateStrategy. OME merges it field by field, so an InferenceService that sets only the type keeps the runtime's budgets.
3. The operator default for the component, from the chart's ome.controller.updateStrategy. It covers type, maxSurge and maxUnavailable.
4. SurgeThenDrain for the type.

To give every InferenceService that uses a runtime the same strategy, set it in the runtime:

spec:
  engineConfig:
    lifecycle:
      updateStrategy:
        type: RecreatePod
        rollingUpdate:
          maxUnavailable: 1

The ome-resources chart sets these defaults for the engine, and the same ones for the router and decoder:

ome:
  controller:
    updateStrategy:
      engine:
        type: SurgeThenDrain
        maxSurge: 25%
        maxUnavailable: 25%

OME fills in only the budget that the strategy reads. The InferenceService in The four strategies sets only the type, so its engine gets maxUnavailable from the chart:

kubectl get inferencereplica llama-3-2-3b-instruct-engine -n llama-demo \
  -o jsonpath='{.spec.lifecycle.updateStrategy}'

{"rollingUpdate":{"maxUnavailable":"25%"},"type":"InPlaceIfPossible"}

An invalid default, such as a zero budget, stops the manager from starting. See The manager rollout doesn't finish.

OME keeps the revisions that a component uses and, with the chart, its 10 most recent other revisions. The chart's ome.controller.lifecycle.revisionHistoryLimit sets that number, and the InferenceService's ome.io/revision-history-limit annotation overrides it. With neither set, OME keeps every revision.

When a rollout doesn't move¶

While no Instance of a component can move, the component's rollout hold names the gate that stops it. For an engine that runs InPlaceIfPossible with maxUnavailable: 0:

kubectl get inferenceservice llama-3-2-3b-instruct -n llama-demo \
  -o jsonpath='{.status.components.engine.lifecycle.rolloutHold}'

{"gate":"Budget","reason":"per-Component unavailability budget 0 exhausted (would become 1)","since":"2026-09-27T08:14:02Z","target":"llama-3-2-3b-instruct-engine-7b4c2e19"}

RolloutHold lists the gates that rollout groups add.

Extra pods hold surge slots¶

Under SurgeThenDrain with a capped maxSurge, every pod that the API still lists for an Instance beyond the Instance's own pod count holds a surge slot. That keeps the ceiling that maxSurge sets: the component never carries more pods than its Instances plus the budget allow, so a slot an old pod still holds is not a slot the next update can take. When those slots deny an update, the Budget hold names the pods:

kubectl get inferenceservice llama-3-2-3b-instruct -n llama-demo \
  -o jsonpath='{.status.components.engine.lifecycle.rolloutHold}'

{"gate":"Budget","reason":"per-Component surge budget 1 exhausted (would become 2); Terminating pod llama-3-2-3b-instruct-engine-0-default-1 still counts against the surge budget until its deletion grace elapses","since":"2026-09-27T08:14:02Z","target":"llama-3-2-3b-instruct-engine-7b4c2e19"}

• A Terminating pod is one OME already deleted but the API still lists, such as the replacement of a failed attempt or a pod of an Instance the component scaled away. It counts until it exits or its deletion grace elapses, at the pod's metadata.deletionTimestamp. The pod stops counting then even if it lingers, for example on a stuck finalizer, and OME wakes at that moment to retry the waiting update. Don't raise maxSurge for this hold: it clears by itself.
• A live extra pod, which the reason reports as extra pod llama-3-2-3b-instruct-engine-1-default-1 holds a surge slot until a start on its Instance replaces it, is a replacement that a failed update attempt left running. Its slot frees when OME next starts an update on its Instance: that start evicts the pod before it creates anything, so it takes the slot the pod held and isn't charged twice. Until then, updates on the component's other Instances can't use the slot.

A pod that an in-flight update still owns isn't named: a drained old pod holds its Instance's surge slot through the update itself, which doesn't finish until the pod is gone. For a leader and its workers, extra pods count in whole Instances: their number is divided by the Instance's pod count and rounded up, so the eight draining pods of a scaled-away eight-pod Instance hold one slot.

Changing the pod template mid-surge¶

Changing the pod template again while a SurgeThenDrain update is in flight records a newer revision and withdraws the one the update is moving to. OME doesn't keep driving a surge toward a withdrawn revision: a new pod built for it has nothing to promote to. Each single-pod surge OME ends this way records a Normal SurgeAbandoned event on the InferenceService, not a warning: the withdrawn revision never got the chance to fail.

While a single-pod Instance's old pod still serves, OME abandons the surge at once, whether or not the new pod is ready:

1. OME takes the new pod out of rotation if it's serving, then deletes it. A pod that never served is deleted with the chart's ome.controller.lifecycle.abandonedReplacementGracePeriod, 30 seconds, instead of its own terminationGracePeriodSeconds.
2. Once the pod is gone, the Instance returns to Ready on the revision it runs. The old pod has kept serving throughout, so capacity never drops.
3. The Instance then moves toward the newest revision as a new update, through the same budgets and gates as any other, under the strategy the component has now.

kubectl get events -n llama-demo --field-selector reason=SurgeAbandoned \
  -o custom-columns=TYPE:.type,MESSAGE:.message

TYPE     MESSAGE
Normal   OMENative component=engine instance=0 abandoned superseded surge to llama-3-2-3b-instruct-engine-7b4c2e19; re-surging toward llama-3-2-3b-instruct-engine-9d1f4a02

When your change takes the Instance back to the revision it already runs, there's nothing left to move to: the message reads abandoned surge to withdrawn revision …; the Instance stays on revision … instead, and no new update starts. And when the new pod is ready but the old pod is already gone, OME finishes the surge instead: deleting the only pod the Instance has left would leave it nothing to serve with, so that pod promotes on the revision it was built for, and the newest revision rolls from there.

Once the Instance hands over — the new pod has taken over serving, and OME has taken the old pod out of rotation to drain it — the surge is past the point of no return. It finishes on the revision it started toward, and the Instance then starts a second update toward the newest one.

If the new pod is lost after that hand-over, OME doesn't rebuild it for the withdrawn revision. The update ends where the Instance stands, with a SurgeAbandoned event:

• An old pod that isn't already terminating returns to rotation: OME lifts the hold that was draining it, and the Instance is Ready on the revision that pod runs. The message ends with the replacement was lost after it took over serving and nothing is rebuilt at it; the source returns to rotation on revision ….
• With the old pod gone too, nothing is left to serve. The Instance re-enters Failed with no operation, and OME rebuilds it at the newest revision from scratch — a fresh start, not a repair. The message ends with no source pod stands, so the Instance re-enters as a fresh start at revision ….

An update that already failed needs no abandoning: its next attempt commits to the newest revision. A multi-pod Instance follows the same rules with its whole replacement Instance, and its events use the reason GangSurgeAbandoned — a Normal event here, unlike the warning after a failed update. A strategy change withdraws nothing: see Changing the strategy mid-rollout.

Changing the strategy mid-rollout¶

An Instance that's already moving finishes with the strategy it started with, which its operation.strategy records. Every other Instance uses the new strategy when it next moves, including one that InPlaceOnly refused or whose update failed. Changing the strategy doesn't reset the revision's retries.

Next steps¶

• Instances: pod names, indices, and the phases an update moves an Instance through.
• Rollout groups: roll several components out together.
• Pause and resume a rollout: stop rollout work and continue it later.
• Release a held revision: retry a revision that OME stopped retrying.