As an alternative to updating an Endpoint all at once, Verta offers the ability to roll out a new model incrementally while monitoring its behavior to prevent a problematic model from being deployed completely.

This is the principle behind a canary update shown in this tutorial.

Using the client

Before, a new model was deployed using a direct update strategy. This time, a CanaryUpdateStrategy will be used:

from verta.endpoint.update import CanaryUpdateStrategy
from verta.endpoint.update.rules import MaximumRequestErrorPercentageThresholdRule
​
strategy = CanaryUpdateStrategy(interval=10, step=0.2)
strategy.add_rule(MaximumRequestErrorPercentageThresholdRule(0.1))
endpoint.update(model_version, strategy)  # or endpoint.update(run, strategy)

To perform a canary update, it must be provided with an interval (in seconds) describing how often to update the deployment, and a step (as a ratio between 0 and 1) describing how much of the deployment should be updated per interval.

A canary update strategy must also have at least one rule associated with it. In this case, the update will monitor the request error percentage; if it exceeds the threshold we have set (10%), the rollout will be halted. See the canary-rules API documentation for additional rules that can be used.

Using the CLI

As with the direct strategy, a canary update can be performed via the CLI:

verta deployment update endpoint /some-path --model-version-id "<id>" \
    --strategy canary \
    --canary-interval 10 \
    --canary-step 0.2 \
    --canary-rule '{"rule": "error_4xx_rate", "rule_parameters": [{"name": "threshold", "value": "0.1"}]}'

--canary-rule takes a JSON string value that represents the rule and threshold to follow. You can find a reference for accepted values here.