Updating endpoints with canary

Search…

As an alternative to updating an Endpoint all at once, Verta offers the ability to roll out a new model incrementally while monitoring its behavior to prevent a problematic model from being deployed completely.
This is the principle behind a canary update shown in this tutorial.
Using the client
Before, a new model was deployed using a direct update strategy. This time, a CanaryUpdateStrategy will be used:
from verta.endpoint.update import CanaryUpdateStrategy
from verta.endpoint.update.rules import MaximumRequestErrorPercentageThresholdRule
​
strategy = CanaryUpdateStrategy(interval=10, step=0.2)
strategy.add_rule(MaximumRequestErrorPercentageThresholdRule(0.1))
endpoint.update(model_version, strategy)  # or endpoint.update(run, strategy)
To perform a canary update, it must be provided with an interval (in seconds) describing how often to update the deployment, and a step (as a ratio between 0 and 1) describing how much of the deployment should be updated per interval.
A canary update strategy must also have at least one rule associated with it. In this case, the update will monitor the request error percentage; if it exceeds the threshold we have set (10%), the rollout will be halted. See the canary-rules API documentation for additional rules that can be used.

Creating custom models with dependencies

Endpoint autoscaling

Last modified 2mo ago

Copy link