Deploying a multi-framework model

Within Verta, a "Model" can be any arbitrary function: a traditional ML model (e.g., sklearn, PyTorch, TF, etc); a function (e.g., squaring a number, making a DB function etc.); or a mixture of the above (e.g., pre-processing code, a DB call, and then a model application.

This tutorial provides an example of how to deploy models using multiple frameworks on Verta. In this case, we will consider a model that uses scikit-learn and XGBoost.

The key concept in Verta for model deployment is an Endpoint. An endpoint is a URL where a deployed model becomes available for use. Deploying a model is therefore a 2-step process:

  1. Create an endpoint

  2. Update the endpoint with a model

We'll look at these in turn.

1. Create an endpoint

Users can create an endpoint using Client.create_endpoint() as follows:

endpoint = client.create_endpoint(path="/some-path")

2. Updating the endpoint with a deployed model

To deploy a model utilizing multiple frameworks, we wrap the logic into a class that extends VertaModelBase.

from verta.registry import VertaModelBase
class CensusTwoStep(VertaModelBase):
def __init__(self, artifacts):
import cloudpickle
self.hpw_model = cloudpickle.load(
open(artifacts["sklearn_model"], "rb"))
self.income_model = cloudpickle.load(
open(artifacts["xgboost_model"], "rb"))
def predict(self, batch_input):
import numpy as np
results = []
for one_input in batch_input:
output = self.hpw_model.predict(one_input)
output = np.concatenate((np.array(one_input), np.reshape(output, (-1,1))), axis=1)
output = self.income_model.predict(output)
results.append(output)
return results

Note that the different frameworks likely expect input/output in different formats and your class needs to account for that.

Once the class has been defined, we create a Registered Model Version with it and update the endpoint.

from verta.environment import Python
model_version = registered_model.create_standard_model(
model_cls=CensusTwoStep,
environment=Python(requirements=["sklearn", "xgboost"]),
artifacts={
"hpw_model" : hpw_model,
"income_model" : income_model
},
name="v6"
)
census_multiple_endpoint = client.get_or_create_endpoint("census-multiple")
census_multiple_endpoint.update(model_version, wait=True)
deployed_model = census_multiple_endpoint.get_deployed_model()
deployed_model.predict([X_train_hpw.values.tolist()[:5]])

See the full code for this tutorial here.