Verta
Search…
Deploying a multi-framework model
Within Verta, a "Model" can be any arbitrary function: a traditional ML model (e.g., sklearn, PyTorch, TF, etc); a function (e.g., squaring a number, making a DB function etc.); or a mixture of the above (e.g., pre-processing code, a DB call, and then a model application.
This tutorial provides an example of how to deploy models using multiple frameworks on Verta. In this case, we will consider a model that uses scikit-learn and XGBoost.
The key concept in Verta for model deployment is an Endpoint. An endpoint is a URL where a deployed model becomes available for use. Deploying a model is therefore a 2-step process:
    1.
    Create an endpoint
    2.
    Update the endpoint with a model
We'll look at these in turn.

1. Create an endpoint

Users can create an endpoint using Client.create_endpoint() as follows:
1
endpoint = client.create_endpoint(path="/some-path")
Copied!

2. Updating the endpoint with a deployed model

To deploy a model utilizing multiple frameworks, we wrap the logic into a class that extends VertaModelBase.
1
from verta.registry import VertaModelBase
2
3
class CensusTwoStep(VertaModelBase):
4
def __init__(self, artifacts):
5
import cloudpickle
6
self.hpw_model = cloudpickle.load(
7
open(artifacts["sklearn_model"], "rb"))
8
self.income_model = cloudpickle.load(
9
open(artifacts["xgboost_model"], "rb"))
10
11
def predict(self, batch_input):
12
import numpy as np
13
results = []
14
for one_input in batch_input:
15
output = self.hpw_model.predict(one_input)
16
output = np.concatenate((np.array(one_input), np.reshape(output, (-1,1))), axis=1)
17
output = self.income_model.predict(output)
18
results.append(output)
19
return results
Copied!
Note that the different frameworks likely expect input/output in different formats and your class needs to account for that.
Once the class has been defined, we create a Registered Model Version with it and update the endpoint.
1
from verta.environment import Python
2
model_version = registered_model.create_standard_model(
3
model_cls=CensusTwoStep,
4
environment=Python(requirements=["sklearn", "xgboost"]),
5
artifacts={
6
"hpw_model" : hpw_model,
7
"income_model" : income_model
8
},
9
name="v6"
10
)
11
12
census_multiple_endpoint = client.get_or_create_endpoint("census-multiple")
13
census_multiple_endpoint.update(model_version, wait=True)
14
deployed_model = census_multiple_endpoint.get_deployed_model()
15
deployed_model.predict([X_train_hpw.values.tolist()[:5]])
Copied!
See the full code for this tutorial here.
Last modified 3mo ago