Accessing Request Headers From Predict

Introduction
This guide shows how to access request headers from the predict method of a custom model. This is useful if you want to pass additional information to your model at inference time. For example, you may want to JWT tokens into your prediction in order to authenticate secondary processes.
This feature is entirely optional; you can continue defining predict()methods without including the headers parameter.
This feature requires the 2023_07 release of the Verta platform.
Prerequisites
Before you can use the model schema feature, you must meet the following prerequisites:
Verta Python library (version 0.24.0 or higher): pip install "verta>=0.24.0".
Getting Started
To access request headers from the predict method, follow these steps:
1.Import the necessary libraries:
from verta import Client
from verta.environment import Python
from verta.registry import VertaModelBase, verify_io
2.Create a Model class that subclasses Verta's VertaModelBase and defines the predict() method with the new headers parameter:
class Model(VertaModelBase):
    def __init__(self, artifacts=None):
        pass
    
    @verify_io
    def predict(self, input, headers):
        # note that headers can be accessed case-insensitively
        jwt_token = headers['grpc-metadata-bearer-access-token']
        # replace this with your own prediction logic
        return output
In the predict() method, you should replace the example code with your own prediction logic that uses your trained model to make predictions on the input data.
3.Create a new model version and deploy it to an endpoint:
client = Client()
​
model_ver = client.get_or_create_registered_model("My Model").create_standard_model(
    Model,
    code_dependencies=[],
    environment=Python(requirements=[]),
)
endpoint = client.get_or_create_endpoint("my-model")
endpoint.update(model_ver, wait=True)
In this step, you should replace "My Model" with the name of your registered model, "my-model" with the name of the endpoint you want to deploy the model to, and customize the requirements parameter of the Python environment to fit your model's needs.
4.Make prediction cURLs. Here is an example with the Grpc-Metadata-Bearer-Access-Token header:
curl -H "Access-Token: <ACCESS_TOKEN>" -X POST https://<VERTA_DOMAIN>/api/v1/predict/my-model -d '<INPUT_BODY>' -H "Content-Type: application/json" -H "Grpc-Metadata-Bearer-Access-Token: <JTW_TOKEN>"
In this step, you should replace <ACCESS_TOKEN> with your Verta access token, <VERTA_DOMAIN> with your Verta domain, <INPUT_BODY> with your input request body, and <JTW_TOKEN> with your JWT token. Add additional headers as needed.
At this time, headers can be passed into the predict method via cURLs but not via the Verta client.
Model Schema Specification and Validation with Pydantic
Monitoring
Last modified 1mo ago