Accessing Request Headers From Predict

Introduction

This guide shows how to access request headers from the predict method of a custom model. This is useful if you want to pass additional information to your model at inference time. For example, you may want to JWT tokens into your prediction in order to authenticate secondary processes.

This feature is entirely optional; you can continue defining predict()methods without including the headers parameter.

This feature requires the 2023_07 release of the Verta platform.

Prerequisites

Before you can use the model schema feature, you must meet the following prerequisites:

• Verta Python library (version 0.24.0 or higher): pip install "verta>=0.24.0".

Getting Started

To access request headers from the predict method, follow these steps:

1. Import the necessary libraries:

   from verta import Client
   from verta.environment import Python
   from verta.registry import VertaModelBase, verify_io

2. Create a Model class that subclasses Verta's VertaModelBase and defines the predict() method with the new headers parameter:

   class Model(VertaModelBase):
       def __init__(self, artifacts=None):
           pass
       
       @verify_io
       def predict(self, input, headers):
           # note that headers can be accessed case-insensitively
           jwt_token = headers['grpc-metadata-bearer-access-token']
           # replace this with your own prediction logic
           return output

   In the predict() method, you should replace the example code with your own prediction logic that uses your trained model to make predictions on the input data.

3. Create a new model version and deploy it to an endpoint:

   client = Client()
   
   model_ver = client.get_or_create_registered_model("My Model").create_standard_model(
       Model,
       code_dependencies=[],
       environment=Python(requirements=[]),
   )
   endpoint = client.get_or_create_endpoint("my-model")
   endpoint.update(model_ver, wait=True)

   In this step, you should replace "My Model" with the name of your registered model, "my-model" with the name of the endpoint you want to deploy the model to, and customize the requirements parameter of the Python environment to fit your model's needs.

4. Make prediction cURLs. Here is an example with the Grpc-Metadata-Bearer-Access-Token header:

   curl -H "Access-Token: <ACCESS_TOKEN>" -X POST https://<VERTA_DOMAIN>/api/v1/predict/my-model -d '<INPUT_BODY>' -H "Content-Type: application/json" -H "Grpc-Metadata-Bearer-Access-Token: <JTW_TOKEN>"

   In this step, you should replace <ACCESS_TOKEN> with your Verta access token, <VERTA_DOMAIN> with your Verta domain, <INPUT_BODY> with your input request body, and <JTW_TOKEN> with your JWT token. Add additional headers as needed.

At this time, headers can be passed into the predict method via cURLs but not via the Verta client.