Running batch predictions

Introduction

The batch predict feature allows you to make predictions against a large Pandas DataFrame. The client's batch_predict() method will split the DataFrame into batches of the specified size and send each batch as a smaller DataFrame to the endpoint for prediction. The client will then collect the predictions and return them as a single DataFrame. This feature is useful when you have a DataFrame that is too big to send over the wire.

To use the batch prediction feature, you'll need to follow these steps:

  1. Create a model with the VertaModelBase.batch_predict() method defined.

  2. Deploy the model to an endpoint.

  3. Use the client's DeployedModel.batch_predict() method to make a prediction.

Running batch predictions against endpoints requires the 2023_04 release of the Verta platform.

This documentation will provide a detailed guide on how to use the batch predict feature. Let's get started!

Prerequisites

Before you can use the batch prediction feature, you must meet the following prerequisites:

  • Verta Python library (version 0.22.2 or higher): pip install verta>=0.22.2.

  • Pandas Python library: pip install pandas.

Getting Started

To use the batch predict feature, follow these steps:

  1. Import the necessary libraries:

    import pandas as pd
    import numpy as np
    from verta import Client
    from verta.environment import Python
    from verta.registry import VertaModelBase
  2. Create a Model class that subclasses Verta's VertaModelBase and defines the batch_predict() method. This method accepts and returns Pandas DataFrames. For example:

    class Model(VertaModelBase):
        def __init__(self, artifacts=None):
            pass
    
        def batch_predict(self, df: pd.DataFrame) -> pd.DataFrame:
            # Replace this with your own prediction logic
            return pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]}, index=['a', 'b', 'c'])
            
        
        def predict(self, input):
            # As of now, the client still expects this method to be present
            raise NotImplementedError
    

    In the batch_predict() method, you should replace the example code with your own batch prediction logic that uses your trained model to make predictions on the input data.

  3. Create a new model version and deploy it to an endpoint:

    client = Client()
    
    model_ver = client.get_or_create_registered_model("My Batch Model").create_standard_model(
        Model,
        code_dependencies=[],
        environment=Python(requirements=["pandas", "numpy"]),
    )
    endpoint = client.get_or_create_endpoint("my-batch-model")
    endpoint.update(model_ver, wait=True)

    In this step, you should replace "My Batch Model" with the name of your registered model, "my-batch-model" with the name of the endpoint you want to deploy the model to, and customize the requirements parameter of the Python environment to fit your model's needs.

  4. Load the data you want to make predictions against into a Pandas DataFrame. Here is a link to the Pandas documentation.

  5. Make a batch prediction:

    deployed_model = endpoint.get_deployed_model()
    out_df = deployed_model.batch_predict(df, batch_size=5)

    Replace df with the name of your Pandas DataFrame. The batch_size parameter is optional and defaults to 100.

The endpoint model playground and curl command present in the endpoint summary page in the web app are not supported for batch predict endpoints.

Last updated