Concepts

Monitored Model

The name of the model that you are monitoring. Each monitored model has its own set of monitoring configurations and set up. Every endpoint deployed in Verta has a 1:1 mapping with a monitored model. A monitored model is automatically created as soon as an endpoint is deployed.

Registered Model and Registered Model Version

A model in Verta is a function that takes input, produces some output and is associated with artifacts, code, and software dependencies. When a model is registered in Verta using Verta Model specification it is called a Register Model. Example: Census-model

A registered model has a name, permissions and metadata associated with it. A registered model contains a collection of Registered Model Versions. Each model can have many versions.

Example: Census-model, v1.0.0

Learn more

Endpoint

An endpoint in Verta can be deployed from a Registered Model Version. As the models get updated with new versions, a live endpoint can be updated with a new Registered Model Version. Example: /Census-prod-model (previous release: Census-model, v1.0.0 and latest release: Census-model, v1.0.1)

Model API

This is the model schema that includes model input and output column and data type definitions. When you register a model in Verta, we provide a function to log your model schema. Every registered model version in Verta captures Model API information. Please note that the model schema may not be static and as new features are added, Verta captures the updated schema via model registry and applies it to the monitoring system.

Reference data

Verta model monitoring system automatically monitors data drift for both input features and output predictions. The drift is computed based on how much production data has changed from a reference data distribution. You can use a subset of your training data that closely represents the distribution of the data the model is trained on as the reference data. Reference data is logged for every Registered Model Version.

Ground truth

Ground truth is the labeled data that represents the actual labels and can be compared against predicted labels in order to compute model performance metrics like accuracy, precision etc. You can ingest delayed ground truth into Verta model monitoring system. Every prediction is uniquely identified using a UUID and the id field is used to map the ground truth with a prediction.