Verta
Latest
Search
⌃K

Model Data Logging - Quick User Guide

Please contact Verta at [email protected] to set up model data logging in your system.
Quick guide to logging data during model predictions.
Verta’s Model Data Logging capability allows users to log arbitrary key-value pairs, with the key being a string and the value being any JSON-compatible object, during model predictions and have those logs stored in a data-lake compatible format.
See the complete, comprehensive guide HERE

Basics

  • The verta.runtime.log() python client API is inserted within the scope of a model's predict() method with the format verta.runtime.log('alphanumeric_string', <Any JSON Compatible object>)
  • Logs are collected during the prediction and written to S3 after results are returned to the requestor.

The Code

from verta.registry import VertaModelBase, verify_io
from verta import runtime
class LoudEcho(VertaModelBase):
""" Takes a string and makes it LOUD!!! """
def __init__(self, artifacts=None):
pass
@verify_io
def predict(self, input: str) -> str:
runtime.log('model_input', input) # log pre-processing data
echo: str = input.upper() + '!!!'
runtime.log('model_output', echo) # log post-processing data
return echo
# Logs are finalized on exiting the predict function
Given a prediction:
client = verta.Client() # create client connection
endpoint = client.get_or_create_endpoint('loud_echo_endpoint_name') # fetch relevant endpoint
deployed_model = endpoint.get_deployed_model() # fetch model
deployed_model.predict('roar') # make a prediction
Output:
"ROAR!!!"
Logs:
{
"model_input": "roar",
"model_output": "ROAR!!!"
}
Locally inspect logs produced by your model by wrapping calls to predict() inside an instance of verta.runtime.context() and printing the logs.
deployed_model = endpoint.get_deployed_model() # fetch model
with runtime.context() as ctx:
result = LoudEcho().predict('yell')
print(ctx.logs()) # print the logs to see how they would be stored
Output:
{'model_input': 'yell', 'model_output': 'YELL!!!'}

Viewing the Logs

Logs are stored as JSON objects in S3 and should be immediately available for access.
Each log file is written twice:
  • s3://BUCKET/PREFIX/by_endpoint/endpoint_id=NNN/prediction_id=XXX/key=YYY/value.json
  • s3://BUCKET/PREFIX/by_time/key=YYY/endpoint_id=NNN/year=YYYY/month=MM/day=DD/prediction_id=XXX/value.json