Quickstart

Verta provides a set of tools for active data and model monitoring, complete with charts and alerts.

Models and data pipelines are represented as monitored entities:

from verta import Client
client = Client()
monitored = client.monitoring.get_or_create_monitored_entity(
"Auto Insurance Cross-selling",
)

Summaries group related samples of data. For example, a summary can represent a particular column of a data frame, and each sample logged to it contains profiles of that column in a particular batch:

from datetime import datetime
from verta.monitoring.profiler import ContinuousHistogramProfiler
from verta.data_types import FloatHistogram
summary = client.monitoring.summaries.create(
"Annual Premium",
FloatHistogram,
monitored,
)
profile = ContinuousHistogramProfiler(columns=["Annual Premium"]).profile(df)
for histogram in profile.values():
summary.log_sample(
histogram,
labels={"source": "reference"},
time_window_start=datetime(2020, 5, 4),
time_window_end=datetime(2021, 5, 11),
)

Alerts can be configured to propagate messages to notification channels if samples exceed defined thresholds:

from verta.monitoring.summaries.queries import SummarySampleQuery
from verta.monitoring.notification_channel import SlackNotificationChannel
from verta.monitoring.alert import ReferenceAlerter
from verta.monitoring.comparison import GreaterThan
ref_sample = summary.find_samples(
SummarySampleQuery(labels={"source":"reference"})
)[0]
threshold = 0.2
channel = client.monitoring.notification_channels.get_or_create(
"Auto Insurance Cross-selling Alerts",
SlackNotificationChannel(webhook_url),
)
alerter = ReferenceAlerter(
GreaterThan(threshold),
ref_sample,
)
alert = summary.alerts.create(
summary.name + " exceeding reference by {}".format(threshold),
alerter,
notification_channels=[channel],
labels={"source": ["low_premium", "high_premium"]}
)