Quickstart

Verta provides a set of tools for active data and model monitoring, complete with charts and alerts.
Models and data pipelines are represented as monitored entities:
from verta import Client
​
client = Client()
monitored = client.monitoring.get_or_create_monitored_entity(
    "Auto Insurance Cross-selling",
)
Summaries group related samples of data. For example, a summary can represent a particular column of a data frame, and each sample logged to it contains profiles of that column in a particular batch:
from datetime import datetime
from verta.monitoring.profiler import ContinuousHistogramProfiler
from verta.data_types import FloatHistogram
​
summary = client.monitoring.summaries.create(
    "Annual Premium",
    FloatHistogram,
    monitored,
)
​
profile = ContinuousHistogramProfiler(columns=["Annual Premium"]).profile(df)
for histogram in profile.values():
    summary.log_sample(
        histogram,
        labels={"source": "reference"},
        time_window_start=datetime(2020, 5, 4),
        time_window_end=datetime(2021, 5, 11),
    )
Alerts can be configured to propagate messages to notification channels if samples exceed defined thresholds:
from verta.monitoring.summaries.queries import SummarySampleQuery
from verta.monitoring.notification_channel import SlackNotificationChannel
from verta.monitoring.alert import ReferenceAlerter
from verta.monitoring.comparison import GreaterThan
​
ref_sample = summary.find_samples(
    SummarySampleQuery(labels={"source":"reference"})
)[0]
threshold = 0.2
​
channel = client.monitoring.notification_channels.get_or_create(
    "Auto Insurance Cross-selling Alerts",
    SlackNotificationChannel(webhook_url),
)
alerter = ReferenceAlerter(
    GreaterThan(threshold),
    ref_sample,
)
alert = summary.alerts.create(
    summary.name + " exceeding reference by {}".format(threshold),
    alerter,
    notification_channels=[channel],
    labels={"source": ["low_premium", "high_premium"]}
)
Monitoring
Concepts
Last updated 2 months ago