2023.09

Logging and visualizing rich metadata

With rich metadata logging and visualization, you can analyze a variety of information about mode performance and dataset distribution for better model debugging.

Attributes enable you to log a comprehensive set of metadata about your training and test datasets, plot feature frequencies, subcategories, etc. You can associate advanced metadata about your experiment run and model quality and generate a variety of visualizations in our Web UI. You can log and visualize histograms, correlation matrix, confusion matrix, line plots, single and/or multiple variable bar charts, log complex tables, etc.

Here are the steps to log and visualize custom attributes:

  • When you are logging an experiment run, record additional metadata about your dataset and model metrics using the client library.

Import from a variety of supported Verta data types and start logging key attributes of your experiment run

Log confusion matrix

run = client.set_experiment_run()
    
# log confusion matrix

from verta.data_types import ConfusionMatrix
data = ConfusionMatrix(
value=[
    [650000, 100000],
    [24000, 3330000],
    ],
labels=["High Income", "Low Income"],
    )
run.log_attribute("Income_Confusion_Matrix", data)

The chart will be available in the web UI under experiment run detail view.

Log discrete histograms

run = client.set_experiment_run()

# log discrete histogram
    
from verta.data_types import DiscreteHistogram
data = DiscreteHistogram(
buckets=list(itertools.chain(*df.index.tolist())),
  data=df.tolist(),
  )
run.log_attribute("Response_Histogram", data)

The chart will be available in the web UI under experiment run detail view.

Log float histograms

run = client.set_experiment_run()

# log float histogram

from verta.data_types import FloatHistogram
data = FloatHistogram(
bucket_limits=[1, 13, 25, 37, 49, 61,72,89],
data=[1500, 5300, 9100, 3400, 700, 1700, 2700],
    )
run.log_attribute("Age_Histogram", data)

The chart will be available in the web UI under experiment run detail view.

Log line chart

run = client.set_experiment_run()

# log line chart
    
from verta.data_types import Line
data = Line(
 x=[1,2,3,4,5,6,7,8,9,10],
 y=[191100,196000,205128,208568,214462,225251,237034,244088,248728,270002],
 )
run.log_attribute("Top_Income_Over_Time", data)

The chart will be available in the web UI under experiment run detail view.

Log Table

run = client.set_experiment_run()

# log table

from verta.data_types import Table
data = Table(
data=[["No formal qualification", 1020, "20%", "5%"], ["High school", 6730, "33%", "5%"], ["College", 8440, "39%", "18%"], ["Bachelors", 12100, "20%", "25%"], ["Advanced", 18400, "30%", "50%"]],
columns=["User-Segment", "Average-Capital-Gain", "Self-Employed", "Occupation-Specialty"],
 )
run.log_attribute("Measurements", data)

The chart will be available in the web UI under experiment run detail view.

Log basic key:value strings

run = client.set_experiment_run()

# log attributes
run.log_attributes({
  'library': "scikit-learn",
   'model_type': "logistic regression",
 })

The chart will be available in the web UI under experiment run detail view.

PreviousQuerying model metadataNextHow to capture training loss over time?

Last updated