Verta
Search…
Querying model metadata
For revisiting past experiment runs, the Verta API offers the ability to find runs based on their properties and logged values.
To start, you can obtain a collection of experiment runs under a project or an experiment using their expt_runs attribute:
1
proj = client.set_project("Project Banana")
2
proj.expt_runs
3
# <ExperimentRuns containing 24 runs>
4
expt = client.set_project("Experiment Coconut")
5
expt.expt_runs
6
# <ExperimentRuns containing 8 runs>
Copied!
These are ExperimentRuns objects, which have a find() method for filtering their contents:
1
expt.expt_runs
2
# <ExperimentRuns containing 8 runs>
3
expt.expt_runs.find("metrics.acc > .95")
4
# <ExperimentRuns containing 3 runs>
5
expt.expt_runs.find("hyperparameters.dropout < .8")
6
# <ExperimentRuns containing 4 runs>
7
expt.expt_runs.find("hyperparameters.optimizer == 'adam'")
8
# <ExperimentRuns containing 6 runs>
Copied!
Multiple queries can be specified together, which returns the intersection of their results.
1
expt.expt_runs.find([
2
"metrics.acc > .95",
3
"hyperparameters.dropout < .8",
4
])
5
# <ExperimentRuns containing 2 runs>
Copied!

Query syntax

Queries are Python strings that follow this syntax:
1
"<field>.<key> <operator> <value>"
Copied!
For example, lets say I have several runs, and I've been logging each multi-layer perceptron's hidden layer size as a hyperparameter with the key "hidden_size".
To find all runs with a hidden size greater than 512, the query would look like this:
1
"hyperparameters.hidden_size > 512"
Copied!
If the value is a string, it should be surrounded by quotes. As examples:
1
"hyperparameters.optimizer == 'adam'"
2
"attributes.team == \"sales\""
Copied!

Fields

The fields currently supported by the client are:
    id
    project_id
    experiment_id
    name
    date_created
    attributes
    hyperparameters
    metrics
date_created is represented as a Unix timestamp, in milliseconds.
Some fields inherently do not have keys, such as id and name, in which case their queries are even more straightforward:
1
"<field> <operator> <value>"
Copied!
For example:
1
"name == 'Run Dragonfruit'"
Copied!

Operators

The operators currently supported by the client are:
    ==
    !=
    >
    >=
    <
    <=
It is recommended to only use == and != with string values.
Last modified 4mo ago