How to manage model dependencies

Capturing exact dependencies is critical to ensure model reproducibility and versioning.

Here is an example on how to capture metadata about Python libraries, installed packages, and system environment variables:

pass  # model training here

run = client.set_experiment_run()
    
# log environment
from verta.environment import Python
run.log_environment(
    Python(
        requirements=[
            "scikit-learn==0.23.2",
            "pandas",  # gets version pin from current environment
        ],
        env_vars={"OMP_NUM_THREADS": "4"},
    ),
    overwrite=True,
)

This information can be visualized in the web UI.

OS-level dependencies

Capturing OS level dependencies are helpful in the following scenario:

• If you have a python dependency that requires a specific package to be installed on the image to build or to run (e.g. GCC)

• If you need a specific version of a package that for python dependency and python doesn't have a way to specify OS-level dependencies

Here is an example to capture OS level dependencies (e.g. for computer vision models):

pass  # model training here

run = client.set_experiment_run()

# log environment
from verta.environment import Python
run.log_environment(
    Python(
        requirements=["scikit-learn", "pandas"],
        apt_packages=["python3-opencv"],  # OS-level dependency
    ),
    overwrite=True,
)

This information can be visualized in the web UI.