Verta
Latest
Search
K

Exporting metrics for an endpoint

Each endpoint publishes a number of performance metrics suitable for monitoring behavior and performance.

Supported Integrations

Datadog

The Verta platform supports exporting endpoint metrics as Custom Metrics to Datadog with the labels indicated below. For more information about using custom metrics within Datadog, please consult the Datadog documentation.

CloudWatch

Endpoint metrics can also be exported to Amazon CloudWatch as Metrics. The labels for each metric below are associated to CloudWatch Metrics as Dimensions. Please see the CloudWatch documentation for information about using Metrics in CloudWatch.

Metric Types

Endpoint State

Name
Description
Min Value
Max Values
Labels
state_up
indicates worker is available to service requests
0
1
worker, model_name, model_version, endpoint_path
state_pending
indicates worker is in 'pending' state
0
1
worker, model_name, model_version, endpoint_path
state_allocated
indicates worker is in 'running' state
0
1
worker, model_name, model_version, endpoint_path
state_restart_count
count of the number of worker restarts
0
worker, model_name, model_version, endpoint_path
When an endpoint is being updated, its worker has the following state transition:
pending (waiting for resources) -> allocated (waiting to start) -> up (running)

Endpoint Utilization

Name
Description
Min Value
Max Values
Labels
api_throughput
number of requests made to a worker (requests/second)
0
worker, model_name, model_version, endpoint_path
api_latency_avg
average request latency for a worker (seconds)
0
worker, model_name, model_version, endpoint_path
api_latency_p99
99th percentile upper bound for request latency for a worker (seconds)
0
worker, model_name, model_version, endpoint_path

Endpoint Resources

Name
Description
Min Value
Max Values
Labels
resources_cpu
[deprecated]
0
worker, model_name, model_version, endpoint_path
resources_cpu_cores
amount of cpu utilized by worker (cores)
0
worker, model_name, model_version, endpoint_path
resources_cpu_ratio
amount of cpu utilized by worker (ratio)
0
1
worker, model_name, model_version, endpoint_path
resources_memory
[deprecated]
0
worker, model_name, model_version, endpoint_path
resources_memory_bytes
amount of memory utilized by worker (bytes)
0
worker, model_name, model_version, endpoint_path
resources_memory_ratio
amount of memory utilized by worker (ratio)
0
1
worker, model_name, model_version, endpoint_path
resources_rx_bytes
amount of received network traffic by worker (bytes/sec)
0
worker, model_name, model_version, endpoint_path
resources_tx_bytes
amount of transmitted network traffic by worker (bytes/sec)
0
worker, model_name, model_version, endpoint_path

Label Definitions

  • worker: name of the worker for the endpoint
  • model_name: name of the registered model configured in the endpoint
  • model_version: version of registered model configured in endpoint
  • endpoint_path: the URI path suffix configured for the endpoint