Monitoring your deployment endpoints

Quick introduction to deployments

  • Valohai deployments are primarily used for online inference

  • All deployments are served from a Kubernetes cluster

  • Each deployment can contain multiple versions, and each version can contain multiple HTTP endpoints

  • For more details, see Core Concepts: Deployments

Under each deployment version, you can view the deployment logs from your deployment endpoints.

You can collect additional metrics from your deployments by printing JSON from your deployment endpoint. Valohai will collect these metrics, and allow you to chart them in both time series and histogram modes.

So we can recognize which outputs you might want to chart out, you must wrap those structures in {"vh_metadata": {}}, something like this:

import json

print(json.dumps({"vh_metadata": {"accuracy": 0.9247000813484192, "best_guess": "dog"}}))

In most use-cases, each request would log out one of these metrics log rows, but we don’t limit that.

Monitoring Valohai Deployments