Logging Metrics

Track and visualize training metrics in Valohai by printing JSON from your scripts. Compare metrics across executions.

💡 About this tutorial: We use YOLOv8 as a practical example to demonstrate Valohai's features. You don't need computer vision knowledge—the patterns you learn here apply to any ML framework. This tutorial focuses on collecting performance metrics while ensuring proper versioning and tracking of your ML workflows.

Valohai automatically tracks metrics whenever your script prints JSON. No SDK needed, just print() your metrics as JSON and we'll handle the rest.

Why metrics matter

• Sort executions by metric values in your dashboard

• Compare performance across multiple training runs

• Set early stopping rules to halt underperforming jobs

• Control pipelines by advancing stages based on metric thresholds

Print final metrics

YOLOv8 already outputs metrics in JSON format. Here's how to capture them:

import shutil
from ultralytics import YOLO
import argparse
import json

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('--epochs', type=int, default=3)
    parser.add_argument('--verbose', type=bool, default=False)
    return parser.parse_args()

args = parse_args()

# Load a model
model = YOLO("yolov8n.pt")  # Load a pretrained model (recommended for training)

# Use the model
model.train(data="coco128.yaml", epochs=args.epochs, verbose=args.verbose)  # Train the model
path = model.export(format="onnx")  # Export the model to ONNX format

metadata = {}
# Loop through the metrics
for metric in model.metrics.results_dict:
    # Some metrics have a 'metrics/' prefix (e.g., metrics/precision)
    # We split it to isolate the actual metric name.
    metric_name = metric.split("metrics/")[-1]
    metric_value = model.metrics.results_dict[metric]

    metadata[metric_name] = metric_value

# Print the JSON dictionary to register metrics and their values in Valohai
print(json.dumps(metadata))

# Copy the exported model to the Valohai outputs directory
shutil.copy(path, '/valohai/outputs/')

# Define a JSON dictionary containing a friendly name
# You can then reference this file with datum://latest-model
file_metadata = {
    "valohai.alias": "latest-model"
}

# Attach the metadata to the file
with open("/valohai/outputs/best.onnx.metadata.json", "w") as f:
    f.write(file_metadata)

Run and view metrics

Launch your training and watch the metrics appear in your execution logs:

vh execution run yolo --adhoc --open-browser

Navigate to the Executions tab to see your metrics in table format:

💡 Customize your view: Click the controls above the table (marked as #2) to choose which columns to display and set decimal precision.

Track metrics during training

Print metrics after each epoch to monitor training progress in real-time. YOLOv8 uses callbacks for this:

import shutil
from ultralytics import YOLO
import argparse
import json

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('--epochs', type=int, default=2)
    parser.add_argument('--verbose', type=bool, default=False)
    return parser.parse_args()

args = parse_args()

def print_valohai_metrics(trainer):
    metadata = {
        "epoch": trainer.epoch,
    }
    # Loop through the metrics
    for metric in trainer.metrics:
        metric_name = metric.split("metrics/")[-1]
        metric_value = trainer.metrics[metric]

        metadata[metric_name] = metric_value

    print(json.dumps(metadata))

# Load a model
model = YOLO("yolov8n.pt")  # Load a pretrained model (recommended for training)
model.add_callback("on_train_epoch_end", print_valohai_metrics)

# Use the model
model.train(data="coco128.yaml", epochs=args.epochs, verbose=args.verbose)  # Train the model
path = model.export(format="onnx")  # Export the model to ONNX format

metadata = {}
# Loop through the metrics
for metric in model.metrics.results_dict:
    # Some metrics have a 'metrics/' prefix
    # Splitting it yields the actual metric name
    metric_name = metric.split("metrics/")[-1]
    metric_value = model.metrics.results_dict[metric]

    metadata[metric_name] = metric_value

# Valohai metrics are collected as JSON key:value pairs
print(json.dumps(metadata))

# Copy the exported model to the Valohai outputs directory
shutil.copy(path, '/valohai/outputs/')

# Define a JSON dictionary containing a friendly name
# You can reference this file with datum://latest-model
file_metadata = {
    "valohai.alias": "latest-model"
}

# Attach the metadata to the file
with open("/valohai/outputs/best.onnx.metadata.json", "w") as f:
    f.write(json.dumps(file_metadata))

Run with more epochs to see live metric updates:

vh execution run yolo --epochs=10 --adhoc --open-browser

Open the Metadata tab and select:

• X-axis: epoch

• Y-axis: any metric you want to track

Watch your training curves update in real-time:

Compare executions

Need to compare hyperparameter experiments?

1. Go to the Executions tab

2. Check the boxes next to executions you want to compare

3. Click Compare above the table

You'll see all metrics side-by-side for easy analysis.

---

Related topics

• Experiment Tracking & Visualizations

• Early stopping with metrics

• Pipeline conditions