Metadata: Track and Compare Your Metrics

Print your metrics as JSON and Valohai automatically tracks, visualizes, and lets you compare them across experiments. No special libraries required. Moreover, Valohai doesn’t restrict what metrics you can track, let it be accuracy, loss, f1_score, ROC, AUC, etc. The metrics are not restricted to numeric values.

💡 Already logging metrics? Just wrap them in JSON format and print. Valohai does the rest.

How It Works

1. Print JSON to stdout — Any valid JSON key value pairs

2. Valohai captures automatically — Real-time collection during training

3. View and compare — Sort experiments, plot graphs, set alerts

import json

# That's it - just print JSON
print(json.dumps({"accuracy": 0.92, "loss": 0.08}))

Quick Example

Training Loop with Metrics

import json

for epoch in range(epochs):
    # Your training code...
    train_loss = train_one_epoch()
    val_acc = validate()
    
    # Track metrics - just print as JSON
    print(json.dumps({
        "epoch": epoch,
        "train_loss": train_loss,
        "val_accuracy": val_acc,
    }))

Now you can:

• Sort experiments by the metric values

• Plot timeseries grapsh across multiple runs

• Set early-stopping rules based on the metrics

  • E.g. “stop execution when accuracy is over 0.9”

Optional: Use the valohai-utils Python helper tool

The valohai-utils helper library offers a simpler syntax:

import valohai

# Log incrementally
for epoch in range(epochs):
    with valohai.logger() as log:
        log.log("accuracy", accuracy)
        log.log("loss", loss)

Common Patterns

Multiple Metrics

import json

# Track everything you care about
metrics = {
    "accuracy": 0.94,
    "precision": 0.92,
    "recall": 0.95,
    "f1_score": 0.93,
    "auc_roc": 0.97,
    "inference_time_ms": 23.5,
}
print(json.dumps(metrics))

Metrics Over Time

import json

for step in range(training_steps):
    # Training...
    
    if step % log_interval == 0:
        print(json.dumps({
            "step": step,
            "loss": current_loss,
            "learning_rate": scheduler.get_lr()[0],
            "gradient_norm": grad_norm,
        }))

How to Use the Metrics in Valohai

Execution Table

See latest values for each execution:

• Sort by any metric

• Filter by thresholds

• Compare at a glance

Automatic Visualizations

• Time series plots per execution

• Compare multiple executions

• Download data as CSV

Best Practices

Log Regularly

# Log every N batches, not just epochs
if batch_idx % 100 == 0:
    print(json.dumps({
        "batch": batch_idx,
        "loss": loss.item(),
    }))

Include Context

# Add helpful context
print(json.dumps({
    "epoch": epoch,
    "phase": "validation",  # training/validation/test
    "accuracy": acc,
    "samples_seen": epoch * len(train_loader),
}))

Quick Reference

Basic Pattern

import json
print(json.dumps({"metric_name": value}))

What Works

• Any value

• Multiple metrics in one print

What to Track

• Model performance (accuracy, loss, F1)

• Training dynamics (learning rate, gradients)

• Any metadata you want log from executions

---