Compare Images

Stack and compare output images from multiple executions. Use blend modes, side-by-side sliders, and color overlays to spot differences in computer vision experiments, quality control testing, and visual analysis.

---

When to Use Image Comparison

Computer vision experiments:

• Compare predictions from different models

• Evaluate segmentation or detection results

• Track improvement across training iterations

Quality control:

• Compare defect detection across different thresholds

• Evaluate image processing pipeline changes

• Validate improvements in image quality

Medical imaging:

• Compare diagnostic outputs

• Evaluate segmentation accuracy

• Track changes in processing algorithms

Before/after analysis:

• Preprocessing effects

• Model architecture changes

• Hyperparameter impact on visual outputs

---

Quick Start (Manual Comparison)

1. Save Images as Outputs

Save images to /valohai/outputs/ in your code:

import matplotlib.pyplot as plt

# Generate and save prediction image
plt.figure(figsize=(10, 10))
plt.imshow(prediction)
plt.savefig("/valohai/outputs/prediction.png")
plt.close()

# Save ground truth
plt.figure(figsize=(10, 10))
plt.imshow(ground_truth)
plt.savefig("/valohai/outputs/ground_truth.png")
plt.close()

---

2. Select Executions

From your project's Executions tab:

1. Select multiple executions using checkboxes

2. Click Compare

---

3. Switch to Images Tab

In the comparison view:

1. Look for the tabs at the top: Metadata and Images

2. Click the Images tab

3. All output images from selected executions appear

---

4. Build Comparison Stacks in Images Tab

Manually add images to stacks:

1. Find the image you want to compare

2. Click the + button next to the image name

3. Select "Add to new stack" or choose an existing stack

4. Repeat for other images you want to compare

Images in the same stack overlay on top of each other, letting you toggle visibility, adjust transparency, and use blend modes.

---

Use Preview Grid

The Preview Grid lets you compare image outputs side-by-side across multiple executions.

Each column represents one execution, and images are aligned in a grid so you can easily spot visual differences. When you hover over an image in one column, the corresponding images in other columns highlight in sync.

You can filter outputs to focus on specific images, making it easy to review model changes, compare checkpoints, or validate visual consistency across runs.

---

Advanced: Automatic Grouping with Metadata

For complex comparisons with many images, use special metadata properties to automatically organize images into comparison groups.

Basic Setup

Add comparison metadata when saving images:

import json
import matplotlib.pyplot as plt

# Save your image
plt.savefig("/valohai/outputs/prediction.png")

# Add comparison metadata
metadata = {
    "vhic_group": "main group/secondary group/subgroup",
    "vhic_base": "name_of_the_comparison_base_file.png",
    "vhic_truth": "name_of_the_comparison_truth_file.png",
    "vhic_name": "Display name for this image",
    "vhic_blend": "multiply",
    "vhic_color": "red",
}

with open("/valohai/outputs/prediction.png.metadata.json", "w") as f:
    json.dump(metadata, f)

---

Comparison Properties

vhic_group (required for auto-grouping) Groups related images together. Use / for hierarchical grouping.

"vhic_group": "main/sub-group/specific"

vhic_base (required for auto-grouping) Filename of the base/reference image for comparison.

"vhic_base": "base_file.png"

vhic_truth (required for auto-grouping) Filename of the ground truth image (often same as base).

"vhic_truth": "ground_truth.png"

vhic_name (optional) Display name in the UI instead of filename.

"vhic_name": "Model A - Best Prediction"

vhic_blend (optional) Initial blend mode for the layer. You can scroll through the different blend modes in the UI: the property just sets the initial value used when the stack is built.

"vhic_blend": "multiply"  # Options: normal, multiply, screen, difference, subtract

vhic_color (optional) Initial color overlay for comparison. You can override this by setting the vhic_color property to a color name, or none to use the image as is.

"vhic_color": "red"  # Options: none, red, yellow, green

---

Complete Example

import json
import matplotlib.pyplot as plt
import numpy as np

# Generate prediction and ground truth
prediction = model.predict(input_image)
ground_truth = load_ground_truth()

# Save ground truth
plt.imsave("/valohai/outputs/ground_truth.png", ground_truth)

# Save prediction with metadata
plt.imsave("/valohai/outputs/prediction.png", prediction)

metadata = {
    "vhic_group": "segmentation/building-detection",
    "vhic_base": "base_image.png",
    "vhic_truth": "ground_truth.png",
    "vhic_name": "ResNet50 Prediction",
    "vhic_blend": "multiply",
    "vhic_color": "red",
}

with open("/valohai/outputs/prediction.png.metadata.json", "w") as f:
    json.dump(metadata, f)

---

Hierarchical Grouping

Organize images into a hierarchy using / separators:

# Group 1: Building detection, top view
metadata = {
    "vhic_group": "segmentation/building-detection/top-view",
    "vhic_base": "base_image_top.png",
    "vhic_truth": "ground_truth_top.png",
}

# Group 2: Building detection, side view
metadata = {
    "vhic_group": "segmentation/building-detection/side-view",
    "vhic_base": "base_image_side.png",
    "vhic_truth": "ground_truth_side.png",
}

# Group 3: Road detection
metadata = {
    "vhic_group": "segmentation/road-detection",
    "vhic_base": "base_image_road.png",
    "vhic_truth": "ground_truth_road.png",
}

In the UI, you'll see a dropdown with:

• segmentation

  • building-detection

    • top-view

    • side-view

  • road-detection

---

Using the Comparison Interface

Layer Controls

Each image in a stack has controls:

Transparency slider: Adjust opacity to blend with layers below Blend mode buttons: Cycle through blend modes (normal, multiply, screen, difference, subtract) Visibility toggle: Show/hide the layer Remove: Take the layer out of the stack

---

Blend Modes

Normal (default): Standard image display. Use transparency to see layers below.

Multiply: Darkens the image. Useful for highlighting differences where prediction is darker than ground truth.

Screen: Lightens the image. Opposite of multiply.

Difference: Shows absolute difference between layers. Bright areas indicate differences, dark areas indicate similarity.

Subtract: Subtracts pixel values. Useful for seeing where prediction exceeds or falls short of ground truth.

---

Color Overlays

Apply color tints for easier visual comparison:

Red: Tint the layer red Yellow: Tint the layer yellow Green: Tint the layer green None: Use original image colors

Use case: Compare two predictions by tinting one red and one green. Where they overlap and agree, you'll see yellow.

---

Side-by-Side Slider

For direct pixel-level comparison:

1. Select two images in a stack

2. Use the slider widget

3. Drag left/right to reveal one image or the other

4. Perfect for spotting subtle differences

---

Pre-Built Comparison Views

When using metadata properties, Valohai provides four automatic views:

Browse: All Images

One stack per image in the group. Base image first, then target, then comparison images.

Use when: You want to see every image individually stacked with its base.

---

Overview: Base + Individual Comparisons

One base stack (base + truth images), then one stack per comparison image (base + that comparison).

Use when: Comparing multiple predictions against the same ground truth.

---

Focus: Base + Switchable Comparison

Same as View 2, but only one comparison stack visible at a time. Quick switch widget to change which comparison you're viewing.

Use when: Many comparison images and you want a cleaner view.

---

Compare: Side-by-Side

Base stack (base + truth) and one combined stack with all comparison images. Side-by-side slider to compare two images. Quick switch to change which images appear in the slider.

Use when: Pixel-perfect comparison is important.

---

Common Use Cases

Compare Model Predictions

# Run multiple models on the same test image
for model_name in ["resnet50", "efficientnet", "vit"]:
    model = load_model(model_name)
    prediction = model.predict(test_image)

    # Save prediction
    output_path = f"/valohai/outputs/{model_name}_prediction.png"
    plt.imsave(output_path, prediction)

    # Add metadata
    metadata = {
        "vhic_group": "test-image-1",
        "vhic_base": "base_image.png",
        "vhic_truth": "ground_truth.png",
        "vhic_name": f"{model_name} Prediction",
    }

    with open(f"{output_path}.metadata.json", "w") as f:
        json.dump(metadata, f)

---

Track Training Progress

# Save predictions at different epochs
for epoch in [10, 50, 100]:
    checkpoint = load_checkpoint(epoch)
    prediction = checkpoint.predict(test_image)

    output_path = f"/valohai/outputs/prediction_epoch_{epoch}.png"
    plt.imsave(output_path, prediction)

    metadata = {
        "vhic_group": "training-progress",
        "vhic_base": "base_image.png",
        "vhic_truth": "ground_truth.png",
        "vhic_name": f"Epoch {epoch}",
    }

    with open(f"{output_path}.metadata.json", "w") as f:
        json.dump(metadata, f)

---

Quality Control Comparison

# Before/after image processing
base_image = load_base()
input_image = load_input()
processed_image = process_pipeline(input_image)

# Save both
plt.imsave("/valohai/outputs/base_image.png", base_image)
plt.imsave("/valohai/outputs/input.png", input_image)
plt.imsave("/valohai/outputs/processed.png", processed_image)

# Add metadata
metadata = {
    "vhic_group": "qc/batch-123",
    "vhic_base": "base_image.png",
    "vhic_truth": "input.png",
    "vhic_name": "Processed Output",
    "vhic_blend": "difference",  # Show differences clearly
}

with open("/valohai/outputs/processed.png.metadata.json", "w") as f:
    json.dump(metadata, f)

---

Best Practices

Use Consistent Image Sizes

All images in a stack should have the same dimensions:

#  Good: Resize to consistent size
from PIL import Image

img = Image.open("prediction.png")
img_resized = img.resize((512, 512))
img_resized.save("/valohai/outputs/prediction.png")

#  Avoid: Mismatched sizes
# Image 1: 640x480
# Image 2: 1024x768

Mismatched sizes make visual comparison difficult.

---

Save at the Same Scale

Keep pixel value ranges consistent:

#  Good: Normalize to [0, 1] or [0, 255]
prediction_normalized = (prediction - prediction.min()) / (prediction.max() - prediction.min())
plt.imsave("/valohai/outputs/prediction.png", prediction_normalized)

#  Avoid: Arbitrary scales
# Some images in [0, 1]
# Others in [0, 255]
# Others in [-10, 10]

---

Use Descriptive Filenames

Choose clear, descriptive names:

#  Good: Clear meaning
"ground_truth.png"

"resnet50_prediction.png"
"epoch_100_output.png"

#  Avoid: Generic names
"output.png"
"img1.png"
"result.png"

Or use the vhic_name property for custom display names.

---

Group Related Images

Use the vhic_group property to organize:

# Group by test image
"vhic_group": "test-image-1"
"vhic_group": "test-image-2"

# Or by experiment
"vhic_group": "experiment-a/segmentation"
"vhic_group": "experiment-b/segmentation"

---

Choose Appropriate Blend Modes

Different blend modes highlight different features:

Normal: General comparison, use transparency Multiply: Darken overlaps, good for detecting false positives Difference: Highlight all differences, both bright and dark Subtract: Directional differences (over-prediction vs under-prediction)

Experiment to find what works best for your use case.

---

Troubleshooting

Images Tab Not Appearing

Symptom: Only Metadata tab visible, no Images tab

Cause: No image outputs in selected executions

Solution: Ensure your code saves images to /valohai/outputs/:

import matplotlib.pyplot as plt

plt.savefig("/valohai/outputs/my_image.png")

---

Images Not Grouping Automatically

Symptom: Images appear ungrouped even with metadata

Causes & Fixes:

Missing required properties:

#  Incomplete metadata
metadata = {
    "vhic_group": "test",  # Missing vhic_base and vhic_truth
}

#  Complete metadata
metadata = {
    "vhic_group": "test",
    "vhic_base": "base_image.png",
    "vhic_truth": "ground_truth.png",
}

Filename mismatch:

#  Wrong: Base file doesn't exist
"vhic_base": "baseimage.png"  # Actual file: "baseimage.png"

#  Correct: Exact filename match
"vhic_base": "base_image.png"

Inconsistent base across group:

#  Wrong: Different base files in same group
Image 1: "vhic_base": "base1.png"
Image 2: "vhic_base": "base2.png"  # Both in same group

#  Correct: Same base file
Image 1: "vhic_base": "base_image.png"
Image 2: "vhic_base": "base_image.png"

---

Images Look Wrong When Stacked

Symptom: Colors or brightness look incorrect

Cause: Images have different value ranges or color spaces

Solution: Normalize before saving:

import numpy as np

# Normalize to [0, 1]
img_normalized = (img - img.min()) / (img.max() - img.min())
plt.imsave("/valohai/outputs/normalized.png", img_normalized)

# Or clip to valid range
img_clipped = np.clip(img, 0, 1)
plt.imsave("/valohai/outputs/clipped.png", img_clipped)

---

Can't See Differences

Symptom: Images look identical when stacked

Try:

• Use Difference blend mode to highlight even small differences

• Adjust transparency to see subtle variations

• Use color overlays (red/green) to spot misalignments

• Use side-by-side slider for pixel-perfect comparison

---

Next Steps

• Visualize metrics alongside image comparisons

• Compare executions to see which model produces better images

• Save comparison stacks for documentation and reporting

• Back to Experiment Tracking overview