Compare Images

Stack and compare output images from multiple executions. Use blend modes, side-by-side sliders, and color overlays to spot differences in computer vision experiments, quality control testing, and visual analysis.

When to Use Image Comparison

Computer vision experiments:

Compare predictions from different models
Evaluate segmentation or detection results
Track improvement across training iterations

Quality control:

Compare defect detection across different thresholds
Evaluate image processing pipeline changes
Validate improvements in image quality

Medical imaging:

Compare diagnostic outputs
Evaluate segmentation accuracy
Track changes in processing algorithms

Before/after analysis:

Preprocessing effects
Model architecture changes
Hyperparameter impact on visual outputs

Quick Start (Manual Comparison)

1. Save Images as Outputs

Save images to /valohai/outputs/ in your code:

import matplotlib.pyplot as plt

# Generate and save prediction image
plt.figure(figsize=(10, 10))
plt.imshow(prediction)
plt.savefig('/valohai/outputs/prediction.png')
plt.close()

# Save ground truth
plt.figure(figsize=(10, 10))
plt.imshow(ground_truth)
plt.savefig('/valohai/outputs/ground_truth.png')
plt.close()

2. Select Executions

From your project's Executions tab:

Select multiple executions using checkboxes
Click Compare

3. Switch to Images Tab

In the comparison view:

Look for the tabs at the top: Metadata and Images
Click the Images tab
All output images from selected executions appear

4. Build Comparison Stacks

Manually add images to stacks:

Find the image you want to compare
Click the + button next to the image name
Select "Add to new stack" or choose an existing stack
Repeat for other images you want to compare

Images in the same stack overlay on top of each other, letting you toggle visibility, adjust transparency, and use blend modes.

Advanced: Automatic Grouping with Metadata

For complex comparisons with many images, use special metadata properties to automatically organize images into comparison groups.

Basic Setup

Add comparison metadata when saving images:

import json
import matplotlib.pyplot as plt

# Save your image
plt.savefig('/valohai/outputs/prediction.png')

# Add comparison metadata
metadata = {
  "vhic_group": "main group/secondary group/subgroup",
  "vhic_base": "name_of_the_comparison_base_file.png",
  "vhic_truth": "name_of_the_comparison_truth_file.png",
  "vhic_name": "Display name for this image",
  "vhic_blend": "multiply",
  "vhic_color": "red"
}

with open('/valohai/outputs/prediction.png.metadata.json', 'w') as f:
    json.dump(metadata, f)

Comparison Properties

vhic_group (required for auto-grouping) Groups related images together. Use / for hierarchical grouping.

"vhic_group": "main/sub-group/specific"

vhic_base (required for auto-grouping) Filename of the base/reference image for comparison.

"vhic_base": "base_file.png"

vhic_truth (required for auto-grouping) Filename of the ground truth image (often same as base).

"vhic_truth": "ground_truth.png"

vhic_name (optional) Display name in the UI instead of filename.

"vhic_name": "Model A - Best Prediction"

vhic_blend (optional) Initial blend mode for the layer. You can scroll through the different blend modes in the UI: the property just sets the initial value used when the stack is built.

"vhic_blend": "multiply"  # Options: normal, multiply, screen, difference, subtract

vhic_color (optional) Initial color overlay for comparison. You can override this by setting the vhic_color property to a color name, or none to use the image as is.

"vhic_color": "red"  # Options: none, red, yellow, green

Complete Example

import json
import matplotlib.pyplot as plt
import numpy as np

# Generate prediction and ground truth
prediction = model.predict(input_image)
ground_truth = load_ground_truth()

# Save ground truth
plt.imsave('/valohai/outputs/ground_truth.png', ground_truth)

# Save prediction with metadata
plt.imsave('/valohai/outputs/prediction.png', prediction)

metadata = {
    "vhic_group": "segmentation/building-detection",
    "vhic_base": "base_image.png",
    "vhic_truth": "ground_truth.png",
    "vhic_name": "ResNet50 Prediction",
    "vhic_blend": "multiply",
    "vhic_color": "red"
}

with open('/valohai/outputs/prediction.png.metadata.json', 'w') as f:
    json.dump(metadata, f)

Hierarchical Grouping

Organize images into a hierarchy using / separators:

# Group 1: Building detection, top view
metadata = {
    "vhic_group": "segmentation/building-detection/top-view",
    "vhic_base": "base_image_top.png",
    "vhic_truth": "ground_truth_top.png"
}

# Group 2: Building detection, side view
metadata = {
    "vhic_group": "segmentation/building-detection/side-view",
    "vhic_base": "base_image_side.png",
    "vhic_truth": "ground_truth_side.png"
}

# Group 3: Road detection
metadata = {
    "vhic_group": "segmentation/road-detection",
    "vhic_base": "base_image_road.png",
    "vhic_truth": "ground_truth_road.png"
}

In the UI, you'll see a dropdown with:

segmentation
- building-detection
  - top-view
  - side-view
- road-detection

Using the Comparison Interface

Layer Controls

Each image in a stack has controls:

Transparency slider: Adjust opacity to blend with layers below Blend mode buttons: Cycle through blend modes (normal, multiply, screen, difference, subtract) Visibility toggle: Show/hide the layer Remove: Take the layer out of the stack

Blend Modes

Normal (default): Standard image display. Use transparency to see layers below.

Multiply: Darkens the image. Useful for highlighting differences where prediction is darker than ground truth.

Screen: Lightens the image. Opposite of multiply.

Difference: Shows absolute difference between layers. Bright areas indicate differences, dark areas indicate similarity.

Subtract: Subtracts pixel values. Useful for seeing where prediction exceeds or falls short of ground truth.

Color Overlays

Apply color tints for easier visual comparison:

Red: Tint the layer red Yellow: Tint the layer yellow Green: Tint the layer green None: Use original image colors

Use case: Compare two predictions by tinting one red and one green. Where they overlap and agree, you'll see yellow.

Side-by-Side Slider

For direct pixel-level comparison:

Select two images in a stack
Use the slider widget
Drag left/right to reveal one image or the other
Perfect for spotting subtle differences

Pre-Built Comparison Views

When using metadata properties, Valohai provides four automatic views:

Browse: All Images

One stack per image in the group. Base image first, then target, then comparison images.

Use when: You want to see every image individually stacked with its base.

Overview: Base + Individual Comparisons

One base stack (base + truth images), then one stack per comparison image (base + that comparison).

Use when: Comparing multiple predictions against the same ground truth.

Focus: Base + Switchable Comparison

Same as View 2, but only one comparison stack visible at a time. Quick switch widget to change which comparison you're viewing.

Use when: Many comparison images and you want a cleaner view.

Compare: Side-by-Side

Base stack (base + truth) and one combined stack with all comparison images. Side-by-side slider to compare two images. Quick switch to change which images appear in the slider.

Use when: Pixel-perfect comparison is important.

Common Use Cases

Compare Model Predictions

# Run multiple models on the same test image
for model_name in ['resnet50', 'efficientnet', 'vit']:
    model = load_model(model_name)
    prediction = model.predict(test_image)
    
    # Save prediction
    output_path = f'/valohai/outputs/{model_name}_prediction.png'
    plt.imsave(output_path, prediction)
    
    # Add metadata
    metadata = {
        "vhic_group": "test-image-1",
        "vhic_base": "base_image.png",
        "vhic_truth": "ground_truth.png",
        "vhic_name": f"{model_name} Prediction"
    }
    
    with open(f'{output_path}.metadata.json', 'w') as f:
        json.dump(metadata, f)

Track Training Progress

# Save predictions at different epochs
for epoch in [10, 50, 100]:
    checkpoint = load_checkpoint(epoch)
    prediction = checkpoint.predict(test_image)
    
    output_path = f'/valohai/outputs/prediction_epoch_{epoch}.png'
    plt.imsave(output_path, prediction)
    
    metadata = {
        "vhic_group": "training-progress",
        "vhic_base": "base_image.png",
        "vhic_truth": "ground_truth.png",
        "vhic_name": f"Epoch {epoch}"
    }
    
    with open(f'{output_path}.metadata.json', 'w') as f:
        json.dump(metadata, f)

Quality Control Comparison

# Before/after image processing
base_image = load_base()
input_image = load_input()
processed_image = process_pipeline(input_image)

# Save both
plt.imsave('/valohai/outputs/base_image.png', base_image)
plt.imsave('/valohai/outputs/input.png', input_image)
plt.imsave('/valohai/outputs/processed.png', processed_image)

# Add metadata
metadata = {
    "vhic_group": "qc/batch-123",
    "vhic_base": "base_image.png",
    "vhic_truth": "input.png",
    "vhic_name": "Processed Output",
    "vhic_blend": "difference"  # Show differences clearly
}

with open('/valohai/outputs/processed.png.metadata.json', 'w') as f:
    json.dump(metadata, f)

Best Practices

Use Consistent Image Sizes

All images in a stack should have the same dimensions:

#  Good: Resize to consistent size
from PIL import Image

img = Image.open('prediction.png')
img_resized = img.resize((512, 512))
img_resized.save('/valohai/outputs/prediction.png')

#  Avoid: Mismatched sizes
# Image 1: 640x480
# Image 2: 1024x768

Mismatched sizes make visual comparison difficult.

Save at the Same Scale

Keep pixel value ranges consistent:

#  Good: Normalize to [0, 1] or [0, 255]
prediction_normalized = (prediction - prediction.min()) / (prediction.max() - prediction.min())
plt.imsave('/valohai/outputs/prediction.png', prediction_normalized)

#  Avoid: Arbitrary scales
# Some images in [0, 1]
# Others in [0, 255]
# Others in [-10, 10]

Use Descriptive Filenames

Choose clear, descriptive names:

#  Good: Clear meaning
"ground_truth.png"
"resnet50_prediction.png"
"epoch_100_output.png"

#  Avoid: Generic names
"output.png"
"img1.png"
"result.png"

Or use the vhic_name property for custom display names.

Use the vhic_group property to organize:

# Group by test image
"vhic_group": "test-image-1"
"vhic_group": "test-image-2"

# Or by experiment
"vhic_group": "experiment-a/segmentation"
"vhic_group": "experiment-b/segmentation"

Choose Appropriate Blend Modes

Different blend modes highlight different features:

Normal: General comparison, use transparency Multiply: Darken overlaps, good for detecting false positives Difference: Highlight all differences, both bright and dark Subtract: Directional differences (over-prediction vs under-prediction)

Experiment to find what works best for your use case.

Troubleshooting

Images Tab Not Appearing

Symptom: Only Metadata tab visible, no Images tab

Cause: No image outputs in selected executions

Solution: Ensure your code saves images to /valohai/outputs/:

import matplotlib.pyplot as plt
plt.savefig('/valohai/outputs/my_image.png')

Images Not Grouping Automatically

Symptom: Images appear ungrouped even with metadata

Causes & Fixes:

Missing required properties:

#  Incomplete metadata
metadata = {
    "vhic_group": "test"  # Missing vhic_base and vhic_truth
}

#  Complete metadata
metadata = {
    "vhic_group": "test",
    "vhic_base": "base_image.png",
    "vhic_truth": "ground_truth.png"
}

Filename mismatch:

#  Wrong: Base file doesn't exist
"vhic_base": "baseimage.png"  # Actual file: "baseimage.png"

#  Correct: Exact filename match
"vhic_base": "base_image.png"

Inconsistent base across group:

#  Wrong: Different base files in same group
Image 1: "vhic_base": "base1.png"
Image 2: "vhic_base": "base2.png"  # Both in same group

#  Correct: Same base file
Image 1: "vhic_base": "base_image.png"
Image 2: "vhic_base": "base_image.png"

Images Look Wrong When Stacked

Symptom: Colors or brightness look incorrect

Cause: Images have different value ranges or color spaces

Solution: Normalize before saving:

import numpy as np

# Normalize to [0, 1]
img_normalized = (img - img.min()) / (img.max() - img.min())
plt.imsave('/valohai/outputs/normalized.png', img_normalized)

# Or clip to valid range
img_clipped = np.clip(img, 0, 1)
plt.imsave('/valohai/outputs/clipped.png', img_clipped)

Can't See Differences

Symptom: Images look identical when stacked

Try:

Use Difference blend mode to highlight even small differences
Adjust transparency to see subtle variations
Use color overlays (red/green) to spot misalignments
Use side-by-side slider for pixel-perfect comparison

Next Steps

Visualize metrics alongside image comparisons
Compare executions to see which model produces better images
Save comparison stacks for documentation and reporting
Back to Experiment Tracking overview

PreviousCompare Executions NextTasks & Parallel Execution

Last updated 5 hours ago

Was this helpful?