Manual Sweeps

Manual Search lets you define exact parameter combinations instead of generating them automatically.

This gives you precise control over which executions run, useful when you have domain knowledge about promising configurations or want to avoid testing irrelevant combinations.

When to use Manual Search

You have domain knowledge: You know which parameter combinations matter and want to skip the rest.

Benchmarking: You're comparing specific models, datasets, or code versions and need exact control over what gets tested.

Targeted experiments: Grid Search would generate too many combinations, but you know which specific ones to test.

Create a Manual Search Task

1. Open your project in Valohai

2. Go to the Tasks tab

3. Click Create Task

4. Select the step with your parameters

5. Scroll to Parameters

6. Select Manual search as the Task type

7. For each parameter:

   • Choose Single or Multiple as the type

   • Enter the value(s) for this parameter set

8. Click Add new set to define another parameter combination

9. Repeat until you've defined all desired combinations

10. Click Create task

Each parameter set becomes one execution in the Task.

Example: Benchmarking models

Suppose you want to compare three architectures on the same dataset:

Parameter Set 1:

• model_name: resnet50

• learning_rate: 0.001

Parameter Set 2:

• model_name: efficientnet

• learning_rate: 0.001

Parameter Set 3:

• model_name: vit

• learning_rate: 0.001

This creates 3 executions—one for each model. You can compare results in the Task view to see which architecture performs best.

Why Manual Search? Grid Search would create many more combinations if you had other parameters. Manual Search lets you test only what matters.

Example: Testing on multiple datasets

You have 100 customer datasets and want to train a model on each.

Step 1: Define a store_id parameter in your valohai.yaml:

- step:
    name: train-model
    parameters:
      - name: store_id
        type: string
    inputs:
      - name: dataset
        default: s3://bucket/data/stores/{parameter:store_id}/data/*

Step 2: Create a Manual Search Task with 100 parameter sets:

• Set 1: store_id = store-001

• Set 2: store_id = store-002

• ...

• Set 100: store_id = store-100

Each execution downloads its corresponding dataset from S3 and trains independently. This approach is faster than running 100 separate executions manually.

Manual Search vs. Grid Search

Manual Search:

• You define exact combinations

• Best when you know which combinations matter

• Fewer executions, targeted results

Grid Search:

• Valohai generates all combinations

• Best for exhaustive exploration

• More executions, comprehensive results

Use Manual Search when precision matters more than coverage.

Next steps

• Task Blueprints to define Manual Search Tasks in YAML

• Grid Search for automatic combination generation

• Early Stopping to halt Tasks based on results