# Pipeline Parameters

Pipeline parameters let you control multiple nodes with a single value. Instead of updating each node individually, define shared parameters that automatically propagate to their targets.

## When to use pipeline parameters

Use pipeline parameters when:

* Multiple nodes need the same configuration value (e.g., batch size, data version)
* You want to experiment with different settings across an entire pipeline
* You need centralized control over distributed processing parameters

## Configure in valohai.yaml

Define pipeline parameters with their target nodes in your `valohai.yaml`:

```yaml
- step:
    name: preprocess-dataset
    image: python:3.9
    command:
      - pip install numpy valohai-utils
      - python ./preprocess_dataset.py
    parameters:
      - name: exec_id
        type: string
      - name: filters
        type: string
        default: ["low-pass"]
    inputs:
      - name: dataset
        default: https://valohaidemo.blob.core.windows.net/mnist/mnist.npz

- step:
    name: train-model
    image: tensorflow/tensorflow:2.6.0
    command:
      - pip install valohai-utils
      - python ./train_model.py {parameters}
    parameters:
      - name: exec_id
        type: string
      - name: train_param
        type: integer
        default: 5

- pipeline:
    name: shared-parameters-example
    parameters:
      - name: id
        targets:
          - preprocess.parameters.exec_id
          - train.parameters.exec_id
      - name: training_parameter
        targets:
          - train.parameters.train_param
        default: 3
      - name: filters
        target: preprocess.parameters.filters
        default: ["remove-outliers", "normalize"]
    nodes:
      - name: preprocess
        step: preprocess-dataset
        type: execution
      - name: train
        step: train-model
        type: execution
      - name: train_in_task
        step: train-model
        type: task
    edges:
      - [preprocess.output.preprocessed_mnist.npz, train.input.dataset]
```

### Key concepts

**Target syntax**: `<node-name>.parameters.<step-parameter-name>`

**Multiple targets**: One pipeline parameter can override multiple node parameters:

```yaml
- name: id
  targets:
    - preprocess.parameters.exec_id
    - train.parameters.exec_id
```

**Selective targeting**: Nodes without targets keep their default values. In the example, `train_in_task` uses the step's default `exec_id`, not the pipeline parameter.

## Access in your code

Pipeline parameters work exactly like regular parameters in your code:

### Command-line parsing

```python
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--exec_id", type=str)
parser.add_argument("--train_param", type=int, default=5)
args = parser.parse_args()
```

### Python with valohai-utils

```python
import valohai

# Access the parameter value - doesn't matter if it's pipeline or node parameter
exec_id = valohai.parameters("exec_id").value
filters = valohai.parameters("filters").value
```

> 💡 *Your code doesn't need to know whether a parameter comes from the pipeline or node level.*

## Multi-value parameters for tasks

Pipeline parameters can distribute multiple values across task nodes:

```yaml
- pipeline:
    name: parallel-processing
    parameters:
      - name: task_configs
        target: process_batch.parameters.config_id
        default: [100, 200, 300, 400]
    nodes:
      - name: process_batch
        step: batch-processor
        type: task  # Creates 4 parallel executions
```

Each value creates a separate execution:

* Execution 1: `config_id=100`
* Execution 2: `config_id=200`
* Execution 3: `config_id=300`
* Execution 4: `config_id=400`

## Web interface behavior

<figure><img src="/files/OzvkEx3dzHq31T8ahLwA" alt=""><figcaption></figcaption></figure>

In the web interface:

1. **Shared Parameters section** shows all pipeline parameters and their targets
2. **Overridden parameters** appear grayed out in node configurations
3. **Non-targeted parameters** remain editable at the node level
4. **Task nodes** allow multiple value entry for parallel execution

## Common patterns

### Experiment tracking

Share a unique ID across all nodes for unified logging:

```yaml
- name: experiment_id
  targets:
    - preprocess.parameters.exp_id
    - train.parameters.exp_id
    - evaluate.parameters.exp_id
```

### Resource scaling

Adjust compute resources uniformly:

```yaml
- name: batch_size
  targets:
    - preprocess.parameters.batch_size
    - train.parameters.batch_size
  default: 32
```

## Next steps

* Learn about [conditional execution](/pipelines/dynamic-conditions.md) based on parameter values
* Set up [parameter sweeps](/notebook-executions/tune-from-notebook.md) for hyperparameter optimization


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.valohai.com/pipelines/pipeline-parameters.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.