# Run a Pipeline

This guide covers the three ways to create and run pipelines in Valohai: through the web interface, command-line, or API.

### Prerequisites

Before creating a pipeline, ensure you have:

* Individual steps defined and tested in your `valohai.yaml`

### Define a pipeline in valohai.yaml

Pipelines are defined in your `valohai.yaml` configuration file. Here's a complete example:

```yaml
# Define individual steps first
- step:
    name: preprocess-dataset
    image: python:3.9
    command:
      - pip install numpy valohai-utils
      - python ./preprocess_dataset.py
    inputs:
      - name: dataset
        default: https://valohaidemo.blob.core.windows.net/mnist/mnist.npz

- step:
    name: train-model
    image: tensorflow/tensorflow:2.6.0
    command:
      - pip install valohai-utils
      - python ./train_model.py {parameters}
    parameters:
      - name: epochs
        default: 5
        type: integer
      - name: learning_rate
        default: 0.001
        type: float
    inputs:
      - name: dataset
        default: https://valohaidemo.blob.core.windows.net/mnist/preprocessed_mnist.npz

- step:
    name: batch-inference
    image: tensorflow/tensorflow:2.6.0
    command:
      - pip install pillow valohai-utils
      - python ./batch_inference.py
    inputs:
      - name: model
      - name: images
        default:
        - https://valohaidemo.blob.core.windows.net/mnist/four-inverted.png
        - https://valohaidemo.blob.core.windows.net/mnist/five-inverted.png
        - https://valohaidemo.blob.core.windows.net/mnist/five-normal.jpg

# Define the pipeline structure
- pipeline:
    name: training-pipeline
    nodes:
      - name: preprocess
        type: execution
        step: preprocess-dataset
      - name: train
        type: execution
        step: train-model
        override:
          inputs:
            - name: dataset  # Replace default inputs with values from the edge
      - name: evaluate
        type: execution
        step: batch-inference
    edges:
      - [preprocess.output.preprocessed_mnist.npz, train.input.dataset]
      - [train.output.model*, evaluate.input.model]
```

#### Key configuration elements

**Nodes**: Reference your existing steps and give them names within the pipeline context.

**Edges**: Define data flow between nodes using the format:

* `[source_node.output.filename, target_node.input.input_name]`
* Use wildcards (`*`) to pass multiple files matching a pattern

**Override**: Remove default values from inputs that will receive data via edges.

### Create via web interface

Once your `valohai.yaml` with the pipeline definition is committed and Valohai has fetched the latest changes, you can create a pipeline run.

1. Navigate to your project
2. Click the **Pipelines** tab
3. Click **Create pipeline**
4. Select your pipeline blueprint from the dropdown (populated from `valohai.yaml`)
5. Review and modify the configuration:
   * Adjust parameters for any node
   * Change input URLs if needed
   * Select different environments or Docker images
6. Click **Create pipeline** to launch

> 💡 *The web interface pre-fills configurations from your YAML but allows runtime overrides without changing code.*

### Create via command-line

Run a pipeline using the Valohai CLI:

```shell
# Run with current directory's valohai.yaml
vh pipeline run training-pipeline --adhoc

# Run with latest fetched commit in Valohai
vh pipeline run training-pipeline

```

#### CLI options

* `--adhoc`: Use local uncommitted changes
* `--commit`: Specify a Git commit/branch/tag

### Create via API

For programmatic pipeline creation, send a POST request to the pipelines endpoint.

#### Basic example

```shell
curl -X POST https://app.valohai.com/api/v0/pipelines/ \
  -H "Authorization: Token YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d @pipeline.json
```

#### Pipeline configuration format

The API expects a JSON payload with complete pipeline specification:

```json
{
  "project": "YOUR_PROJECT_ID",
  "title": "training-pipeline",
  "nodes": [
    {
      "name": "preprocess",
      "type": "execution",
      "template": {
        "environment": "ENVIRONMENT_ID",
        "commit": "main",
        "step": "preprocess-dataset"
      }
    },
    {
      "name": "train",
      "type": "execution",
      "template": {
        "commit": "main",
        "step": "train-model",
        "parameters": {
          "epochs": 10,
          "learning_rate": 0.001
        }
      },
      "on_error": "stop-all"
    }
  ],
  "edges": [
    {
      "source_node": "preprocess",
      "source_key": "preprocessed_mnist.npz",
      "source_type": "output",
      "target_node": "train",
      "target_type": "input",
      "target_key": "dataset"
    }
  ]
}
```

> 💡 *Find your project ID and environment IDs via the web interface or API endpoints.*

### Pipeline execution behavior

Once created, pipelines execute automatically:

* Nodes start when all their input edges are satisfied
* Parallel execution occurs when dependencies allow
* Failed nodes can be configured to stop the entire pipeline or allow others to continue

### Next steps

* Learn about [reusing nodes](/pipelines/reuse-nodes.md) to avoid re-running successful steps
* Explore [conditional execution](/pipelines/dynamic-conditions.md) for dynamic workflows
* Set up [scheduled pipelines](/automation-overview/triggers/scheduled-triggers.md) for production automation


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.valohai.com/pipelines/run-basic-pipeline.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.