Building Pipelines

Connect multiple execution steps into automated pipelines. Learn to define nodes, edges, and run pipelines from CLI.

💡 About this tutorial: We use YOLOv8 as a practical example to demonstrate Valohai's features. You don't need computer vision knowledge—the patterns you learn here apply to any ML framework. This tutorial focuses on chaining multiple steps into a pipeline while ensuring proper versioning and tracking of your ML workflows.

Connect your execution steps into automated workflows. Pipelines let you chain training, evaluation, and inference jobs with automatic data handoffs between them.

Core Concepts

Nodes — Individual execution steps in your pipeline Edges — Data connections between nodes (outputs → inputs, metrics → parameters, parameters → parameters)

Your pipeline is fully defined in valohai.yaml. No separate orchestration needed.

Define a Pipeline

Add this pipeline definition to your valohai.yaml:

- pipeline:
    name: train-and-predict
    nodes:
      - name: train
        step: yolo
        type: execution
      - name: inference
        step: inference
        type: execution
        override:
          inputs:
            - name: model
            - name: images
              default: https://ultralytics.com/images/bus.jpg
    edges:
    - [train.output.best.onnx, inference.input.model]

What's happening here?

1. Two nodes: train runs your YOLO training step, inference runs predictions

2. One edge: Connects the trained model output to the inference input

3. Override: Sets default values for the inference step

💡 The edge syntax [source.output.filename, target.input.name] automatically passes files between steps.

Run Your Pipeline

Execute the pipeline from your terminal:

vh pipeline run train-and-predict --adhoc

The --adhoc flag runs using your local valohai.yaml without committing to Git first.

Monitor Pipeline Progress

Open the Valohai UI to track your pipeline:

1. Click nodes to view individual job details, logs, and metrics

2. Check outputs in the top-right corner for pipeline-level results

When the training job completes, inference automatically starts with the trained model.

Related Content

• Add conditional logic to pipelines

• Debug failed pipeline nodes