Reusable Step Libraries

Reusable Step Libraries let you define steps once and use them across all projects in your organization.

How they differ from project steps

Project steps are defined in each project's valohai.yaml:

• Specific to one project

• Tied to your codebase

• Modified by anyone with project access

Library steps are defined in separate Git repositories:

• Shared across your entire organization

• Managed centrally by admins

• Reusable without duplication

When to use libraries

Create library steps for workflows you run repeatedly across projects:

• Database queries (Redshift, BigQuery, Snowflake)

• Image preprocessing

• Model evaluation scripts

• Docker image builds

• Data validation checks

• Report generation

If you find yourself copying the same step definition into multiple projects, it belongs in a library.

Two types of libraries

Valohai Ecosystem Libraries

Pre-built steps maintained by Valohai:

• Database connectors (AWS Redshift, GCP BigQuery, Snowflake)

• Common ML workflows

• Infrastructure utilities (Docker Image Builder)

See Using Valohai Ecosystem Libraries to get started.

Custom Libraries

Your own reusable steps:

• Organization-specific workflows

• Custom integrations

• Internal tooling

See Build Your Own Library to create one.

How it works

1. Admins connect Git repositories as libraries

2. Library YAML defines steps with metadata (category, icon, description)

3. All users see library steps when creating executions

4. Steps run just like project steps—same parameters, same environments

Library steps appear in the execution creation UI alongside your project steps, organized by category.

What makes a good library step?

Good candidates:

• Generic and reusable across projects

• Well-documented with clear parameters

• Stable and rarely changes

• Solves a common problem

Poor candidates:

• Highly project-specific logic

• Frequently changing implementations

• Tightly coupled to one codebase

Next steps

Use existing libraries: Start with Valohai Ecosystem Libraries for database connectors and common utilities.

Build your own: Follow Build Your Own Library to create custom steps for your organization.