Define Valohai parameters¶

See also

This how-to is a part of our Bring your existing projects to Valohai series.

A short introduction parameters

Defining parameters allows you to easily rerun, sort, and keep track of executions based on the parameter values used to run them.
You can easily create (or copy) an execution and change the parameters in the UI, without changing your code.
Defining parameters allows you to start creating Tasks where you run multiple parallel executions with different parameter combinations.
Each step has it’s own parameter configuration in valohai.yaml.
A parameter can be type of a string, integer, float, and flag (=boolean).
Parameters are passed as command-line arguments to your code.
- Edit how parameters are passed to your code using pass-as.
- You can also parse parameter values from YAML and JSON files inside your execution. See file-based configuration.

valohai-utils (Python)

import valohai

# Define inputs available for this step and their default location
# The default location can be overriden when you create a new execution (UI, API or CLI)
default_inputs = {
    'myinput': 's3://bucket/mydata.csv'
}

# Define parameters in a dictionary
default_parameters = {
    'iterations': 10,
}

# Open the CSV file from Valohai inputs
with open(valohai.inputs("myinput").path()) as csv_file:
    reader = csv.reader(csv_file, delimiter=',')

# Create a step 'train' in valohai.yaml with a set of parameters
valohai.prepare(step="train", image="tensorflow/tensorflow:2.6.1-gpu", default_inputs=default_inputs, default_parameters=default_parameters)

# Access the parameters in your code
for i in range(valohai.parameters('iterations').value):
    print("Iteration %s" % i)

Generate or update your existing YAML file by running

vh yaml step myfile.py

The generated valohai.yaml configuration file looks like:

Python

import argparse

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('--iterations', type=int, default=10)
    return parser.parse_args()

args = parse_args()

print(args.iterations)

Create a valohai.yaml configuration file and define your step in it:

library(optparse)

option_list <- list(
    make_option("--iterations", default = 10),
)

parser <- OptionParser(option_list = option_list)
args <- parse_args(parser)

iterations <- args$iterations
write(iterations, stdout())

Create a valohai.yaml configuration file and define your step in it:

- step:
    name: Train model
    image: tensorflow/tensorflow:1.13.1
    command: python myfile.py {parameters}
    parameters:
      - name: iterations
        type: integer
        default: 100