Define Valohai parameters¶
See also
This how-to is a part of our Bring your existing projects to Valohai series.
A short introduction parameters
Defining parameters allows you to easily rerun, sort, and keep track of executions based on the parameter values used to run them.
You can easily create (or copy) an execution and change the parameters in the UI, without changing your code.
Defining parameters allows you to start creating Tasks where you run multiple parallel executions with different parameter combinations.
Each step has it’s own parameter configuration in
valohai.yaml
.A parameter can be type of a
string
,integer
,float
, andflag
(=boolean).Parameters are passed as command-line arguments to your code.
Edit how parameters are passed to your code using pass-as.
You can also parse parameter values from YAML and JSON files inside your execution. See file-based configuration.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | import valohai
# Define inputs available for this step and their default location
# The default location can be overriden when you create a new execution (UI, API or CLI)
default_inputs = {
'myinput': 's3://bucket/mydata.csv'
}
# Define parameters in a dictionary
default_parameters = {
'iterations': 10,
}
# Open the CSV file from Valohai inputs
with open(valohai.inputs("myinput").path()) as csv_file:
reader = csv.reader(csv_file, delimiter=',')
# Create a step 'train' in valohai.yaml with a set of parameters
valohai.prepare(step="train", image="tensorflow/tensorflow:2.6.1-gpu", default_inputs=default_inputs, default_parameters=default_parameters)
# Access the parameters in your code
for i in range(valohai.parameters('iterations').value):
print("Iteration %s" % i)
|
Generate or update your existing YAML file by running
vh yaml step myfile.py
The generated valohai.yaml
configuration file looks like:
import argparse
def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument('--iterations', type=int, default=10)
return parser.parse_args()
args = parse_args()
print(args.iterations)
Create a valohai.yaml
configuration file and define your step in it:
library(optparse)
option_list <- list(
make_option("--iterations", default = 10),
)
parser <- OptionParser(option_list = option_list)
args <- parse_args(parser)
iterations <- args$iterations
write(iterations, stdout())
Create a valohai.yaml
configuration file and define your step in it:
- step:
name: Train model
image: tensorflow/tensorflow:1.13.1
command: python myfile.py {parameters}
parameters:
- name: iterations
type: integer
default: 100
See also