This tutorial is a part of our Valohai Fundamentals series.
In this section you will learn how to
Import Valohai in your Python scripts
Save Valohai outputs
Run executions in Valohai
A sample training script¶
Download this mnist.npz file to your working directory
Create a new file
valohai-quickstart/train.pyand paste the following example script:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
import numpy as np import tensorflow as tf input_path = 'mnist.npz' with np.load(input_path, allow_pickle=True) as f: x_train, y_train = f['x_train'], f['y_train'] x_test, y_test = f['x_test'], f['y_test'] x_train, x_test = x_train / 255.0, x_test / 255.0 model = tf.keras.models.Sequential([ tf.keras.layers.Flatten(input_shape=(28, 28)), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(10, activation='softmax') ]) loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True) model.compile(optimizer='adam', loss=loss_fn, metrics=['accuracy']) model.fit(x_train, y_train, epochs=5) model.evaluate(x_test, y_test, verbose=2) output_path = 'model.h5' model.save(output_path)
Define a Valohai configuration file¶
Now we’ll create a valohai.yaml - Config File for your project. Run the following commands from your command line:
# Initialize a new Valohai project in this directory vh init # Confirm the directory (y) # valohai-cli will look for any Python files in your directory # it will find our train.py and # ask if that's the script file you want to use as the command # Choose train.py by typing 1 # Confirm selection (y) # Choose a Docker image from the list, or input your own image # Choose tensorflow/tensorflow:2.1.0-py3 by typing 12 # You'll see a preview of the valohai.yaml configuration file # accept it (y) to generate the file
Now edit the file and change the name of the step to
1 2 3 4 5 6 7 8 9 10 11 12 13 14
--- - step: name: train-model image: tensorflow/tensorflow:2.1.0-py3 command: python train.py #inputs: # - name: example-input # default: https://example.com/ #parameters: # - name: example # description: Example parameter # type: integer # default: 300
Run in Valohai¶
Finally, we run the following commands in the the command line, on your own computer:
vh exec run train-model --adhoc
vh exec run <step-name>creates a new Valohai execution with the step that we defined.
--adhoctells Valohai that the code for this execution is coming from our local machine instead of our code repository. Valohai will package the local directory and upload it for an execution.