Getting started

Note

This tutorial is a part of our Valohai Fundamentals series.

In this section you will learn how to

  • Import Valohai in your Python scripts

  • Save Valohai outputs

  • Run executions in Valohai

A sample training script

  • Download this mnist.npz file to your working directory

  • Create a new file valohai-quickstart/train.py and paste the following example script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import numpy as np
import tensorflow as tf

input_path = 'mnist.npz'
with np.load(input_path, allow_pickle=True) as f:
    x_train, y_train = f['x_train'], f['y_train']
    x_test, y_test = f['x_test'], f['y_test']

x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation='softmax')
])

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer='adam',
              loss=loss_fn,
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)

model.evaluate(x_test,  y_test, verbose=2)

output_path = 'model.h5'
model.save(output_path)

Import Valohai

Now we’ll import valohai and define a new Step in the train.py file we just created..

  • Create a new the step called train-model

  • Define a default Docker image for this step. The Docker image should contain the packages we need to run our code, like Tensorflow.

1
2
3
4
5
6
7
8
import numpy as np
import tensorflow as tf
import valohai

valohai.prepare(
    step='train-model',
    image='tensorflow/tensorflow:2.6.0',
)

Run in Valohai

Finally, we run the following commands in the command line, on your own computer:

vh yaml step train.py
vh exec run train-model --adhoc
  • vh yaml step uses the command-line tools to generate a valohai.yaml - Config File and a requirements.txt that contains valohai-utils (which is need to run the Python script).

  • vh exec run <step-name> creates a new Valohai execution with the step that we defined.

  • --adhoc tells Valohai that the code for this execution is coming from our local machine instead of our code repository. Valohai will package the local directory and upload it for an execution.

View in Valohai

  • Open your project on app.valohai.com

  • Open the latest execution

  • The details type contains information about the executions

  • The logs tab contains all logs from the execution