Getting started

Note

This tutorial is a part of our Valohai fundamentals series.

In this section you will learn how to

  • Import Valohai in your Python scripts

  • Save Valohai outputs

  • Run executions in Valohai

A sample training script

  • Download this mnist.npz file to your working directory

  • Create a new file valohai-quickstart/train.py and paste the following example script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import numpy as np
import tensorflow as tf

input_path = 'mnist.npz'
with np.load(input_path, allow_pickle=True) as f:
    x_train, y_train = f['x_train'], f['y_train']
    x_test, y_test = f['x_test'], f['y_test']

x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation='softmax')
])

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer='adam',
              loss=loss_fn,
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)

model.evaluate(x_test,  y_test, verbose=2)

output_path = 'model.h5'
model.save(output_path)

Create a configuration file

Add a new file called valohai.yaml to define a new Valohai step called train. Whenever we launch this step in Valohai we want to run the train.py script inside a TensorFlow 2.6.0 Docker image.

- step:
    name: train-model
    command: python train.py
    image: tensorflow/tensorflow:2.6.0

Run in Valohai

Finally, we can start a new Valohai job directly from your command-line.

vh exec run train-model --adhoc
  • vh exec run <step-name> creates a new Valohai execution with the step that we defined.

  • --adhoc tells Valohai that the code for this execution is coming from our local machine instead of our code repository. Valohai will package the local directory and upload it for an execution.

View in Valohai

  • Open your project on app.valohai.com

  • Open the latest execution

  • The details type contains information about the executions

  • The logs tab contains all logs from the execution