Getting started

Note

This tutorial is a part of our Valohai Fundamentals series.

In this section you will learn how to

  • Import Valohai in your Python scripts

  • Save Valohai outputs

  • Run executions in Valohai

A sample training script

  • Download this mnist.npz file to your working directory

  • Create a new file valohai-quickstart/train.py and paste the following example script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import numpy as np
import tensorflow as tf

input_path = 'mnist.npz'
with np.load(input_path, allow_pickle=True) as f:
    x_train, y_train = f['x_train'], f['y_train']
    x_test, y_test = f['x_test'], f['y_test']

x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation='softmax')
])

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
model.compile(optimizer='adam',
              loss=loss_fn,
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)

model.evaluate(x_test,  y_test, verbose=2)

output_path = 'model.h5'
model.save(output_path)

Define a Valohai configuration file

Now we’ll create a valohai.yaml - Config File for your project. Run the following commands from your command line:

# Initialize a new Valohai project in this directory
vh init

# Confirm the directory (y)

# valohai-cli will look for any Python files in your directory
# it will find our train.py and
# ask if that's the script file you want to use as the command

# Choose train.py by typing 1
# Confirm selection (y)

# Choose a Docker image from the list, or input your own image
# Choose tensorflow/tensorflow:2.1.0-py3 by typing 12

# You'll see a preview of the valohai.yaml configuration file
# accept it (y) to generate the file

Now edit the file and change the name of the step to train-model:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
---

- step:
    name: train-model
    image: tensorflow/tensorflow:2.1.0-py3
    command: python train.py
    #inputs:
    #  - name: example-input
    #    default: https://example.com/
    #parameters:
    # - name: example
    #   description: Example parameter
    #   type: integer
    #   default: 300

Run in Valohai

Finally, we run the following commands in the the command line, on your own computer:

vh exec run train-model --adhoc
  • vh exec run <step-name> creates a new Valohai execution with the step that we defined.

  • --adhoc tells Valohai that the code for this execution is coming from our local machine instead of our code repository. Valohai will package the local directory and upload it for an execution.

View in Valohai

  • Open your project on app.valohai.com

  • Open the latest execution

  • The details type contains information about the executions

  • The logs tab contains all logs from the execution