Quick Start - Jupyter Notebooks (MNIST)¶
In this tutorial, we will use the Valohai Jupyter Notebook Extension to build a machine learning model using the MNIST dataset.
You can also
Installation¶
Install Valohai’s Jupyhai package to run notebooks from your local machine on Valohai.
pip install jupyhai
jupyhai install
Start the notebook server¶
Navigate to the folder where your notebooks are located and start notebooks.
jupyter notebook
Run your own code and notebooks
- This mounts your current working directory into the container.
- All the files in your current working directory will be available within the container
- All changes within the mounted folder will persist after shutting down
- You can run install additional libraries to the environment by adding a new cell at the top of your notebook and running
!pip install mylibrary
Sign in¶
Press the Valohai button in the toolbar and login using your Valohai credentials.
Select the following:
- Project: Valohai project where the executions will be version controlled
- Environment: Environment type for the cloud executions (E.g. AWS p2.xlarge)
- Docker Image: Docker image that provides the required libraries (E.g. TensorFlow)
- Use valohai/pypermill as the base when building your custom Docker image to be used as a part of Valohai Notebook executions.
These are the same settings you would choose when using Valohai website, CLI or valohai.yaml.
Once you are happy with your selections. Press save.
Create an execution & watch the results¶
Press the Run remote button on the toolbar
The gizmo for the new execution will appear to the right.
You can click the #1 > Notebook button to download the finished notebook back to your local machine.
Open a Notebook from a previous execution
Each of the colored gizmos on the right side of the page signify a single Valohai execution. You can click on any of the completed executions and select Notebook to launch the Notebook version that was used to run the execution.
Parameterize the notebook¶
Parameterizing a notebook happens through cell tags. Tags are a standard Jupyter feature.
Here we mark the first cell with a parameters tag, which means all variables are considered to be
Valohai parameters, just like you would define in the valohai.yaml.
Download the inputs¶
Here we marked the first cell with inputs tag and ran it in Valohai.
All the variables in this cell will be considered as Valohai input URIs for the execution, just like in the valohai.yaml.
Reusing the parameterized notebook¶
Now you can run notebook based experiments without a notebook!
Because the learning_rate is parameterized, you can change it via Valohai web interface and run
additional experiments without even opening the notebook.
FAQ¶
When I try to download my outputs back from a finished execution, I get 404: Not Found
Always use the notebook server through http://127.0.0.1:8888 instead of http://localhost:8888.