Work with Data

Download data from cloud storage directly into your notebook environment using Valohai inputs. Configure inputs when launching your notebook, then access them in your code.

Configure Inputs

When creating a notebook execution, add inputs in the configuration panel. An input can point to a single file or multiple files using wildcards.

Input URL Formats

Public cloud storage

https://valohaidemo.blob.core.windows.net/mnist/mnist.npz

Downloads a single file from a public URL.

Private data stores

s3://mybucket/orders/may_2022.csv

Downloads a single file from your configured S3 data store. You can use also other object data stores (azure://, gs:// etc.)

Multiple files with wildcards

s3://mybucket/pytorch-sample/*

Downloads all files in the pytorch-sample folder.

Valohai downloads these files before your notebook starts. They're available immediately when you open Jupyter.

Access Files in Your Notebook

Use environment variables to locate your input files. This prevents errors from hardcoded paths.

💡 Recommended: Always use VH_INPUTS_DIR and VH_OUTPUTS_DIR environment variables. This keeps your code consistent between notebooks and executions.

Using valohai-utils

import valohai
import csv

# Open a single CSV file
with open(valohai.inputs("dataset").path()) as csv_file:
    reader = csv.reader(csv_file, delimiter=',')
    for row in reader:
        print(row)

The 🐍 valohai-utils library handles path resolution automatically.

Using environment variables

import os
import pandas as pd

# Get the inputs directory
VH_INPUTS_DIR = os.getenv('VH_INPUTS_DIR', '.inputs')

# Build the full path to your file
# Result: /valohai/inputs/dataset/may_2022.csv
path_to_file = os.path.join(VH_INPUTS_DIR, 'dataset/may_2022.csv')

# Load the data
df = pd.read_csv(path_to_file)

This approach works in both notebooks and standard executions, making it easier to migrate code later.

Working with Multiple Files

If your input uses a wildcard pattern, all matching files appear in the input directory:

import os
import valohai

# Get all files from the input
input_dir = valohai.inputs("images").path()

# List all downloaded files
for filename in os.listdir(input_dir):
    file_path = os.path.join(input_dir, filename)
    print(f"Processing {file_path}")

Next Steps

• Save Your Work — Write outputs and log metrics

• Ingest Files — Deep dive into data loading patterns