Save additional context

You can save additional information with each file you saved. The structure of the information is purely up for you to decide.

For example, if you’re reading images from a certain factory, you could save information about the factory, date and the conditions around the images.

To save additional information with each file, you’ll need to attach extra metadata to your output files by saving a .metadata.json file alongside your data.

Store the metadata in JSON format within a *.metadata.json file. Valohai will associate this metadata with the respective output file.

Example

import valohai
import json

metadata = {
    "valohai.tags": ["prod", "lemonade"], # creates Valohai tags for the file
    "valohai.alias": "model-prod", # creates or updates a Valohai data alias to point to this output file
    "factory": "eu-02",
    "product": "katti"
}

save_path = '/valohai/outputs/model.pkl'
model.save(save_path)

metadata_path = '/valohai/outputs/model.pkl.metadata.json'
with open(metadata_path, 'w') as outfile:
    json.dump(metadata, outfile)

Read data

You can access the metadata that you’ve attached to a file either through the Valohai API or during execution.

Any metadata created with datums is available during runtime under the /valohai/config/inputs.json file.

import json

with open('/valohai/config/inputs.json') as json_file:
    vh_inputs_config = json.load(json_file)

# Print metadata from each file that is in the input named "myinput"
for data in vh_inputs_config['myinput']['files']:
    print(data["metadata"])