Datum Properties – Valohai Docs

Datum Properties help you save additional information with each file you save. Properties can be in JSONFormat and can be used to store any information you want to associate with your data.

Add Properties

Programmatically

How to use the Valohai APIs

If you’re new to using the Valohai APIs, you find more information here: Make calls to the Valohai API

You can save properties to your output datums by saving an additional JSON file *.metadata.json for each of your execution’s output files.

import valohai
import json

metadata = {
    "valohai.tags": ["production", "lemonade"]
    "meta1": "value1",
    "meta2": "value2"
    "sigma": 0.1
}

save_path = '/valohai/outputs/model.h5'
model.save(save_path)

metadata_path = '/valohai/outputs/model.h5.metadata.json'
with open(metadata_path, 'w') as outfile:
    json.dump(metadata, outfile)

You can save properties to the datums by using the Valohai API. There are 3 different endpoints available to do that
/api/v0/data/${id}/metadata/ (to apply a single properties object to a single datum)

import os
import valohai
import json
import requests
properties = {
    "chocolate": "bar",
    "coffee": 2,
    "lemonade": "bar",
    "sheeps": ["fluffy", "white"],
    "valohai.tags": ["tag1", "tag2"],
    "valohai.alias": "datum-alias-custom",
}

datum_id = "datum-id-1" # Put the valid datum id here
response = requests.post(
    f'https://app.valohai.com/api/v0/data/{datum_id}/metadata/',
    json=properties,
    headers={
          'Authorization': 'Token ' + os.getenv('VH_TOKEN'),
          'Content-Type': 'application/json'
        }
)
print(response.status_code)

/api/v0/data/metadata/apply/; (to apply each properties object to each datum provided in payload)

import os
import valohai
import json
import requests


# Replace <datum-id-*> with valid datum ids 

payload = {
    "datum_metadata": {
        "<datum-id-1>":{ 
            "metaC": 123456789
        },
        "<datum-id-2>":{
            "metaC": 23456789,
            "valohai.tags": [
                "tag1",
                "tag2"
            ],
            "valohai.alias": "production-alias"
        },
        "<datum-id-3>":{
            "metaC": 3,
            "valohai.tags": [
                "tag3",
                "tag4"
            ],
            "valohai.alias": "development-alias"
        }
    }
}


response = requests.post(
    f'https://app.valohai.com/api/v0/data/metadata/apply/',
    json=payload,
    headers={
          'Authorization': 'Token ' + os.getenv('VH_TOKEN'),
          'Content-Type': 'application/json'
        }
)
print(response.status_code)

/api/v0/data/metadata/apply-all/ (to apply single properties objects to all datums provided in payload)

import os
import valohai
import json
import requests
payload = {
    "metadata": {
        "resolution": "100",
        "snake": 2,
        "valohai.tags": ["a","b","c","d"]
    },
    "datum_ids": [
        "<datum-id-1>", # Put the valid datum ids here
        "<datum-id-2>",
        "<datum-id-3>"
    ]
}

response = requests.post(
    f'https://app.valohai.com/api/v0/data/metadata/apply-all/',
    json=payload,
    headers={
          'Authorization': 'Token ' + os.getenv('VH_TOKEN'),
          'Content-Type': 'application/json'
        }
)
print(response.status_code)

To remove/update properties keys of a datum

Programmatically

If you want to remove any key from datum properties you can send it as None in the payload. If you want to update any key you can send the new value in the payload.

import os
import valohai
import json
import requests
properties = {
    "chocolate": None, # This will remove the key chocolate from the datum properties
    "coffee": None, # This will remove the key coffee from the datum properties
    "lemonade": "new_bar" # This will update the value of lemonade key to new_bar
}

datum_id = "datum-id-1" # Put the valid datum id here
response = requests.post(
    f'https://app.valohai.com/api/v0/data/{datum_id}/metadata/',
    json=properties,
    headers={
          'Authorization': 'Token ' + os.getenv('VH_TOKEN'),
          'Content-Type': 'application/json'
        }
)
print(response.status_code)

See and find Properties

Web Application

You can see all the attached properties to a datum in the Valohai web UI by clicking on the datum in Project Data tab. All existing key-value pairs are displayed there. Additionally, you can also search through them to see if some specific property exists or not. The full key-value pairs can be seen by hovering over the properties.

Find datum properties