Each Valohai execution will contain the following environment variables, giving you more context about the execution your job is being executed in.
# info about the execution itself VH_CONFIG_DIR=/valohai/config # downloaded files will be saved here VH_INPUTS_DIR=/valohai/inputs # save files to be uploaded here VH_OUTPUTS_DIR=/valohai/outputs # your git repository code, also the working directory VH_REPOSITORY_DIR=/valohai/repository # the following will change between each execution; it includes the execution UUID VH_JOB_ID=exec-016eb6ec-50cb-0031-3f48-d556e47b1c78 # the execution ID VH_EXECUTION_ID=016eb6ec-50cb-0031-3f48-d556e47b1c78 # the project ID VH_PROJECT_ID=04a37c09-dbe1-4c01-b715-0a3223c50188 # the task ID (if any) VH_TASK_ID=f9c97759-513e-44a1-9666-97cf198cde80 # the pipeline ID (if any) VH_PIPELINE_ID=f403603b-ad11-4cc4-a90d-3118f51c8dcd # the pipeline node ID (if any) VH_PIPELINE_NODE_ID=972834e2-23b5-429a-9f6d-80b8c4a75c8a
Special environment variables
There are a handful of special environment variables that affect the behavior of the Valohai execution agent.
Setting this to a true value (1, yes, true) will have the execution agent ignore any pre-existing cached data. This is useful if you have reused the same URL for a given datum (which, for reproducibility reasons, you should not generally do). The datum will still be written to the agent machine’s local cache.
Setting this to a true value (1, yes, true) will have the execution agent ignore existing Docker images and force pulling them anew from the source.
Setting this to a true value (1, yes, true) will have the execution agent forcibly clean all Docker images and cached data off the agent machine before and after the execution. This will take additional time.
Setting this to a false value (0, no, false) will have the execution write all /tmp operations on the disk instead of the memory. A standard Valohai execution will say in the logs “/tmp is a memory filesystem” and write to memory when you write to /tmp. This might cause out of memory errors in some cases. Note: If you change the default behaviour and /tmp is written to disk, both reading and writing operations will be slower.
Will increase the size of the shared memory directory of the container. For example, if you’d like to increase the SHM size to 16GB then you can use 16G, 16 GB, 16 gb, or 16GB as the value
For private environments where Google Cloud TPU is enabled, this environment variable will contain the GRPC endpoint(s) of the allocated Cloud TPU(s), separated by spaces. This may be passed in as the argument for TPUClusterResolver, for example:
When TPUs are not enabled, this variable will not be set.