Environment Variables

Environment variables are dynamic key-value pairs that running processes can write and read. This also holds true in the context of Valohai.

You can define environment variables in three ways:

• Simply write export MY_KEY=”my-value” in your commands. This is the most flexible way but won’t be tracked or easily searchable in Valohai.
• Define expected environment variables in the valohai.yaml. You can define a default value that can be overwritten by the execution creator.
• Define environment variables in the project settings. Any execution in the project can optionally be instructed to inherit the project environment variables. Project environment variables can be set to secret so they are not shown anywhere on the UI.
• Define environment variable groups on the organization level, so they can be easily accessed by multiple projects.

Execution environment variables

Environment variables in step commands

You can define your environment variables in the commands like this:

- step:
    name: train-model
    image: python:3.6
    command:
    - export MODE="1"
    - export POST="clip"
    - python train.py

Environment variables in valohai.yaml

A better way is to explicitly define what environmental variables the execution expects:

- step:
    name: train-model
    image: python:3.6
    command: python train.py
    environment-variables:
      - name: MODE
        default: "1"
      - name: POST
        default: "clip"

A variable in environment-variables has four potential properties:

• name: The environment variable name how it will be passed to the execution.
• description: (optional) More detailed human-readable description of the variable.
• default: (optional) Default value for the environment variable.
• optional: (optional) Whether this environment variable is optional. All environment variables are optional by default so only optional: false would make sense.

This way the environment variables can easily be tracked and edited when creating executions either through the graphical user interface, command-line client or API.

Project environment variables

Project level environment variables are excellent for private keys when doing more advanced data source integrations; or when you have some project-wide settings you want to share between executions.

1. Go to your project settings
2. Open the “Environment Variables” tab.
3. Enter the environment variable name and value in their respective fields, then save them with the “Add” button.
4. To keep a value secret, check the “Secret” checkbox.
5. Save the changes made to the group by clicking on the “Save” icon

Make sure that the [X] Inherit project’s environment variables and secrets checkbox is ticked when creating the execution.

Organization environment variables

Administrative members have the capability to define organization-level environment variables. These variables can be grouped and associated with various projects. This streamlines the management of shared environment variables across multiple projects, eliminating the need for redundant definitions.

This centralized approach simplifies tasks like key rotation, as the information is stored in one location for all relevant projects, improving efficiency and security.

Creating Environment Variable Groups

1. Click on “Hi, <username>!” in the top-right corner.
2. Select “Manage <organization>.”
3. Open the “Environment Variables” tab.
4. Name your environment variable group and create it by clicking “Create new environment variable group.”
5. Choose the projects that should have access to this group.
6. Enter the environment variable name and value in their respective fields, then save them with the “Add” button.
7. To keep a value secret, check the “Secret” checkbox.
8. Save the changes made to the group by clicking “Save”

Example

In the example below, there are two environment variable groups: “production” and “staging.”

• “Production” is used in projects “fun-tensorflow” and “production.”
• “Staging” is used exclusively in the “staging” project.

Both groups contain two environment variables, with one marked as secret and another having a public value.

Using Environment Variable Groups

The environment variable groups you’ve created will be visible under the project’s “Env Variables” tab, as shown below. Non-admin users can view them but cannot modify their values. Values for secret environment variables are hidden in the user interface.