Valohai Self-Hosted on OpenShift

This guide contains YAML templates and instructions for setting up a self-hosted Valohai installation on an OpenShift cluster. Depending on your organization’s current infrastructure, you may need to adjust these steps to fit your environment.

Note
If you have questions about custom configurations for example, specific login options or email server connections—please contact your Valohai representative.

Prerequisites

1. Existing OpenShift cluster Ensure you have a functioning OpenShift cluster to which you have administrative or sufficient privileges.
2. kubectl or oc CLI installed Install and configure kubectl or the oc client to manage resources on your OpenShift cluster.
3. Sufficient OpenShift node resources We recommend at least 4 CPUs and 16 GB of RAM on a node for running the Valohai core services (e.g., Postgres, Redis, the main Valohai pods, etc.). Adjust based on your usage and scale.

Plan Your Deployment

Valohai’s self-hosted setup comprises four core components:

• Valohai application – The main web app (roi).
• PostgreSQL – Database for Valohai metadata and records.
• Redis – Job queue and caching layer.
• Optimo – Bayesian optimization service used by Valohai.

These components typically run inside the same namespace (e.g., valohai or default). Ensure there are appropriate NetworkPolicies (if enabled) so these pods can communicate:

• Valohai ↔ Redis on port 6379
• Valohai ↔ Postgres on port 5432
• Valohai ↔ Optimo on port 80 (or your custom configuration)

Set Up Configuration

In the YAML templates (or the repository provided by Valohai), you’ll find the following ConfigMaps and Deployments that require custom values:

db-config-configmap.yaml
- POSTGRES_PASSWORD: The password for the Postgres database. Allowed characters: uppercase, lowercase letters, and numbers.

optimo-deployment.yaml
- OPTIMO_BASIC_AUTH_PASSWORD: Basic auth password for the Optimo service. Must match the one used in the Valohai ConfigMap.

roi-config-configmap.yaml
- PASSWORD in DATABASE_URL: Must match the POSTGRES_PASSWORD in db-config-configmap.yaml. - SECRET_KEY, REPO_PRIVATE_KEY_SECRET, STATS_JWT_KEY:
- Must be set to secure values (uppercase, lowercase letters, and numbers allowed). - OPTIMO_BASIC_AUTH_PASSWORD: Must match the OPTIMO_BASIC_AUTH_PASSWORD in optimo-deployment.yaml. - URL_BASE: The external URL through which users will access the Valohai web UI (e.g., https://valohai.yourdomain or http://valohai.yourdomain).

Tip
Customize any additional settings (e.g., SMTP/email details, SSO) in roi-config-configmap.yaml as needed, in consultation with your Valohai representative.

Prepare the Valohai Docker Image

Obtain the Valohai web application image
- Your Valohai contact will provide a Docker image (e.g., via a presigned S3 URL).

Push the image to your OpenShift registry (or another accessible registry)
- For instance, using OpenShift’s internal registry:

oc login --token=<your-openshift-token> --server=<openshift-api-url>
docker login -u <user> -p <token> <registry-url>
docker pull <valohai-image-from-s3>
docker tag <valohai-image-from-s3> <your-openshift-registry>/<namespace>/<valohai-image>:<tag>
docker push <your-openshift-registry>/<namespace>/<valohai-image>:<tag>

Update the valohai-deployment.yaml
- Replace <valohai-image> with your actual image name and tag. Ensure the pull secret (if needed) is properly configured on your OpenShift cluster.

Deploy the Valohai Setup

Clone or download the YAML templates from your Valohai repository (e.g., valohai-self-hosted-k8).

Create a project/namespace

oc new-project valohai
# or
oc create namespace valohai

Apply all YAML files

kubectl apply -f . -n <namespace>
# or
oc apply -f . -n <namespace>

Verify that the resources are up

oc get pods -n <namespace>
oc get deployments -n <namespace>
oc get services -n <namespace>

You should see pods for valohai, postgres, redis, and optimo running.

Create an Admin user

After the Valohai pods are running, you need an admin user to log into the web interface:

Shell into the valohai pod

oc rsh <valohai-pod-name> -n <namespace>

Run the initialization command

python manage.py roi_init --mode dev

This will create an admin account with credentials that are printed to stdout. Press Ctrl + C or type exit when done.

Expose the Valohai Web App

In OpenShift, you can use Routes to expose services externally. A quick way is via oc expose

oc expose svc/valohai -n valohai

Then, check the route

oc get routes -n valohai

You’ll see a host name generated by OpenShift. Access your Valohai WebUI at that address.

HTTPS Note By default, oc expose creates an HTTP route. For HTTPS/TLS, you’ll need to configure TLS certificates. Refer to OpenShift’s documentation on Creating a Secure Route.

Set Up the Valohai Workers

Valohai needs workers to actually run your data science workloads (e.g., training and inference jobs). You have a few options:

OpenShift/Kubernetes Workers

In order to set up the workers that will run your workloads, you can refer to the respective guides in this documentation:

• Kubernetes workers
• On-premises servers: Ubuntu installer or manual install
• Autoscaled EC2 instances

Note that regardless of the installation method, the workers need to be able to connect to the redis queue on port 6379 set up in your cluster during this installation.

As you’re running the self-hosted installation on OpenShift, it might make sense to do an OpenShift installation of workers as well. For easier installation of OpenShift workers, we recommend using ‘helm’.

Helm Install

Helm is a package manager for OpenShift and Kubernetes. It allows installing and upgrading OpenShift applications with ease.

A Helm chart is available to install Valohai workers to OpenShift clusters.

It installs the necessary service accounts, roles and other resource definitions for Valohai.

Please contact your Valohai representative to receive the required custom-values.yaml file. There are various details that can be configured so remember to voice any needs and limitations.

helm repo add valohai --force-update https://dist.valohai.com/charts/
helm upgrade --install \
    -n valohai-workers \
    --create-namespace \
    valohai-workers \
    valohai/valohai-workers \
    -f custom-values.yaml

Once the installation is complete, please supply the installer output to the Valohai team as well as connection information to your Kubernetes API (e.g., hostname, port) to complete the integration.

Set up a data store

Valohai will require an S3 compatible data store. This can be for example a MinIO running on the cluster or an S3 bucket in your AWS. Discuss with your Valohai contact which option would best fit your needs.