Kubernetes Workers on OVH Cluster

Setting up your Valohai infrastructure with OVHcloud might seem a bit daunting, but don’t worry—we’ve got you covered. This guide will walk you through creating and configuring a private network, a Redis instance, and a Kubernetes cluster. By the end, you’ll have Valohai and OVHcloud working together like a charm, ready to handle your data science projects with ease.

Create Private Network

1. Go to Private network (vRack) and click on Add Private Network.
2. On step 1, choose the region.
3. During step 2 (Configure your private network), choose Create a Gateway and connect to the private network.

After creating the private network, ensure it was set up correctly by clicking on Gateway in the left-hand navigation bar to verify the gateway has been created.

More details on private network are available in OVH docs.

Create Redis

1. Log in to your OVHcloud Control Panel and open your Public Cloud project.
2. Click on Databases in the left-hand navigation bar and select Caching service (Redis instance).
3. Make sure your database is public.
4. After the service is created, create a user with these flags:

Commands:

+exists +setnx +blpop +lpush +llen +lrange +get +rpush +lset +info +config +hsetnx +expire +persist +multi +lrem +hget +ping +evalsha +exec +set +auth +del +client +hexists +hset +script

For Keys and Channels set * - use allkeys and allchannels

5. Do not forget to save the password of the user.

Lastly, go to Authorised API and allow app.valohai.com to connect to the Redis with this IP: 34.248.245.191/32.

More details on Redis are available in OVH docs.

Create Kubernetes Cluster

1. Go to OVH Kubernetes cluster and press Create Kubernetes cluster.
2. On step 1, choose the region. Make sure the region of private network created before and this cluster is the same.
3. On step 2, get the recommended version of Kubernetes.
4. On step 3, choose the private network created in step 1. Also, make sure you enable OVHcloud Internet Gateway.
5. On step 4, choose the node type that you require.
6. On step 5, when configuring the node pool size, enable Autoscaling, choose the minimum to 1 and the maximum to a value that you require.
7. Proceed with the name of the Kubernetes Cluster.

More details on Kubernetes cluster are available in OVH docs.

Install Valohai Workers on Your Kubernetes Cluster

After the cluster has been created, wait until your node pool status is OK and autoscaling is enabled.

1. Go to the Service tab of your cluster and download the kubeconfig file.
2. In your terminal, run:

export KUBECONFIG=<path-to-file>/kubeconfig.yml

Install Valohai workers using Helm on your cluster by running:

helm repo add valohai --force-update https://dist.valohai.com/charts/
helm upgrade --install \
    -n valohai-workers \
    --create-namespace \
    valohai-workers \
    valohai/valohai-workers \
    -f custom-values.yaml

More details on Valohai Kubernetes workers are available in Valohai docs.

Update Redis Authorized IP

1. Go back to the Gateway created earlier, copy the public IP of the gateway.
2. Add the Getaway IP as another Authorized IP in your Redis settings.

This ensures the Kubernetes cluster can communicate with Redis.

Final Steps

After installation, securely send the following information to Valohai support:

• The URI that OVH generates for your Redis in the login information.

• The redis user credentials.

• The kubeconfig.yml file generated by OVH when creating Kubernetes Cluster.

After the environment is set up by Valohai team, you will be able to create executions using OVH Kubernetes cluster.