How to open a shell to your Valohai Kubernetes execution

How to open a shell to your Valohai Kubernetes execution

Pre-requisites

You need access to the Kubernetes cluster itself to open a debug shell.

1. You can reach the cluster’s control plane using kubectl from your local machine. This can mean for example a client certificate or a service account with an access token.
2. Your Kubernetes cluster user has sufficient Roles assigned to get pods and create pods/exec resources in the valohai-workers namespace
   • Example Role definition:

apiVersion: rbac.authorization.k8s.io/v1 
kind: Role 
metadata: 
  namespace: valohai-workers
  name: debug-executions 
rules:
  - apiGroups: [“”] 
    resources: [“pods”] 
    verbs: [“get”]
  - apiGroups: [“”] 
    resources: [“pods/exec”] 
    verbs: [“create”]

Steps

As long as the pre-requisites remain fulfilled, you can debug any execution in that environment.

1. Start an execution using the Kubernetes cluster environment
2. Read (and copy) the unique execution pod name from the execution log:
   • 
   • The container name should be workload and the namespace should be valohai-workers. If they are not, change them in the command below.
3. Open shell: kubectl exec -it {pod} -c workload --namespace valohai-workers -- bash
   • Substitute the unique pod name copied above in place of {pod}