PyCharm

In this guide, we’ll look at attaching the PyCharm remote debugger to a Valohai execution.

Create a new Debug Configuration in PyCharm

The IDE host name should be localhost and the port should be the port that you want to have listening to incoming debug connections on your local machine (here 1234).

You can also now start the created Debug Server and leave it to wait for the process connection.

Python Example

Our example uses pydevd_pycharm, a Python package that enabled remote debugging with PyCharm.

Import pydevd_pycharm and set it to wait for connection with the debugger.

import pydevd_pycharm
import numpy as np
import time

# The script is halted here, until a debugger is attached
connected = False

while not connected:
    try:
        # The arbitrary container port 1234 on the worker machine is set to connect with the debugger.
        # Note that this is not the same port you specified during step 1 as that is on your local machine.
        pydevd_pycharm.settrace('localhost', port=1234, stdoutToServer=True, stderrToServer=True)
        connected = True
    except Exception:
        print("Waiting for connection...")
        time.sleep(10)

for x in range(1,20) :
    print(f"Doing computation {x}")
    data = np.random.random((50,50))
    sum = np.sum(data)
    time.sleep(2)

valohai.yaml

- step:
  name: train
  image: python:3.9
  command:
    - pip install numpy pydevd_pycharm~=<PYCHARM-VERSION>
    - python train.py

Then launch a new Valohai execution with SSH debugging enabled. Follow the Remote Access (SSH) how-to guide for detailed instructions.

Connect from PyCharm to a remote execution

You will need to establish a reverse SSH tunnel between your local machine and the Valohai worker instance. You’ll get the IP of the worker machine from the Valohai execution logs

ssh -i <PATH-TO-YOUR-PRIVATE-SSH-KEY> <IP-FROM-VALOHAI> -p 2222 -R 1234:localhost:1234 -t /bin/bash

You might see a prompt saying that the file path on the worker machine cannot be found in the project. This happens because the path mapping was not defined when creating the debugging configuration. You can just click on the “Auto detect” to set the mapping.

Finally, you should see a message in the Debugger console saying that the connection was established.

The debugger will wait for you to tell it to continue with the execution by clicking on the green arrow. When you hit a breakpoint, you’ll also be able to see your local variables and edit them on the fly.