Ingest files to an inference job

We’ve created a training job that produces a model and saves it to Valohai inputs. Now, let’s examine how to ingest that model into another type of job (a step) that uses the model to make predictions based on the images we provide.

Define a step with inputs in valohai.yaml

Begin by defining a new step with two input groups in your valohai.yaml. Insert this step either before or after the previously defined step.

- step:
    name: inference
    image: docker.io/ultralytics/ultralytics:8.0.180-python
    command: python inference.py
    inputs:
        - name: model
          default: datum://latest-model
          filename: best.onnx
        - name: images
          default: https://ultralytics.com/images/bus.jpg

Access the input files in your script

Next, create a new Python script called inference.py and include the following code:

from ultralytics import YOLO
import os

path_to_model = "/valohai/inputs/model/best.onnx"
path_to_images = "/valohai/inputs/images/"

model = YOLO(path_to_model)

for image in os.listdir(path_to_images):
    image_path = os.path.join(path_to_images, image)

    if os.path.isfile(image_path):
        # Run prediction on the image
        # Save the result as an image in /valohai/outputs/
        results = model.predict(image_path, save=True, project="/valohai/outputs", name="predictions")

        # Print the Boxes object containing the detection bounding boxes
        for r in results:
            print(r.boxes)

Run from the command line

You can run an inference job directly from the command line:

vh execution run inference --adhoc --open-browser

On the execution details page, you’ll see an inputs section. Here, you can verify that the model originates from a previously executed Valohai job. You’ll also find a link to the image file used for prediction, accompanied by a preview icon that allows you to view the image in your browser.

After the inference job completes, you can review the results in the logs and in the outputs tab.

Rerun with another set of images

It’s easy to launch another inference job with different data. Simply upload a new image, copy your latest inference job, update the inputs with the new image, and create a new execution.

1. Upload a new image from the project’s data tab. Once uploaded, it will automatically appear in the data catalog.
2. Open the Executions tab, select your latest inference job, and click the “Copy” button in the top-right corner.
3. Scroll down to the inputs section and add the new image to the images input. You can either provide a URL or select the image from the project’s data catalog under the “Data” tab.
4. Scroll to the bottom and click “Create Execution” to start the new inference job.

You will now have two inputs and corresponding files in the outputs section.