We’ve created a training job that generates a model and saves it to Valohai inputs. Let’s next look how we can ingest that model into another type of a job (=step) that’ll use the model to provide predictions based on an image that we pass in.
Define a step with inputs in valohai.yaml
Start by defining a new step with two input groups in your
valohai.yaml. Add this before or after the first step we’ve defined.
- step: name: inference image: docker.io/ultralytics/ultralytics:8.0.180-python command: python inference.py inputs: - name: model default: datum://latest-model filename: best.onnx - name: images default: https://ultralytics.com/images/bus.jpg
Access the input files in your script
Now create a new Python script called
inference.py and place in it the following:
from ultralytics import YOLO import os path_to_model = "/valohai/inputs/model/best.onnx" path_to_images = "/valohai/inputs/images/" model = YOLO(path_to_model) for image in os.listdir(path_to_images): image_path = os.path.join(path_to_images, image) if os.path.isfile(image_path): # Run prediction on image # Save result as an image in /valohai/outputs/ results = model.predict(image_path, save=True, project="/valohai/outputs", name="predictions") # Print the Boxes object containing the detection bounding boxes for r in results: print(r.boxes)
Run from the command-line
You can run an inference job from the command-line
vh execution run inference --adhoc --open-browser
On the execution details page, you’ll see an input section. Here, you can observe that the model we’re utilizing originates from a previously executed Valohai job. You’ll also find a link to the image file that we’re predicting with. Next to the link, there’s a preview icon that enables you to view the image directly in the browser.
After the inference job has finished, you can inspect the results within the log tab and in the outputs tab.
Rerun with another set of images
Easily run another inference job with different data by uploading a new image, copying your latest inference job, adding the new image to the inputs, and creating a new execution.
- Begin by uploading a new image from the project’s data tab. Once uploaded, it will be automatically added to the data catalog.
- Access your most recent inference job from the user interface via the Executions tab. When you have it open click the “Copy” button in the top right corner.
- Scroll down to the inputs section and add a new image to the images input. You can provide a URL or click on the “Data” tab to select the image you just uploaded to the project’s data catalog.
- Scroll to the bottom and click “Create Execution” to initiate the new inference job.
Now you’ll find two inputs, and two files in the outputs section.