Launch pipelines with webhooks – Valohai Docs

Quickstart Example

Add the following steps and pipeline to your valohai.yaml configuration file, commit the changes, push them to Git, and fetch the new commit to Valohai.

Add the following code to a file called webhook_entry.py:

import json
import valohai

from urllib.parse import parse_qs

valohai.prepare(
    step="Webhook Entry",
)

with open(valohai.inputs('webhook-payload').path()) as f:
    post_parameters = {k: v[0] for k, v in parse_qs(f.read()).items()}

print("Received these parameters from the webhook:")
for k, v in post_parameters.items():
    print(f"{k}: {v}")
if "prompt" in post_parameters:
    # Submit metadata
    print(json.dumps({'parsed_prompt': post_parameters['prompt']}))

Add the following code to a file called webhook_followup.py:

import valohai


valohai.prepare(
    step="Webhook Followup",
)

prompt = valohai.parameters("prompt").value
print("Received prompt:", prompt)
print("---")
print("This is just a demonstration to pass values around in a pipeline, so not doing anything useful with it.")

Add the following steps and pipeline definitions to your project’s valohai.yaml file:

- step:
    name: Webhook Entry
    image: python:3.11
    command:
    - pip install valohai-utils
    - python ./webhook_entry.py
    inputs:
    - name: webhook-payload
      optional: false
- step:
    name: Webhook Followup
    image: python:3.11
    command:
    - pip install valohai-utils
    - python ./webhook_followup.py {parameters}
    parameters:
    - name: prompt
      optional: false
      type: string

- pipeline:
    name: Webhook Pipeline
    edges:
    - configuration: {}
      source: entry.metadata.parsed_prompt
      target: followup.parameter.prompt
    nodes:
    - name: entry
      on-error: stop-all
      step: Webhook Entry
      type: execution
    - name: followup
      on-error: stop-all
      step: Webhook Followup
      type: execution

Generate a new secret and store it securely; you’ll need it when configuring and using the webhook. A passphrase or a random string of characters is fine; your password manager can do this for you. This is referred to as the Webhook Secret.

We can now define the webhook trigger. Go to Project settings -> open Triggers tab -> create New Trigger

For Trigger Type, select Webhook
Using the condition column’s + Add menu, add a Web Request Authentication condition with the following fields:
1. Auth Type: select “Static Secret Token”
2. Token Lookup Namespace: select “HTTP Header”
3. Token Lookup Key: type “Authorization”
4. Secret Key: type the Webhook Secret.
5. Value Prefix: type secret=
Using the action column’s + Add menu, add a Run Pipeline action:
1. Source Commit Reference: type the primary branch of your Valohai project in Git. This is usually either main or master
2. Pipeline Name: type “Webhook Pipeline”
3. Payload Input Name: type entry.webhook-payload.
Click on Save Trigger
You are now on the Triggers list again. Open the ... menu of the trigger we just created, and select Edit to return to editing the trigger.

As the trigger has now been created, the permanent webhook URL is displayed below Webhook Type:

Webhook URL

Click on the copy icon. We’ll paste it in a bit.

We can now launch the pipeline with a specially crafted Web request. Let’s use curl, but any custom request creator will do.

curl -X POST <paste the webhook URL here> -H "Authorization: secret=<paste the Webhook Secret here>" -d "hello=valohai&prompt=draw+me+a+shark"

You can also launch webhooks with Python, such as from inside your executions. Using urllib (no install required):

import urllib.request
import urllib.parse

urllib.request.urlopen(urllib.request.Request(
   '<paste the webhook URL here>', 
   data=urllib.parse.urlencode({
      'hello': 'valohai',
      'prompt': 'draw me a shark'
   }).encode(), 
   headers={
      'Authorization': 'secret=<paste the Webhook Secret here>'
   }
))

Requests?

There are more examples in the next chapter, Webhooks triggers reference, including with Requests

We can now see a new pipeline being launched in the project! Let’s check the second execution’s logs.

Execution output, entry

Execution output, followup

Webhooks can be used to connect events from various online services directly to Valohai without needing to run any middleware or custom services. You can either go from here to develop your own custom integrations, or read our guides below for detailed instructions for a number of services.

Quickstart Example

Further reading