Route Prefixes – Valohai Docs

In Valohai, deployment prefixes are essential for routing requests to the right endpoints in your applications. Here’s what you need to know:

VH_DEFAULT_PREFIX Environment Variable: This variable contains the default prefix under which your application is served. It typically follows the format: organization/project/deployment/version/endpoint. It does not account for any deployment aliases you may have.
X-VH-Prefix HTTP Header: Regardless of whether your application is served using the default version moniker or an alias, the application will receive an X-VH-Prefix HTTP header. This header contains the root path under which the application is served. For example, it could look like: organization/project/deployment/versionalias/endpoint.
Path-Based Routing Consideration: If your deployment code relies on path-based routing (as opposed to other routing styles like RPC), you may need to use the VH_DEFAULT_PREFIX environment variable or the X-VH-Prefix header to ensure proper routing of requests. Alternatively, you can configure your app’s routes to accept any prefix if that suits your use case.

Understanding these deployment prefixes is crucial for handling routing effectively in your Valohai applications.

FastAPI example

For FastAPI applications, handling deployment prefixes involves using a middleware function that can rewrite the internal path based on either the VH_DEFAULT_PREFIX environment variable or the X-VH-Prefix HTTP header. Below is a middleware function that accomplishes this task:

import os
from fastapi import FastAPI
from starlette.requests import Request

app = FastAPI()

@app.middleware("http")
async def process_valohai_prefix(request: Request, call_next):
    path = request.scope["path"]
    for prefix in (
        request.headers.get("X-VH-Prefix"),
        os.environ.get("VH_DEFAULT_PREFIX"),
    ):
        if not prefix:  # Could have no header or no envvar, so skip
            continue
        if path.startswith(prefix):  # If the path starts with this prefix,
            # ... tell FastAPI (0.56.0+) that that is the prefix we're mounted under...
            request.scope["root_path"] = prefix
            # ... then strip the prefix out as far as FastAPI is concerned.
            request.scope["path"] = "/" + path[len(prefix) :].lstrip("/")
            break
    return await call_next(request)

@app.get("/predict")
def predict(name: str):
    return {"predicted_first_letter": name[:1].lower()}

You can easily integrate this middleware function into your FastAPI app. It will help you manage deployment prefixes effectively, whether they come from the VH_DEFAULT_PREFIX environment variable or the X-VH-Prefix HTTP header.

Flask example

Below you’ll find a simple Flask app and a generic WSGI middleware that rewrites the internal request path based on the prefix environment variable or headers.

import os
from functools import wraps
from flask import Flask, request, jsonify
from werkzeug import run_simple

def handle_valohai_prefix(environ):
    path = environ["PATH_INFO"]
    for prefix in (
        environ.get("HTTP_X_VH_PREFIX"),
        os.environ.get("VH_DEFAULT_PREFIX"),
    ):
        if not prefix:  # Could have no header or no envvar, so skip
            continue
        if path.startswith(prefix):  # If the path starts with this prefix,
            # ... then strip the prefix out as far as WSGI is concerned.
            environ["PATH_INFO"] = "/" + path[len(prefix) :].lstrip("/")
            break

def manage_prefixes(app):
    """
    Decorator to apply Valohai prefix management to a WSGI app callable.
    """
    @wraps(app)
    def prefix_managed_app(environ, start_response):
        handle_valohai_prefix(environ)
        return app(environ, start_response)

    return prefix_managed_app

app = Flask(__name__)

@app.route("/predict")
def predict():
    name = request.args.get("name", "")
    return jsonify({"predicted_first_letter": name[:1].lower()})

app = manage_prefixes(app)

if __name__ == "__main__":
    run_simple("0", 8000, use_reloader=True, application=app)