End to end RAG pipeline with Documentation

Run a Retrieval-Augmented Generation (RAG) workflow for interacting with technical documentation on Valohai.

Overview

This example shows how to build your own RAG Doctor 👩‍⚕️:

• Build an embeddings database from documentation CSVs

• Query the database with LLMs

• Serve the RAG model as an API endpoint

Steps

1

Data Preparation

Compile CSV documentation and create an embeddings database, ensuring data is organized for efficient retrieval.

2

Embedding Database

Utilize Large Language Models (LLMs) to build and query the embeddings database effectively.

3

API Deployment

Configure and deploy the RAG model as an API endpoint to facilitate seamless interaction and integration.

GitHub Repository

The repository walks you through how to go through the above steps:

GitHub - valohai/rag-doc-example: RAG system for documentation assistant using Qdrant and OpenAI 🌀GitHub