Implement on-prem RAG with Qdrant and Ollama for a self-hosted KB

Try It This n8n template provides a self hosted RAG implementation.

How it works Provides one workflow to maintain the knowledge base and another one to query the knowledge base. Uploaded documents are saved into the Qdrant vector store. When a query is made, the most relevant documents are retrieved from the vector store and sent to the LLM as context for generating a response.

How to use Start the workflow by clicking Execute workflow Use the file upload form to upload a document into the knowledge base (Qdrant db). Click Open chat to start asking questions related to the uploaded documents.

Setup steps Below steps show how to setup on Amazon Linux. Consult your OS for respective steps

Install Ollama on prem mkdir ollama cd ollama curl -fsSL https://ollama.com/install.sh | sh ollama --version Install required models ( in Amazon Linux)

ollama pull llama3:8b ollama pull mistral:7b ollama pull nomic-embed-text:latest Access ollama via http://localhost:11434 Fire up Qdrant (e.g. via docker) docker run -p 6333:6333 qdrant/qdrant. Access Qdrant via http://localhost:6333/dashboard Create a Qdrant collection named knowledge-base configured with vector length of 768. NB: Do not forget a persistent docker volume for Qdrant if you want to keep the data when using docker. Point the nodes to the respective on premise Qdrant and Ollama runtimes.

Need Help? Join the Discord or ask in the Forum!

Happy RAGing!