Question-Answer Chatbot with Self-hosted LLMs & RAG

• Setup the complete infrastructure stack for a Question-Answer chatbot for your private data in just a few minutes!
• Your stack will be powered by Self-hosted Open-Source Large Language Models and Retrieval Augmented Generation running on Kubernetes Cloud clusters.

Overview

The Question-Answer Chatbot is powered by these technologies:

1. Open-Source Large Language Models
2. Retrieval Augmented Generation (RAG)
3. Vector Stores
4. Ray AI/ML compute framework
5. Elotl Luna

Retrieval Augmented Generation

The graphic below shows how RAG is used to determine an answer to the end-user's question about a specific knowledge base.

Installation

• Cluster Setup Summary
• Install Infrastructure Tools
• Install Model Serve Stack
• Model Serving
• Retrieval Augmented Generation using FAISS
• Creation of the Vector Store
• Install the RAG & LLM querying service
• Send a question to your LLM with RAG
• Query your LLM with RAG using a Chat UI
• Uninstall

Jump to complete install doc available here.