Retrieval-Augmented Generation backend system answering legal queries with citations using FAISS, sentence-transformers, and Mistral-7B via Together API.
Features
Natural language legal question answering
Citation-backed responses using document chunks
FAISS vector similarity retrieval
PDF & DOCX preprocessing pipeline
Structured prompt generation with metadata
Architecture
Core System Components
Retriever module using FAISS vector search
Generator module using Together API and Mistral-7B
Document loader & chunker
App Layer
FastAPI HTTP API
Structured response formatting
Chunk metadata tracking (filename, source)
Project Structure
app
generator.pyPrompt building and LLM calls
main.pyFastAPI app entry point and /query endpoint
retriever.pyDocument retrieval from FAISS index
utils.pyPDF/DOCX loading, text chunking
data
index
Key Challenges
Building accurate RAG pipeline for legal reasoning
Maintaining citation relevance and traceability
Managing embedding & token efficiency
Key Learnings
RAG requires deep coordination between retrieval and prompt engineering
Chunking strategy dramatically affects accuracy
Metadata is crucial for real-world citation traceability