Why
Law firms have years of useful case files they cannot send to a cloud model. Privacy was not a setting; it decided the architecture.
What
I handled backend logic and integration as part of a team. The prototype ingests a firm’s files, finds the relevant passages, and grounds a local model’s answer in them.
How
Python chunks the documents, ChromaDB stores embeddings, and Ollama runs LLaMA locally. Streamlit shows the answer and its source context; Docker packages the private deployment.
Move the whole pipeline inside
Ingestion, embeddings, retrieval, and generation all stay inside the same controlled boundary.
Separate ingestion from answering
New files can enter the knowledge base without retraining the model.
Show what the model used
Retrieved context stays visible, and RAG can be switched off to check what the model actually knows.