Showcase: ai-rag-enterprise-knowledge-base — A Dependency-Free RAG API with Real BM25 Retrieval

Open-source showcase · github.com/Shivay00001/ai-rag-enterprise-knowledge-base

TL;DR: ai-rag-enterprise-knowledge-base is a complete retrieval-augmented generation API in five Python files: upload .txt/.md documents, they're chunked and indexed with a from-scratch Okapi BM25 implementation (standard library only — no ML framework), and POST /ask answers questions with a real OpenAI-compatible LLM call grounded in retrieved chunks, with numbered citations. Its most distinctive feature is honesty: without an API key it returns HTTP 503 and refuses to fabricate an answer, and it surfaces real upstream errors instead of masking them. Status: working — tests pass against a real stack, Docker Compose ships it, and it's deliberately small enough to audit fully.

1. What it is

The whole system is main.py (FastAPI routes), retrieval.py (BM25 index + chunking), store.py (on-disk document store), llm.py (OpenAI-compatible HTTP client), and tests/test_rag.py — roughly 11 KB of application code plus tests. Endpoints: POST /documents/upload, GET /documents, POST /search (raw retrieval), and POST /ask (grounded Q&A with citations). A docker-compose.yml runs it on port 8001 with a named volume for the document store and a healthcheck on /health.

No embeddings, no vector database, no framework bloat: dependencies are fastapi, uvicorn[standard], httpx, python-multipart. The retrieval engine is ~90 lines of pure Python you can read and verify by hand.

2. The problem it solves

Every business sits on documents — support articles, policy PDFs, onboarding docs, contracts — that employees and customers need answers from, not links to. Naive LLM chatbots hallucinate answers from that corpus; keyword search returns documents, not answers. RAG (retrieval-augmented generation) is the standard fix: retrieve the relevant passages, then make the LLM answer strictly from them.

Most RAG demos cheat: they embed a toy dataset, hardcode answers, or — worst — silently fabricate when the retrieval misses. This repo's design centers on the failure modes instead: chunking that's transparent about its rules, a scoring function you can audit (BM25 has a formula, not a black box), and error paths that say "I don't know" instead of inventing.

3. How it works — the pipeline

1. Ingest. POST /documents/upload accepts a file, rejects anything that isn't .txt or .md (HTTP 400), rejects non-UTF-8 content (400), rejects empty documents (400). The text goes through chunk_text(text): split on blank lines into paragraphs, merge small paragraphs up to ~800 characters, hard-split paragraphs longer than the limit. Each chunk gets an id of the form <doc_id>:<n>.
2. Index. Every chunk is tokenized ([a-z0-9]+, lowercased) and added to a BM25Index — a real Okapi BM25 implementation with the canonical hyperparameters (k1=1.5, b=0.75), IDF with the standard +0.5 smoothing, and length normalization against a running average chunk length. search() scores every chunk, sorts descending, and returns the top-k with scores above zero, rounded to 4 decimals. Only hits with positive score are returned — no zero-score filler.
3. Persist. store.py keeps everything in ./data/docs.json: full text, metadata (filename, upload timestamp, char count), and chunk ids. Document ids are 12-hex-character uuid prefixes. At startup, store.rebuild_index(index) re-chunks every document deterministically and repopulates the in-memory index — so a container restart doesn't lose the corpus.
4. Ask. POST /ask retrieves the top-k chunks, formats them as numbered context blocks ([1] (doc abc123) …), and calls the LLM with a strict system prompt: "Answer ONLY from the context below. Cite every factual claim with [1], [2], … If the context does not contain the answer, say so explicitly." The response returns the answer plus a citations array — chunk id, doc id, BM25 score, and a 300-character excerpt — so every claim is traceable to source text.
5. Fail honestly. No relevant chunks → HTTP 404 ("Upload documents first."). No API key → HTTP 503 with "Refusing to fabricate an answer." Provider error (e.g. a bad key → 401, provider outage → 502): the real status and message are surfaced, never masked.

The LLM client (llm.py) is a real httpx POST to {OPENAI_BASE_URL}/chat/completions with temperature=0.2 and a 60-second timeout — and because it speaks the OpenAI-compatible protocol, it works against OpenAI, any OpenAI-compatible gateway, or a local server like Ollama/LM Studio without code changes.

4. The test suite — proof it isn't a stub

tests/test_rag.py covers the claims that matter: upload a document, then verify /search actually retrieves the right chunk (proving BM25 works end-to-end); call /ask with no key and assert the honest 503; call /ask with a dummy key and assert the real upstream 401 comes back — which proves the HTTP client genuinely hits the network instead of returning canned text. There's also CI (.github/workflows/ci.yml, python-ci.yml) so the suite runs on every push.

4b. Why "no ML framework" is a deliberate engineering choice

It's worth pausing on what this repo doesn't depend on, because it's the core of its design philosophy. There's no sentence-transformers, no FAISS, no PyTorch — the retrieval engine is ~90 lines of standard-library Python implementing a published, peer-reviewed ranking function (Okapi BM25, Robertson & Zaragoza). That buys three things most RAG demos can't claim:

• Installability: pip install -r requirements.txt completes in seconds with four small packages. No 2 GB torch wheel, no CUDA questions, no version-hell.
• Debuggability: when a query returns the wrong chunk, you can compute the BM25 score by hand — term frequency, document frequency, average length are all inspectable. Try doing that with a 768-dimensional embedding.
• Determinism: the same corpus and query always produce the same ranking. Embedding models change behavior across versions; BM25 doesn't.

The honest tradeoff is recall: keyword matching misses paraphrases and synonyms. The repo's answer is architectural rather than defensive — keep BM25 as the transparent, always-correct baseline, and add embeddings as a second signal later (hybrid search) rather than replacing the auditable path. For internal knowledge bases where "why did it return this?" is a compliance question, that's the right order.

# Local
pip install -r requirements.txt
uvicorn main:app --port 8001

# With a key (OpenAI, or any OpenAI-compatible endpoint)
OPENAI_API_KEY=sk-... uvicorn main:app --port 8001

# Or Docker Compose (persists ./data, healthchecks /health)
docker compose up

curl -F "[email protected]" http://localhost:8001/documents/upload
curl -X POST http://localhost:8001/search \
  -H 'Content-Type: application/json' -d '{"query":"refund window","top_k":3}'
curl -X POST http://localhost:8001/ask \
  -H 'Content-Type: application/json' -d '{"question":"What is the refund window?"}'