- Applied AI
- RAG
- Team of three
A research assistant
that checks itself.
An adaptive RAG system for computer science papers that scored 35% higher on correctness than a naive baseline.
- Role
- Co-developer in a team of three; built the evaluation harness
- Project
- Team project, PES University
- Output
- A local Streamlit app and an IEEE-format paper
- Stack
- Python, FAISS, MiniLM embeddings, Ollama, Streamlit
Python · FAISS · Ollama · Streamlit
01 · Context
Ask a question, get an answer grounded in the papers.
ADA, the Academic Document Advisor, is a local research assistant for computer science papers. It retrieves the relevant abstracts, writes an answer from them, and says so when the evidence is thin.
02 · The problem
Naive RAG treats every question the same.
- Pain point
A request for a summary and a request for a critique are retrieved and answered identically.
- Pain point
When retrieval comes back weak, the answer sounds just as confident as when it is strong.
- Risk
Nothing checks the finished answer against the sources it claims to rest on.
03 · How it adapts
Classify the question, then change the plan.
- 01ClassifyEach query is sorted into summary, comparison, methodology or critique, and scored for complexity.
- 02RetrieveA FAISS index over MiniLM embeddings fetches the abstracts, and the pipeline rates how good the retrieval was.
- 03AdaptRetrieval and generation change with the query type, its complexity and that retrieval quality.
- 04Self-critiqueThe answer is checked against the retrieved abstracts, and unsupported claims are flagged.
- 05Degrade gracefullyWhen retrieval is weak, ADA signals its uncertainty instead of bluffing.
04 · Evaluation
Scored against a naive baseline by an LLM judge.
I built an automated LLM-as-judge harness that runs the baseline and ADA over the same 100-question benchmark, across models and repeats, and scores every answer. The results went into an IEEE-format paper we co-authored.
- 35%higher correctness score (1.24 to 1.68 out of 2)
- 84%faithfulness, up from 63%
- 34%better on keyword coverage
05 · What I took from it
An answer is only as good as its check.
- Lesson
Build the evaluation before tuning anything. Without it, every improvement is a feeling.
- Lesson
Letting a system say it is unsure is a feature, not a weakness.
Next case study
EK Desk: the fee desk two preschools run on
Need a developer
who ships?
Talk to me.
Open to SDE internships and 2027 full-time roles. Remote or relocation.
Download resume1{2 "name": "Tarun Basavaraju",3 "title": "Software engineer",4 "email": "tarunbasavaraju25@gmail.com",5 "phone": "+91 98458 59937",6 "location": "Bengaluru, India"7}