All work
  • Applied AI
  • RAG
  • Team of three

A research assistant
that checks itself.

An adaptive RAG system for computer science papers that scored 35% higher on correctness than a naive baseline.

Role
Co-developer in a team of three; built the evaluation harness
Project
Team project, PES University
Output
A local Streamlit app and an IEEE-format paper
Stack
Python, FAISS, MiniLM embeddings, Ollama, Streamlit
View the code

Python · FAISS · Ollama · Streamlit

01 · Context

Ask a question, get an answer grounded in the papers.

ADA, the Academic Document Advisor, is a local research assistant for computer science papers. It retrieves the relevant abstracts, writes an answer from them, and says so when the evidence is thin.

02 · The problem

Naive RAG treats every question the same.

  • Pain point

    A request for a summary and a request for a critique are retrieved and answered identically.

  • Pain point

    When retrieval comes back weak, the answer sounds just as confident as when it is strong.

  • Risk

    Nothing checks the finished answer against the sources it claims to rest on.

03 · How it adapts

Classify the question, then change the plan.

  1. 01ClassifyEach query is sorted into summary, comparison, methodology or critique, and scored for complexity.
  2. 02RetrieveA FAISS index over MiniLM embeddings fetches the abstracts, and the pipeline rates how good the retrieval was.
  3. 03AdaptRetrieval and generation change with the query type, its complexity and that retrieval quality.
  4. 04Self-critiqueThe answer is checked against the retrieved abstracts, and unsupported claims are flagged.
  5. 05Degrade gracefullyWhen retrieval is weak, ADA signals its uncertainty instead of bluffing.

04 · Evaluation

Scored against a naive baseline by an LLM judge.

I built an automated LLM-as-judge harness that runs the baseline and ADA over the same 100-question benchmark, across models and repeats, and scores every answer. The results went into an IEEE-format paper we co-authored.

  • 35%higher correctness score (1.24 to 1.68 out of 2)
  • 84%faithfulness, up from 63%
  • 34%better on keyword coverage

05 · What I took from it

An answer is only as good as its check.

  • Lesson

    Build the evaluation before tuning anything. Without it, every improvement is a feeling.

  • Lesson

    Letting a system say it is unsure is a feature, not a weakness.

Next case study

EK Desk: the fee desk two preschools run on

Need a developer
who ships?

Talk to me.

Open to SDE internships and 2027 full-time roles. Remote or relocation.

Download resume
contact.json
1{2  "name": "Tarun Basavaraju",3  "title": "Software engineer",4  "email": "tarunbasavaraju25@gmail.com",5  "phone": "+91 98458 59937",6  "location": "Bengaluru, India"7}