Evaluating a Grounded AI Assistant Before Changing Its Model

An assistant can produce a polished answer from the wrong evidence. That makes a model change difficult to assess by reading a handful of successful conversations.

For a CV assistant or an internal knowledge tool, I would define success around the information the application is allowed to use and the questions it is expected to answer.

Evaluate Retrieval and Answers Separately

Retrieval chooses evidence; generation turns that evidence into an answer. When the answer is wrong, those stages need separate inspection. Microsoft's RAG design and evaluation guidance treats preparation, retrieval, and evaluation as parts of the overall system design.

If a question asks about payroll experience but retrieval returns only frontend projects, adjusting the model's tone will not repair the missing evidence. Conversely, retrieving the correct passage does not guarantee that every generated claim follows from it.

Start With a Small, Reviewable Set

For a first evaluation of a CV assistant, I would write around thirty cases covering direct questions, paraphrases, ambiguous requests, missing facts, sensitive questions, and attempts to override the assistant's scope.

Each case would record the question, expected handling, supporting source IDs, required facts, and claims that must not appear. The expected answer would usually be a set of criteria rather than an exact sentence.

For example, “Has this person worked with ERP systems?” should identify relevant supported experience. It should not invent a customer count, revenue impact, or implementation duration to make the answer sound stronger.

Make Abstention an Expected Result

A deliberately narrow assistant should sometimes decline to answer. I would include questions about salary, unrecorded availability, and unrelated technical tasks, with an explicit expected response for each category.

I would also include legitimate questions near those boundaries. A question about remote collaboration should not automatically be treated as a request for private employment terms.

That helps reveal both unsupported answers and unnecessary refusals.

Compare One Change at a Time

My first comparison would hold the knowledge snapshot and evaluation questions fixed while changing the model integration. Any necessary request-format changes would be recorded. Prompt revisions and retrieval tuning would follow as separate experiments.

For each run, I would record the model identifier, prompt version, knowledge version, retrieved sources, answer, elapsed time, token usage, and fallback reason. Evaluation logs should use synthetic or appropriately sanitized questions where sensitive information is involved.

I would repeat difficult cases to check whether an apparently good result is consistent. Automated checks can flag excessive length or missing fields; a reviewer still needs to judge whether the claims are supported.

Set Release Criteria Before Reading the Results

For this kind of assistant, my initial gate would require no unsupported personal claims in the reviewed set, correct handling of scope cases, and acceptable response time and cost. Passing a small set is evidence for a limited rollout, not proof that every future answer will be correct.

A model upgrade earns its place when it improves the application's actual behavior. Fluency is only one part of that behavior.