A support copilot that cites its answers

Support answered the same questions every day while the documentation that contained the answers went unread. An earlier chatbot experiment had been switched off after it invented a refund policy, so the bar for a second attempt was high: no answer was better than a confident wrong one.

We built retrieval first: hybrid keyword and vector search over structurally chunked documentation, then a reranking pass. Every generated sentence must carry a chunk id, and a verification step drops any answer whose citations do not resolve, falling back to the source passages and a handover to a human.

An evaluation set of real support questions runs on every prompt or model change, and support staff rate answers in place, which gives the team a labelled dataset and an early warning when quality drifts.