01
What the fixed graph did
The first version was a LangGraph pipeline with one path: classify the question, pick a view, write SQL, run it, format the answer. Each step ran once, in order. For a question that fitted one view, it worked. More than half of real user questions did not fit one view.
02
Where it broke
Follow-ups lost the thread."And for Germany?" started a new run with no memory of the first question.
Two views meant failure.The pipeline picked one view, so any question that needed a join failed at that step.
No answer still produced SQL.When the data held no answer, the graph still wrote a query, and the user got an empty or wrong table.
Every new question type meant a new branch.The graph grew with each exception.
03
What replaced it
A ReAct agent reasons, calls a tool, reads the result and decides the next step. It stops when it can answer, or when it can say why it cannot. The tools decide what it may do, not the prompt.
Simplified. In production the checkpointer is SQL Server, LangSmith traces every run, and guard and catalog are the parts we spent most time on.
04
Why seven views, not the raw schema
The agent never sees the source tables. Seven views carry the definitions: what counts as a reimbursement decision, which date is the decision date, how a country is named. The model chooses between seven well-described things instead of 500+ tables, most of them junction tables, and the definitions live in one place the data team owns.
05
Enrichment and fallback, in a fixed order
SQL first, documents second, the web last and always cited. The system prompt states the order and the tool descriptions repeat it. Documents are less a fallback than enrichment: about 30% of answers draw on them. The web is the true fallback, used in under 5% of answers.
06
Talking to other agents
The agent is exposed over A2A, so other agents on the platform can ask it questions and it can ask them. Agents built by other internal teams call it when they need the data it can reach.
07
What we would do the same way again
Guardrails in code.The SQL guard runs on whatever the model writes, so a bad query never reaches the database.
An evaluation set before the switch.Run side by side with the live system on the same questions: completed without an error, 91% → 100%.
Speed measured on the same questions.10% quicker than the live system.
Related use case
Enterprise AI searchAnswers from your own documents and records.