Evals Are the New Tests
Vishal Sachar
Co-Founder & CEO of CLRT
In traditional software you write tests. You assert that a given input produces a given output, and if it does not, the build fails. That model breaks the moment your system is an AI agent, because agents are probabilistic. Run the same task twice and you may get two different good answers. So the question changes from "did it produce exactly this" to "did it produce something that meets the goal." The thing that answers that question is an eval, and writing evals is the new core skill.
There are two kinds, and you need both. An output eval asks whether the result is correct, complete, and safe. A trajectory eval asks whether the agent got there in an acceptable way: the right tools, no forbidden actions, no lucky guess that happened to land but will not repeat. The first checks the destination. The second checks the route, because an agent that reaches a right answer by a wrong path is a failure waiting for its second run.
This forces a shift that catches teams off guard. Your specification becomes your eval criteria. A vague spec cannot be evaluated, which means it cannot be trusted, which means writing a precise specification is no longer paperwork. It is the highest-value work in the process, because the same document that tells the agent what to build also tells the eval what to check.
That is why specification quality is the new bottleneck. The agent can build quickly. The limit is how well you defined what good means.
Tests checked the code. Evals check the goal and the path to it. Write the spec well and the eval writes itself.
A deeper dive
An eval is built from a set of representative cases, each with expected properties rather than one exact expected output, since exactness is the wrong demand for a probabilistic system. Output evals score the result against those properties, and for anything subjective a separate model often acts as the grader, which is the maker-and-checker split applied to evaluation itself. Trajectory evals inspect the run: which tools were called, in what order, whether any guardrail was touched, whether the agent took an unrepeatable shortcut. In a mature setup the specification feeds both the build and the eval, the agent self-corrects against any failing case before a human sees it, and deployment gates block anything that regresses. The effect is that the bottleneck moves left, away from building and toward specifying, because the clearer the spec, the sharper the eval, and the more of the work you can safely leave running unattended.
Key terms
- Output eval
- Asks whether the result is correct, complete, and safe. It checks the destination.
- Trajectory eval
- Asks whether the agent got there acceptably, with the right tools and no forbidden actions. It checks the route.
Work with CLRT
Trustworthy agents in production are built on evals, not hope. CLRT builds the specification and evaluation layer that lets an agent run unattended without becoming a liability. Let us show you what that looks like for your work.
Vishal Sachar
Vishal Sachar is the Co-Founder and CEO of CLRT, where he helps UAE businesses make sense of applied agentic AI and put it to work. He writes on agentic systems, AI governance, and the economics of automation. Reach him at vishal@clrtstudio.com or on LinkedIn.
