Skip to main content
Click a candidate in the experiment detail to open the candidate view. This is where you read what the candidate actually changes, build confidence with deep tests, and go all the way to Publish.

​
Reading a candidate

  • Header the candidate number, the Best badge, the gain, and the [Publish] button at the top right
  • Results the score plus Improved, Broke, and Cases tested counts. A “nothing broke” badge appears when nothing regressed. The score comes from the criteria in Settings → Review & Scores and the same claim-level verification that flags hallucinations on a trace.
  • What this changes the surface tag (prompt_replace and so on), the target agent, and the full changed content. For a prompt-replacement candidate, the new prompt is shown verbatim.
  • Original before → after expand the collapsed comparison to see the original side by side.
  • Case-by-case each case is marked Kept or Still failing, with improvements and regressions distinguished. Expand a row for the case detail.

​
Deep test

When one number is not enough to publish on, re-examine the candidate through four checks.
  • Accuracy & safety Do answers improve without breaking what worked? Answers are re-verified against their evidence, the same way a trace is.
  • Cost Same quality for less money, on which model?
  • Reliability Run k times. Do answers agree?
  • Holdout Do previously-working cases stay unbroken?
Pick a check and press the [Start deep test] button. The replay is logged step by step, and when it finishes you get accuracy, reliability, and regression numbers. A “safe uplift” badge appears when the improvement holds with no regressions. The [Run again] button repeats the test, and the [Mark for re-experiment] button flags the candidate when the results don’t convince you.

​
Publish

Press the [Publish] button to apply the candidate’s changes. Published improvements are managed on the Improvements tab of Versions: an approved improvement ships as a numbered version with its diff, author, eval score, and one-click rollback (see Versions).

​
Learn more

Guarding against regressions

The three layers of regression defense before publishing.

Versions

Published improvements and rollback.