Sonata LabsLong-horizon evals
for frontier models

Real-world measures of alignment

Sonata Labs is a research and product lab building long-horizon evaluations for frontier models.

The failures that matter appear hours into a task. Here is how we look for them:

I.

Long horizons

The failures that matter appear hours into a task, not in the first reply. Our evaluations run long enough to find them.

II.

The whole trajectory

We score every step an agent takes, not only where it ends up. Reaching the right result by a path you could not trust is still a failure.

III.

Who we build with

Companies building environments, evals and benchmarks, to make them for alignment‑specific tasks. And model builders, to measure alignment on real-world tasks over long horizons.

Research

Coverage

  • “The experiment exposes a structural gap: agents are evaluated on whether they complete tasks, not on whether they remain honest, proportionate, or restrained when the task is impossible.”

    Gadget Review,

  • “If personal AI assistants are deployed at scale, they could overwhelm government agencies and businesses with automated requests.”

    The Decoder,

  • “The setup matters because it is close to how enterprises are starting to deploy agents, not how labs benchmark them.”

    Technspire,

The Vals AI thread was viewed 13,299 times, and the study reached 107 points and 126 comments on Hacker News.

Work with us

We would like to hear from teams building environments, evals and benchmarks, and from model builders evaluating alignment over long horizons.