Traditional debugging workflows fall apart the moment software begins interacting natively with user interfaces. When an agent (Refresh uses Computer-1) executes a long-horizon task across a browser or a desktop, traditional cloud logs only capture isolated inputs and outputs. Without a continuous record of the entire run, this lack of context leaves developers with a massive blind spot.
A failure at step 100 is almost always the compounding result of a subtle visual misinterpretation that occurred all the way back at step 10. Without a continuous timeline, standard text logs and static screenshots force you to blindly guess how the agent actually lost its way.
We built trajectories.sh to eliminate that blind spot. Instead of treating agent execution loops as throwaway text files (ATIF), it tracks them as continuous, multimodal paths over time, giving you a complete map of the run's historical arc.
Interactive trajectory replay: scroll through agent actions on the right, or click any step to jump to that moment. View full run →
The Failure Trace: Debugging Real Agent Behavior
To see how this changes the debugging workflow, look closely at the complex NASA data-retrieval run featured in the player above. The task requires the agent to filter and count specific satellite coordinates, which creates a fragile, multi-step pipeline.
Instead of guessing why the run ultimately diverged from the expected path, you can scrub straight to Step 16. There, you can watch the model hit a friction point with the native application interface. Rather than blindly forcing its way through the structured UI, the log captures its exact cognitive pivot:
“Let me search for deep space spacecraft that were active in July 2009.”
You can track the exact millisecond it abandons the internal app mechanics entirely, clicks up into the browser's address bar, and pivots to an unscripted reward-hacking strategy. It completely bypasses the intended workflow by looking up the raw data directly on the open web.
The Diagnostic Suite
Fixing these behavioral regressions requires engineers to see exactly what the browser and the model were doing at any given millisecond. trajectories.sh unifies the entire execution arc into a single timeline, giving you three distinct diagnostic angles to dissect a failure:
Whether you are diagnosing subtle rendering issues, infinite loops, or instances of reward hacking, trajectories.sh immediately isolates the behavior. Instead of wasting engineering hours spinning up identical local setups just to reproduce a single bug, you can capture the entire failure mode into a single, self-contained trace file that records every visual and stateful layer of the run.
Collaborative Agent Review
Since CUAs operate across real-world human interfaces, their failures are rarely simple code bugs. Rather, they tend to be complex behavioral deviations, and diagnosing them requires the same rigor as traditional code reviews.
trajectories.sh turns that chaotic debugging loop into a shared workspace where engineering teams can drop comments directly onto an active execution path, pin recurring edge cases, and patch behavioral drift in real time. When it is time to ship, you can generate cryptographically verifiable execution links to give external stakeholders unforgeable, step-by-step proof of exactly how your model handled a live production environment.