Built for one agent, breaking under a thousand

Our enterprise customers run whole teams of AI agents — software that carries out work on its own — completing tens of thousands of tasks a day. Any task can fail, get flagged for a human to review, or pause for a human to approve. At 15,000 tasks a day, even a 5% failure rate is 750 fires a day. Workforce — the part of the product for managing all those agents — had been built for a handful of them; at this volume it fell apart. The operators running the agents couldn’t see what was breaking, how widely it had spread, or how to fix it in one move. They were accountable for outcomes they had no way to watch.

‹ ›

It started small — then the real problem opened up

The task came in narrow: operators couldn’t easily find or review approvals. The first fix lived inside Agent Builder — a toolbar and timeline to move between escalations and approvals in a single agent. But it only ever showed one agent at a time, buried deep in the app.

The TaskOps prototype, built in Cursor on real data

I prototyped in Cursor pulling real customer data — hedging on where the feature should live, in the Analytics page or the Tasks page. After internal testing, we doubled down on one direction to put in front of customers. That testing is where the core insight surfaced.

‹ ›

It would live in the Tasks page, not Analytics — Analytics is for high-level ROI and trends, while the Tasks page is where operators go to catch fires and manage the workforce in real time.

Operators don’t manage tasks. They manage patterns.

Across those testing sessions, every operator reasoned the same way: start at a time window (“past week”, “past 24 hours”), narrow to a specific agent or workforce, then drill to the task. They reason in clusters — same error, same tool, same window — but the product forced them to treat each task as a one-off.

So the new experience would start at the macro, drill to the cause, and land at the fix — the mental model operators already used.

From one fix to a TaskOps system operators run at scale

Detection had to work the way operators think — start at the macro, drill to the cause, act on the fix. So the dashboard was built around three principles:

Every design decision traced back to a call

Those principles came straight from the operators. Each pain point they named has a matching move in the design — what they said, and what I did about it:

I chunked the build down with engineers

With no dedicated PM, I drove delivery myself. I read the codebase in Claude Code, wrote the engineering-style design docs the work usually depends on, then broke each feature into small, reviewable chunks — the engineers reviewed and commented in Linear, where an agent turned each chunk into a directional ticket to build on.

Grouping surfaces repeating failures at a glance, instead of burying them in the scroll

Feature 4

Group by workforce, agent, or error

Cluster tasks by workforce, agent or error, so operators triage by root cause — the next milestone, on the same grouping foundation.

“The workforce and agent toggles are what I really wanted — I can finally see who’s causing what, and spot which agents keep breaking.”Lena · operator

From a side project to a team I was chosen to lead

What began as a customer-led side project I drove with a handful of engineers became a formal squad inside Agent Operations — one of the company’s two core teams — and a top priority, with a large share of engineering behind it. I was chosen to lead it as Product Owner + Product Designer — now setting the strategy and roadmap for how the company detects, resolves and proves ROI on agents at scale. Quite possibly the best work of my life.

One prototype. Several strategic bets, de-risked at once.

With the team in place, the next question was what to build next — so I built a mobile prototype — but it did double duty. The obvious question was do operators even want to manage agents from a phone? Beyond that, I used it to put concepts we hadn’t built yet in front of all three enterprise customers: proactive alerts, flagging matching errors in bulk, and richer at-a-glance analytics. One artefact, several product bets answered at once — and it’s where the Q2 roadmap took shape.

Agents run around the clock; operators don’t — so the strategic questions were:

Four problems, all pointing upstream to detection

Testing the mobile concepts with all three enterprise customers surfaced four problems — and the biggest ask was the simplest: “tell me when something breaks.” Every one of them traced back to the same place — detection.

The test confirmed it. Detection is still the bottleneck.

That gap — detection — is the Q2 bet. The concept testing put it beyond doubt: across all three customers, operators still get stuck at the very first step. So as PO I framed the roadmap as a funnel and made detection the Q2 bet again — the biggest bottleneck first — and sequenced what comes next. We’ll likely stay on detection through Q3, with the metrics guiding us.

Each stage of that funnel has its own backlog of concepts. Here’s how I sequenced them — foundation first, then the highest-leverage bet per stage — and why each one earns its place:

The team formed halfway through Q2, so nothing had been measured before — the first job was setting up the baselines themselves.

Value delivered iteratively — and a platform priority reshaped

A few outcomes already stand, whatever the long-term numbers say:

Agent products climb the same ladder of autonomy we use for self-driving cars — from hands-on to hands-off:

Operators are already writing the L4 rules themselves — “rerun only if it didn’t email anyone; auto-heal the flaky API.” That rule-writing is what makes L4 (agents that fix themselves) possible — and the future I’m building toward.