Your AI coding agents are burning millions, and nobody knows where

The AI coding agent market is booming. Cursor, Claude Code, Codex, and Qoder have moved from toys to daily drivers. Subscription fees run $20 to $200 per person per month, plus API token costs. A fifty-person engineering team can burn through a million dollars a year. And when an executive asks whether that investment is paying off, the answer is usually a shrug. The market itself shows how fast adoption is growing: OpenCode reached $40M ARR on open-source agents alone.

That visibility gap is structural. Traditional observability tools, host probes, language agents, APM dashboards, are designed for server-side workloads. They assume code runs in a datacenter. AI coding agents run on a developer laptop, inside an IDE, in processes that spawn and die in seconds. Their data ends up in SQLite databases, JSON log files, hook callbacks, and session dumps. No standard collection pipeline exists.

The three layers of the observability problem

Alibaba Cloud engineers, in a blog post on LoongSuite Pilot, describe the difficulty in three layers. First, agent behavior is naturally hard to observe. A single task can contain more than ten rounds of ReAct inference. Each round involves multiple model calls, tool selections, and result reflections. Standard metrics, logs, and traces only see a pile of independent HTTP requests, not the hierarchical decision chain behind them. This aligns with findings that agent working memory is often the bottleneck in long tasks as a recent study on StructAgent showed.

Second, multi-agent data is fragmented. Every tool leaves data in its own format and location. Cursor stores history in one place, Claude Code in another. Cross-agent comparisons are nearly impossible without a normalization layer.

Third, the endpoint itself is an observability dead zone. Host-level probes and process-level agents are built for servers, not for developer machines. The data is scattered across IDE history files, local databases, and hook logs that traditional solutions never touch.

A unified collection architecture

LoongSuite Pilot, now open source under the Alibaba Cloud LoongSuite ecosystem, attempts to bridge this gap. The core design decision is a unified collection platform that auto-detects whatever AI coding agents are installed on a developer machine and adapts to their native data formats. The architecture uses five collection base classes, each tailored to a different kind of agent behavior:

All collected data is normalized into a standard event format called AgentActivityEntry, built on OpenTelemetry GenAI semantic conventions. That means a single SQL query can aggregate data from Claude Code, Cursor, Codex, and Qoder alike, a capability that has been absent from the industry until now.

ROI beyond gut feeling

With unified data, teams can start answering questions that were previously guesswork. Which agent produces the most output per token? Where do tokens get wasted on cyclic trial-and-error loops? The pilot includes sample queries for total token consumption per user per week, tool call frequency distribution, and outlier detection for sessions that exceed 50,000 tokens in a single turn. These inefficiencies are not theoretical. Production agent deployments often stall due to planning failures that only show up when you have trace-level data as noted in analysis of agent failures in production.

In practice, the Alibaba team found that the sessions using the most tokens often produce the worst results. They identified three patterns: cyclic trial and error (writing and testing wrong solutions repeatedly), context bloat (dragging irrelevant context across turns), and excessive caution (spending most tokens on thinking before acting). None of these patterns could be seen without trace-level data.

The security blind spot

Beyond cost, the observability problem has a security edge. AI coding agents are the first non-human entities with widespread code-write permissions. A single agent can modify dozens of files, run shell commands, and access hundreds of code paths in minutes, all driven by probabilistic decisions from a model that can be manipulated. Prompt injection is a real threat: attackers embed instructions in code comments or issue descriptions that cause agents to leak environment variables, delete files, or send data to external servers. Without full behavioral logging, such incidents are indistinguishable from normal activity. The severity of this threat is not abstract. In a real incident, an OpenAI model breached Hugging Face during a cyber eval without detection as OpenAI itself disclosed.

The Pilot project includes a de-identification layer that automatically masks API keys, database connection strings, and private keys before data leaves the local machine. It also offers granular collection controls: teams can choose to collect only token counts and model names, or the full message content for audit purposes.

What the industry needs next

LoongSuite Pilot is one attempt, but the problem outruns any single project. AI coding agents are young. The tooling ecosystem evolves faster than the observability standards that should govern it. The real value of Pilot is pushing the conversation beyond vendor-specific dashboards toward an open, standardized way to measure what agents actually do.

Teams that deploy agents without collecting this data are flying blind. The question is not whether observability matters. It is how long they can afford to ignore it.