Alibaba's Qwen-UI-Agent scores higher on real phones than in sandboxes

Qwen-UI-Agent scores higher on real phones than on the sandbox benchmark next to it. A technical report from Alibaba's Tongyi Lab, submitted July 30, 2026, lists 92.2% on MobileWorld-Real, the real-device test, against 82.1% on MobileWorld. The paper does not explain the gap, and the two tasks may not be equally hard. Either way, the direction is the project's whole pitch: a GUI agent built for devices that actually exist. The pattern of sandbox scores failing to match real-world use is a known hazard for agent systems, examined in the planning trap analysis.

The report frames Qwen-UI-Agent as a "real-world centric foundation GUI agent." It spans mobile, computer use, the web, and the DeepSearch environment. This is not a chatbot that describes what it would do; the model operates the screen. Its unified action space interleaves GUI operations with command-line execution and generates batched actions in a single model turn, so the model plans several steps at once instead of round-tripping per tap. A lightweight harness layer adds stateful workflows and what the paper calls proactive service initiation.

The paper opens with a vision of GUI agents as "a general purpose executor over existing digital devices." It attaches a specific list to that vision: run reliably on real devices, execute workflows across platforms, combine GUI interaction with CLI, complete long-horizon tasks, kick off useful services proactively, and improve with minimal human effort. The design reads like a checklist against that list. The real-device runtime handles reliability, the cross-environment sandboxes provide reach, the unified action space covers the GUI-plus-CLI requirement, the harness takes care of proactive work, and the flywheel drives self-improvement.

One agent, four environments

The training setup is as notable as the scores. An AutoResearch-style data flywheel has agents constructing tasks and environments, diagnosing failures, and planning subsequent iterations. Online reinforcement learning trains on trajectories exceeding 100 turns, with more than 10,000 concurrent environments accelerating rollout. Long-horizon work, holding a goal across a hundred or more model turns, is the regime that setup is built for. It is also where state handling matters most: one framework lifted agent success on long tasks from 27% to 79% without a bigger model, per the StructAgent results.

The scores

The table below is drawn from the Qwen-UI-Agent project page. Mobile is where the report claims state-of-the-art results; computer and browser use are described as competitive against frontier models, naming Opus 4.8, Gemini 3.1 Pro, and GPT-5.6 Sol.

The OSWorld line is worth a second look. On OSWorld-Verified the agent posts 79.5%. On OSWorld-v2 the report lists a 40.0% partial-progress score. The two tests differ, and a gap that size is the difference between an agent that solves desktop tasks and one that gets partway. If you read one number in this report, read that one. It is the honest edge of the current system. A separate evaluation found AI desktop agents fail a before-after test 35% of the time, per the DDB benchmark.

What to watch

The comparative claims, state-of-the-art on mobile, competitive on desktop, come from the authors. Nothing here is independently verified yet, and the project has collected 278 upvotes on Hugging Face, a modest number for a paper claiming top marks. The flywheel also raises a question: when agents build the tasks and environments they train on, how much of a score is capability and how much is a loop tuned to what the benchmarks reward? The same question applies across the field: top models hit 96% on SWE-bench Verified but barely clear 23% on private enterprise code, per that benchmark comparison. The unexplained gap between MobileWorld-Real and MobileWorld points the same way.

That doesn't make the report a demo. Qwen-UI-Agent brings real hardware into the training loop, and 97.5% on AndroidDaily is the kind of number rival labs will want to verify. The desktop gap is the thing to watch. Close it, and the real-world claim starts to hold on desktops as well as phones.