Publications

Computer-use agents tackle both everyday desktop tasks and long-horizon professional workflows. Interfaces centered on screenshots and mouse-and-keyboard actions require repeated visual interpretation while leaving semantic and programmatic capabilities underused. Hybrid Computer Use (HCU) brings three complementary interaction channels into a general agent: programmatic tools such as code and CLI, semantic interaction through accessibility (AX), and visual interaction through graphical user interfaces (GUI). HCU adds AX and GUI tools to the agent's existing tools through MCP, leaving channel selection and combination to the model. Its AX tools return compact observations, support batched actions with postconditions, and report changes as AX diffs. Cross-channel action guards check targets and layouts against live state before input. A supplementary skill explains how to use and coordinate the channels. On OSWorld 2.0, HCU achieves the highest Partial and Binary scores among the compared systems. Trajectory analysis shows that all evaluated models improve with HCU while dividing work across the channels differently.

Effective computer-use agents should combine visual GUI interaction with code and terminal execution within a single policy. Mage-VL-Cowork is a 4B hybrid computer-use agent trained from a general multimodally aligned checkpoint through a five-stage curriculum across midtraining, SFT, and RL. Mage-CUA-Forge replaces per-task agent construction with reusable, skill-conditioned generation workflows and application-specific preparers. It synthesizes 293,806 desktop and web tasks across 6,915 skills and reduces recurring per-task expenditure by 82.6%. Instead of directly distilling a default teacher, the work evolves the teacher harness and uses specialized harnesses to produce high-quality trajectories and stage-specific supervision. Mage-VL-Cowork-4B reaches 66.0% on OSWorld-Verified with lower reasoning cost, the best reported result at its parameter size.

Standard vision-language models (VLMs) suffer from Moravec's paradox: they excel at complex offline visual reasoning but struggle with simple streaming perception tasks and process them inefficiently. We present Mage-VL, an efficient codec-native streaming foundation model for real-time multimodal understanding and interaction. At its core, our custom tokenizer, Mage-ViT, replaces uniform frame sampling by selectively encoding dynamic, entropy-rich regions using motion vectors and residual energy across sparse anchor (I) and predicted (P) frames. Operating at a 16 x 16 patch level, this reduces visual token consumption by over 75% while preserving spatiotemporal context. Trained from scratch on approximately 560M unlabeled images and 100M unlabeled video frames, Mage-ViT matches or outperforms flagship encoders trained on billions of image-text pairs. We establish AI4AI data pipelines encompassing prompt-code joint optimization for multimodal captioning and AI-driven performance diagnosis to guide training recipes. Furthermore, through a bio-inspired dual-system architecture - a lightweight System 1 event gate and a causal System 2 decoder - Mage-VL enables proactive streaming perception. Extensive evaluations show that Mage-VL-4B matches Qwen3-VL-4B on static tasks while achieving strong gains in video understanding and 2D/3D spatial reasoning, with up to a 3.5x wall-clock inference speedup, and comprehensively surpasses the 15B Phi-4-reasoning-vision baseline. Beyond model artifacts, we deliver seven key empirical findings covering pre-training data efficiency, variable-resolution scaling, codec system acceleration, VideoQA SFT redundancy, motion-spatial synergy, AI4AI data pipelines, and Zero-Vision SFT for multimodal RL.

Live websites introduce network failures, access restrictions, and content drift into GUI Agent training and evaluation. WebHarbor uses coding agents to recreate real websites as local Docker environments with authentication, databases, and multimodal content. These environments support interactive tasks and state resets between runs. The environment collection can expand with new websites and tasks as agent capabilities develop.

The rapid growth of online video platforms has created an urgent demand for real-time video understanding. Unlike offline tasks, online video understanding emphasizes proactive responsiveness, where models must detect when sufficient evidence has appeared in a stream to answer a given question (trigger) and respond immediately. We introduce TV-Online (Trigger Video-Online), a large-scale dataset with 50K videos, 200K questions, and 500K time-stamped answers, covering progressively complex trigger-based tasks. We further propose a streaming-oriented modeling paradigm with protocol-level tagging and structured state management, trained via a progressive strategy using difficulty annotations and reinforcement objectives. Finally, we introduce a unified evaluation metric integrating semantic, temporal, and coverage dimensions.