I want to explore how to build agents that work autonomously in the digital world: using computers more effectively, completing longer tasks more independently, and improving through their work. My main current research entry point is Hybrid Computer Use. Through Work Agent engineering and product development, I also try to understand real users' needs.
As an opening post about my research, this is a record of my interests, lessons from practice, and expectations for the future as of September 2026. Some are active explorations; others are goals I have not reached. I hope to return to these ideas and see which judgments held up and which need to change.
1. Three connected research directions
I organize my interests into three parallel but connected directions:
| Research direction | Core focus |
|---|---|
| General Computer Use Agents | Use computers more intelligently and effectively to complete different kinds of tasks. |
| Long-Horizon Agents | Complete longer tasks more autonomously, with less continual human guidance and intervention. |
| Self-Improving Agents | Use information and experience from work to improve subsequent capabilities and understanding of users and environments. |
These concern how to act, how to sustain work through completion, and how to improve through that work. Harnesses, personalization, product interaction, and evaluation run across them.
1.1 General Computer Use: use the digital world's interfaces
I understand Computer Use broadly, with coding as one part of it. Writing code, working with files, calling APIs, using CLIs, and interacting with graphical interfaces are all ways to complete tasks.
My near-term focus is Hybrid Computer Use: bringing these approaches together within one agent environment. My personal preference is to make use of code and machine-callable interfaces first. The digital world already offers shells, files, APIs, SDKs, and tools connected through MCP. I want agents to take full advantage of them.
AX, or accessibility interfaces, is another valuable example. It depends on support from applications and operating systems, but I see the semantic information already exposed as something like a free lunch. We can provide infrastructure that lets models understand interfaces and act through text, reducing unnecessary visual inference.
Using software with a graphical interface is distinct from using purely visual GUI interaction. An agent that works with specialized applications and websites does not have to rely exclusively on screenshots and coordinate clicks. AX can also provide ways to observe and act on those interfaces.
My preference should not become a fixed order models must follow. I would rather expose useful tools and interfaces and let models choose and combine them for the task. I currently tend to regard vision as a fallback, but hybrid training may lead models to discover better approaches, including choosing vision more often in some situations.
This is one aspect of how I understand the Bitter Lesson: introduce fewer human preferences that restrict exploration. Similarly, I see some fixed screenshot-verification steps today as compensation for models not yet being adapted to hybrid interaction. Over time, I want models to decide when to observe, how to verify, and whether the task is complete.
1.2 Long-Horizon: complete longer tasks more autonomously
The basic goal of Long-Horizon Agents is to keep progressing toward completion as tasks involve more steps and take longer.
I believe this requires both model capability and better harnesses. Task management, context compaction, and progress retention need to support sustained work together, helping an agent know what it has done and what remains, with fewer reminders and interventions.
This is also why I want to build Hybrid Computer Use within mature harnesses. Real tasks may alternate between code, AX, vision, search, and other tools. These capabilities need shared task progress and context in service of completing the whole task. Tool breadth and sustained execution need to be considered together.
Beyond that foundation, I particularly care about another question: can an agent understand intent accurately and complete a longer task when the user has not spelled out every requirement?
In actual use, I often encounter two situations. Either the agent keeps asking questions before it can proceed, or it guesses when the request is brief and takes the work in the wrong direction. I want smarter agents to use past user behavior, earlier conversations, preferences, and the contents and progress of the current workspace.
A request of a few sentences does not mean those sentences are all the agent can know about the person or task.
I want fewer questions that existing information could answer, and fewer unsupported guesses, so sustained execution becomes work that users can confidently delegate.
1.3 Self-Improving: use experience to improve subsequent work
I do not want agents to start from scratch every time, repeat mistakes, or keep asking me to introduce myself and the environment. The improvements I care about include:
- Task capability: learning from attempts, failures, and feedback to improve how problems are solved.
- Understanding the user: learning habits, preferences, and expectations to reduce repeated explanations.
- Understanding the workspace: becoming familiar with its contents, conventions, and task progress.
I include personalization within this direction. Understanding users and adapting to their working environments can both develop through ongoing work and help agents interpret the next request more accurately.
I especially care about context and exploration history: what an agent tried, why it failed, and what feedback it received. This Experience does not all need to become long-term Memory. Even information useful only for a while or within the current task can reduce wasted effort.
One basic idea is therefore to preserve digital records wherever possible so later agents can read them. Once information is retained, there is room for improvement and DIY experimentation; how to organize, extract, and use it can remain open. This is also why I moved from ChatGPT's web interface to local Codex early on. Even for everyday questions, I wanted interactions to leave records useful to later work.
Skill Optimizer 
I do not limit "internalizing" information to one mechanism. Experience can remain readable information or affect Memory, Skills, tools, and harnesses. Updating model parameters is not my only criterion for improvement.
In the longer term, I am also interested in people and agents taking part together throughout new-model development to improve research quality and efficiency. This currently still needs ongoing human review. As capabilities improve, people may focus more on setting goals and defining standards, with their involvement decreasing further.
These three directions interact through use. Using existing information to understand and complete this task is distinct from learning through it to improve the next one, but both can connect through continued work.
2. How engineering and products shape my research
Engineering and product development have given me many research ideas. They expose real needs and problems that can be less apparent in research settings. The gap between Computer Use research and practical deployment is the most direct example.
2.1 Starting from ordinary people's needs
I have long wanted tools to let people be a little "lazier." As a child, I watched my mother work with spreadsheets and type formulas. It looked tedious. I wanted an easier, faster way to do those things and free up her time and hands.
In October 2024, shortly after starting university, I used Cursor to
build Lazy Calendar 
Later, while building
Open Cowork 
Codex Desktop had not yet been released then. The main entry points I had encountered were Codex CLI and Claude Code. They were useful to programmers comfortable with terminals, but unwelcoming to ordinary users. I wanted an open-source Cowork that let people who did not code experience this convenience. Open Cowork included Skills and GUI capabilities to cover gaps in CLI-only operation.
With Open CoDesign 
These projects keep me focused on what ordinary people want agents to do, how they express needs and follow progress, and how they use the results. I do not want to stay within the world of programmers and coding, assuming everyone shares the same needs.
2.2 Why Work Agents need broader Computer Use
In my view, programming benefits from established tools and ecosystems. Many development tasks can be done through CLIs, code, and files, with additional tools connected through mechanisms such as MCP. This gives agents a relatively complete path through the work.
Broader work involves specialized applications and websites, many without convenient, complete machine interfaces. An agent that handles only interface-friendly work and asks people to take over elsewhere cannot easily take responsibility for the whole task.
More general Work Agents with less dependence on human intervention therefore need to use these graphical interfaces. They can do so through AX, vision, or other available interfaces. I want to retain the advantages of code and machine interfaces while also covering software and websites that are less accommodating to agents.
2.3 Gaps encountered in deployment
In Maka 
Many Computer Use research settings I encountered centered on pure GUI interaction. From studying Codex's implementation, I saw an emphasis on AX in products. Its value went beyond avoiding cursor contention; I also saw benefits in token efficiency and action accuracy. Meanwhile, speed, compatibility, and action errors remained issues in my work on Maka. Models did not seem fully adapted to this method, and interface quality depended on upstream support.
Another gap concerns the execution environment. Compared with mature coding agents, many research implementations I encountered felt closer to an observation, decision, and action loop. Integrating MCP, Skills, and other capabilities, and supporting sustained work, often required further adaptation.
These experiences encouraged me to consider Hybrid Computer Use together with mature harnesses. I want training, evaluation, and actual use to share more execution infrastructure. I also want research questions to come from real development and use, as well as existing experimental settings.
3. Reflections from evaluation
3.1 Working like a real coworker
I worked on an evaluation project at MSRA in which an agent faces a real workspace and a brief instruction. It needs to combine workspace contents and clues from different messages to understand the actual intent, including what the instruction leaves implicit, and work like a digital worker or coworker.
The task is long-horizon and needs mixed Computer Use, including GUI, CLI, and search. Understanding requests, finding information, and sustaining execution are connected.
This scenario brings together my interests in computer interaction and long-horizon work. It also raises a further question: how can understanding and experience gained during this work be retained to improve subsequent performance? That is a question about self-improvement; reading workspace information alone does not establish learning from experience.
3.2 Professional tasks need professional experience
One lesson from this project is that evaluations of specialized work need people who have actually done that work. Studying knowledge, processes, and materials does not reveal all the practical details without firsthand experience.
Knowing broadly what a profession does is still some distance from constructing realistic tasks and judging whether results are usable. This made me value practitioners' involvement in task design and assessment.
3.3 Exploring evaluation of everyday use
For future Work Agent evaluation, I would like to start from an ordinary user's perspective. Can agents perform well in sufficiently real tasks and environments that do not depend on a particular professional specialization? Turning class-group announcements into calendar events is one need I have personally encountered.
I feel these everyday scenarios deserve more attention, though existing coverage and overlooked needs require further investigation. I wonder whether they receive less attention because the productivity value of any single task is less conspicuous. Yet these may be things people handle frequently, with a direct effect on everyday experience.
I am interested in different evaluation methods and ways of presenting results. I want to understand both the limits of specific capabilities and how much of that capability users actually experience, including how much effort they save. Questions from real use will continue to shape the evaluations I want to explore.
4. How I think deployment may change
4.1 Sharing a computer is an important constraint today
In the near term, people and agents may still primarily share a device in an interaction pattern similar to Vibe Coding. Whether actions are observable and whether they disrupt normal computer use are direct product concerns.
Cursor contention from pure-GUI operation in Open Cowork made this tangible to me. I value less disruptive execution and understand why engineering and products prioritize background operation.
However, running in the background and choosing an interface are different dimensions. AX is an observation and action method I care about; background operation is an interaction requirement. I do not want today's shared-device constraint to become a fixed premise for all future Computer Use research.
4.2 If workspaces move to the cloud
One future I am willing to bet on is that more workspaces move to the cloud, or that files, authenticated sessions, and working environments can be synchronized and shared. Agents could have their own cloud computers or isolated environments, receive tasks, keep working, and deliver results.
In that setting, people and agents would no longer compete for the same desktop or cursor, and multiple agents could work in parallel more easily. At least the constraint of not disturbing someone using the current computer would weaken considerably. Interface selection could focus more on task outcomes, efficiency, and reliability. Interaction would still matter, but cursor contention would no longer be the central issue.
This is why local-first is a choice shaped by current conditions for me. Files, authenticated sessions, and applications are local today, making local operation convenient. What I care about is agents accessing the information and tools needed for work, with records available for subsequent use.
This is a possibility I favor, not a certain trajectory. I also think about more agent-native systems or environments, exposing more suitable interfaces and supporting collaboration and parallel work. Their specific form remains open.
4.3 Hybrid as a choice for that future
Looking in this direction, I do not want exploration fixed at either of two ends: the purely visual GUI agents in some research settings I have encountered, or the unobtrusive background, AX-first approach I associate with engineering efforts such as CUA Driver. These are two emphases I currently see, not a classification of all research and products.
I choose Hybrid not only as a compromise between today's emphases, but because it fits the future I favor.
Independent environments reduce constraints from sharing a device with people. We should provide agents with a rich set of tools and interfaces and let their intelligence guide how they observe and act.
Code, CLIs, AX, vision, and other tools can all play a role. I do not want to predetermine which one should ultimately dominate, nor do I think every task must mix several methods. What matters is giving models choices and helping them use those choices better through training and practice.
This connects my current work with the longer-term goal. Building Hybrid tools and harnesses addresses deployment problems today while leaving room for smarter models and more independent working environments.
5. What comes next
Hybrid Computer Use remains my main current research entry point. Long-Horizon and Self-Improvement are parallel, mutually supporting interests. Work Agent development and evaluation keep exposing new needs and questions.
This post is a starting point. I hope to keep recording concrete work, attempts, and changes in judgment so the influence between research, engineering, and products remains visible. The longer-term goal is still agents that take on real tasks more autonomously and improve through work, helping more people spend less time on tedious things.