Most builders in AI today work downstream—assembling products on top of existing models, APIs, and platforms. It is no secret that the responsibility of building foundational models rests on the shoulders of Big Tech billionaires.
However, this makes startups vulnerable to being rendered obsolete, fearing that their products can be replicated.
So, what is a durable path for smaller players to build something foundational? Twenty-year-old engineer Dev Mandal thinks the answer lies in datasets—those that sit at the base of what companies like OpenAI, Anthropic, and Google need to make good on their stated ambition of automating the bulk of white-collar work.
“All of this work is done through a computer,” explains Mandal, the founder of Markov Studios, which has built the largest open-source computer use dataset.
“Most of this work is done across a range of software platforms—and to get AI to also be able to do so, it has to learn how to use the software in the first place,” he tells AIM.
While companies like OpenAI, Anthropic, and Perplexity have already shipped their own computer-use agents, those systems were trained on proprietary data. Today, open-weight LLMs are comparable to the performance of proprietary ones. However, developers still need quality datasets to fine-tune open-weight models and build something like, say, a Kimi K2.5 that works like Claude Computer Use or Cowork.
Until now, very little of that data has been available at this scale, leaving Mandal with an opportunity to fill the gap.
Why Mandal is Building Markov
Markov’s dataset consists of roughly 48,000 screen recording videos, totalling around 12,300 hours of real software usage.
The dataset, hosted on Hugging Face under a CC-BY-4.0 license, covers six software platforms: AutoCAD (10,059 videos), Blender (11,493), Excel (8,111), Photoshop (10,704), Salesforce (7,807), and VS Code (304).
The licensing allows reuse and modification, making it easier for developers to build on top of it. It crossed 1 lakh downloads on March 21, up from around 45,000 just days earlier on March 14, pointing to early developer demand for computer-use agents.
According to the company, all videos have been trimmed to remove non-screen content, such as intros, talking heads, and transitions. Audio has been stripped from these videos, leaving only raw interaction footage.
Mandal, who is studying aerospace engineering at IIT Madras, says the idea for Markov came from a simple observation.
The internet already contains large-scale demonstrations of how people use software, particularly on platforms like YouTube, but that data is not directly usable for training. “The main reason we are doing this is that this data isn’t really structured, and we have to convert it into a format which allows us to train better models,” he adds.
He notes that raw screen recordings alone are insufficient. “You need more than just a screen recording of what a person is doing—you need how one is using the keyboard and the mouse as well.”
Built for the RL Era
Markov’s dataset attempts to move in that direction by pairing visual sequences with action-level signals, which involves capturing where interactions occur on the screen. It also captures how inputs like clicks and cursor movements translate into changes in the interface.
The dataset is designed to align with how computer use fits into existing training paradigms. Each action produces an immediate and observable change in the interface, creating tight feedback loops.
Besides, this setup is highly compatible with reinforcement learning, which trains an AI by rewarding correct behaviour.
In this context, Markov’s dataset provides a strong, verifiable reward signal because each action can be clearly evaluated based on the interface's response and the overall progress toward completing a task. It focuses on widely used, production-grade software. Mandal sees this dataset as part of a larger shift in how AI systems will be trained.
Mandal has a “deep conviction” that in the era of Artificial General Intelligence, or AGI, models will rely heavily on video data, largely because other data formats are reaching their limits. Even if datasets are created manually, reaching a meaningful scale would take years, if not decades.
Video already captures human behaviour at scale, which is state, action, and sequence in a single format. Mandal focuses on computer use as an entry point because of the immediate demand and the economic value tied to automating enterprise workflows.
However, Mandal does not frame dataset building as a mission-driven pursuit. “I feel building data isn’t the most noble of efforts… at the end of the day, it all comes down to money,” he says candidly.
The demand for training data from AI labs has turned datasets into a separate layer of the ecosystem, with its own incentives. “Claude, Perplexity, and others in the computer use ecosystem are geared towards consumers, just general work like sending an email, calendar, and so on. But eventually, they will focus on automating enterprise tasks,” Mandal adds, suggesting that he may have already captured an early opportunity.
“We know there’s a gap, and we’re just working to fill that demand,” he remarks, explaining that open source datasets are easier to implement as compared to getting access to computer-use data directly from the enterprises. This often involves complex and capital-intensive partnerships, such as Snowflake and Salesforce partnering in 2024 for bidirectional, zero-ETL data sharing .
However, even if building on open source can appear thankless, the incentives are shifting.
Some of the most closely watched moves in AI over the past year have been the acquisition of open-source-led resources, wherein individual developers or small teams built widely adopted tools before being absorbed by larger companies.
OpenAI acquired Astral, Anthropic acquired Bun, and even Peter Steinberger, the creator of the now-famous OpenClaw project, was hired by OpenAI.
And Mandal recognises this. While he says acquisition is one of the definitions of success, he also says that open-source development, besides solving problems, can lead to more tangible incentives.
“Most hiring in the future is going to be done on acqui-hires rather than general hiring,” he notes.
He explains that, given how individual developers are building in public to solve their own problems, once their projects are also seen as a potential solution to a Big Tech company’s problem, it wouldn’t take much time to hire them.
Your reaction