The AI-as-a-copilot company, Microsoft, is moving beyond merely integrating generative AI on Azure and is planning to deploy it in Windows applications. This time, it is not just adding ChatGPT to everything, but much more. 

According to a recent report by Windows Central, the company is striving to integrate Windows 11 applications, including Photos, Snipping Tool, and Paint with generative AI capabilities. Undoubtedly, the image-generation capabilities would be powered by OpenAI’s DALL-E, the same technology that the company employs for Bing Image Generator.

Regarding the upgrade to the Camera and Photos app, there’s a possibility that Microsoft will incorporate optical character recognition (OCR) technology into both applications. This advancement has the potential to empower each app to recognise text, individuals, and objects within photos or screenshots, thereby significantly simplifying the process of copying and pasting information from these sources. 

Snipping Tool is also expected to get an OCR technology update. This strategic enhancement would enable Windows 11 to promptly recognise and replicate text from screenshots onto the clipboard, streamlining information utilisation. Sounds interesting, but all of this comes with a mild hit of security issues.

All the way to the cloud

Another recent development by Microsoft, when it announced the integration of Python into Excel, has been met with some questions due to issues related to privacy and security. Developers have raised concerns as the program can not run locally but only on Azure Cloud. And the community is clearly not completely on board with this decision. 

Same can be expected from the Paint application. Running text-to-image models locally is a Herculean task and requires a lot of computation within a single device. On the other hand, Paint has always been a locally running software with minimal computation requirements. Being the ideal way to keep the load less on the device, Microsoft will link it to the cloud as well. 

Coupling this with OCR on Snipping Tool, Photos, and Camera, people are concerned about sharing all of this information with Microsoft through the cloud connection. Ideally, it would make sense for Microsoft to run towards building edge capable products before deploying these on the Windows ecosystem. 

The timeline for the rollout of these AI integrations remains ambiguous as Microsoft is meticulously fine-tuning these concepts. An imminent ‘special event’ scheduled for September 21, is anticipated to unveil new Surface hardware while shedding further light on Microsoft’s AI roadmap for Windows.

Microsoft is not that bad

To Microsoft’s credit, the company has been in talks with Intel, Qualcomm, AMD, and NVIDIA for developing support for AI on the silicon level, allowing edge computing on Windows devices. When it comes to current LLMs, such as Llama 2, the computation requirements have gone down significantly, but not enough to make it available for everyone on their devices locally without occupying massive amounts of storage and RAM usage. Releasing Llama 2 with Meta on Microsoft cloud was possibly a hint towards optimising it in the future as well. 

Over the past year, the tech giant has consistently spotlighted AI functionalities across its product portfolio. Notably, the trend has transcended to its flagship operating system, Windows 11. Two prominent AI-driven features were announced at Microsoft Build 2023,  Windows Studio Effects and the eagerly anticipated Windows Copilot, have already been announced, with the latter set to debut this fall.

Arguably, the hype around products like ChatGPT has been on the decline. It may well be just a handful of people eagerly waiting for their Windows 11 computer to be able to generate ChatGPT-like text locally, though it would definitely prove to be very useful. 

Even if Microsoft ends up integrating all of these products on Windows 11 locally, the only question that remains is, will MineSweeper or Notepad get an AI upgrade as well? 

Funnily enough, this move might even mark the revival of the loved and hated Clippy assistant on Office products. This time, with the integration of generative AI, it might be even more helpful, or even more annoying. Well, the company has already announced plans to invest in it, so who knows?