Microsoft is in its AI age. The company has gone on an integration spree, putting OpenAI’s models into Office, Bing, and even Windows. Now, the tech giant is open sourcing a set of tools to bring this level of AI integration to all developers. The Redmond giant recently released TypeChat, a library of software tools aimed at augmenting traditional UI with the power of AI. 

By targeting the code generation capabilities of LLMs, researchers at Microsoft have created a system that can help AI algorithms to communicate with applications. This way, the user can rely on natural language inputs to interface with software in addition to existing UI elements. What’s more, in true open source fashion, the toolkit can be used with any LLM of the developers’ choosing, opening the floodgates of AI-powered applications to the world. 

Just as the company is trying to create the future of computing through the tight integration of AI into applications used by millions every day, it is also enabling a developer movement to enable the larger integration of AI into everyday applications. 

TypeChat explained

One of the biggest issues that developers have been dealing with in terms of integrating LLMs into software is that language models rarely give machine-readable text. Even when the user prompts the model to provide structured data, LLMs fall short. However, the team at Microsoft found that these generative AI algorithms perform moderately well at translating user queries into JSON (JavaScript Object Notation) format, which works great for machines. 

In the example provided by Microsoft in its blog, the LLM was able to accurately transcribe a customer’s order into a largely error-free JSON output. To ensure that this happens repeatedly, they further refined the technique by adding a type requirement to the output. For example, when asked to respond in a predefined output type called ‘Response’, the LLM creates a far more refined and structured output for the query. 

This code can also be validated by the TypeScript compiler, which the researchers were using. This paves the way for a clean, structured, and machine-readable output to be used in an app’s workflow. This method, tentatively termed a ‘response schema’, can be used for a variety of different applications by clearly defining a schema for each use-case. Some applications showcased were sentiment analysis, application creation through an ‘API schema’, and a ‘data schema’ for structured outputs. 

As mentioned previously, the project is not only open source, but also has plumbing to plug into different LLMs. While it was built with OpenAI API and Azure OpenAI service, the researchers clearly mentioned that it can be used with “any chat completion-style API”. However, they also stated that TypeChat works best with models that have been trained on both prose and code. TypeChat is just the latest announcement in an environment that is currently seeing a high degree of growth, enabling developers to create the apps of the future.

AI tooling for the future

TypeChat is just the latest in a long line of tooling for LLM integration. Arguably kick-started with the launch of LangChain, this field has seen extreme innovation over the past few months. Even AI agents, which had their time in the Sun with offerings like AutoGPT and Baby AGI, can be classified under the category of AI tooling. Also, LLMs have become more capable at parsing large databases thanks to vector database systems like PineCone and Weavite. 

Any successful innovation in the software field is only amplified by movements in the developer ecosystem. The power of open dev tools is extremely apparent, as seen by the explosive innovation surrounding Meta’s LLaMA model. Even top executives at tech giants have acknowledged that top AI companies have no moat, and that open source will eventually win out. 

According to a survey by Sequoia Capital, only 15% of its respondents have built custom language models from scratch or open source. However, 38% of them are interested in the app development framework around LLMs, and a whopping 94% use pre-trained models. Another interesting takeaway was that 88% of respondents stated that retrieval mechanisms, like vector databases, are a key part of the tech stack. 

This shows the hunger that the field has for more developer-focused LLM tooling; an appetite Microsoft seems eager to sate. During the 2023 Microsoft Build conference, the company announced a spate of dev tools, from adopting OpenAI’s plugin standard to super powering the WinML API. 

These tools also serve the same vertical as TypeChat, targeted squarely at developers looking to build AI applications. With the rise of this ecosystem, developers will soon have the tools to create applications that can integrate AI with a simple text box, and chaining together multiple models will create an AI the likes of which can only be matched in science fiction.