Microsoft launched a range of Copilot Plus PCs yesterday that includes new AI features built directly into Windows 11. Behind the scenes, the company now has more than 40 AI models running on Windows 11 thanks to a new Windows Copilot Runtime that will also allow developers to use these models for their apps.
At Microsoft Build today, the company is providing a lot more details about exactly how this Windows Copilot Runtime works. The runtime includes a library of APIs that developers can tap into for their own apps, with AI frameworks and toolchains that are designed for developers to ship their own on-device models on Windows.
“Windows Copilot Library consists of ready-to-use AI APIs like Studio Effects, Live Captions Translations, OCR, Recall with User Activity, and Phi Silica, which will be available to developers in June,” explains Windows and Surface chief Pavan Davuluri.
Developers will be able to use the Windows Copilot Library to integrate things like Studio Effects, filters, portrait blur, and other features into their apps. Meta is adding the Windows Studio Effects into WhatsApp, so you’ll get features like background blur and eye contact during video calls. Even Live Captions and the new AI-powered translation feature can be used by developers with little to no code.
Microsoft demonstrated its Recall AI feature yesterday, allowing Copilot Plus PCs to document and store everything that you do on your PC so you can recall memories and search through a timeline. This is all powered by a new Windows Semantic Index that stores this data locally, and Microsoft plans to allow developers to build something similar.
“We will make this capability available for developers with Vector Embeddings API to build their own vector store and RAG within their applications and with their app data,” says Davuluri.
Photo: Allison Johnson / The Verge
Developers will also be able to improve Windows’ new Recall feature by adding contextual information to their apps that feeds into the database powering this feature. “This integration helps users pick up where they left off in your app, improving app engagement and users’ seamless flow between Windows and your app,” says Davuluri.
All of these improvements inside Windows for developers are the very early building blocks for more AI-powered apps on top of its new Arm-powered systems and the NPUs coming from AMD and Intel soon. While Microsoft is building the platform for developers to create AI apps for Windows, it’s now banking on this being an important part of the next decade of Windows development. Onstage at Build today, Davuluri stood in front of a slide that read “Windows is the most open platform for AI,” signaling just how important this moment is for Microsoft.
+ There are no comments
Add yours