Microsoft Copilot AI will soon run locally on PCs

Microsoft's Copilot AI service is set to run locally on PCs, Intel told Tom's Hardware. The company also said that next-gen AI PCs would require built-in neural processing units (NPUs) with over 40 TOPS (trillion operations per second) of power — beyond the capabilities of any consumer processor on the market. 

Intel said that the AI PCs would be able to run "more elements of Copilot" locally. Currently, Copilot runs nearly everything in the cloud, even small requests. That creates a fair amount of lag that's fine for larger jobs, but not ideal for smaller jobs. Adding local compute capability would decrease that lag, while potentially improving performance and privacy as well. 

Microsoft was previously rumored to require 40 TOPS on next-gen AI PCs (along with a modest 16GB of RAM). Right now, Windows doesn't make much use of NPUs, apart from running video effects like background blurring for Surface Studio webcams. ChromeOS and macOS both use NPU power for more video and audio processing features, though, along with OCR, translation, live transcription and more, Ars Technica noted.

So far, the processor with the fastest NPU speed is Apple M3, which offers 18 TOPS across the lineup (M3, M3 Pro and M3 Ultra). AMD's Ryzen 8040 and 7040 laptop chips are next with 16 and 10 TOPS respectively, while Intel's Meteor Lake laptop hits 10 TOPS as well. Qualcomm may offer the first processor with enough power for Copilot via the Snapdragon X Elite, which will offer 45 TOPS of AI compute speed. 

Intel's Lunar Lake chips, set to arrive in 2025, will ship with triple its current NPU speeds. Yesterday, the company introduced 300 new AI features optimized specifically for its own OpenVino platform. The chip giant also announced an AI PC development kit based on the the ASUS NUC Pro that uses its current Meteor Lake silicon. 

"From a desktop standpoint, we have plans on the desktop side, what we would say [is an] AI PC. And then there's also the next-gen AI PC, the 40 TOPS requirements; we have all of our different steps in our roadmap on how we cover all the different segments," the company told Tom's Hardware.

This article originally appeared on Engadget at https://www.engadget.com/microsoft-copilot-ai-will-soon-run-locally-on-pcs-130642514.html?src=rss https://www.engadget.com/microsoft-copilot-ai-will-soon-run-locally-on-pcs-130642514.html?src=rss