With Windows ML, Intel AI to Invade Mobile PCs

　　It might not be too long before your average mobile PC will feature — on its motherboard — not just CPUs and GPUs but also an embedded AI inference chip, like the Intel/Movidius Vision Processor Unit (VPU).

　　The first clue for this scenario unfolded in Microsoft Corp.’s launch announcement today, at its Windows Developer Day, of Windows ML, an open-standard framework for machine-learning tasks in the Windows OS. Microsoft said that it is extending Windows OS native support for the Intel/Movidius VPU. Implied in the message is that Intel/Movidius has taken a step closer to finding a home not just in embedded applications, such as drones and surveillance cameras, but also in Windows-based laptops and tablets.

　　In a telephone interview with EE Times, Gary Brown, director of marketing at Movidius/Intel, confirmed, “Although today’s announcement isn’t about that [VPU integration on a mobile PC], yes, you will see VPU migrating into a PC motherboard.”

　　Windows ML is expected to bring Windows up to date in the fast-heating AI world. It will “dynamically determine the most suitable hardware for any given AI workload and intelligently distribute across multiple hardware types — now including Intel VPUs,” according to Intel/Movidius.

　　Brown explained, “Our VPU can offload heavy-duty AI processing tasks such as vision, face recognition, voice, biometrics, and others from CPUs and GPUs on PCs. VPU can help free up their processing resources.”

　　Intel/Movidius VPU goes mainstream

　　Kevin Krewell, principal analyst at Tirias Research, told us, “Adding native Windows support will flag developers that the Movidius VPU is going more mainstream.”

　　However, Krewell isn’t quite sure if a PC is the right home for VPU. “I can see the VPU as a good addition to AR/VR products like a next-generation Hololens. I’m not sure if it makes sense in a PC — there’s plenty of processing power in a PC, including the CPU and GPU to process video. The Movidius VPU works best where the unit is power-constrained, like drones.”

　　He added, “Perhaps this is a first step for Microsoft extending Windows into new areas such as drones and robots.”

　　The world of AI is expanding fast, putting Microsoft under pressure to catch up. Last year, Khronos started work on its own low-level ML framework. Like graphics APIs, this is meant to be a generic API. However, as seen in the Windows ML announcement, it’s clear that Microsoft would have still needed one that’s specific to its Windows OS.

　　Mike Demler, senior analyst at The Linley Group, observed that Windows ML looks to be like any other neural-network runtime API. However, he added, “It’s about time that Microsoft caught up with Arm platforms.”

　　Windows OS native support “gives Movidius better access to the Windows laptop/tablet market,” he said.

　　Asked what’s in it for Microsoft, Demler explained, “Developers could already run machine-learning apps on Windows platforms using the CPU, GPU, or custom peripherals like [Intel/Movidius] Myriad, but Windows ML gives them a standard method.” For PC users, Windows ML will “help drive the migration of machine-learning apps to client devices — PCs in this case.”

　　What AI applications?

　　If an AI processor is being designed into specific embedded systems, applications and tasks assigned to the AI chip are clear. They can do object tracking, collision avoidance in drones, or forensic analysis in surveillance cameras. The goal of pushing AI to the edge is to enable embedded systems for “sense, assess, and decide” actions, said former Movidius CEO Remi El-Ouazzane, now Intel’s vice president and general manager.

　　If so, what exactly are the AI applications on a PC?

　　Intel’s Brown suggested many. “Suppose you walk into a room and there is a Windows tablet on a table. It can see you, recognize your voice or face, and help you with a variety of personal-assistant types of work, including smart music search or classifications of your photos. The vision-based AI can also help enhance images in video conferencing.” Of course, it’s up to app developers’ imaginations to come up with new AI apps on PCs.

　　Demler believes that AI apps on PCs will be no different from apps on other mobile devices. “They include biometrics, AR/VR, image processing, object recognition, and others,” he said.

　　For now, Intel’s Brown acknowledged that Movidius/Intel’s Myriad X is the first AI processor for Windows ML to take advantage of on mobile PCs. Asked about the timing for Myriad X to be on the motherboard, Brown said, “Soon.”

　　However, the deal obviously isn’t exclusive to Intel/Movidius.

　　Asked to speculate on other possible AI accelerator candidates on PCs, Demler stressed that the conversation focuses on mobile PCs. “There are plenty of Nvidia GPUs running machine-learning apps in desktop PCs. Linley [Gwennap] recently covered an AI accelerator startup called Gyrfalcon that has an AI-accelerator chip they build into a USB stick, just like Movidius. The trick is getting onto the motherboard. In China, you have AI chip companies like Cambricon, which has Lenovo as one of its investors.”

　　Myriad X

　　The AI inference chip expected on Windows ML first is Intel/Movidus’s Myriad X. When the company first unveiled it last summer, El-Ouazzne told us, in designing the company’s third-generation VPU, “We were looking to anything that allows us to increase the performance of neural networks without increasing power.”

　　With many more hardware acceleration blocks, Myriad X architecture today can do a trillion operations per second (TOPS) of compute performance on deep-neural-network inferences while keeping power use within a watt. The Myriad X, housed in an 8 x 9-mm package, is capable of 4 TOPS in total.

　　In designing Myriad X, Movidius explained that it increased the number of its Streaming Hybrid Architecture Vector Engine (SHAVE) DSP Cores from 12 [in Myriad 2] to 16. Movidius added a neural-compute engine with more than 20 enhanced hardware accelerators. These accelerators are designed to perform specific tasks without additional compute overhead. Tasks include depth-mapping to extract edges (a key to drone landings, for example), a de-warping engine for sensors enabling a wider field of view, and optical flow for super-high-performance motion estimation critical in tracking and counting people with surveillance cameras, explained El-Ouazzane at the time of the Myriad X announcement last year.