Nvidia's RTX Spark has an NPU it barely mentions, and I spent a week figuring out what it actually is

Key Points:

• The HP OmniBook Ultra 16 features an NPU (Neural Processing Unit) in its RTX Spark platform, unlike the similar GB10 hardware in Nvidia's DGX Spark, which appears to have the NPU disabled or absent. This NPU is a 55 TOPS accelerator intended solely for Microsoft Copilot+ features, but Nvidia has provided minimal public information about it.
• Investigations reveal the NPU is a descendant of Nvidia's Deep Learning Accelerator (DLA) architecture with RISC-V processors, running neural networks independently of the GPU, confirmed by tests showing the GPU idle during NPU computations. However, software support is limited, with Nvidia's TensorRT for Windows on Arm only in beta and lacking full INT8 compute support.
• Performance benchmarks show the NPU is faster than the CPU but significantly slower and less power-efficient than the GPU, especially in INT8 mode where the expected throughput and efficiency gains are absent. The NPU runs INT8 data but effectively at FP16 throughput, leaving Nvidia's advertised 50-55 TOPS unexplained.
• Firmware and driver analysis suggest the NPU shares architecture with Nvidia's 2024 RISC-V DLA design, but the DGX Spark's GB10 platform explicitly disables the DLA in firmware, and no Linux support exists for it. The NPU's presence and activation appear tied to Windows on Arm platforms needing to meet Microsoft's Copilot+ NPU requirements.
• Nvidia's software support for the NPU is minimal and primarily aimed at fulfilling Microsoft's Copilot+ demands, with no DirectML access and limited execution providers. The NPU seems included mainly to satisfy Microsoft's hardware criteria rather than as a strategic AI hardware focus for Nvidia, as evidenced by scarce documentation and limited functionality at launch.