An NPU is a block, either inside a system on chip or on a separate module, built to execute neural network layers efficiently. It typically runs integer arithmetic on many multiply-accumulate units in parallel, which is why models are usually quantized before they target it. Phone chips, edge AI boxes and even some microcontrollers now include one.
For IoT the NPU is what makes real-time computer vision or audio analysis possible within a small power budget. A camera module with an NPU can run object detection on every frame without a fan or a mains supply. The trade-off is toolchain lock-in: each vendor’s NPU supports a specific set of operators and formats, so a model that runs on one may need conversion or retraining for another.
Part of the IoT Glossary. Spotted an error or a missing term? Tell us.