The main use is enabling developers to write their own NPU kernels according to their respective need, instead of picking workaround from what Intel provides in OpenVINO. I think the project would benefit three scenarios:
1. Implement uncommon NN operators.
2. Implement a high-precision (FP32, etc.) version of existing OpenVINO operators for numerically sensitive usage.
3. Implement a mega kernel that fuse several small kernels together to reduce SHAVE invocations, and potentially improve performance.
But most importantly, it gives you more control over the hardware you own. That control IMO should have been yours from day one you bought the machine.
Author here. Feel free to ask any questions :)
What is this useful for?
The main use is enabling developers to write their own NPU kernels according to their respective need, instead of picking workaround from what Intel provides in OpenVINO. I think the project would benefit three scenarios:
1. Implement uncommon NN operators.
2. Implement a high-precision (FP32, etc.) version of existing OpenVINO operators for numerically sensitive usage.
3. Implement a mega kernel that fuse several small kernels together to reduce SHAVE invocations, and potentially improve performance.
But most importantly, it gives you more control over the hardware you own. That control IMO should have been yours from day one you bought the machine.