“To put this in perspective, I ran a 56k parameter custom CNN model, with 3 convolutional layers and the 13 TOPS AI HAT was able to process up to 2,400 FPS - very good!!
It is not possible to run a meaningful transformer model on here, like an LLM. For two reasons: model size being fit into RAM and (even if it does fit into RAM) token rate.”
“The hat itself is great but the included gpio header is useless. It is too short for gpio pass through if the hat is mounted above the pi as advertised in the photos. The included standoffs and pcie cable are also too short they do not clear the CPU cooler fan. Instead I have had to mount the hat below the pi as the hat which works but I don't think is the intended way to use it.”
“Only useful for vision AI. Which is disappointing as I have audio and text projects I was hoping to use it on.
https://community.hailo.ai/t/raspberry-pi-ai-hat-disappointment/18099”
“This is an amazing piece of kit - if you go from running computer vision (and other inference AI tasks) on a CPU to the NPU, the performance boost is incredible. I'm using the Hailo libraries via C which is perhaps a bit more trouble than using Python, but there's a lot of useful info on the Internet. Perfact for some very cool projects.”
