Raspberry Pi Gets Generative AI with New AI HAT+ 2

So you bought a Raspberry Pi. Then you used the former children’s project board to create a home server, or perhaps a functional robot. What’s next? Adding artificial intelligence, obviously. The company behind the modular board has announced the Raspberry Pi AI HAT+ 2, an add-on that “delivers 40 TOPS (INT4) of inferencing performance” and allows on-device inference for your at-home AI ambitions. The cost of this is fairly cheap, too. Artificial Raspberry Pi Putting a new AI HAT+ 2 into your Pi project costs just R2,150 ($130), which suggests that Asus’s similarly-skilled plug-in AI chipset might prove at least semi-affordable when it launches. Compatible with the Pi 5 board, the new hardware uses Hailo-8 (26-TOPS) and Hailo-8L (13-TOPS) neural network accelerators. According to the company, the previous version of this add-on “provides best-in-class acceleration for vision-based neural network models, including object detection, pose estimation, and scene segmentation,” though it lacked enough grunt for full-on generative AI. That’s where the second version of the board comes in. 8GB of onboard RAM and support for three different LLMs (DeepSeek-R1-Distill, Llama3.2, and three Qwen2 options) permit the AI HAT+ 2 to run large language models, vision-language models, and other genAI functions. Support for more complicated models will be added later via updates. Buying one of these isn’t like locking ChatGPT in your basement, however. The folks at Raspberry Pi point out that its newest add-on board is limited. Compared to the big boys like “cloud-based LLMs from OpenAI, Meta, and Anthropic [which] range from 500 billion to 2 trillion parameters; the edge-based LLMs running on the Raspberry Pi AI HAT+ 2 […] are sized to fit into the available on-board RAM, typically run at 1–7 billion parameters. Smaller LLMs like these are not designed to match the knowledge set available to the larger models, but rather to operate within a constrained dataset.”