Hailo Announces Hailo-10H M.2 Generative AI Module (40 TOPS) Running Llama-3.2-3B at 6.5W
The M.2 2242 Key-M module drops into standard Linux mini-PCs and Raspberry Pi 5 PCIe carriers to offload local LLM and VLM inference.
⚡ Key Engineering & Supply-Chain Takeaways
- Hailo launched retail availability of the Hailo-10H M.2 2242/2280 Generative AI accelerator module delivering 40 TOPS of INT4 compute.
- Unlike the vision-only Hailo-8 (26 TOPS), the Hailo-10H integrates 8GB of dedicated on-module LPDDR4X memory to run local LLMs and VLMs independently of host RAM.
- Executes Llama-3.2-3B and Qwen2.5-3B at 14.5 tokens/sec while consuming just 6.5W over a single PCIe Gen 3 x2/x4 slot.
🔬 Technical & Supply-Chain Specification Matrix
| Module Form Factor | M.2 2242 / 2280 Key-M (PCIe Gen 3 x2 / x4 Interface) |
|---|---|
| Peak AI Compute | 40 TOPS (INT4) / 20 TOPS (INT8) Dataflow Architecture |
| On-Module Dedicated RAM | 8GB LPDDR4X-4266 Dedicated to NPU Weights & KV Cache |
| 3B LLM Throughput & Power | 14.5 tokens/sec on Llama-3.2-3B at 6.5W Typical Module Power |
| Retail Kit Price | $139 Standalone M.2 Module / $165 Raspberry Pi 5 AI HAT+ Gen 2 |
Why Dedicated On-Module LPDDR4X Unlocks Generative AI on Raspberry Pi 5
The popular Hailo-8 (26 TOPS) AI HAT+ transformed the Raspberry Pi 5 into a powerhouse for 60fps YOLOv11 vision pipelines, but because the Hailo-8 relies on SRAM-only dataflow without dedicated external DRAM, it could not hold multi-gigabyte Large Language Models.
The new Hailo-10H M.2 Generative AI Module ($139) pairs Hailo's second-generation 40 TOPS dataflow NPU with 8GB of dedicated on-module LPDDR4X memory right on the 22×42mm card.
14.5 Tokens/Sec at 6.5W With Zero Host CPU Load
When plugged into a Raspberry Pi 5 or an x86 industrial mini-PC via M.2 Key-M:
- Zero Host Memory Contention: The entire 3B LLM or 2B Vision-Language Model resides inside the Hailo-10H's private 8GB LPDDR4X pool, leaving the Raspberry Pi 5's CPU and system RAM 100% free for Docker containers, Home Assistant, and Frigate NVR streams.
- Energy Efficiency: Running
Qwen2.5-3B-Instructat 14.5 tokens/sec draws just 6.5W on the M.2 rail—over 3x more energy-efficient per token than CPU-only inference.
🔗 Related AI Hardware in RECIWO Database
All 38 Devices →Raspberry Pi 5 (16GB) + Hailo-8 26 TOPS AI Hat+ Kit
16GB Raspberry Pi 5 bundled with the PCIe Gen 3 Hailo-8 (26 TOPS) neural accelerator HAT, delivering 60fps multi-stream YOLOv11 and Pose estimation at just 9W total system draw.
Radxa Fogwise AirBox Q900 (48 TOPS Edge AI Box)
Compact CNC-milled fanless aluminum micro-server powered by the 4nm Qualcomm QCS9075 / SG2300x series with 48 INT8 TOPS and dual 2.5GbE ports for home AI gateways.
NVIDIA Jetson Orin NX Super Edge AI DevKit
Palm-sized generative AI supercomputer delivering 157 TOPS (Super Mode) and 102 GB/s memory bandwidth to run 8B multimodal LLMs and VLA robotics policies locally.
📚 Related Deep-Dive Engineering Teardowns
Browse All 83 Articles →Raspberry Pi 5 (16GB) + Hailo-8 (26 TOPS) AI HAT+ Review: 60FPS Vision & Whisper at 9 Watts
We benchmarked the 16GB Raspberry Pi 5 paired with the 26 TOPS Hailo-8 PCIe Gen 3 AI HAT+ across YOLOv11m, SCRFD face recognition, and CLIP semantic video search.
Edge 3B–7B LLM Silicon Architecture: Why Memory Bandwidth Matters More Than Peak TOPS
RECIWO Labs' silicon analysis proves that external DRAM memory bandwidth—not peak TOPS—is the primary bottleneck for running 3B–7B LLMs on edge AI hardware.
2026 Small Language Models (SLMs) on Edge Hardware: Qwen3-1.7B vs. Llama-4-Scout-3B vs. Gemma-3-4B
RECIWO Labs benchmarked five sub-4B parameter language models quantized to INT4 across tool-calling JSON accuracy, summarization hallucination rate, and RAM footprint.
