Daily Roundup

Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

Needle 2 is a 14MB agentic LLM optimized for edge devices like phones, wearables, and IoT devices, delivering 500 tokens/sec on Raspberry Pi 5 and 300-700 tokens/sec on budget phones. Built on Simple Attention Networks, it uses 2-bit compression with 45M parameters and achieves comparable performance to models 5-70x larger on tool-calling and device-use benchmarks. The model excels at structured extraction and function calling with typed parameters, requiring minimal world knowledge, and supports fine-tuning on consumer hardware in minutes to hours with automated data generation.

• Needle 2 is a 14MB binary requiring only 28MB RAM for full sessions, enabling deployment on resource-constrained devices including sub-$200 phones, Raspberry Pi, wearables, and microcontrollers
• Achieves 500 tokens/sec on Raspberry Pi 5, 400-1,500 tokens/sec on VR devices (Meta Quest 3S, Apple Vision Pro), and 300-700 tokens/sec on budget Android phones
• Uses Simple Attention Networks architecture with 2-bit compression (45M parameters) spending only 70 MFLOPs per token—7-85x fewer than comparable performant LLMs
• +7 more key points...