Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots

Needle 2 is a 14MB agentic LLM optimized for edge devices like phones, wearables, and IoT devices, delivering 500 tokens/sec on Raspberry Pi 5 and 300-700 tokens/sec on budget phones. Built on Simple Attention Networks, it uses 2-bit compression with 45M parameters and achieves comparable performance to models 5-70x larger on tool-calling and device-use benchmarks. The model excels at structured extraction and function calling with typed parameters, requiring minimal world knowledge, and supports fine-tuning on consumer hardware in minutes to hours with automated data generation.

Key Points

• •Needle 2 is a 14MB binary requiring only 28MB RAM for full sessions, enabling deployment on resource-constrained devices including sub-$200 phones, Raspberry Pi, wearables, and microcontrollers
• •Achieves 500 tokens/sec on Raspberry Pi 5, 400-1,500 tokens/sec on VR devices (Meta Quest 3S, Apple Vision Pro), and 300-700 tokens/sec on budget Android phones
• •Uses Simple Attention Networks architecture with 2-bit compression (45M parameters) spending only 70 MFLOPs per token—7-85x fewer than comparable performant LLMs
• •Trades wins with LFM2.5 230M and Apple Foundation Model on tool-calling and mobile device-use benchmarks while being 5-70x smaller
• •Optimized for structured extraction and function calling with typed parameters—framed as mapping messy sentences to specific functions with values, eliminating need for open-ended prose generation
• •Supports fine-tuning on Mac/PC in minutes to hours with automated data-generation pipeline requiring only a few sample inputs for custom task optimization
• •Includes learned confidence scoring via Cactus Hybrid technique to determine when to act locally vs. escalate to cloud or larger models
• •Addresses the 21 billion IoT devices globally, particularly in emerging markets where most phones lack NPUs and have limited computational resources
• •Can function as text classifier, summarization model, or structured extraction engine by passing schemas instead of tool definitions
• •Designed for always-on assistant use cases within strict power budgets where every MFLOP translates to milliwatt-hours of energy consumption

Workflow Diagram

Concepts