Overview

Needle 2 is a groundbreaking 45-million parameter foundation model engineered for exceptional performance on resource-constrained devices. This compact, 14MB binary runs efficiently, requiring only ~28MB of RAM, making it ideal for deployment on everything from mobile phones and wearables to smart home devices and robots. It excels at tool calling, structured data extraction, and device interaction with a focus on simplicity and powerful capabilities.

Key Features

• Ultra-Compact & Efficient: A single, self-contained 14MB binary that operates within ~28MB of RAM, minimizing resource overhead.
• Powerful Tool Calling: Seamlessly integrate and call your custom tools with a simple, structured contract (text in, JSON out) governed by a byte-level grammar derived from your schemas.
• Confidence-Gated Responses: Every output includes a calibrated confidence score, enabling robust decision-making and graceful escalation for low-confidence results.
• Intelligent Tool Retrieval: Manage large tool catalogues efficiently; Needle automatically identifies and presents the top five most relevant tools per turn, optimizing performance.
• Bounded & Predictable Memory: Utilizes a sliding window with pinned KV sinks to maintain consistent memory usage, regardless of conversation length.
• Flexible Tool Definition: Define tools with simple decorators, detailed argument descriptions, or advanced constraints using needle.Field for precise control.
• Streamlined Data Extraction: Extract structured data into Pydantic models or raw JSON with the same powerful mechanism as tool calling.
• LoRA Fine-Tuning: Easily fine-tune the model with LoRA for customized performance, producing a single, deployable .cact file.
• System Facts Integration: Enhance context awareness by providing environment state (date, locale, device, battery, etc.) for more accurate and responsive interactions.

Typical Use Cases

• On-Device Assistants: Power intelligent agents directly on smartphones, smartwatches, and earbuds without relying on cloud connectivity for core functionalities.
• Smart Home Automation: Enable voice control and automation for a wide range of smart home devices, with minimal processing power required.
• Robotics & Embedded Systems: Implement sophisticated control and decision-making capabilities in robots and other embedded systems where resources are severely limited.
• Structured Data Extraction: Reliably extract specific information from text inputs for applications like form processing, log analysis, and data summarization on edge devices.
• Offline Functionality: Provide essential tool-calling and extraction features even in environments with intermittent or no network access.
• Specialized Device Control: Build deeply integrated applications that interact with device-specific APIs and functionalities efficiently.