Cactus has released Needle 3, a foundation model that packs a full AI assistant into a single 8-29 MB binary. The company claims it can match DeepSeek V4 Flash on extraction tasks and beat models ten times its size on mobile tool calls. The model runs on a Raspberry Pi and trades general chat capacity for focused performance.
The key facts are simple to state:
- Size: 8-29 MB CQ2-bit binary, with developers choosing from a 2-layer (2L) subnetwork to a 20-layer (20L) network
- Training data: 360B tokens of proprietary structured dataset
- Inference: 400-4k tokens/s decode and 1-10k tokens/s prefill on a Raspberry Pi 5
What Needle 3 Actually Does
Needle 3 handles three main jobs: tool calls, structured extraction, and text embeddings. The model is built on a Simple Attention Network architecture and quantized to CQ2 bit depth. That compression is the whole pitch: a full AI assistant that fits on a microcontroller or a wearable without needing a cloud connection.
The company frames the design as intelligence laddering. Each layer of the model is a sub-network with increasing capacity. Developers pick the right size from the smallest 2L subnetwork to the largest 20L subnetwork. Fine-tuning is supported, so a four-layer model can match DeepSeek V4 Flash after one epoch of training on downstream tasks.
Tool Calls and Extraction
The model’s main trick is picking the right function to call based on what the user asks. The example given involves a weather tool defined with a docstring and a function signature. The assistant recognizes the intent, fills in the arguments, and hands the call back to the developer’s code.
python
import needle @needle.tool def get_weather(city: str): "Get the current weather for a city." return {"city": city, "temp_c": 27, "sky": "clear"} agent = needle.Needle(tools=[get_weather]) print(agent.run("what's it like in Lagos right now?")["results"]) # [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]
If the model cannot find a matching function, it returns an empty list instead of guessing. That behavior is the company’s explicit promise: no guesses when no tool covers the request.
Extraction works in the opposite direction. Instead of calling a function, the model takes messy text and produces a structured output. The company says the decode grammar guarantees the output parses, meaning the result always fits the declared shape.
python
from pydantic import BaseModel class Invoice(BaseModel): vendor: str total: float due_date: str invoice = needle.extract("Invoice from Acme Corp, $1,200.00, due 2026-09-01", Invoice) print(invoice.vendor, invoice.total) # -> Acme Corp 1200.0
The example shows an invoice being pulled from a string into a typed object. The model handles invoices, bookings, notifications, and forms. The company also notes that the extraction framework generalized well to classification problems.
Routing and Confidence
Every response includes a confidence score from a calibrated head. The engine applies gating automatically, though the exact threshold is not stated. The company emphasizes that describing tools well is the whole game.
The routing system lets developers match requests to specific functions using regular expressions. A match restricts the decode to the matched tools and requires a call, so the request reaches the tool you named instead of being refused or misrouted. The example shows a trigger pattern that excludes common words other tools might own.
python
from typing import Literal @needle.tool(triggers=[r"\b(turn|switch|power|flip)\b.*\b(on|off)\b", r"\btoggle\b"]) def control_device(device: str, action: Literal["on", "off", "toggle"]): "Switch or toggle any named smart-home device." return {"device": device, "action": action} agent = needle.Needle(tools=[control_device, get_weather]) agent.complete("toggle the garage door") # function_calls [{"name": "control_device", "arguments": {"device": "garage door", "action": "toggle"}}]
What It Runs On
The Raspberry Pi 5 is the inference platform. The model runs at 400-4k tokens per second for decoding and 1-10k tokens per second for prefill. The company says the model supports a wide range of tiny devices, from microcontrollers to smart home hubs.
The model fits on devices with limited storage. The company’s examples cover phones, wearables, robots, smart homes, cars, and computers. The embeddings allow local search and alert merging, so a watch can merge near-duplicate alerts without sending anything to the cloud.
Getting Started
Installation is straightforward. The Python package pulls the inference engine from Hugging Face once and caches it. There is nothing else to build. The company calls this a foundation model for mobile, wearables, robots, smart home, automotive, and microcontrollers.
The deployment loop is simple. The model reads your tool descriptions, picks the right call, fills the arguments, and executes the function. The result comes back as JSON with the executed tool results attached.
The example workflow completes the loop: the model picks the call, Needle executes the function, feeds the result back, and returns the final response with the executed tool results attached as results.
Why This Matters
The claim is bold: a model one tenth the size of its competitors matches their extraction performance and beats them on tool calls. The calibration and gating mean the model knows when it does not know. That is a useful trait for embedded systems, where wrong answers are worse than no answers.
The company’s framing is practical. This is not a general conversational model. It is an automation model that maps natural language to specific actions. The tool-call interface is the whole point.
The Raspberry Pi 5 benchmark shows the model running at 400-4k tokens per second for decoding and 1-10k tokens per second for prefill. A model that runs at those speeds on a Raspberry Pi can support real-time interactions on a device that costs under $100. That is a low barrier to entry.
The company’s examples show the model in action across every category it supports. A smart home assistant can dim the bedroom and lock up in one command. A robot can clean the kitchen while leaving the bedroom untouched. A phone assistant can make an album from last weekend’s photos or dim the screen. A wearable can read a card charge into merchant, amount, and date. An AR glass can navigate from a short request. A car can handle climate, media, navigation, and calls from the cabin. A computer can take plain-English commands like drafting a mail, starting a timer, copying an address, or opening a tab.
The embeddings enable semantic search over notes, messages, and documents. A query matched to the closest of hundreds of tools. Near-duplicate alerts merged on a watch.
Needle 3 is a foundation model for mobile, wearables, robots, smart home, automotive, and microcontrollers. It is a model that fits on a Raspberry Pi and beats models ten times its size on mobile tool calls. It is a model that trades general chat capacity for focused performance. It is a model that knows exactly what it is supposed to do and does it well.
Source material: “Show HN: Cactus Needle 3: 8-29MB automation models can match DeepSeek V4 Flash,” cactuscompute.com.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.