Needle 3 Automation Foundation Model For Tiny Devices
One set of weights, every depth from 2 to 20 layers a model of its own: an intelligence ladder.
Model
Intelligence laddering. Every layer of Needle 3 is a sub-network with monotonically increasing capacity. Developers can choose the right size from the 2-layer (2L) subnetwork to 20 layers (20L). Each subnetwork is amenable to fine-tuning, such that 4L can match DeepSeek V4 Flash when tuned on downstream tasks for one epoch. Intelligence laddering produces 9 to 29 MB CQ2-bit binaries and supports a wide range of tiny devices.
Laddered Simple Attention Networks
Get started
Install the Python package. The inference engine is fetched once from Hugging Face and cached; there is nothing else to build.
Needle reads your tool descriptions to decide what to call and how to fill arguments, so describing them well is the whole game.
Simple : decorate a function. The signature gives the argument types, the docstring is the tool description, and run() completes the loop: the model picks the call, Needle executes your function, feeds the result back, and returns the final response with the executed tool results attached as results .
import needle @needle.tool def get_weather(city: str): "Get the current weather for a city." return { "city" : city, "temp_c" : 27 , "sky" : "clear" } agent = needle.Needle(tools=[get_weather]) print(agent.run( "what's it like in Lagos right now?" )[ "results" ]) # [{'city': 'Lagos', 'temp_c': 27, 'sky': 'clear'}]
Route by pattern : when a description cannot enumerate every phrasing, give a tool triggers , regular expressions matched against each request. A match restricts the decode to the matched tools and requires a call, so the request reaches the tool you named instead of being refused or misrouted, and the call ships even below the confidence floor. A match restricts the whole turn, so a catch-all should exclude the nouns other tools own, e.g. ^(?![\s\S]*\b(lights?|doors?)\b)[\s\S]*\b(turn|switch)\b[\s\S]*\b(on|off)\b ; then "switch the fan on and dim the kitchen lights" still reaches both tools.
from typing import Literal @needle.tool (triggers=[r "\b(turn|switch|power|flip)\b.*\b(on|off)\b" , r "\btoggle\b" ]) def control_device(device: str, action: Literal[ "on" , "off" , "toggle" ]): "Switch or toggle any named smart-home device." return { "device" : device, "action" : action} agent = needle.Needle(tools=[control_device, get_weather]) agent.complete( "toggle the garage door" ) # function_calls [{"name": "control_device", "arguments": {"device": "garage door", "action": "toggle"}}]
Extraction : to pull structured data out of text, declare the shape and call extract() . Pass a Pydantic model and you get a typed object back.
from pydantic import BaseModel class Invoice(BaseModel): vendor: str total: float due_date: str invoice = needle.extract( "Invoice from Acme Corp, $1,200.00, due 2026-09-01" , Invoice) print(invoice.vendor, invoice.total) # -> Acme Corp 1200.0
{ "type" : "call" , "success" : true , "error" : null , "error_code" : null , "function_calls" : [ { "name" : "set_lights" , "arguments" : { "room" : "living room" , "on" : true , "brightness" : 30 } } ], "reasoning" : "'living room' -> room; 'dim' -> on true, brightness 30" , "confidence" : 0.94 , "prefill_tps" : 4300.0 , "decode_tps" : 850.0 , "peak_ram_mb" : 28.5 }
Confidence gating and routing : every response carries a confidence score from a calibrated head, and the engine already applies a floor of 0.1. Below it, the call is withheld into suppressed_calls and function_calls is empty. Above it, the score is yours to route on: act at once when it is high, show the call and ask when it is middling, and treat an empty result as a refusal. A tool with triggers always produces a call for a matching request, so the score is what tells you whether to run it or confirm it.
r = agent.complete(user_text) calls = r[ "function_calls" ] held = r[ "suppressed_calls" ] if calls and r[ "confidence" ] >= 0.7 : execute(calls) # sure: act elif calls or held: confirm(calls or held, r[ "reasoning" ]) # unsure: show the call, ask else : say( "I can't do that here" ) # nothing to do: refuse
Writing tools : the model reads a schema literally, so a narrow tool with a plain description beats a broad one. One tool per action, described by the actions it covers ("Turn a room's lights on or off") rather than a category. Name enum options after what a user says ( action: ["increase", "decrease"] ) and keep synonyms in the description. Give a required argument a default when a request may leave it out; a required argument with no default and no evidence in the request is withheld rather than guessed. Put value formats in descriptions ( "City, ST" , "e.g. T-1042" ). Add triggers to intents that must always reach a tool, and keep the toolset per turn small, since every extra tool is a chance to misroute.
Fine-tune : the Python package is the quick path. LoRA on the frozen base at the full 20 layers, then a 4-bit .cact of any subnetwork that runs on the same engine.
needle finetune data.jsonl --epochs 10 --out adapter.safetensors needle build --lora adapter.safetensors --out tuned.cact needle build --lora adapter.safetensors --platform linux-arm64 --layers 2 --out ./device
The guides go deeper: designing tools , confidence , extraction , fine-tuning , the Python reference , supported devices , the .cact format and porting Needle . Source is on GitHub .
Cactus Platform
Needle was designed to be customised. Its capacity is a ladder, and a subnetwork as small as 2 layers, fine-tuned on one product's tools, runs optimally on devices far smaller than the full model needs. Constraining the capacity to a narrow, well-defined task is what lets it reach frontier-level accuracy there: fine-tuning on DroidCall lifts every subnetwork by 18 to 36 points, and from 4 layers up the tuned subnetwork passes DeepSeek V4 Flash, starting at 29M parameters (Figure 3).
Every subnetwork, fine-tuned on the platform
The Cactus Platform is the full path: Cactus datasets, the 2-bit quantisation behind the shipped model, evaluation design and tracking, full-depth fine-tunes and dataset management, all on our infrastructure and training pipeline, no need to build your own.
Deploy
Every deployment target ships a prebuilt engine under 1 MB that loads the needle3.cact weights at start. needle build fetches the engine for a platform and puts the weights beside it, at the full 20 layers or any smaller subnetwork:
# engine, header and weights for this Mac needle build --platform macos-arm64 # an 8-layer subnetwork for a Pi needle build --platform linux-arm64 --layers 8 --out ./pi # a tuned archive needle build --lora adapter.safetensors --out tuned.cact