Cactus Compute has released Needle 2, an open 45-million-parameter model for tool calling, device use and structured extraction. What stands out is not its capability but its size.

The numbers

The entire model ships as a single 14MB binary and runs a full session in about 28MB of RAM. Weights are trained and deployed at CQ2-bit using Cactus Quants, and the model is sealed inside the company's own C++ engine, so there is no runtime to install and no download at inference time.

Reported decode throughput:

  • 500 tokens per second on a Raspberry Pi 5
  • 400-1,500 tokens per second on Meta Quest 3S and Apple Vision Pro
  • 300-700 tokens per second on sub-$200 phones

The design premise

The team states its reasoning plainly: mapping a messy sentence onto a typed function signature needs no world knowledge and no open-ended prose.

That framing explains why 45 million parameters are enough here, and why the model targets hardware with no GPU and no NPU. When the task is defined narrowly, the model can stay narrow too.

The architecture

Needle 2 uses what the team calls a Simple Attention Network. The recipe replaces the feed-forward network with a Hadamard MLP, keeps GQA attention, adds engram key-value memory from hashed n-gram tables, and uses multi-lane hyper-connections. The network is 27 layers and 512 wide.

Pretraining used a proprietary 115-billion-token corpus, with 38 billion tokens of post-training. The model spends 70 MFLOPs per token, with 35 million of its 45 million parameters matmul-active.

The comparison points are striking: LFM2.5-230M spends 460 MFLOPs per token and FunctionGemma 270M spends 540, while Apple's foundation model sits near 6,000.

Where it runs

Needle 2 ships as prebuilt binaries and a static library for macOS, Linux (x86-64, ARM64, ARMv7, RISC-V, MIPS32el), Windows, Android, iOS, watchOS, tvOS and WebAssembly.

According to the company, Pebble already runs Needle locally in the Index 01 app for offline voice actions.

The engine's behaviour is unusual as well: weights never decompress into RAM. The 2-bit codes expand inside vector registers and fuse into integer dot products, so the arithmetic path stays int8. A single binary probes the CPU at startup and selects the right path.

Who it is for

The audience for a model at this scale is not cloud-first software companies but teams shipping on constrained hardware: wearable and IoT startups, consumer-electronics makers, robotics teams and large device manufacturers that need an offline fallback.

The applications follow from that: voice-to-action on screenless devices, offline appliance control, receipt and invoice field extraction, and local routing that escalates to the cloud only on low confidence.

Regulated settings where audio cannot leave the device also belong on that list. There, a model that runs without ever touching the network is not a performance question but a compliance one.