Thinking Machines Releases Inkling 2 Mini Transformer Designed to Run Locally on Laptops
Thinking Machines Lab has launched Inkling 2 Mini, a 26.4-billion-parameter multimodal AI model that uses a novel looped architecture to match the reasoning capabilities of much larger systems. The open-weights model is designed to run entirely offline on consumer hardware like a 32-gigabyte MacBook Pro.
By Wei Zhang
Share this story
Thinking Machines Lab, the San Francisco artificial intelligence startup led by former OpenAI chief technology officer Mira Murati, has challenged the cloud-computing paradigm. On November 20, 2026, the company released a compact multimodal model engineered specifically to execute offline on consumer hardware. The release targets developers building autonomous agents.[1][2]
The release marks a deliberate shift in how AI laboratories extract reasoning performance from small hardware footprints. Rather than scaling parameter counts upward, the company rethought how a compact transformer spends its compute. The model ties Alibaba's dense 27-billion-parameter flagship on aggregate intelligence benchmarks.[1][2]
"A model that lives up to the promise of local deployment—not by stripping away agentic capability, but by rethinking how a compact transformer spends its compute," VentureBeat reported on November 21. The model has already drawn rapid support from local-AI runtimes.[1]
Inkling 2 Mini requires an approximately 20-gigabyte memory footprint to operate. This allows it to run comfortably on an Apple MacBook Pro with 32 gigabytes of unified memory or a single desktop graphics card. It frees developers from relying on expensive data center clusters.[1][2][3]
The Looped Architecture
The core innovation driving this efficiency is a looped transformer architecture. Standard transformer models process each token in a single, linear pass from the first layer to the last. If a problem requires more sequential reasoning steps, standard models must generate long chains of thought in text.[2]
Inkling 2 Mini departs from that standard decoder-only stack. It contains 24 layers of weights, but the middle 12 layers are reused. The model runs those middle layers three times in a loop with shared weights, giving the effective depth of a 48-layer model.[1][2]
"Today we are releasing Inkling 2 Mini, a fast, powerful model that's small enough to run on your laptop," Thinking Machines noted in its release. "Inkling 2 Mini is a looped transformer, running the middle layers of the model three times for every token."[2]
This design allows the model to apply more reasoning and reach the correct answer with fewer generated tokens. The system holds 26.4 billion total parameters. However, it operates as a sparse Mixture-of-Experts, routing each token to just eight of 128 available experts.[1][2]
As a result, it only activates 3.4 billion unique parameters on a standard pass. When accounting for the three loop passes through the shared middle layers, it uses 5.4 billion effective weight activations per token. This keeps the memory bandwidth requirements low enough for laptops.[2][4]
Benchmark Performance
On standardized evaluations, the model punches well above its weight class in agentic coding and tool use. It leads all open-weights models under 100 billion parameters on the DeepSWE v1.1 benchmark, scoring 48.2 percent. It also achieved 46.5 percent on the Vals Index v2.2.[1][2]
Artificial Analysis evaluated the model on its Intelligence Index, placing it at a score of 20. Inkling 2 Mini matches the performance of Qwen3.8-27B and Nanbeige4.3-33B. However, it achieves this with roughly five times and six times fewer active parameters, respectively.[2][4]
The looped design adds internal computation without adding output tokens. On a compute proxy measure multiplying active parameters by tokens, Inkling 2 Mini sits on the Pareto frontier. It uses approximately 2.8 times less compute than Qwen3.8-27B to achieve the same Intelligence Index score.[4]
This efficiency translates directly into speed and cost savings for developers. Artificial Analysis found that Inkling 2 Mini costs $0.59 and takes 9.0 minutes per Intelligence Index task. In contrast, Qwen3.8-27B costs $3.53 and requires 38.2 minutes to complete the same workload.[4]
Multimodal Capabilities
Beyond text generation, Inkling 2 Mini is natively multimodal. It accepts UTF-8 text alongside pixel-based images, optimally sized between 40 and 4,096 pixels. The vision encoder supports image and document understanding alongside traditional text-based agent tasks.[1][2]
The model supports a massive one-million-token context window. This allows developers to feed entire codebases, extensive documentation, or multiple high-resolution images into the prompt. The reasoning effort is adjustable, giving users control over how much compute is spent per task.[1][2]
Thinking Machines demonstrated these offline capabilities by running the model in airplane mode on a 48-gigabyte M5 Pro MacBook Pro. The system built a playable flight simulator in minutes. It also powered 20 persistent autonomous villagers simultaneously in a local town simulation.[1]
Precision and Interpretability
The model was trained using four-bit NVFP4 precision in the expert layers, bolstered by hardware acceleration on NVIDIA's GB300 platform. The final four layers ran in eight-bit MXFP8. Inkling 2 Mini's weights are deployed in the exact same numeric precision they were trained on.[2]
Because 20 layers remain in NVFP4 and four in MXFP8, there is no conversion loss from post-training quantization. This preserves the model's reasoning quality while keeping the file size compact. The downloadable GGUF format requires at least 18 gigabytes of system memory.[2][3]
The research firm Goodfire published an analysis of the model's internal mechanics on November 20. They noted that Inkling 2 Mini uses attention residuals rather than reading from a cumulative residual stream. Each layer reads from previous layer outputs using depth-wise attention over blocks.[5]
"The model's 24 layers are divided into 4 blocks. That means there is no single residual stream running through the network," Goodfire explained. The cross-loop attention allows information to pass across the loops, creating a highly complex but efficient routing system.[5]
Open Source Ecosystem
Thinking Machines released Inkling 2 Mini under the permissive Apache 2.0 license. This open-weights approach allows developers to fine-tune the model and integrate it into commercial products without paying licensing fees. The weights are available for download through the Hugging Face repository.[2][3]
Infrastructure partners quickly followed the release with local speed benchmarks and compatibility updates. The model supports inference using frameworks like SGLang, vLLM, and NVIDIA Dynamo. The Ollama library also added immediate support for running the model locally.[3][6]
The optimization platform Unsloth published quantized builds alongside llama.cpp support. "Run Inkling 2 Mini, the strongest 26.4B model that runs on 16GB RAM," Unsloth announced. "Thinking Machines' Apache 2.0 MoE model uses a looped transformer arch & scores 48.2% on DeepSWE v1.1, the best for size."[1]
The release fundamentally alters the trajectory of local artificial intelligence. By proving that architectural innovations like looped layers can substitute for raw parameter scale, Thinking Machines has decoupled advanced reasoning from data center hardware. Developers can now run frontier-level coding agents entirely offline.[7]
Key points
- Thinking Machines Lab released Inkling 2 Mini, a 26.4-billion-parameter multimodal model designed to run offline on consumer hardware.
- The model utilizes a looped transformer architecture, running its middle 12 layers three times to achieve 48 effective layer applications.
- Inkling 2 Mini matches the performance of Alibaba's Qwen3.8-27B on intelligence benchmarks while using 2.8 times less compute.
- The open-weights model requires a 20-gigabyte memory footprint, allowing it to operate on a 32-gigabyte MacBook Pro or a single desktop GPU.
Unanswered questions
- How the looped transformer architecture will scale to models exceeding 100 billion parameters.
- Whether the increased compute time per token in looped layers will introduce latency bottlenecks in real-time voice applications.
- Local AI Advocates
- Prioritize data privacy and zero-cost inference on consumer hardware.
- AI Efficiency Researchers
- Focus on maximizing intelligence per joule through architectural innovations.
- Enterprise Developers
- Seek to deploy agentic coding models without sending proprietary code to cloud providers.
Perspectives this story doesn't cover
- Cloud Infrastructure Providers
- Hardware Manufacturers
Sources
[1]VentureBeatEnterprise DevelopersThinking Machines wows with Inkling 2 Mini looped transformer model that runs locally on laptops
Read on VentureBeat →
[2]Thinking Machines LabEnterprise DevelopersIntroducing Inkling 2 Mini
Read on Thinking Machines Lab →
[3]Hugging FaceLocal AI AdvocatesInkling 2 Mini Model Card
Read on Hugging Face →
[4]Artificial AnalysisAI Efficiency ResearchersInkling 2 Mini: Intelligence, Performance & Price Analysis
Read on Artificial Analysis →
[5]GoodfireAI Efficiency ResearchersInterpreting Inkling 2 Mini
Read on Goodfire →
[6]OllamaLocal AI Advocatesinkling-2-mini
Read on Ollama →
[7]Factlen Editorial TeamEnterprise DevelopersSynthesis by Factlen editorial team
Read on Factlen Editorial Team →
More in Technology
See all →AI Infrastructure
Nvidia and PreussenElektra Scout Decommissioned German Nuclear Plants for AI Hubs
5 sources
Synthetic Training Data
Unitree Unveils Self-Evolving Robot Software to Challenge Tesla Optimus
6 sources
AI Infrastructure
Marvell Unveils 32TB Optical Memory Appliance to Break AI GPU Bottlenecks
4 sources
AI Infrastructure
Meta Details Prometheus Supercluster Architecture Scaled for One Million GPUs
7 sources
Comments
Every angle. Every day.
Get Technology stories with full source coverage and perspective breakdowns, free every day.




