Skip to main content
Local AIThinking Machines Lab· 6 min read· in Technology

Thinking Machines Releases Inkling 2 Mini Transformer Designed to Run Locally on Laptops

Thinking Machines Lab has launched Inkling 2 Mini, a 26.4-billion-parameter multimodal AI model that uses a novel looped architecture to match the reasoning capabilities of much larger systems. The open-weights model is designed to run entirely offline on consumer hardware like a 32-gigabyte MacBook Pro.

By Wei Zhang

Share this story

Listen

Thinking Machines Lab, the San Francisco artificial intelligence startup led by former OpenAI chief technology officer Mira Murati, has challenged the cloud-computing paradigm. On November 20, 2026, the company released a compact multimodal model engineered specifically to execute offline on consumer hardware. The release targets developers building autonomous agents.[1][2]

The release marks a deliberate shift in how AI laboratories extract reasoning performance from small hardware footprints. Rather than scaling parameter counts upward, the company rethought how a compact transformer spends its compute. The model ties Alibaba's dense 27-billion-parameter flagship on aggregate intelligence benchmarks.[1][2]

"A model that lives up to the promise of local deployment—not by stripping away agentic capability, but by rethinking how a compact transformer spends its compute," VentureBeat reported on November 21. The model has already drawn rapid support from local-AI runtimes.[1]

Inkling 2 Mini requires an approximately 20-gigabyte memory footprint to operate. This allows it to run comfortably on an Apple MacBook Pro with 32 gigabytes of unified memory or a single desktop graphics card. It frees developers from relying on expensive data center clusters.[1][2][3]

Inkling 2 Mini achieves its small footprint by activating only a fraction of its parameters per token.

The Looped Architecture

The core innovation driving this efficiency is a looped transformer architecture. Standard transformer models process each token in a single, linear pass from the first layer to the last. If a problem requires more sequential reasoning steps, standard models must generate long chains of thought in text.[2]

Inkling 2 Mini departs from that standard decoder-only stack. It contains 24 layers of weights, but the middle 12 layers are reused. The model runs those middle layers three times in a loop with shared weights, giving the effective depth of a 48-layer model.[1][2]

"Today we are releasing Inkling 2 Mini, a fast, powerful model that's small enough to run on your laptop," Thinking Machines noted in its release. "Inkling 2 Mini is a looped transformer, running the middle layers of the model three times for every token."[2]

This design allows the model to apply more reasoning and reach the correct answer with fewer generated tokens. The system holds 26.4 billion total parameters. However, it operates as a sparse Mixture-of-Experts, routing each token to just eight of 128 available experts.[1][2]

As a result, it only activates 3.4 billion unique parameters on a standard pass. When accounting for the three loop passes through the shared middle layers, it uses 5.4 billion effective weight activations per token. This keeps the memory bandwidth requirements low enough for laptops.[2][4]

Benchmark Performance

On standardized evaluations, the model punches well above its weight class in agentic coding and tool use. It leads all open-weights models under 100 billion parameters on the DeepSWE v1.1 benchmark, scoring 48.2 percent. It also achieved 46.5 percent on the Vals Index v2.2.[1][2]

Artificial Analysis evaluated the model on its Intelligence Index, placing it at a score of 20. Inkling 2 Mini matches the performance of Qwen3.8-27B and Nanbeige4.3-33B. However, it achieves this with roughly five times and six times fewer active parameters, respectively.[2][4]

The looped design adds internal computation without adding output tokens. On a compute proxy measure multiplying active parameters by tokens, Inkling 2 Mini sits on the Pareto frontier. It uses approximately 2.8 times less compute than Qwen3.8-27B to achieve the same Intelligence Index score.[4]

Inkling 2 Mini leads open-weights models under 100 billion parameters on software engineering benchmarks.

This efficiency translates directly into speed and cost savings for developers. Artificial Analysis found that Inkling 2 Mini costs $0.59 and takes 9.0 minutes per Intelligence Index task. In contrast, Qwen3.8-27B costs $3.53 and requires 38.2 minutes to complete the same workload.[4]

Multimodal Capabilities

Beyond text generation, Inkling 2 Mini is natively multimodal. It accepts UTF-8 text alongside pixel-based images, optimally sized between 40 and 4,096 pixels. The vision encoder supports image and document understanding alongside traditional text-based agent tasks.[1][2]

The model supports a massive one-million-token context window. This allows developers to feed entire codebases, extensive documentation, or multiple high-resolution images into the prompt. The reasoning effort is adjustable, giving users control over how much compute is spent per task.[1][2]

Thinking Machines demonstrated these offline capabilities by running the model in airplane mode on a 48-gigabyte M5 Pro MacBook Pro. The system built a playable flight simulator in minutes. It also powered 20 persistent autonomous villagers simultaneously in a local town simulation.[1]

Precision and Interpretability

The model was trained using four-bit NVFP4 precision in the expert layers, bolstered by hardware acceleration on NVIDIA's GB300 platform. The final four layers ran in eight-bit MXFP8. Inkling 2 Mini's weights are deployed in the exact same numeric precision they were trained on.[2]

Because 20 layers remain in NVFP4 and four in MXFP8, there is no conversion loss from post-training quantization. This preserves the model's reasoning quality while keeping the file size compact. The downloadable GGUF format requires at least 18 gigabytes of system memory.[2][3]

The research firm Goodfire published an analysis of the model's internal mechanics on November 20. They noted that Inkling 2 Mini uses attention residuals rather than reading from a cumulative residual stream. Each layer reads from previous layer outputs using depth-wise attention over blocks.[5]

The looped architecture reuses the middle 12 layers three times, generating 48 effective layer applications.

"The model's 24 layers are divided into 4 blocks. That means there is no single residual stream running through the network," Goodfire explained. The cross-loop attention allows information to pass across the loops, creating a highly complex but efficient routing system.[5]

Open Source Ecosystem

Thinking Machines released Inkling 2 Mini under the permissive Apache 2.0 license. This open-weights approach allows developers to fine-tune the model and integrate it into commercial products without paying licensing fees. The weights are available for download through the Hugging Face repository.[2][3]

Infrastructure partners quickly followed the release with local speed benchmarks and compatibility updates. The model supports inference using frameworks like SGLang, vLLM, and NVIDIA Dynamo. The Ollama library also added immediate support for running the model locally.[3][6]

The optimization platform Unsloth published quantized builds alongside llama.cpp support. "Run Inkling 2 Mini, the strongest 26.4B model that runs on 16GB RAM," Unsloth announced. "Thinking Machines' Apache 2.0 MoE model uses a looped transformer arch & scores 48.2% on DeepSWE v1.1, the best for size."[1]

The model's efficiency translates to significantly lower compute costs per task compared to similarly sized models.

The release fundamentally alters the trajectory of local artificial intelligence. By proving that architectural innovations like looped layers can substitute for raw parameter scale, Thinking Machines has decoupled advanced reasoning from data center hardware. Developers can now run frontier-level coding agents entirely offline.[7]

Key points

  • Thinking Machines Lab released Inkling 2 Mini, a 26.4-billion-parameter multimodal model designed to run offline on consumer hardware.
  • The model utilizes a looped transformer architecture, running its middle 12 layers three times to achieve 48 effective layer applications.
  • Inkling 2 Mini matches the performance of Alibaba's Qwen3.8-27B on intelligence benchmarks while using 2.8 times less compute.
  • The open-weights model requires a 20-gigabyte memory footprint, allowing it to operate on a 32-gigabyte MacBook Pro or a single desktop GPU.

Unanswered questions

  • How the looped transformer architecture will scale to models exceeding 100 billion parameters.
  • Whether the increased compute time per token in looped layers will introduce latency bottlenecks in real-time voice applications.
Local AI Advocates 35%AI Efficiency Researchers 35%Enterprise Developers 30%
Local AI Advocates
Prioritize data privacy and zero-cost inference on consumer hardware.
AI Efficiency Researchers
Focus on maximizing intelligence per joule through architectural innovations.
Enterprise Developers
Seek to deploy agentic coding models without sending proprietary code to cloud providers.

Perspectives this story doesn't cover

  • Cloud Infrastructure Providers
  • Hardware Manufacturers

Sources

Source coverage

7 outlets

3 viewpoints surfaced

Local AI Advocates 35%AI Efficiency Researchers 35%Enterprise Developers 30%
  1. [1]VentureBeatEnterprise Developers

    Thinking Machines wows with Inkling 2 Mini looped transformer model that runs locally on laptops

    Read on VentureBeat →
  2. [2]Thinking Machines LabEnterprise Developers

    Introducing Inkling 2 Mini

    Read on Thinking Machines Lab →
  3. [3]Hugging FaceLocal AI Advocates

    Inkling 2 Mini Model Card

    Read on Hugging Face →
  4. [4]Artificial AnalysisAI Efficiency Researchers

    Inkling 2 Mini: Intelligence, Performance & Price Analysis

    Read on Artificial Analysis →
  5. [5]GoodfireAI Efficiency Researchers

    Interpreting Inkling 2 Mini

    Read on Goodfire →
  6. [6]OllamaLocal AI Advocates

    inkling-2-mini

    Read on Ollama →
  7. [7]Factlen Editorial TeamEnterprise Developers

    Synthesis by Factlen editorial team

    Read on Factlen Editorial Team →
Listen

Comments

Stay informed

Every angle. Every day.

Get Technology stories with full source coverage and perspective breakdowns, free every day.