How Inkling 2 Mini Brings Cutting-Edge AI To Local Setups
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: How Inkling 2 Mini Brings Cutting-Edge AI To Local Setups on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model that the company describes as having about 26 billion total parameters and activating about 3 billion per token. It supports several inference runtimes and includes a research version with adjustable layer passes, but the supplied information does not independently verify benchmark claims or specify practical hardware requirements.

Thinking Machines Lab has released Inkling 2 Mini, a multimodal model designed for local and other inference setups that the company says has about 26 billion total parameters and activates about 3 billion per token, as detailed in the original analysis. The release supports several inference runtimes and includes a research checkpoint with adjustable layer passes. The company reports competitive benchmark results, but the material available for this article does not include independent validation or specific hardware requirements.

According to the release, Inkling 2 Mini accepts text and image inputs and has a claimed 1 million-token context window. Its architecture uses 24 physical decoder layers: the first six and last six run once, while 12 middle layers are reused in three passes. This produces 48 logical layer visits per token, the company says, without a separate set of weights for each visit.

The model is a mixture of experts. At each layer visit, its routing system selects eight experts from a pool of 128, according to the technical description. Thinking Machines says it is distributing expert weights in NVFP4 and MXFP8 mixed-precision formats and includes a DSpark drafter intended to speed generation with a small memory cost. These are design and performance claims from the company, not independent measurements in the supplied material. For context on local AI hardware, see whether 64 GB can help a Mac mini run local AI.

The fixed-pass model is available through Transformers, llama.cpp and MLX, while SGLang has day-one inference support for datacenter deployments. A separate research artifact, TML-d24-varloop-research, allows users to change the number of passes. The company says fewer passes may make inference faster, with a possible quality trade-off. Readers exploring local workloads can also consult this guide to AI mini PCs.

At a glance
announcementWhen: Announced in November 2026, according t…
The developmentThinking Machines Lab released Inkling 2 Mini, with local-inference runtime support and a research checkpoint that can adjust repeated-layer passes.
AI Models · Briefing

Inkling 2 Mini for Local AI

Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.

26B
Total parameters, about
3B
Parameters active per token, about
1M
Token context window

A loop reuses model layers

The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.

1
Start with 24 layers

The first six and last six physical decoder layers run once.

2
Repeat the middle

The same 12 middle layers run in three passes, for 48 logical layer visits per token.

3
Route to experts

At each visit, the model selects eight experts from a pool of 128.

Options for inference

The release offers several runtimes and a research checkpoint with adjustable loop passes.

Fixed model

Multiple runtimes

Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.

Research variant

Fewer passes

The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.

Claims need outside checks

The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.

Reported results
  • Leads in 32 of 48 benchmarks against named models.
  • Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
  • Trails on AutomationBench-AA v1.0.6.
Still unclear
  • Complete memory requirements and device-specific speed.
  • Independent image and long-context performance.
  • Quality at reduced loop counts.

What local users should assess

Runtime availability is a starting point; actual usefulness depends on hardware and workload.

Memory useGeneration speedImage handlingLong-context behaviorOutput quality
CaveatThe release does not specify a complete file size or minimum hardware configuration.
The takeaway
Inkling 2 Mini brings an interesting local inference design, but independent tests and hardware-specific measurements will show whether its reported promise holds in practice.

Local AI Access Depends on Real Hardware

Inkling 2 Mini adds another multimodal option for local inference, with ports for several runtimes used across personal-computing and datacenter environments. Its repeated-layer design reuses weights, and expert routing selects only part of the available expert pool for each token. Those choices are intended to combine model capacity with more selective computation.

But the architecture alone does not show whether the model will run comfortably on a particular desktop, mini PC or graphics card. Users need measured memory use and generation speeds across specific hardware before they can judge the practical cost of running it locally. Image understanding, long-context behavior and output quality also matter; the supplied release information does not provide independent results on those questions.

The adjustable-pass research checkpoint could also be useful to developers studying the relationship between compute and quality. If fewer passes reduce latency without an unacceptable drop in results, that could offer a way to tune inference for different setups. That outcome remains a possibility, not an established result in the material provided.

Products Worth Considering

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)

As an affiliate, we earn on qualifying purchases.

How Inkling Reuses Its Layers

The key architectural choice is the reuse of the 12 middle physical layers. They run three times, rather than having a distinct set of weights at each of the model’s 48 logical layer visits. The company says learned messages carry information between passes, and a learned gate controls how much information is passed forward. Its fixed model uses three passes; the research checkpoint can be set to one or two.

Thinking Machines compares Mini with its Inkling 2 Small model. Both execute 48 layer visits per token, but Mini has a hidden width of 2,304, compared with 4,608 for Small, and routes among 128 experts per layer rather than 384. The supplied material describes Mini as substantially smaller, but does not give a complete file size or minimum configuration. It also notes earlier work on looping architectures, including Nanbeige.

The release says each pass through the repeated layers begins with the same input from the first six layers, with learned messages conveying information between passes. It also says each visit retains its own key-value and short-convolution state. Reusing layer weights therefore does not mean the model skips computation across the repeated visits.

“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”

— Thinking Machines Lab, in the supplied release material

Benchmark and Hardware Questions Remain

Thinking Machines calls Inkling 2 Mini state of the art in its class on the Artificial Analysis Intelligence Index v5 and says it leads in 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. The company also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0, and a lower result on AutomationBench-AA v1.0.6. These comparisons are company-reported; the supplied material does not contain the full score table or independent replication.

It remains unclear how much memory the complete model requires, what generation speeds users can expect on consumer devices, or how well image inputs and the claimed million-token context perform in outside testing. The quality of the one- and two-pass research settings is also not fully established here. The full model card may contain further technical details, but it was not included in the source material provided.

Independent Tests Will Clarify Trade-Offs

The next useful evidence would include the full model card, reproducible benchmark scores and hands-on reports that name the hardware and runtime used. Tests of memory use, generation speed, image tasks and long-context performance would help users compare the model with alternatives on like-for-like terms.

Further evaluation of the research checkpoint could show how changing the number of passes affects both latency and output quality. Until such details are available, the release establishes that the model and runtime support have been announced, but not how well it will perform on a given local setup.

Key Questions

What is Inkling 2 Mini?

It is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company says it accepts text and image inputs and has about 26 billion total parameters, with about 3 billion activated per token.

Can Inkling 2 Mini run locally?

The release lists support for Transformers, llama.cpp and MLX, which can be used for inference in different environments. SGLang is listed for datacenter deployments. The supplied information does not specify minimum hardware or confirm performance on particular consumer devices.

What does the research checkpoint change?

The TML-d24-varloop-research artifact allows the number of passes through the reused middle layers to be adjusted. Thinking Machines says fewer passes may increase speed, but could reduce quality; the supplied material does not provide independent results quantifying that trade-off.

Are the reported benchmark results independently verified?

Not in the material provided. Thinking Machines reports several benchmark comparisons, but the full score table and independent replication were not included.

What should users check before trying it?

Users will need to check the model files, runtime compatibility and hardware demands. Independent measurements of memory use, speed and task quality would help establish whether it fits a particular local setup.

Primary source: Hugging Face · via ThorstenMeyerAI.com

You May Also Like

Critical PHP Flaws, PHP2Shell, And The Dutch Government Websites Going Offline

PHP released fixes for two critical PHP2Shell flaws. Dutch officials took several government websites offline as a precaution, not after confirmed breaches.

How To Think About Privacy With XREAL AURA And Gemini

XREAL AURA launched with Gemini view sharing. Here is what the glasses can see, what Google says about processing, and what remains unclear.

Why AI Agents, Tests And Trust Belong In The Same Conversation

UK and Anthropic reports detail unauthorized online actions during an AI cyber test and raise questions about safeguards, evaluation and trust.

Questions To Ask Before Choosing From The 10 Best Network Attached Storage Devices For 2027

Compare 10 NAS devices by drive bays, included storage, expansion, setup needs and backup plans before choosing one for home or office use.