🔍 Read the full analysis: How Inkling 2 Mini Brings Cutting-Edge AI To Local Setups on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model that the company describes as having about 26 billion total parameters and activating about 3 billion per token. It supports several inference runtimes and includes a research version with adjustable layer passes, but the supplied information does not independently verify benchmark claims or specify practical hardware requirements.
Thinking Machines Lab has released Inkling 2 Mini, a multimodal model designed for local and other inference setups that the company says has about 26 billion total parameters and activates about 3 billion per token, as detailed in the original analysis. The release supports several inference runtimes and includes a research checkpoint with adjustable layer passes. The company reports competitive benchmark results, but the material available for this article does not include independent validation or specific hardware requirements.
According to the release, Inkling 2 Mini accepts text and image inputs and has a claimed 1 million-token context window. Its architecture uses 24 physical decoder layers: the first six and last six run once, while 12 middle layers are reused in three passes. This produces 48 logical layer visits per token, the company says, without a separate set of weights for each visit.
The model is a mixture of experts. At each layer visit, its routing system selects eight experts from a pool of 128, according to the technical description. Thinking Machines says it is distributing expert weights in NVFP4 and MXFP8 mixed-precision formats and includes a DSpark drafter intended to speed generation with a small memory cost. These are design and performance claims from the company, not independent measurements in the supplied material. For context on local AI hardware, see whether 64 GB can help a Mac mini run local AI.
The fixed-pass model is available through Transformers, llama.cpp and MLX, while SGLang has day-one inference support for datacenter deployments. A separate research artifact, TML-d24-varloop-research, allows users to change the number of passes. The company says fewer passes may make inference faster, with a possible quality trade-off. Readers exploring local workloads can also consult this guide to AI mini PCs.
Inkling 2 Mini for Local AI
Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.
A loop reuses model layers
The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.
The first six and last six physical decoder layers run once.
The same 12 middle layers run in three passes, for 48 logical layer visits per token.
At each visit, the model selects eight experts from a pool of 128.
Options for inference
The release offers several runtimes and a research checkpoint with adjustable loop passes.
Multiple runtimes
Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.
Fewer passes
The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.
Claims need outside checks
The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.
- Leads in 32 of 48 benchmarks against named models.
- Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
- Trails on AutomationBench-AA v1.0.6.
- Complete memory requirements and device-specific speed.
- Independent image and long-context performance.
- Quality at reduced loop counts.
What local users should assess
Runtime availability is a starting point; actual usefulness depends on hardware and workload.
Local AI Access Depends on Real Hardware
Inkling 2 Mini adds another multimodal option for local inference, with ports for several runtimes used across personal-computing and datacenter environments. Its repeated-layer design reuses weights, and expert routing selects only part of the available expert pool for each token. Those choices are intended to combine model capacity with more selective computation.
But the architecture alone does not show whether the model will run comfortably on a particular desktop, mini PC or graphics card. Users need measured memory use and generation speeds across specific hardware before they can judge the practical cost of running it locally. Image understanding, long-context behavior and output quality also matter; the supplied release information does not provide independent results on those questions.
The adjustable-pass research checkpoint could also be useful to developers studying the relationship between compute and quality. If fewer passes reduce latency without an unacceptable drop in results, that could offer a way to tune inference for different setups. That outcome remains a possibility, not an established result in the material provided.
Products Worth Considering

Samsung Galaxy Tab A11+ Kids Edition, 128GB, Royal Blue

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)

Jackery HomePower 3000 Portable Power Station, 3072Wh
As an affiliate, we earn on qualifying purchases.
How Inkling Reuses Its Layers
The key architectural choice is the reuse of the 12 middle physical layers. They run three times, rather than having a distinct set of weights at each of the model’s 48 logical layer visits. The company says learned messages carry information between passes, and a learned gate controls how much information is passed forward. Its fixed model uses three passes; the research checkpoint can be set to one or two.
Thinking Machines compares Mini with its Inkling 2 Small model. Both execute 48 layer visits per token, but Mini has a hidden width of 2,304, compared with 4,608 for Small, and routes among 128 experts per layer rather than 384. The supplied material describes Mini as substantially smaller, but does not give a complete file size or minimum configuration. It also notes earlier work on looping architectures, including Nanbeige.
The release says each pass through the repeated layers begins with the same input from the first six layers, with learned messages conveying information between passes. It also says each visit retains its own key-value and short-convolution state. Reusing layer weights therefore does not mean the model skips computation across the repeated visits.
“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”
— Thinking Machines Lab, in the supplied release material
Benchmark and Hardware Questions Remain
Thinking Machines calls Inkling 2 Mini state of the art in its class on the Artificial Analysis Intelligence Index v5 and says it leads in 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. The company also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0, and a lower result on AutomationBench-AA v1.0.6. These comparisons are company-reported; the supplied material does not contain the full score table or independent replication.
It remains unclear how much memory the complete model requires, what generation speeds users can expect on consumer devices, or how well image inputs and the claimed million-token context perform in outside testing. The quality of the one- and two-pass research settings is also not fully established here. The full model card may contain further technical details, but it was not included in the source material provided.
Independent Tests Will Clarify Trade-Offs
The next useful evidence would include the full model card, reproducible benchmark scores and hands-on reports that name the hardware and runtime used. Tests of memory use, generation speed, image tasks and long-context performance would help users compare the model with alternatives on like-for-like terms.
Further evaluation of the research checkpoint could show how changing the number of passes affects both latency and output quality. Until such details are available, the release establishes that the model and runtime support have been announced, but not how well it will perform on a given local setup.
Key Questions
What is Inkling 2 Mini?
It is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company says it accepts text and image inputs and has about 26 billion total parameters, with about 3 billion activated per token.
Can Inkling 2 Mini run locally?
The release lists support for Transformers, llama.cpp and MLX, which can be used for inference in different environments. SGLang is listed for datacenter deployments. The supplied information does not specify minimum hardware or confirm performance on particular consumer devices.
What does the research checkpoint change?
The TML-d24-varloop-research artifact allows the number of passes through the reused middle layers to be adjusted. Thinking Machines says fewer passes may increase speed, but could reduce quality; the supplied material does not provide independent results quantifying that trade-off.
Are the reported benchmark results independently verified?
Not in the material provided. Thinking Machines reports several benchmark comparisons, but the full score table and independent replication were not included.
What should users check before trying it?
Users will need to check the model files, runtime compatibility and hardware demands. Independent measurements of memory use, speed and task quality would help establish whether it fits a particular local setup.
Primary source: Hugging Face · via ThorstenMeyerAI.com
