🔍 Read the full analysis: Can LFM2.5-24B-A2B Power Fast, Private Agents On Your Device? on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Liquid AI has introduced LFM2.5-24B-A2B, a reasoning model designed for tool-using agents and local deployment. The company reports strong results on several agent and instruction-following benchmarks, but the figures have not been independently replicated in the supplied material and the model trails competitors on some coding, web and knowledge tests.
Liquid AI has introduced LFM2.5-24B-A2B, a reasoning model designed to run local agents that use tools across multiple steps, as described in the original analysis. The company says the model has 24 billion total parameters but activates 2 billion per token, a design intended to reduce inference costs; its reported benchmark strengths are concentrated in some agent and instruction-following tasks, not across every evaluation.
Liquid AI describes LFM2.5 as a reasoning-only model that generates a reasoning phase before its final answer and can make tool calls over multiple turns. The company says it trained the model within agent harnesses including OpenClaw, Hermes Agent, Pi and OpenCode. Its context window is 131,000 tokens, up from 32,000 in the predecessor LFM2-24B-A2B.
In results reported by Liquid AI across 16 benchmarks, LFM2.5 scored 48.09 on LiveClaw, compared with 38.19 for Qwen3.6-35B-A3B, and 27.69 on AutomationBench. The next-highest score listed for AutomationBench was 8.72. The company says LFM2.5 led or matched the comparison models on five of eight agent and tool-use tests and led on all three instruction-following tests.
The results were not uniformly in LFM2.5’s favor. Liquid AI’s comparison shows Qwen scoring higher on WebArena Verified, GPQA Diamond and SWE-Bench Verified, among other tests; Gemma-4-26B-A4B-it led on HLE, while Nemotron-3.5-Lightning had the least negative score on AA-Omniscience. Liquid AI says it ran evaluations using vLLM and each benchmark’s recommended generation settings.
Local Agents and Compute Trade-Offs
Agent systems often make repeated model calls while planning, using tools and responding to results. A model that activates 2 billion parameters per token, rather than all 24 billion, could make those repeated calls less demanding to serve than a model that activates its full parameter count. That is the design rationale, not proof of a particular cost or hardware requirement: the supplied release information does not quantify total workflow cost or establish which consumer devices can run the model at useful settings.
Local inference can keep prompts and outputs on a user’s device or organization’s infrastructure, depending on deployment and configuration. That can matter for privacy and control, but local use does not automatically guarantee that data stays on-device; connected tools and surrounding software also affect where information goes. For developers, the benchmark profile offers a reason to test LFM2.5 on tool-heavy and instruction-following tasks, while the weaker results on some coding, web and knowledge evaluations argue against assuming it is a general-purpose winner.
Liquid AI also reports decoding speeds of 184 tokens per second on an Apple M5 Max and 114 on a Ryzen AI Max+ 395. It reports more than 21,000 output tokens per second on one NVIDIA B300 under high concurrency. Those company figures may help developers decide what to test, but the B300 result describes a high-concurrency setup, not a typical single-user rate.
As an affiliate, we earn on qualifying purchases.
What Changed From LFM2
LFM2.5 retains the architecture of LFM2-24B-A2B, which Liquid AI describes as a 40-layer mixture-of-experts model with 64 experts per block and top-four routing. The company says the design combines gated short convolutions with grouped-query attention. The main stated changes include a longer context window and expanded training.
According to Liquid AI, pre-training grew from about 17 trillion to 33 trillion tokens, followed by mid-training in reasoning, math, code, instruction following, multilingual text and long-context material. The vocabulary increased from about 65,000 to 128,000 tokens, which the company says is intended to improve non-Latin tokenization. Post-training included supervised fine-tuning, teacher specialization, two stages of multi-domain on-policy distillation and agentic reinforcement learning.
The model is listed for inference through llama.cpp, MLX, vLLM, SGLang and ONNX, among other deployment options described in the release. Framework availability can make testing easier, but it does not by itself establish performance, memory needs or ease of use across different hardware.
““Competitive with Qwen3.6-35B-A3B, a model with ~50% more total and active parameters, on agentic and tool-use tasks.””
— Liquid AI
AI reasoning model for private deployment
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Independent Tests Still Needed
The supplied results are Liquid AI’s own evaluations. The material does not provide independent replications, confidence intervals or full hardware configurations for each test. It also does not specify the quantization, memory footprint, power use or quality settings behind the reported device speeds, so readers cannot directly infer how the figures will translate to a particular computer.
The high-concurrency B300 figure is not directly comparable to a single-user local workload. Performance may also vary with the agent harness, tools, prompts and inference settings. The supplied information does not state license terms, pricing or a complete deployment cost comparison; those details could shape whether the model suits personal devices, organizational infrastructure or hosted use.
As an affiliate, we earn on qualifying purchases.
Deployment Results to Watch
The next useful evidence will come from developers testing LFM2.5 in their own agent setups and publishing comparable results for task completion, latency, memory and power use. Liquid AI says the model has day-one support in several inference frameworks and provides instructions for serving it behind a local OpenAI-compatible endpoint for agent harnesses such as Hermes Agent.
Independent replications, full benchmark settings and clear availability and licensing information would help establish how the reported results translate beyond company evaluations. Until then, teams can treat the release as a candidate for task-specific testing rather than evidence that it will outperform alternatives across local agent workloads.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is LFM2.5-24B-A2B?
It is a reasoning model from Liquid AI designed for tool-using agents. The company describes it as a 24-billion-parameter mixture-of-experts model that activates 2 billion parameters per token.
Can it run locally?
Liquid AI is positioning the model for local deployment and lists support for frameworks including llama.cpp, MLX, vLLM, SGLang and ONNX. The supplied material does not specify the memory or quantization requirements needed for particular devices.
Does it outperform competing models?
Not across all reported tests. Liquid AI reports strong results on some agent and instruction-following benchmarks, while comparison models score higher on several web, coding and knowledge evaluations.
Are the reported speed and benchmark results independently verified?
The supplied material presents them as Liquid AI’s evaluations and does not include independent replications. It also omits some testing details, including per-device memory use and power consumption.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
