Liquid AI Ships LFM2.5-230M with llama.cpp, MLX, vLLM, SGLang, and ONNX Support for On-Device Inference

Editor's take

Liquid AI's release of the 230 million parameter LFM2.5-230M model, boasting broad framework support including llama.cpp and MLX, allows for efficient on-device inference. This development is significant for edge AI applications, enabling complex language tasks on resource-constrained devices like smartphones and single-board computers, a critical step towards democratizing AI capabilities beyond cloud infrastructure.

The performance figures, achieving 213 tokens/sec on a Galaxy S25 Ultra and 42 tokens/sec on a Raspberry Pi 5, underscore the model's viability for real-time tool use and data extraction in mobile and embedded environments. This pushes the boundaries of what's feasible locally, directly impacting the development of offline AI assistants and smart devices.

Future attention should focus on the model's scalability and its ability to maintain performance across a wider range of hardware, particularly as more specialized edge AI accelerators emerge. Demonstrating robust on-device fine-tuning capabilities would further solidify its position as a practical solution for distributed AI deployments.