Thinking Machines Lab Releases Inkling-Small: A 276B Total, 12B Active Open Weights Multimodal MoE Model

Editor's take

Thinking Machines Lab has unveiled Inkling-Small, a 276 billion parameter multimodal Mixture-of-Experts (MoE) model with 12 billion active parameters, demonstrating performance comparable to its larger predecessor at a fraction of the size. This development is significant as it pushes the boundaries of efficient model deployment, with a specific checkpoint (NVFP4) capable of running on a single NVIDIA B300 GPU, addressing the growing computational demands and accessibility challenges in AI research and application.

The ability to run such a large, capable model on a single high-end GPU has implications for democratizing access to advanced AI, potentially enabling more researchers and smaller organizations to experiment with and deploy sophisticated multimodal capabilities. This contrasts with the typical infrastructure requirements for models of Inkling-Small's scale, which often necessitate distributed computing clusters.

Future developments to monitor include the performance benchmarks of Inkling-Small across a wider range of multimodal tasks and its adoption by the open-source community compared to other large MoE models like Mixtral 8x7B. The long-term impact will depend on whether this efficiency translates into practical, cost-effective deployments for real-world applications.