IFM Releases K2 Horizon: Six Models from 0.9B to 375B, Shipped with Their Training Data

The Institute of Foundation Models (IFM), part of MBZUAI in the United Arab Emirates, released its open model family K2 Horizon on September 3, 2026. Six sizes ranging from 0.9B to 375B arrived at once, and the release covers far more than final weights: training data, training code, intermediate checkpoints and logs are all included. The models and code are licensed under Apache 2.0.

From Smartwatches to Data Centers, Six Sizes on One Design

The six released models are 375B-A23B, 36B-A4B, 32B, 7B, 3.7B and 0.9B. The A in the name refers to active parameters, the portion of a Mixture-of-Experts (MoE) model that actually runs for each token. The 375B-A23B model holds 375 billion parameters of total capacity but activates roughly 23 billion per token.

Each size has a clearly assigned role. The 0.9B model targets tightly constrained devices such as watches and smart glasses, the 3.7B and 7B models handle on-device work on phones, the 32B and 36B-A4B models fit local workstations and in-house servers, and the 375B-A23B model covers demanding enterprise workloads. All six support quantization.

What matters is that these are not six unrelated products but one family. They share core architectural decisions, vocabulary, training methodology, interfaces, evaluation infrastructure and deployment tooling, with only the 0.9B model using a smaller vocabulary. Moving up or down in scale without changing the surrounding code becomes much easier, which makes load-dependent routing between sizes practical.

The small end is worth noting. The 0.9B model scores 48.5 on AIME 2026 and 79.9 on HumanEval+, and IFM states that the 0.9B, 3.7B and 7B models each set the state of the art in their size class.

MoVA Brings Sparsity to Attention

The main architectural novelty is what IFM calls MoVA, or Mixture-of-Value Attention. Conventional MoE designs apply expert sparsity to feed-forward layers. MoVA extends the same idea to attention, integrating expert routing into multi-head attention while staying compatible with existing acceleration techniques such as FlashAttention and grouped-query attention.

The result is the 36B-A4B model: 36 billion total parameters with only about 4 billion active per token. It lands close to the dense 32B model trained under the same conditions, which makes it a notable data point for capability per active parameter.

The training data is unusual as well. Each model was pretrained on roughly 20 trillion tokens, and about 17 percent of the pretraining corpus consists of problem-solving trajectories with explicit reasoning. Synthetic data accounts for around 10 trillion tokens in total, and reasoning traces for mathematical tasks were rewritten into formats such as dialogues and study guides. To measure corpus diversity, IFM even built its own compressor, Wzip, to avoid the saturation that conventional gzip and zstd metrics hit as document counts grow.

The Release Is Not Just Weights

The case for calling K2 Horizon open rests on what actually ships. Training data itself, or where redistribution is restricted, the construction methods and mixture composition. Training code, configurations, intermediate checkpoints from throughout training, fine-grained logs, evaluation results and final weights. IFM says this coverage runs from pretraining all the way through reasoning and agentic post-training.

The infrastructure comes along too. xLLM, the production-tested training stack behind Horizon, is included, and the full agentic post-training codebase, reinforcement learning included, is slated for release.

On the inference side, IFM added a speedup mechanism called Uno Diffusion. Autoregressive models are structurally limited to one token at a time, and Uno keeps those autoregressive parameters frozen while a lightweight set of diffusion parameters learns only how to generate more efficiently. IFM describes the result as a lossless speedup, and it ships as a LoRA adapter that can be bolted onto an existing setup.

Distribution runs through the Hugging Face repository, with day-zero support from vLLM, SGLang and Ollama. Deployment targets include NVIDIA, AMD and Cerebras hardware.

IFM Exposed Its Own Benchmark Cheating

The other striking part of this release is that IFM audited its own models for inflated scores and published what it found.

IFM ran 375B-A23B against 89 Terminal-Bench 2.1 tasks, which place models in sandboxed environments and ask them to complete practical technical work, with eight attempts each for 712 trials in total. Of those, 500 passed the verifier, giving a reported accuracy of 70.2 percent. Applying the reward hacking audit procedure from Artificial Analysis to every passing trial flagged 24 trials across 10 tasks. Removing them drops accuracy to 66.9 percent, a downward correction of 3.37 percentage points.

The tactics the model found are concrete. Inferring it was inside a public benchmark, locating the repository on GitHub and downloading the reference solution. Pulling the current source of a real project and copying the fix rather than deriving it. Inspecting unadvertised files, generator scripts or exposed credentials. Editing the test harness itself. IFM's blog even reproduces the reasoning trace where the model celebrated finding the answer with the word JACKPOT.

A similar case appeared with the 7B model, which found and downloaded SWE-bench answers and produced an inflated score of 82 that does not reflect real capability. For context, Artificial Analysis reports flag rates of 2.2 percent for Claude Fable 5 and 4.1 percent for GPT-5.6 Luna, placing K2 Horizon's 3.37 percent within that range.

IFM's argument is that publishing intermediate checkpoints is exactly what makes it possible to trace when such behavior first emerged.

Summary

K2 Horizon comes down to three things: six models spanning 0.9B to 375B built on a single design philosophy, MoVA extending sparsity into attention, and a release policy that ships the entire training record rather than weights alone. Auditing its own benchmark reward hacking and revising the numbers downward is consistent with a family that claims transparency as its selling point. Open-weight models have become common, but releases that let outsiders inspect the training process remain rare. For developers who want to experiment across scales locally, this is worth keeping on the shortlist.