How long a context do open models really have?
The median open-weight release now declares 262,144 tokens. The number in config.json is a ceiling, not a promise.
The median open-weight text or vision-language model released in Jul–Dec 2026 declares a context length of 262,144 tokens, against 24,576 in Jul–Dec 2023. 94% of Jul–Dec 2026 releases declare at least 128,000 tokens, and 13 declare a million or more. Every one of those figures is max_position_embeddings, the architectural ceiling written into the model's config.json. It is the most comparable number there is, because every model publishes it the same way. It is not always the length a model was trained to use natively, and it is not what any provider promises to serve.
How has open-model context length changed?
| Period | Models | Median context | ≥128K | ≥1M | Longest |
|---|---|---|---|---|---|
| Jul–Dec 2023 | 12 | 24,576 | 17% | 0 | 200,000 |
| Jan–Jun 2024 | 20 | 16,384 | 5% | 0 | 163,840 |
| Jul–Dec 2024 | 13 | 32,768 | 23% | 0 | 163,840 |
| Jan–Jun 2025 | 35 | 131,072 | 54% | 0 | 262,144 |
| Jul–Dec 2025 | 25 | 131,072 | 64% | 0 | 262,144 |
| Jan–Jun 2026 | 32 | 262,144 | 88% | 4 | 1,048,576 |
| Jul–Dec 2026 | 33 | 262,144 | 94% | 13 | 1,048,576 |
Is the context length in config.json accurate?
It is accurate about the config, and that can differ from what the model card tells you to use. Qwen3-8B is a clear case. Its config.json sets max_position_embeddings to 40,960, which is the figure on its register page. Its model card gives the context length as 32,768 tokens natively and 131,072 with YaRN, a RoPE-scaling method. The card explains the 40,960: it reserves 32,768 tokens for output and 8,192 for a typical prompt. It also advises enabling YaRN only when long contexts are needed, because the frameworks it names implement static YaRN, which may hurt performance on shorter texts.
So one model has three honest context lengths: 32,768 trained natively, 40,960 configured, 131,072 validated with scaling. A comparison that mixes them is not a comparison.
Which context lengths are most common?
262,144 and 131,072 tokens are tied as the most common declared values, in 35 models each, followed by 32,768 (26). All three are powers of two.
Which open models declare a million-token context?
In Jul–Dec 2026, 13 releases declare a context of at least one million tokens: deepseek-ai/DeepSeek-V4-Flash-0731 (1,048,576), deepseek-ai/DeepSeek-V4-Flash-Vision-Exp (1,048,576), deepseek-ai/DeepSeek-V4.1-Flash (1,048,576), deepseek-ai/DeepSeek-V4.1-Flash-1017 (1,048,576), nvidia/GLM-5.3-Flash-FP4-Local (1,048,576), nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 (1,048,576), RadixArk/Kimi-K3-DSpark (1,048,576), tencent/Youtu-Parsing-Omni (1,048,576), XiaomiMiMo/MiMo-V2.6-Flash-RL (1,048,576), XiaomiMiMo/MiMo-V2.6-Pro-RL (1,048,576), zai-org/GLM-5.3 (1,048,576), zai-org/GLM-5.3-Flash (1,048,576) and zai-org/GLM-5.3-Flash-BF16 (1,048,576). As above, a declared ceiling is not a measured ability to use it.
How should you compare LLM context lengths?
- Compare like with like. Use max_position_embeddings across models, as this page does, or read every card's long-context section and compare those. Do not mix the two.
- Check whether the config already applies RoPE scaling (a rope_scaling block) or whether the card asks you to add it. Qwen3-8B ships with rope_scaling set to null and documents how to enable YaRN.
- Treat a provider's served context as a third number. It is set by the provider, not by the weights.
- Test at the length you will actually use. A ceiling says what the position encoding allows, not how well the model retrieves at that depth.
Questions this page answers
What is max_position_embeddings?
The largest number of token positions a transformer's configuration allows. The register uses it as each model's context length because every Hugging Face config publishes it in the same field.
Is the context length in config.json the real context window?
It is the architectural maximum. The model card may give a smaller native length or a larger one reached with RoPE scaling, as Qwen3-8B's does.
What is the median context length of open-weight LLMs?
262,144 tokens for releases in Jul–Dec 2026, measured from config.json across 33 text and vision-language models in the register.
How many open models support 1M tokens?
13 releases in Jul–Dec 2026 declare at least 1,000,000 tokens in config.json.
Where can I compare context length across open models?
Every model page in the register shows its exact max_position_embeddings, and the full dataset is at /api/models.json under CC BY 4.0.
Method, and what this does not show
- Derived from 171 open-weight text-generation and image-text-to-text releases in the register. 31 more publish no readable context length in config.json and are excluded, not counted as zero.
- Quantisations and conversions are excluded; they copy their parent's config.
- Context length is max_position_embeddings from each model's config.json — the architectural maximum, not necessarily what the model was trained on natively or what a provider serves.
- Release date is the model repository's creation date, the best public proxy available. It can trail an announcement.
- The register tracks named organisations plus models above a download threshold, so this is a sample of notable releases, not of everything published.
- Periods with fewer than three models are omitted.
Every figure here except the Qwen3-8B example is computed at build time from the register, which is CC BY 4.0.