Tencent's Hy4 preview: 770B parameters, 1M context and Apache 2.0
Tencent's Hy4 preview is a 770B mixture-of-experts model with 49B active and a 1M-token context, under Apache 2.0 after Hy3's preview had territory limits. Tencent calls it an early version.
Tencent open-sourced Hy4 preview on 28 August 2026. It is a 770-billion-parameter mixture-of-experts model with 49 billion parameters active per token and a 1-million-token context window, and it is the largest open model Tencent has released so far.
What is new
The model has 78 layers: one dense feed-forward layer and 77 mixture-of-experts layers, each with 256 routed experts and one shared expert, of which 8 routed experts fire per token. Attention is Gated DeepSeek Sparse Attention with IndexCache, which reuses the sparse index across layers, and a built-in multi-token-prediction layer supports speculative decoding. The model takes text only. Reasoning is on by default, and a request can switch it off for direct answers.
Tencent calls Hy4 preview an early version with known problems: it can reason for longer than a task needs and tends to over-verify its own work. In a blind test, 163 Tencent experts rated outputs on 203 engineering tasks and scored it 2.99 out of 4, against 2.92 for GLM 5.3 and 2.94 for Kimi K3. That is Tencent’s own evaluation, and its benchmark appendix is an image.
Running it
The BF16 checkpoint is about 1.56 TB and the FP8 build is about 814 GB. A single server with eight 80 GB GPUs holds 640 GB, so it cannot hold even the FP8 weights. The model card’s example commands serve the FP8 weights on 8 GPUs with vLLM or SGLang, use the built-in prediction layer for speculative decoding and name ready-made container images. Tencent also publishes fine-tuning instructions and points to its AngelSlim toolkit for quantization.
License
The weights are Apache 2.0, with no territory limit, revenue cap or hosting clause. Tencent used the same license for the final Hy3 release on 6 July 2026. The Hy3 preview of 23 April 2026 was different: its Tencent Hy Community License did not apply in the EU, the UK or South Korea. Terms still vary by Tencent model, because HunyuanOCR-1.5, released on 7 July, carries the Tencent Hunyuan Community License, so check the license file of each repository.
Why it matters
Tencent compares Hy4 preview with GLM-5.3 (753B) and Kimi K3 (2.8T). Their MIT-style licenses add conditions for the largest hosting businesses, while Hy4 preview’s Apache 2.0 license has none. Few teams can run it, though, and it carries a preview label. The practical choice for most is Hy3, which has a final release, 295B parameters with 21B active, a 256K context and about 300 GB of FP8 weights. DeepSeek V4 vs Hy3 compares Hy3 with DeepSeek’s family, and the best open-source LLMs list shows where both fit.
Sources
- Hy4 preview model card (Hugging Face) huggingface.co
- Hy4 preview repository (GitHub) github.com
- Tencent open-sources 770B Hy4 preview model (TechNode, 4 Sep 2026) technode.global
- Hy4 preview LICENSE (Apache 2.0) huggingface.co
- Hy3 preview LICENSE (Tencent Hy Community License) huggingface.co
Models in this story
| Model | Released | Parameters | Context | License | Smallest variant needs |
|---|---|---|---|---|---|
| Hy4 preview Tencent | 770B 49B active | 1M | Apache 2.0 (Permissive) | 531 GB GPU server | |
| Hy3 Tencent | 295B 21B active | 256K | Apache 2.0 (Permissive) | 204 GB Big-memory desktop | |
| GLM-5.3 Z.ai (Zhipu AI) | 753B 40B active | 1M | GLM-5.3 License (Conditional) | 520 GB GPU server | |
| Kimi K3 Moonshot AI | 2.8T 104B active | 1M | Kimi K3 License (Conditional) | 1.72 TB Multi-node cluster |