Hy4 preview
Tencent's 770B MoE preview with 49B active, 1M context, sparse attention and an Apache 2.0 license; early and still verbose.
In short. Hy4 preview is Tencent's biggest open model so far, and an early version of its next generation. It reads up to one million tokens and is aimed at coding, office work and research. Tencent itself says it still thinks too long and double-checks too much, and it needs a large multi-GPU server.
Summary
Tencent open-sourced Hy4 preview in an announcement on 28 August 2026. It is a mixture-of-experts model with 770 billion parameters and 49 billion active per token, built from 78 layers: the first has a dense feed-forward block and the other 77 are mixture-of-experts layers with 256 routed experts and one shared expert, of which 8 routed experts fire per token. The attention is Gated DeepSeek Sparse Attention with IndexCache, which reuses the sparse index across layers, and the residual path uses identity Hyper-Connections. A separate multi-token-prediction layer supports speculative decoding. The context window is 1M tokens, and the model takes text only.
Tencent calls it an early version with known problems: it can spend longer than needed reasoning through complex tasks and tends to over-verify its own work. In a blind comparison, 163 Tencent experts rated outputs on 203 engineering tasks and scored Hy4 preview 2.99 out of 4, against 2.92 for GLM 5.3 (46.8% wins, 12.8% ties, 40.4% losses) and 2.94 for Kimi K3 (51.2% wins, 7.9% ties, 40.9% losses). This is Tencent's own internal evaluation, and the benchmark appendix is an image.
The weights are under Apache 2.0. The BF16 checkpoint is about 1.56 TB and the FP8 build about 814 GB, more than eight 80 GB GPUs can hold, so it needs a large multi-GPU cluster. The repository documents serving with vLLM and SGLang, fine-tuning and quantization with Tencent's AngelSlim toolkit. For a smaller Tencent model with a final release, see Hy3.
Good for
- Research on very large sparse-attention MoE models with a permissive license
- Coding, office and long-context workloads on a large multi-GPU cluster
- Teams that want to test a 1M-token open model against GLM 5.3 and Kimi K3
- Fine-tuning and quantization work, using Tencent's published guidance
Watch out for
- It is labeled a preview, and Tencent lists excessive reasoning and over-verification as known issues
- FP8 weights alone are about 814 GB, so a single 8-GPU 80 GB server cannot hold it
- The expert comparison is Tencent's own and covers selected engineering tasks
- It is text-only, with no image input
Performance claims come from the developer and have not been independently verified.
How much memory does it need?
Estimates for the Hy4 preview (BF16) variant. A mixture-of-experts model must hold all of its experts in memory even though only some are used for each token, so total parameters decide the memory and active parameters decide the speed.
| Variants | Parameters | BF16 | FP8 / INT8 | 4-bit (Q4) | Fits on |
|---|---|---|---|---|---|
| Hy4 preview (BF16) BF16 | 770B 49B active | 1.77 TB | 885 GB | 531 GB | GPU server |
| Hy4 preview-FP8 FP8 | 770B 49B active | 1.77 TB | 885 GB | 531 GB | GPU server |
Estimates: weights plus about 15% overhead. Real usage grows with context length and depends on the software.
The license in plain words
One of the most widely used permissive licenses. You may use, change and sell products built on the model with only light conditions.
See also
- Baidu's ERNIE 5.5 Flash Base: 280B parameters, license excludes the EU
- Tencent's Hy4 preview: 770B parameters, 1M context and Apache 2.0
Frequently asked questions
- What is Hy4 preview?
- Hy4 preview is Tencent's biggest open model so far, and an early version of its next generation. It reads up to one million tokens and is aimed at coding, office work and research. Tencent itself says it still thinks too long and double-checks too much, and it needs a large multi-GPU server.
- Can I use Hy4 preview commercially?
- Yes. Hy4 preview is released under Apache 2.0, a permissive license. You can use it commercially if you follow the standard conditions, such as keeping the license notice.
- Is Hy4 preview open source?
- Hy4 preview is released under Apache 2.0, a license approved by the Open Source Initiative. Training data and training code may not be public, so it is best described as open weights under an open-source license.
- Can I run Hy4 preview on my own computer?
- Not on a desktop. The smallest variant, Hy4 preview (BF16), needs about 531 GB at 4-bit precision, which means a GPU server with several data-center cards.
- When was Hy4 preview released?
- Hy4 preview was released by Tencent on August 28, 2026.
- How long a context window does Hy4 preview have?
- Hy4 preview supports a context window of up to 1M tokens.
Official links
Sources
Specifications come from these primary sources. Benchmark claims, where mentioned, are reported by the developer.
- Hy4 preview model card (Hugging Face) huggingface.co
- Hy4 preview repository (GitHub) github.com
- Tencent open-sources 770B Hy4 preview model (TechNode, 4 Sep 2026) technode.global
- Hy4 preview LICENSE (Apache 2.0) huggingface.co
Release date and facts last verified November 11, 2026.
Related models
| Model | Released | Parameters | Context | License | Smallest variant needs |
|---|---|---|---|---|---|
| Hy3 Tencent | 295B 21B active | 256K | Apache 2.0 (Permissive) | 204 GB Big-memory desktop | |
| GLM-5.3 Z.ai (Zhipu AI) | 753B 40B active | 1M | GLM-5.3 License (Conditional) | 520 GB GPU server | |
| Kimi K3 Moonshot AI | 2.8T 104B active | 1M | Kimi K3 License (Conditional) | 1.72 TB Multi-node cluster | |
| DeepSeek V4 DeepSeek | 284B – 1.6T 49B active | 1M | MIT (Permissive) | 176 GB Big-memory desktop |