Hy4 preview

Tencent's 770B MoE preview with 49B active, 1M context, sparse attention and an Apache 2.0 license; early and still verbose.

In short. Hy4 preview is Tencent's biggest open model so far, and an early version of its next generation. It reads up to one million tokens and is aimed at coding, office work and research. Tencent itself says it still thinks too long and double-checks too much, and it needs a large multi-GPU server.

Summary

Tencent open-sourced Hy4 preview in an announcement on 28 August 2026. It is a mixture-of-experts model with 770 billion parameters and 49 billion active per token, built from 78 layers: the first has a dense feed-forward block and the other 77 are mixture-of-experts layers with 256 routed experts and one shared expert, of which 8 routed experts fire per token. The attention is Gated DeepSeek Sparse Attention with IndexCache, which reuses the sparse index across layers, and the residual path uses identity Hyper-Connections. A separate multi-token-prediction layer supports speculative decoding. The context window is 1M tokens, and the model takes text only.

Tencent calls it an early version with known problems: it can spend longer than needed reasoning through complex tasks and tends to over-verify its own work. In a blind comparison, 163 Tencent experts rated outputs on 203 engineering tasks and scored Hy4 preview 2.99 out of 4, against 2.92 for GLM 5.3 (46.8% wins, 12.8% ties, 40.4% losses) and 2.94 for Kimi K3 (51.2% wins, 7.9% ties, 40.9% losses). This is Tencent's own internal evaluation, and the benchmark appendix is an image.

The weights are under Apache 2.0. The BF16 checkpoint is about 1.56 TB and the FP8 build about 814 GB, more than eight 80 GB GPUs can hold, so it needs a large multi-GPU cluster. The repository documents serving with vLLM and SGLang, fine-tuning and quantization with Tencent's AngelSlim toolkit. For a smaller Tencent model with a final release, see Hy3.

Good for

  • Research on very large sparse-attention MoE models with a permissive license
  • Coding, office and long-context workloads on a large multi-GPU cluster
  • Teams that want to test a 1M-token open model against GLM 5.3 and Kimi K3
  • Fine-tuning and quantization work, using Tencent's published guidance

Watch out for

  • It is labeled a preview, and Tencent lists excessive reasoning and over-verification as known issues
  • FP8 weights alone are about 814 GB, so a single 8-GPU 80 GB server cannot hold it
  • The expert comparison is Tencent's own and covers selected engineering tasks
  • It is text-only, with no image input

Performance claims come from the developer and have not been independently verified.

How much memory does it need?

Estimates for the Hy4 preview (BF16) variant. A mixture-of-experts model must hold all of its experts in memory even though only some are used for each token, so total parameters decide the memory and active parameters decide the speed.

Variants Parameters BF16 FP8 / INT8 4-bit (Q4) Fits on
Hy4 preview (BF16) BF16 770B 49B active 1.77 TB 885 GB 531 GB GPU server
Hy4 preview-FP8 FP8 770B 49B active 1.77 TB 885 GB 531 GB GPU server

Estimates: weights plus about 15% overhead. Real usage grows with context length and depends on the software.

VRAM calculator

The license in plain words

One of the most widely used permissive licenses. You may use, change and sell products built on the model with only light conditions.

More about Apache 2.0

See also

Frequently asked questions

What is Hy4 preview?
Hy4 preview is Tencent's biggest open model so far, and an early version of its next generation. It reads up to one million tokens and is aimed at coding, office work and research. Tencent itself says it still thinks too long and double-checks too much, and it needs a large multi-GPU server.
Can I use Hy4 preview commercially?
Yes. Hy4 preview is released under Apache 2.0, a permissive license. You can use it commercially if you follow the standard conditions, such as keeping the license notice.
Is Hy4 preview open source?
Hy4 preview is released under Apache 2.0, a license approved by the Open Source Initiative. Training data and training code may not be public, so it is best described as open weights under an open-source license.
Can I run Hy4 preview on my own computer?
Not on a desktop. The smallest variant, Hy4 preview (BF16), needs about 531 GB at 4-bit precision, which means a GPU server with several data-center cards.
When was Hy4 preview released?
Hy4 preview was released by Tencent on August 28, 2026.
How long a context window does Hy4 preview have?
Hy4 preview supports a context window of up to 1M tokens.

Official links

Sources

Specifications come from these primary sources. Benchmark claims, where mentioned, are reported by the developer.

Release date and facts last verified November 11, 2026.

Related models

Model Released Parameters Context License Smallest variant needs
Hy3 Tencent 295B 21B active 256K Apache 2.0 (Permissive) 204 GB Big-memory desktop
GLM-5.3 Z.ai (Zhipu AI) 753B 40B active 1M GLM-5.3 License (Conditional) 520 GB GPU server
Kimi K3 Moonshot AI 2.8T 104B active 1M Kimi K3 License (Conditional) 1.72 TB Multi-node cluster
DeepSeek V4 DeepSeek 284B – 1.6T 49B active 1M MIT (Permissive) 176 GB Big-memory desktop