Tencent released Hy4 Preview, a text-only open-weight language model with 770 billion total parameters and 49 billion active parameters, under an Apache 2.0 license on Hugging Face, according to the model card and Simon Willison's blog.
The model uses a mixture-of-experts design with 78 layers and 256 routed experts plus one shared expert, activating the top eight routed experts per token alongside the shared expert, according to the Hugging Face model card. It supports a 1 million token context window and ships as a 1.56 terabyte download.
Hy4 is a large jump from Hy3, released in July, which had 295 billion total parameters, 21 billion active parameters and a 256,000 token context window at 598 gigabytes, Willison wrote.
The model supports two reasoning modes, a default "high" effort setting and a "no_think" option, and Tencent describes it as "an early version" with known limitations still being addressed, according to the model card.
A near tripling of active parameters and a fourfold larger context window, released under a permissive license, gives developers another large open-weight model to self-host or fine-tune without waiting on the leading U.S. labs, provided they have the hardware to run it.