Listen to this Post
Introduction:
Tencent has officially open-sourced Hy4 Preview, a 770-billion-parameter Mixture-of-Experts (MoE) large language model designed specifically for real-world productivity tasks including coding, financial analysis, office work, and scientific research. In internal blind evaluations involving 163 experts across 203 engineering tasks, Hy4 Preview scored 2.99/4.00, edging out Z.ai’s GLM-5.3 (2.92) and Moonshot AI’s Kimi K3 (2.94). What sets this release apart isn’t just the performance metrics—it’s the model’s unprecedented role in its own development: Hy4 Preview participated in automating its training methods, data strategies, evaluation frameworks, and even inference infrastructure optimization, achieving a 31.8% end-to-end throughput improvement.
Learning Objectives & Secrets:
- Objective 1: Deploy Hy4 Preview Locally with vLLM – Learn to serve the 770B MoE model on multi-GPU infrastructure using vLLM’s Docker images and speculative decoding with MTP (Multi-Token Prediction) layers.
- Objective 2: Optimize Inference Throughput Secret – Leverage the model’s autonomous bottleneck analysis capabilities by tuning `–attention-backend FLASHMLA_SPARSE` and enabling the native MTP speculative decoding layer (10B total / 0.7B activated) to reduce latency.
- Objective 3: API Integration Secret – Access Hy4 Preview through Tencent Cloud TokenHub or OpenRouter at $0.834 per million input tokens and $2.501 per million output tokens, with cached-hit tokens at just $0.042 per million ($0.3 RMB).
You Should Know:
- Model Architecture Deep Dive: 78 Layers of MoE with 256 Experts
Hy4 Preview represents a significant architectural leap from its predecessor Hy3 (295B total / 21B active). The backbone comprises 78 layers—the first layer uses a dense FFN, while the remaining 77 layers adopt MoE with 256 routed experts and 1 shared expert per layer, activating the top-8 routed experts plus the shared expert per token. The attention module employs Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache for cross-layer sparse index reuse, and iHC (identity Hyper-Connections) to expand inter-layer information flow. A native MTP layer adds 10B total parameters (0.7B activated) for speculative decoding.
Step-by-step guide to inspect model architecture:
Clone the Hugging Face repository
git lfs install
git clone https://huggingface.co/tencent/Hy4-preview-FP8
Inspect model configuration
cat Hy4-preview-FP8/config.json | jq '{
total_params: .num_parameters,
num_layers: .num_hidden_layers,
num_experts: .num_local_experts,
num_activated_experts: .num_activated_experts,
context_length: .max_position_embeddings
}'
2. Local Deployment with vLLM: Production-Ready Serving
The recommended deployment method uses vLLM’s pre-built Docker image optimized for Hy4 Preview. Minimum hardware requirements: 8× NVIDIA B300 GPUs for FP8 quantization or 16× B200 for BF16 precision.
Step-by-step guide for Docker deployment:
Deploy Hy4 Preview with MTP speculative decoding enabled
docker run --gpus all \
-p 8000:8000 \
--ipc=host \
-e VLLM_ENABLE_HPC_OPS=1 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai:hy4-preview \
tencent/Hy4-preview-FP8 \
--tensor-parallel-size 8 \
--speculative-config '{"num_speculative_tokens":3,"method":"mtp"}' \
--attention-backend FLASHMLA_SPARSE \
--tool-call-parser hy_v4 \
--reasoning-parser hy_v4 \
--enable-auto-tool-choice \
--served-model-1ame hy4-preview
Alternative: Build vLLM from source:
uv venv --python 3.12 --seed
source .venv/bin/activate
git clone https://github.com/vllm-project/vllm.git
cd vllm
uv pip install --editable . --torch-backend=auto
export VLLM_ENABLE_HPC_OPS=1
vllm serve tencent/Hy4-preview-FP8 \
--tensor-parallel-size 8 \
--speculative-config '{"num_speculative_tokens":3,"method":"mtp"}' \
--attention-backend FLASHMLA_SPARSE \
--tool-call-parser hy_v4 \
--reasoning-parser hy_v4
3. API Integration: TokenHub and OpenRouter Access
For production applications without local infrastructure, Hy4 Preview is accessible via Tencent Cloud TokenHub and OpenRouter. The API supports the OpenAI-compatible chat completion format with Hy4’s specialized tool-calling and reasoning parsers.
Python API integration example:
import openai
client = openai.OpenAI(
base_url="https://api.openrouter.ai/v1",
api_key="YOUR_API_KEY"
)
response = client.chat.completions.create(
model="tencent/hy4-preview",
messages=[
{"role": "system", "content": "You are a financial analysis assistant."},
{"role": "user", "content": "Analyze this quarterly report and flag anomalies."}
],
temperature=0.3,
max_tokens=4096,
extra_body={
"tool_choice": "auto",
"reasoning_parser": "hy_v4"
}
)
print(response.choices[bash].message.content)
Pricing structure (RMB per million tokens):
- Input: ¥6 ($0.834)
- Output: ¥18 ($2.501)
- Cache hit: ¥0.3 ($0.042)
4. Performance Optimization: Self-Optimizing Inference
Hy4 Preview demonstrated autonomous inference infrastructure optimization by analyzing system bottlenecks and implementing operator fusion and communication optimizations, achieving 31.8% end-to-end throughput improvement over baseline.
Benchmarking your deployment:
Test throughput with varying context lengths
for ctx in 1024 4096 16384 65536 262144 1048576; do
curl -X POST http://localhost:8000/v1/completions \
-H "Content-Type: application/json" \
-d "{
\"model\": \"hy4-preview\",
\"prompt\": \"Explain the concept of MoE architecture in detail. \",
\"max_tokens\": 512,
\"temperature\": 0.7,
\"extra_body\": {\"context_length\": $ctx}
}" \
-w "Context: $ctx, Time: %{time_total}s\n" \
-o /dev/null -s
done
5. Security and Compliance for Enterprise AI Deployment
When deploying open-source LLMs in enterprise environments, consider these security hardening practices:
API key rotation and access control:
Generate secure API keys openssl rand -base64 32 Set up rate limiting with nginx limit_req_zone $binary_remote_addr zone=llm_api:10m rate=10r/s;
Prompt injection mitigation:
- Implement input sanitization with regex filtering for system prompt override attempts
- Use Hy4’s built-in `hy_v4` reasoning parser which includes structured output validation
- Enable auto-tool-choice with strict schema validation to prevent unauthorized function calls
Data privacy considerations:
- Hy4 Preview is Apache 2.0 licensed, allowing on-premises deployment
- For sensitive financial or healthcare data, deploy locally with the FP8 quantized version to maintain data sovereignty
What Undercode Say:
- Key Takeaway 1: Hy4 Preview’s recursive self-improvement capability—where the model actively participates in optimizing its own training methods, data strategies, and inference infrastructure—represents a paradigm shift in AI development. This isn’t just a model that performs well; it’s a model that learns how to make itself better.
-
Key Takeaway 2: The aggressive pricing strategy (¥6/¥18 per million tokens with ¥0.3 cache hits) combined with Apache 2.0 open-source licensing puts direct pressure on competitors like Z.ai and Moonshot AI. For enterprises, this means enterprise-grade AI capabilities are becoming commoditized faster than anticipated.
Analysis: The 770B parameter scale with only 49B activated per token demonstrates Tencent’s engineering sophistication in MoE optimization. The model’s ability to process 1M token contexts—four times Hy3’s capacity—enables handling of entire codebases, comprehensive financial reports, or extensive research papers in a single pass. The integration with WorkBuddy and CodeBuddy (with two-week free access) suggests Tencent is aggressively building an ecosystem play, not just releasing a standalone model. However, the benchmark results are Tencent’s internal evaluations—independent third-party validation on the Open LLM Leaderboard is still pending. The model’s recognition as an “early iteration” with “known issues in complex task reasoning” indicates that Hy4’s full potential will unfold in subsequent versions.
Prediction:
- +1 The open-sourcing of Hy4 Preview under Apache 2.0 will accelerate enterprise AI adoption, particularly in finance, software engineering, and scientific research sectors, as organizations can now deploy production-grade models without vendor lock-in.
-
+1 The 31.8% throughput gain from autonomous inference optimization sets a precedent for self-improving AI infrastructure—future models may require less manual tuning and optimization from ML engineering teams.
-
-1 The pricing war in China’s LLM market will intensify, potentially forcing smaller AI startups with less capital to consolidate or pivot to niche applications, reducing diversity in the ecosystem.
-
+1 Tencent’s integration of Hy4 across its product suite (WorkBuddy, CodeBuddy, Yuanbao, ima) positions the company to convert model capability into recurring software revenue, strengthening its cloud and AI business against competitors.
-
-1 The lack of independent third-party benchmarks means performance claims remain unverified—enterprises should conduct their own evaluations before committing to production deployment.
-
+1 The recursive self-improvement loop—where Hy4 proposes training methods, runs experiments, and feeds results back into development—could accelerate the iteration cycle beyond the current two-month major version cadence, potentially reshaping how AI models are developed.
▶️ Related Video (84% Match):
https://www.youtube.com/watch?v=0_LPzH_1z1g
🎯Let’s Practice For Free:
🎓 Live Courses & Certifications:
Join Undercode Academy for Verified Certifications
🚀 Request a Custom Project:
Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands
IT/Security Reporter URL:
Reported By: https://lnkd.in/p/e-HfUgMC – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
