Hugging Face's logo

deepseek-ai
/
DeepSeek-V3-Base

Safetensors
deepseek_v3
custom_code
fp8
Model card Files
xet
Community
70

`aux_loss_alpha` should be 1e-4 instead of 1e-3?

#60
by cuichenx - opened
base: refs/heads/main
←
from: refs/pr/60
Discussion Files changed
+1
-1
cuichenx

According to DeepSeekV3 technical report section 4.2

For the balance loss, we set 𝛼 to 0.0001

`aux_loss_alpha` should be 1e-4 instead of 1e-3?41955df7
Cannot merge
This branch has merge conflicts in the following files:
  • config.json

Sign up or log in to comment