We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
2 parents fa7bdcc + b3fe5f8 commit 2c0edb7Copy full SHA for 2c0edb7
1 file changed
src/MaxText/configs/models/qwen3-235b-a22b.yml
@@ -17,6 +17,7 @@
17
# Core Architectural Parameters
18
decoder_block: "qwen3_moe"
19
base_emb_dim: 4096
20
+base_mlp_dim: 1536
21
base_num_query_heads: 64
22
base_num_kv_heads: 4
23
base_num_decoder_layers: 94
0 commit comments