Text Generation
Transformers
Safetensors
PyTorch
English
nvidia
conversational
ameyasunilm commited on
Commit
dfe3fe6
·
verified ·
1 Parent(s): 021924e

Update max position embeddings to 128k from 8k

Browse files
Files changed (1) hide show
  1. config.json +1 -1
config.json CHANGED
@@ -25,7 +25,7 @@
25
  "mamba_hidden_act": "silu",
26
  "mamba_num_heads": 112,
27
  "mamba_proj_bias": false,
28
- "max_position_embeddings": 8192,
29
  "mlp_bias": false,
30
  "mlp_hidden_act": "relu2",
31
  "model_type": "nemotron_h",
 
25
  "mamba_hidden_act": "silu",
26
  "mamba_num_heads": 112,
27
  "mamba_proj_bias": false,
28
+ "max_position_embeddings": 131072,
29
  "mlp_bias": false,
30
  "mlp_hidden_act": "relu2",
31
  "model_type": "nemotron_h",