p1atdev
/

dart-v2-moe-base

Text Generation

text-generation-inference

Model card Files Files and versions

p1atdev commited on May 11, 2024

Commit

fbdcb30

·

verified ·

1 Parent(s): 8170002

Update README.md

Files changed (1) hide show

README.md +3 -3

README.md CHANGED Viewed

@@ -181,7 +181,7 @@ TODO
 #### Training Hyperparameters
 The following hyperparameters were used during training:
-- learning_rate: 0.00025
 - train_batch_size: 1024
 - eval_batch_size: 256
 - seed: 42
@@ -190,7 +190,7 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 1000
-- num_epochs: 4
 ## Evaluation
@@ -204,7 +204,7 @@ The architecture of this model is [Mixtral](https://huggingface.co/docs/transfor
 ### Compute Infrastructure
-Private server.
 #### Hardware

 #### Training Hyperparameters
 The following hyperparameters were used during training:
+- learning_rate: 0.0005
 - train_batch_size: 1024
 - eval_batch_size: 256
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 1000
+- num_epochs: 5
 ## Evaluation
 ### Compute Infrastructure
+Server in a university laboratory
 #### Hardware