Update README.md

README.md

@@ -1,11 +1,13 @@
 ---
 library_name: transformers
-tags: []
+datasets:
+- xfordanita/code-summary-java
 ---
 
 # Model Card for Model ID
 
-<!-- Provide a quick summary of what the model is/does. -->
+This model is a fine-tuned version of **codellama/CodeLlama-7b-hf** on the **QLoRA** by using the method **PEFT** with  library..
+
 
 
 
@@ -13,7 +15,6 @@ tags: []
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
 
 This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
 
@@ -92,8 +93,29 @@ Use the code below to get started with the model.
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 
+Training  on Free Kaggle GPU 2*(15GB VRAM) with the following params:
+
+```py
+training_arguments = TrainingArguments(
+    output_dir='./results',
+    num_train_epochs=8,
+    per_device_train_batch_size=4,
+    gradient_accumulation_steps=2,
+    optim="paged_adamw_32bit",
+    save_steps=0,
+    logging_steps=10,
+    learning_rate=2e-4,
+    weight_decay=0.1,  # Utilisation d'une valeur plus élevée pour la régularisation L2
+    fp16=True,
+    max_grad_norm=1.0,  # Réduire la taille maximale des gradients pour éviter les explosions de gradients
+    max_steps=-1,
+    warmup_ratio=0.1,  # Augmentation du ratio de warmup
+    group_by_length=True,
+    lr_scheduler_type="constant",  # Utilisation d'un taux d'apprentissage constant
+    report_to="tensorboard"
+)
+```
 #### Speeds, Sizes, Times [optional]
 
 <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->