Update README.md

README.md

@@ -1,3 +1,33 @@
 ---
 license: openrail
+datasets:
+- bigcode/the-stack-dedup
+library_name: transformers
+tags:
+- code_generation
+- R programming
+- sas
+- santacoder
 ---
+
+# Statscoder
+
+This model is a fine-tuned version of [bigcode/santacoder](https://huggingface.co/bigcode/santacoder) on `R` and `SAS` language repositories in [the stack ](https://huggingface.co/datasets/bigcode/the-stack-dedup) dataset.
+
+
+## Training procedure
+
+The model was finetuned using the code adapted from [loubnabnl/santacoder-finetuning](https://github.com/loubnabnl/santacoder-finetuning). Adapted to handle multiple subsets of datasets and it is [here](https://github.com/infinitylogesh/santacoder-finetuning).
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- gradient_accumulation_steps: 4
+- optimizer: adafactor
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 100
+- training_steps: 1600
+- seq_length: 1024
+- no_fp16