--- license: apache-2.0 base_model: mosaicml/mpt-7b-instruct tags: - trl - dpo - generated_from_trainer model-index: - name: mpt_1000_STEPS_1e7_rate_01_beta_DPO results: [] --- # mpt_1000_STEPS_1e7_rate_01_beta_DPO This model is a fine-tuned version of [mosaicml/mpt-7b-instruct](https://huggingface.co/mosaicml/mpt-7b-instruct) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.6924 - Rewards/chosen: -0.0146 - Rewards/rejected: -0.0175 - Rewards/accuracies: 0.5275 - Rewards/margins: 0.0029 - Logps/rejected: -21.6159 - Logps/chosen: -20.8410 - Logits/rejected: 14.2241 - Logits/chosen: 14.2267 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-07 - train_batch_size: 2 - eval_batch_size: 1 - seed: 42 - gradient_accumulation_steps: 2 - total_train_batch_size: 4 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 100 - training_steps: 1000 ### Training results | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:| | 0.6908 | 0.05 | 50 | 0.6958 | -0.0024 | 0.0016 | 0.4835 | -0.0040 | -21.5521 | -20.8002 | 14.2618 | 14.2644 | | 0.7007 | 0.1 | 100 | 0.6940 | -0.0004 | -0.0001 | 0.5033 | -0.0003 | -21.5577 | -20.7936 | 14.2508 | 14.2534 | | 0.6945 | 0.15 | 150 | 0.6935 | -0.0010 | -0.0016 | 0.4923 | 0.0006 | -21.5629 | -20.7956 | 14.2501 | 14.2527 | | 0.6911 | 0.2 | 200 | 0.6947 | 0.0111 | 0.0130 | 0.5055 | -0.0019 | -21.5142 | -20.7552 | 14.2536 | 14.2561 | | 0.6944 | 0.24 | 250 | 0.6926 | -0.0007 | -0.0032 | 0.5297 | 0.0025 | -21.5681 | -20.7945 | 14.2489 | 14.2515 | | 0.6893 | 0.29 | 300 | 0.6925 | -0.0029 | -0.0056 | 0.5143 | 0.0027 | -21.5761 | -20.8017 | 14.2454 | 14.2480 | | 0.6964 | 0.34 | 350 | 0.6933 | -0.0031 | -0.0043 | 0.4901 | 0.0012 | -21.5718 | -20.8026 | 14.2500 | 14.2526 | | 0.6846 | 0.39 | 400 | 0.6899 | -0.0142 | -0.0220 | 0.5516 | 0.0078 | -21.6306 | -20.8394 | 14.2259 | 14.2284 | | 0.6823 | 0.44 | 450 | 0.6910 | -0.0143 | -0.0200 | 0.5143 | 0.0056 | -21.6240 | -20.8400 | 14.2294 | 14.2320 | | 0.6838 | 0.49 | 500 | 0.6908 | -0.0099 | -0.0159 | 0.5297 | 0.0059 | -21.6103 | -20.8253 | 14.2237 | 14.2263 | | 0.678 | 0.54 | 550 | 0.6897 | -0.0151 | -0.0234 | 0.5407 | 0.0082 | -21.6354 | -20.8427 | 14.2251 | 14.2277 | | 0.6872 | 0.59 | 600 | 0.6915 | -0.0176 | -0.0223 | 0.5385 | 0.0047 | -21.6318 | -20.8508 | 14.2284 | 14.2311 | | 0.6881 | 0.64 | 650 | 0.6906 | -0.0132 | -0.0196 | 0.5319 | 0.0064 | -21.6228 | -20.8362 | 14.2236 | 14.2262 | | 0.6841 | 0.68 | 700 | 0.6910 | -0.0146 | -0.0202 | 0.5143 | 0.0057 | -21.6249 | -20.8408 | 14.2152 | 14.2178 | | 0.6883 | 0.73 | 750 | 0.6901 | -0.0148 | -0.0223 | 0.5626 | 0.0075 | -21.6317 | -20.8414 | 14.2218 | 14.2244 | | 0.6813 | 0.78 | 800 | 0.6917 | -0.0150 | -0.0192 | 0.5341 | 0.0041 | -21.6213 | -20.8422 | 14.2255 | 14.2281 | | 0.6987 | 0.83 | 850 | 0.6902 | -0.0129 | -0.0204 | 0.5297 | 0.0075 | -21.6253 | -20.8350 | 14.2198 | 14.2223 | | 0.687 | 0.88 | 900 | 0.6928 | -0.0126 | -0.0148 | 0.5121 | 0.0021 | -21.6067 | -20.8343 | 14.2248 | 14.2275 | | 0.6885 | 0.93 | 950 | 0.6924 | -0.0146 | -0.0175 | 0.5275 | 0.0029 | -21.6159 | -20.8410 | 14.2241 | 14.2267 | | 0.6904 | 0.98 | 1000 | 0.6924 | -0.0146 | -0.0175 | 0.5275 | 0.0029 | -21.6159 | -20.8410 | 14.2241 | 14.2267 | ### Framework versions - Transformers 4.39.1 - Pytorch 2.0.0+cu117 - Datasets 2.18.0 - Tokenizers 0.15.2