Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+base_model: bert-base-multilingual-cased
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "bert-base-multilingual-cased",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "value",
+    "query"
+  ],
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7fbbe85a018cbc03df57dcb70d167166206be94ef08498c2fde9528cb9bfc56
+size 1189596

checkpoint-2564/README.md

@@ -0,0 +1,202 @@
+---
+base_model: bert-base-multilingual-cased
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-2564/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "bert-base-multilingual-cased",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "value",
+    "query"
+  ],
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-2564/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7fbbe85a018cbc03df57dcb70d167166206be94ef08498c2fde9528cb9bfc56
+size 1189596

checkpoint-2564/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f5548d42a3a9f84416dc5d962b6816ac066735b52b7510768644636192ad63e
+size 2407802

checkpoint-2564/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0bb6e8e031e8eb48a5dedf63a55b3ce8f7c921ab549210c2db3be7848bd1e62f
+size 14244

checkpoint-2564/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99a0f3b37430e7123db02ed3ebf0928c4e150e4acab827794efa75589b87c3cc
+size 1064

checkpoint-2564/special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

checkpoint-2564/tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

checkpoint-2564/trainer_state.json

@@ -0,0 +1,398 @@
+{
+  "best_metric": 1.386317253112793,
+  "best_model_checkpoint": "./lora-mcqa-model\\checkpoint-2564",
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 2564,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01950078003120125,
+      "grad_norm": 1.9830408096313477,
+      "learning_rate": 0.00019869994799791993,
+      "loss": 1.394,
+      "step": 50
+    },
+    {
+      "epoch": 0.0390015600624025,
+      "grad_norm": 1.4824576377868652,
+      "learning_rate": 0.00019742589703588144,
+      "loss": 1.3896,
+      "step": 100
+    },
+    {
+      "epoch": 0.05850234009360374,
+      "grad_norm": 1.495307445526123,
+      "learning_rate": 0.00019612584503380135,
+      "loss": 1.3846,
+      "step": 150
+    },
+    {
+      "epoch": 0.078003120124805,
+      "grad_norm": 1.9779902696609497,
+      "learning_rate": 0.00019482579303172127,
+      "loss": 1.3801,
+      "step": 200
+    },
+    {
+      "epoch": 0.09750390015600624,
+      "grad_norm": 1.9127660989761353,
+      "learning_rate": 0.00019352574102964121,
+      "loss": 1.3866,
+      "step": 250
+    },
+    {
+      "epoch": 0.11700468018720749,
+      "grad_norm": 1.6987781524658203,
+      "learning_rate": 0.0001922256890275611,
+      "loss": 1.3873,
+      "step": 300
+    },
+    {
+      "epoch": 0.13650546021840873,
+      "grad_norm": 1.5772737264633179,
+      "learning_rate": 0.00019092563702548102,
+      "loss": 1.3981,
+      "step": 350
+    },
+    {
+      "epoch": 0.15600624024961,
+      "grad_norm": 1.6646242141723633,
+      "learning_rate": 0.00018962558502340094,
+      "loss": 1.3819,
+      "step": 400
+    },
+    {
+      "epoch": 0.17550702028081122,
+      "grad_norm": 1.4280552864074707,
+      "learning_rate": 0.00018832553302132086,
+      "loss": 1.3905,
+      "step": 450
+    },
+    {
+      "epoch": 0.19500780031201248,
+      "grad_norm": 1.2003318071365356,
+      "learning_rate": 0.00018702548101924078,
+      "loss": 1.3912,
+      "step": 500
+    },
+    {
+      "epoch": 0.21450858034321374,
+      "grad_norm": 1.2694647312164307,
+      "learning_rate": 0.0001857254290171607,
+      "loss": 1.3887,
+      "step": 550
+    },
+    {
+      "epoch": 0.23400936037441497,
+      "grad_norm": 2.167478084564209,
+      "learning_rate": 0.0001844253770150806,
+      "loss": 1.3849,
+      "step": 600
+    },
+    {
+      "epoch": 0.2535101404056162,
+      "grad_norm": 1.689456820487976,
+      "learning_rate": 0.00018312532501300053,
+      "loss": 1.3918,
+      "step": 650
+    },
+    {
+      "epoch": 0.27301092043681746,
+      "grad_norm": 1.3930853605270386,
+      "learning_rate": 0.00018182527301092045,
+      "loss": 1.3892,
+      "step": 700
+    },
+    {
+      "epoch": 0.2925117004680187,
+      "grad_norm": 1.3363187313079834,
+      "learning_rate": 0.00018052522100884036,
+      "loss": 1.3896,
+      "step": 750
+    },
+    {
+      "epoch": 0.31201248049922,
+      "grad_norm": 1.33578360080719,
+      "learning_rate": 0.00017922516900676028,
+      "loss": 1.3871,
+      "step": 800
+    },
+    {
+      "epoch": 0.33151326053042124,
+      "grad_norm": 1.5101064443588257,
+      "learning_rate": 0.0001779251170046802,
+      "loss": 1.3843,
+      "step": 850
+    },
+    {
+      "epoch": 0.35101404056162244,
+      "grad_norm": 1.1404200792312622,
+      "learning_rate": 0.00017662506500260011,
+      "loss": 1.3869,
+      "step": 900
+    },
+    {
+      "epoch": 0.3705148205928237,
+      "grad_norm": 1.2848296165466309,
+      "learning_rate": 0.00017532501300052003,
+      "loss": 1.3922,
+      "step": 950
+    },
+    {
+      "epoch": 0.39001560062402496,
+      "grad_norm": 1.8385337591171265,
+      "learning_rate": 0.00017402496099843995,
+      "loss": 1.389,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4095163806552262,
+      "grad_norm": 1.3018165826797485,
+      "learning_rate": 0.00017272490899635987,
+      "loss": 1.385,
+      "step": 1050
+    },
+    {
+      "epoch": 0.4290171606864275,
+      "grad_norm": 1.3563063144683838,
+      "learning_rate": 0.00017142485699427976,
+      "loss": 1.3871,
+      "step": 1100
+    },
+    {
+      "epoch": 0.4485179407176287,
+      "grad_norm": 1.234568476676941,
+      "learning_rate": 0.0001701248049921997,
+      "loss": 1.382,
+      "step": 1150
+    },
+    {
+      "epoch": 0.46801872074882994,
+      "grad_norm": 1.41502046585083,
+      "learning_rate": 0.00016882475299011962,
+      "loss": 1.3885,
+      "step": 1200
+    },
+    {
+      "epoch": 0.4875195007800312,
+      "grad_norm": 1.309103012084961,
+      "learning_rate": 0.00016752470098803954,
+      "loss": 1.3942,
+      "step": 1250
+    },
+    {
+      "epoch": 0.5070202808112324,
+      "grad_norm": 1.4687711000442505,
+      "learning_rate": 0.00016622464898595945,
+      "loss": 1.3855,
+      "step": 1300
+    },
+    {
+      "epoch": 0.5265210608424337,
+      "grad_norm": 1.1189204454421997,
+      "learning_rate": 0.00016492459698387937,
+      "loss": 1.388,
+      "step": 1350
+    },
+    {
+      "epoch": 0.5460218408736349,
+      "grad_norm": 1.3603004217147827,
+      "learning_rate": 0.0001636245449817993,
+      "loss": 1.3892,
+      "step": 1400
+    },
+    {
+      "epoch": 0.5655226209048362,
+      "grad_norm": 1.442458152770996,
+      "learning_rate": 0.00016232449297971918,
+      "loss": 1.3839,
+      "step": 1450
+    },
+    {
+      "epoch": 0.5850234009360374,
+      "grad_norm": 0.9751923084259033,
+      "learning_rate": 0.00016102444097763912,
+      "loss": 1.389,
+      "step": 1500
+    },
+    {
+      "epoch": 0.6045241809672387,
+      "grad_norm": 0.9651883244514465,
+      "learning_rate": 0.00015972438897555904,
+      "loss": 1.3842,
+      "step": 1550
+    },
+    {
+      "epoch": 0.62402496099844,
+      "grad_norm": 0.9299620985984802,
+      "learning_rate": 0.00015842433697347893,
+      "loss": 1.3869,
+      "step": 1600
+    },
+    {
+      "epoch": 0.6435257410296412,
+      "grad_norm": 0.9508662819862366,
+      "learning_rate": 0.00015712428497139885,
+      "loss": 1.3946,
+      "step": 1650
+    },
+    {
+      "epoch": 0.6630265210608425,
+      "grad_norm": 1.1460316181182861,
+      "learning_rate": 0.0001558242329693188,
+      "loss": 1.3932,
+      "step": 1700
+    },
+    {
+      "epoch": 0.6825273010920437,
+      "grad_norm": 1.099560022354126,
+      "learning_rate": 0.0001545241809672387,
+      "loss": 1.387,
+      "step": 1750
+    },
+    {
+      "epoch": 0.7020280811232449,
+      "grad_norm": 1.0458426475524902,
+      "learning_rate": 0.0001532241289651586,
+      "loss": 1.3873,
+      "step": 1800
+    },
+    {
+      "epoch": 0.7215288611544461,
+      "grad_norm": 1.1988131999969482,
+      "learning_rate": 0.00015192407696307852,
+      "loss": 1.3831,
+      "step": 1850
+    },
+    {
+      "epoch": 0.7410296411856474,
+      "grad_norm": 1.167151927947998,
+      "learning_rate": 0.00015062402496099846,
+      "loss": 1.3865,
+      "step": 1900
+    },
+    {
+      "epoch": 0.7605304212168487,
+      "grad_norm": 0.9355433583259583,
+      "learning_rate": 0.00014932397295891835,
+      "loss": 1.3899,
+      "step": 1950
+    },
+    {
+      "epoch": 0.7800312012480499,
+      "grad_norm": 1.04149329662323,
+      "learning_rate": 0.00014802392095683827,
+      "loss": 1.3857,
+      "step": 2000
+    },
+    {
+      "epoch": 0.7995319812792512,
+      "grad_norm": 0.8921425938606262,
+      "learning_rate": 0.00014672386895475822,
+      "loss": 1.3882,
+      "step": 2050
+    },
+    {
+      "epoch": 0.8190327613104524,
+      "grad_norm": 1.4201322793960571,
+      "learning_rate": 0.0001454238169526781,
+      "loss": 1.3885,
+      "step": 2100
+    },
+    {
+      "epoch": 0.8385335413416537,
+      "grad_norm": 1.2703887224197388,
+      "learning_rate": 0.00014412376495059802,
+      "loss": 1.3868,
+      "step": 2150
+    },
+    {
+      "epoch": 0.858034321372855,
+      "grad_norm": 1.2008202075958252,
+      "learning_rate": 0.00014282371294851794,
+      "loss": 1.3862,
+      "step": 2200
+    },
+    {
+      "epoch": 0.8775351014040562,
+      "grad_norm": 1.705498456954956,
+      "learning_rate": 0.0001415236609464379,
+      "loss": 1.3837,
+      "step": 2250
+    },
+    {
+      "epoch": 0.8970358814352574,
+      "grad_norm": 1.1787363290786743,
+      "learning_rate": 0.00014022360894435778,
+      "loss": 1.391,
+      "step": 2300
+    },
+    {
+      "epoch": 0.9165366614664586,
+      "grad_norm": 0.9128132462501526,
+      "learning_rate": 0.0001389235569422777,
+      "loss": 1.3829,
+      "step": 2350
+    },
+    {
+      "epoch": 0.9360374414976599,
+      "grad_norm": 1.2079758644104004,
+      "learning_rate": 0.0001376235049401976,
+      "loss": 1.3879,
+      "step": 2400
+    },
+    {
+      "epoch": 0.9555382215288611,
+      "grad_norm": 1.4396191835403442,
+      "learning_rate": 0.00013632345293811753,
+      "loss": 1.3859,
+      "step": 2450
+    },
+    {
+      "epoch": 0.9750390015600624,
+      "grad_norm": 0.9314255118370056,
+      "learning_rate": 0.00013502340093603745,
+      "loss": 1.39,
+      "step": 2500
+    },
+    {
+      "epoch": 0.9945397815912637,
+      "grad_norm": 0.9400072693824768,
+      "learning_rate": 0.00013372334893395736,
+      "loss": 1.3861,
+      "step": 2550
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.386317253112793,
+      "eval_runtime": 16.917,
+      "eval_samples_per_second": 75.782,
+      "eval_steps_per_second": 18.975,
+      "step": 2564
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 7692,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.082997836648448e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2564/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44ca3034cf87097f30254f9aacafe4cc22fa749525bb201546c79db28d1797eb
+size 5240

checkpoint-7692/README.md

@@ -0,0 +1,202 @@
+---
+base_model: bert-base-multilingual-cased
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-7692/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "bert-base-multilingual-cased",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "value",
+    "query"
+  ],
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-7692/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a413fc5b630ab058ab404b2a56b1a9fc9ff0113dc9ad50a922c07e15720ab15
+size 1189596

checkpoint-7692/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2073b2db36698c73e4651c5759064e95a25e91f7530d7e906ece1358eefffa57
+size 2407802

checkpoint-7692/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3f5e7e0e1bb28ff6d17cd246d0d11ca3cbb85631f383d25423ee6843e54833b
+size 14244

checkpoint-7692/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d0767faac71e2e7e1bfc360c83d9ed78d74c8c8b707490b7514ab91c1f5c23f
+size 1064

checkpoint-7692/special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

checkpoint-7692/tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

checkpoint-7692/trainer_state.json

@@ -0,0 +1,1128 @@
+{
+  "best_metric": 1.386317253112793,
+  "best_model_checkpoint": "./lora-mcqa-model\\checkpoint-2564",
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 7692,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.01950078003120125,
+      "grad_norm": 1.9830408096313477,
+      "learning_rate": 0.00019869994799791993,
+      "loss": 1.394,
+      "step": 50
+    },
+    {
+      "epoch": 0.0390015600624025,
+      "grad_norm": 1.4824576377868652,
+      "learning_rate": 0.00019742589703588144,
+      "loss": 1.3896,
+      "step": 100
+    },
+    {
+      "epoch": 0.05850234009360374,
+      "grad_norm": 1.495307445526123,
+      "learning_rate": 0.00019612584503380135,
+      "loss": 1.3846,
+      "step": 150
+    },
+    {
+      "epoch": 0.078003120124805,
+      "grad_norm": 1.9779902696609497,
+      "learning_rate": 0.00019482579303172127,
+      "loss": 1.3801,
+      "step": 200
+    },
+    {
+      "epoch": 0.09750390015600624,
+      "grad_norm": 1.9127660989761353,
+      "learning_rate": 0.00019352574102964121,
+      "loss": 1.3866,
+      "step": 250
+    },
+    {
+      "epoch": 0.11700468018720749,
+      "grad_norm": 1.6987781524658203,
+      "learning_rate": 0.0001922256890275611,
+      "loss": 1.3873,
+      "step": 300
+    },
+    {
+      "epoch": 0.13650546021840873,
+      "grad_norm": 1.5772737264633179,
+      "learning_rate": 0.00019092563702548102,
+      "loss": 1.3981,
+      "step": 350
+    },
+    {
+      "epoch": 0.15600624024961,
+      "grad_norm": 1.6646242141723633,
+      "learning_rate": 0.00018962558502340094,
+      "loss": 1.3819,
+      "step": 400
+    },
+    {
+      "epoch": 0.17550702028081122,
+      "grad_norm": 1.4280552864074707,
+      "learning_rate": 0.00018832553302132086,
+      "loss": 1.3905,
+      "step": 450
+    },
+    {
+      "epoch": 0.19500780031201248,
+      "grad_norm": 1.2003318071365356,
+      "learning_rate": 0.00018702548101924078,
+      "loss": 1.3912,
+      "step": 500
+    },
+    {
+      "epoch": 0.21450858034321374,
+      "grad_norm": 1.2694647312164307,
+      "learning_rate": 0.0001857254290171607,
+      "loss": 1.3887,
+      "step": 550
+    },
+    {
+      "epoch": 0.23400936037441497,
+      "grad_norm": 2.167478084564209,
+      "learning_rate": 0.0001844253770150806,
+      "loss": 1.3849,
+      "step": 600
+    },
+    {
+      "epoch": 0.2535101404056162,
+      "grad_norm": 1.689456820487976,
+      "learning_rate": 0.00018312532501300053,
+      "loss": 1.3918,
+      "step": 650
+    },
+    {
+      "epoch": 0.27301092043681746,
+      "grad_norm": 1.3930853605270386,
+      "learning_rate": 0.00018182527301092045,
+      "loss": 1.3892,
+      "step": 700
+    },
+    {
+      "epoch": 0.2925117004680187,
+      "grad_norm": 1.3363187313079834,
+      "learning_rate": 0.00018052522100884036,
+      "loss": 1.3896,
+      "step": 750
+    },
+    {
+      "epoch": 0.31201248049922,
+      "grad_norm": 1.33578360080719,
+      "learning_rate": 0.00017922516900676028,
+      "loss": 1.3871,
+      "step": 800
+    },
+    {
+      "epoch": 0.33151326053042124,
+      "grad_norm": 1.5101064443588257,
+      "learning_rate": 0.0001779251170046802,
+      "loss": 1.3843,
+      "step": 850
+    },
+    {
+      "epoch": 0.35101404056162244,
+      "grad_norm": 1.1404200792312622,
+      "learning_rate": 0.00017662506500260011,
+      "loss": 1.3869,
+      "step": 900
+    },
+    {
+      "epoch": 0.3705148205928237,
+      "grad_norm": 1.2848296165466309,
+      "learning_rate": 0.00017532501300052003,
+      "loss": 1.3922,
+      "step": 950
+    },
+    {
+      "epoch": 0.39001560062402496,
+      "grad_norm": 1.8385337591171265,
+      "learning_rate": 0.00017402496099843995,
+      "loss": 1.389,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4095163806552262,
+      "grad_norm": 1.3018165826797485,
+      "learning_rate": 0.00017272490899635987,
+      "loss": 1.385,
+      "step": 1050
+    },
+    {
+      "epoch": 0.4290171606864275,
+      "grad_norm": 1.3563063144683838,
+      "learning_rate": 0.00017142485699427976,
+      "loss": 1.3871,
+      "step": 1100
+    },
+    {
+      "epoch": 0.4485179407176287,
+      "grad_norm": 1.234568476676941,
+      "learning_rate": 0.0001701248049921997,
+      "loss": 1.382,
+      "step": 1150
+    },
+    {
+      "epoch": 0.46801872074882994,
+      "grad_norm": 1.41502046585083,
+      "learning_rate": 0.00016882475299011962,
+      "loss": 1.3885,
+      "step": 1200
+    },
+    {
+      "epoch": 0.4875195007800312,
+      "grad_norm": 1.309103012084961,
+      "learning_rate": 0.00016752470098803954,
+      "loss": 1.3942,
+      "step": 1250
+    },
+    {
+      "epoch": 0.5070202808112324,
+      "grad_norm": 1.4687711000442505,
+      "learning_rate": 0.00016622464898595945,
+      "loss": 1.3855,
+      "step": 1300
+    },
+    {
+      "epoch": 0.5265210608424337,
+      "grad_norm": 1.1189204454421997,
+      "learning_rate": 0.00016492459698387937,
+      "loss": 1.388,
+      "step": 1350
+    },
+    {
+      "epoch": 0.5460218408736349,
+      "grad_norm": 1.3603004217147827,
+      "learning_rate": 0.0001636245449817993,
+      "loss": 1.3892,
+      "step": 1400
+    },
+    {
+      "epoch": 0.5655226209048362,
+      "grad_norm": 1.442458152770996,
+      "learning_rate": 0.00016232449297971918,
+      "loss": 1.3839,
+      "step": 1450
+    },
+    {
+      "epoch": 0.5850234009360374,
+      "grad_norm": 0.9751923084259033,
+      "learning_rate": 0.00016102444097763912,
+      "loss": 1.389,
+      "step": 1500
+    },
+    {
+      "epoch": 0.6045241809672387,
+      "grad_norm": 0.9651883244514465,
+      "learning_rate": 0.00015972438897555904,
+      "loss": 1.3842,
+      "step": 1550
+    },
+    {
+      "epoch": 0.62402496099844,
+      "grad_norm": 0.9299620985984802,
+      "learning_rate": 0.00015842433697347893,
+      "loss": 1.3869,
+      "step": 1600
+    },
+    {
+      "epoch": 0.6435257410296412,
+      "grad_norm": 0.9508662819862366,
+      "learning_rate": 0.00015712428497139885,
+      "loss": 1.3946,
+      "step": 1650
+    },
+    {
+      "epoch": 0.6630265210608425,
+      "grad_norm": 1.1460316181182861,
+      "learning_rate": 0.0001558242329693188,
+      "loss": 1.3932,
+      "step": 1700
+    },
+    {
+      "epoch": 0.6825273010920437,
+      "grad_norm": 1.099560022354126,
+      "learning_rate": 0.0001545241809672387,
+      "loss": 1.387,
+      "step": 1750
+    },
+    {
+      "epoch": 0.7020280811232449,
+      "grad_norm": 1.0458426475524902,
+      "learning_rate": 0.0001532241289651586,
+      "loss": 1.3873,
+      "step": 1800
+    },
+    {
+      "epoch": 0.7215288611544461,
+      "grad_norm": 1.1988131999969482,
+      "learning_rate": 0.00015192407696307852,
+      "loss": 1.3831,
+      "step": 1850
+    },
+    {
+      "epoch": 0.7410296411856474,
+      "grad_norm": 1.167151927947998,
+      "learning_rate": 0.00015062402496099846,
+      "loss": 1.3865,
+      "step": 1900
+    },
+    {
+      "epoch": 0.7605304212168487,
+      "grad_norm": 0.9355433583259583,
+      "learning_rate": 0.00014932397295891835,
+      "loss": 1.3899,
+      "step": 1950
+    },
+    {
+      "epoch": 0.7800312012480499,
+      "grad_norm": 1.04149329662323,
+      "learning_rate": 0.00014802392095683827,
+      "loss": 1.3857,
+      "step": 2000
+    },
+    {
+      "epoch": 0.7995319812792512,
+      "grad_norm": 0.8921425938606262,
+      "learning_rate": 0.00014672386895475822,
+      "loss": 1.3882,
+      "step": 2050
+    },
+    {
+      "epoch": 0.8190327613104524,
+      "grad_norm": 1.4201322793960571,
+      "learning_rate": 0.0001454238169526781,
+      "loss": 1.3885,
+      "step": 2100
+    },
+    {
+      "epoch": 0.8385335413416537,
+      "grad_norm": 1.2703887224197388,
+      "learning_rate": 0.00014412376495059802,
+      "loss": 1.3868,
+      "step": 2150
+    },
+    {
+      "epoch": 0.858034321372855,
+      "grad_norm": 1.2008202075958252,
+      "learning_rate": 0.00014282371294851794,
+      "loss": 1.3862,
+      "step": 2200
+    },
+    {
+      "epoch": 0.8775351014040562,
+      "grad_norm": 1.705498456954956,
+      "learning_rate": 0.0001415236609464379,
+      "loss": 1.3837,
+      "step": 2250
+    },
+    {
+      "epoch": 0.8970358814352574,
+      "grad_norm": 1.1787363290786743,
+      "learning_rate": 0.00014022360894435778,
+      "loss": 1.391,
+      "step": 2300
+    },
+    {
+      "epoch": 0.9165366614664586,
+      "grad_norm": 0.9128132462501526,
+      "learning_rate": 0.0001389235569422777,
+      "loss": 1.3829,
+      "step": 2350
+    },
+    {
+      "epoch": 0.9360374414976599,
+      "grad_norm": 1.2079758644104004,
+      "learning_rate": 0.0001376235049401976,
+      "loss": 1.3879,
+      "step": 2400
+    },
+    {
+      "epoch": 0.9555382215288611,
+      "grad_norm": 1.4396191835403442,
+      "learning_rate": 0.00013632345293811753,
+      "loss": 1.3859,
+      "step": 2450
+    },
+    {
+      "epoch": 0.9750390015600624,
+      "grad_norm": 0.9314255118370056,
+      "learning_rate": 0.00013502340093603745,
+      "loss": 1.39,
+      "step": 2500
+    },
+    {
+      "epoch": 0.9945397815912637,
+      "grad_norm": 0.9400072693824768,
+      "learning_rate": 0.00013372334893395736,
+      "loss": 1.3861,
+      "step": 2550
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.386317253112793,
+      "eval_runtime": 16.917,
+      "eval_samples_per_second": 75.782,
+      "eval_steps_per_second": 18.975,
+      "step": 2564
+    },
+    {
+      "epoch": 1.0140405616224648,
+      "grad_norm": 1.151302456855774,
+      "learning_rate": 0.00013242329693187728,
+      "loss": 1.3844,
+      "step": 2600
+    },
+    {
+      "epoch": 1.033541341653666,
+      "grad_norm": 1.0478647947311401,
+      "learning_rate": 0.0001311232449297972,
+      "loss": 1.3897,
+      "step": 2650
+    },
+    {
+      "epoch": 1.0530421216848673,
+      "grad_norm": 1.1034784317016602,
+      "learning_rate": 0.00012982319292771712,
+      "loss": 1.3896,
+      "step": 2700
+    },
+    {
+      "epoch": 1.0725429017160686,
+      "grad_norm": 0.9335495829582214,
+      "learning_rate": 0.00012852314092563703,
+      "loss": 1.3879,
+      "step": 2750
+    },
+    {
+      "epoch": 1.0920436817472698,
+      "grad_norm": 1.020143985748291,
+      "learning_rate": 0.00012722308892355695,
+      "loss": 1.3892,
+      "step": 2800
+    },
+    {
+      "epoch": 1.111544461778471,
+      "grad_norm": 1.0304250717163086,
+      "learning_rate": 0.00012592303692147687,
+      "loss": 1.3872,
+      "step": 2850
+    },
+    {
+      "epoch": 1.1310452418096724,
+      "grad_norm": 1.080430269241333,
+      "learning_rate": 0.0001246229849193968,
+      "loss": 1.386,
+      "step": 2900
+    },
+    {
+      "epoch": 1.1505460218408736,
+      "grad_norm": 1.6232908964157104,
+      "learning_rate": 0.0001233229329173167,
+      "loss": 1.3822,
+      "step": 2950
+    },
+    {
+      "epoch": 1.1700468018720749,
+      "grad_norm": 1.36116361618042,
+      "learning_rate": 0.00012202288091523661,
+      "loss": 1.3874,
+      "step": 3000
+    },
+    {
+      "epoch": 1.1895475819032761,
+      "grad_norm": 1.1913774013519287,
+      "learning_rate": 0.00012072282891315654,
+      "loss": 1.3865,
+      "step": 3050
+    },
+    {
+      "epoch": 1.2090483619344774,
+      "grad_norm": 1.0453519821166992,
+      "learning_rate": 0.00011942277691107644,
+      "loss": 1.3908,
+      "step": 3100
+    },
+    {
+      "epoch": 1.2285491419656787,
+      "grad_norm": 1.23416268825531,
+      "learning_rate": 0.00011812272490899636,
+      "loss": 1.3847,
+      "step": 3150
+    },
+    {
+      "epoch": 1.24804992199688,
+      "grad_norm": 0.9433267116546631,
+      "learning_rate": 0.00011682267290691629,
+      "loss": 1.3895,
+      "step": 3200
+    },
+    {
+      "epoch": 1.2675507020280812,
+      "grad_norm": 1.4030227661132812,
+      "learning_rate": 0.00011552262090483621,
+      "loss": 1.3889,
+      "step": 3250
+    },
+    {
+      "epoch": 1.2870514820592824,
+      "grad_norm": 1.1687849760055542,
+      "learning_rate": 0.00011422256890275611,
+      "loss": 1.3878,
+      "step": 3300
+    },
+    {
+      "epoch": 1.3065522620904837,
+      "grad_norm": 1.3713322877883911,
+      "learning_rate": 0.00011292251690067603,
+      "loss": 1.3817,
+      "step": 3350
+    },
+    {
+      "epoch": 1.3260530421216847,
+      "grad_norm": 1.982105016708374,
+      "learning_rate": 0.00011162246489859596,
+      "loss": 1.3873,
+      "step": 3400
+    },
+    {
+      "epoch": 1.345553822152886,
+      "grad_norm": 1.0167853832244873,
+      "learning_rate": 0.00011032241289651587,
+      "loss": 1.3826,
+      "step": 3450
+    },
+    {
+      "epoch": 1.3650546021840873,
+      "grad_norm": 1.2796357870101929,
+      "learning_rate": 0.00010902236089443578,
+      "loss": 1.3894,
+      "step": 3500
+    },
+    {
+      "epoch": 1.3845553822152885,
+      "grad_norm": 1.0817245244979858,
+      "learning_rate": 0.0001077223088923557,
+      "loss": 1.3892,
+      "step": 3550
+    },
+    {
+      "epoch": 1.4040561622464898,
+      "grad_norm": 1.192622184753418,
+      "learning_rate": 0.0001064222568902756,
+      "loss": 1.3895,
+      "step": 3600
+    },
+    {
+      "epoch": 1.423556942277691,
+      "grad_norm": 1.1314730644226074,
+      "learning_rate": 0.00010512220488819554,
+      "loss": 1.388,
+      "step": 3650
+    },
+    {
+      "epoch": 1.4430577223088923,
+      "grad_norm": 1.026350736618042,
+      "learning_rate": 0.00010382215288611545,
+      "loss": 1.3827,
+      "step": 3700
+    },
+    {
+      "epoch": 1.4625585023400935,
+      "grad_norm": 1.116678237915039,
+      "learning_rate": 0.00010252210088403537,
+      "loss": 1.388,
+      "step": 3750
+    },
+    {
+      "epoch": 1.4820592823712948,
+      "grad_norm": 0.9644101858139038,
+      "learning_rate": 0.00010122204888195527,
+      "loss": 1.384,
+      "step": 3800
+    },
+    {
+      "epoch": 1.501560062402496,
+      "grad_norm": 1.1748864650726318,
+      "learning_rate": 9.99219968798752e-05,
+      "loss": 1.385,
+      "step": 3850
+    },
+    {
+      "epoch": 1.5210608424336973,
+      "grad_norm": 1.0787124633789062,
+      "learning_rate": 9.862194487779511e-05,
+      "loss": 1.3812,
+      "step": 3900
+    },
+    {
+      "epoch": 1.5405616224648986,
+      "grad_norm": 1.315277099609375,
+      "learning_rate": 9.732189287571504e-05,
+      "loss": 1.3858,
+      "step": 3950
+    },
+    {
+      "epoch": 1.5600624024960998,
+      "grad_norm": 1.0303170680999756,
+      "learning_rate": 9.602184087363494e-05,
+      "loss": 1.3911,
+      "step": 4000
+    },
+    {
+      "epoch": 1.579563182527301,
+      "grad_norm": 1.4852935075759888,
+      "learning_rate": 9.472178887155486e-05,
+      "loss": 1.3842,
+      "step": 4050
+    },
+    {
+      "epoch": 1.5990639625585024,
+      "grad_norm": 1.1838001012802124,
+      "learning_rate": 9.344773790951638e-05,
+      "loss": 1.3883,
+      "step": 4100
+    },
+    {
+      "epoch": 1.6185647425897036,
+      "grad_norm": 1.374031662940979,
+      "learning_rate": 9.21476859074363e-05,
+      "loss": 1.3914,
+      "step": 4150
+    },
+    {
+      "epoch": 1.6380655226209049,
+      "grad_norm": 0.98434978723526,
+      "learning_rate": 9.084763390535622e-05,
+      "loss": 1.3883,
+      "step": 4200
+    },
+    {
+      "epoch": 1.6575663026521061,
+      "grad_norm": 1.2400907278060913,
+      "learning_rate": 8.954758190327614e-05,
+      "loss": 1.3924,
+      "step": 4250
+    },
+    {
+      "epoch": 1.6770670826833074,
+      "grad_norm": 0.9704477787017822,
+      "learning_rate": 8.824752990119605e-05,
+      "loss": 1.3915,
+      "step": 4300
+    },
+    {
+      "epoch": 1.6965678627145087,
+      "grad_norm": 0.9739221930503845,
+      "learning_rate": 8.694747789911597e-05,
+      "loss": 1.3907,
+      "step": 4350
+    },
+    {
+      "epoch": 1.71606864274571,
+      "grad_norm": 1.2051200866699219,
+      "learning_rate": 8.564742589703589e-05,
+      "loss": 1.3812,
+      "step": 4400
+    },
+    {
+      "epoch": 1.7355694227769112,
+      "grad_norm": 0.976754903793335,
+      "learning_rate": 8.43473738949558e-05,
+      "loss": 1.3872,
+      "step": 4450
+    },
+    {
+      "epoch": 1.7550702028081124,
+      "grad_norm": 1.115320086479187,
+      "learning_rate": 8.304732189287572e-05,
+      "loss": 1.3873,
+      "step": 4500
+    },
+    {
+      "epoch": 1.7745709828393137,
+      "grad_norm": 1.04775869846344,
+      "learning_rate": 8.174726989079563e-05,
+      "loss": 1.3857,
+      "step": 4550
+    },
+    {
+      "epoch": 1.794071762870515,
+      "grad_norm": 1.0488988161087036,
+      "learning_rate": 8.044721788871556e-05,
+      "loss": 1.3851,
+      "step": 4600
+    },
+    {
+      "epoch": 1.8135725429017162,
+      "grad_norm": 1.1655857563018799,
+      "learning_rate": 7.914716588663546e-05,
+      "loss": 1.383,
+      "step": 4650
+    },
+    {
+      "epoch": 1.8330733229329175,
+      "grad_norm": 1.433255910873413,
+      "learning_rate": 7.784711388455539e-05,
+      "loss": 1.3864,
+      "step": 4700
+    },
+    {
+      "epoch": 1.8525741029641187,
+      "grad_norm": 1.045211672782898,
+      "learning_rate": 7.65470618824753e-05,
+      "loss": 1.3877,
+      "step": 4750
+    },
+    {
+      "epoch": 1.8720748829953198,
+      "grad_norm": 1.2182040214538574,
+      "learning_rate": 7.524700988039521e-05,
+      "loss": 1.3812,
+      "step": 4800
+    },
+    {
+      "epoch": 1.891575663026521,
+      "grad_norm": 1.1119019985198975,
+      "learning_rate": 7.394695787831513e-05,
+      "loss": 1.3874,
+      "step": 4850
+    },
+    {
+      "epoch": 1.9110764430577223,
+      "grad_norm": 1.2834023237228394,
+      "learning_rate": 7.264690587623505e-05,
+      "loss": 1.3887,
+      "step": 4900
+    },
+    {
+      "epoch": 1.9305772230889235,
+      "grad_norm": 1.3699750900268555,
+      "learning_rate": 7.134685387415497e-05,
+      "loss": 1.3833,
+      "step": 4950
+    },
+    {
+      "epoch": 1.9500780031201248,
+      "grad_norm": 1.0018161535263062,
+      "learning_rate": 7.004680187207488e-05,
+      "loss": 1.3847,
+      "step": 5000
+    },
+    {
+      "epoch": 1.969578783151326,
+      "grad_norm": 1.7948187589645386,
+      "learning_rate": 6.87467498699948e-05,
+      "loss": 1.3799,
+      "step": 5050
+    },
+    {
+      "epoch": 1.9890795631825273,
+      "grad_norm": 1.2683899402618408,
+      "learning_rate": 6.744669786791472e-05,
+      "loss": 1.3878,
+      "step": 5100
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.3883001804351807,
+      "eval_runtime": 16.7843,
+      "eval_samples_per_second": 76.381,
+      "eval_steps_per_second": 19.125,
+      "step": 5128
+    },
+    {
+      "epoch": 2.0085803432137284,
+      "grad_norm": 1.0674426555633545,
+      "learning_rate": 6.614664586583464e-05,
+      "loss": 1.385,
+      "step": 5150
+    },
+    {
+      "epoch": 2.0280811232449296,
+      "grad_norm": 2.8433749675750732,
+      "learning_rate": 6.484659386375455e-05,
+      "loss": 1.3951,
+      "step": 5200
+    },
+    {
+      "epoch": 2.047581903276131,
+      "grad_norm": 1.4678908586502075,
+      "learning_rate": 6.354654186167447e-05,
+      "loss": 1.3832,
+      "step": 5250
+    },
+    {
+      "epoch": 2.067082683307332,
+      "grad_norm": 1.1326549053192139,
+      "learning_rate": 6.224648985959438e-05,
+      "loss": 1.3886,
+      "step": 5300
+    },
+    {
+      "epoch": 2.0865834633385334,
+      "grad_norm": 1.351509690284729,
+      "learning_rate": 6.094643785751431e-05,
+      "loss": 1.3835,
+      "step": 5350
+    },
+    {
+      "epoch": 2.1060842433697347,
+      "grad_norm": 1.6677987575531006,
+      "learning_rate": 5.964638585543422e-05,
+      "loss": 1.387,
+      "step": 5400
+    },
+    {
+      "epoch": 2.125585023400936,
+      "grad_norm": 1.6974619626998901,
+      "learning_rate": 5.834633385335414e-05,
+      "loss": 1.395,
+      "step": 5450
+    },
+    {
+      "epoch": 2.145085803432137,
+      "grad_norm": 1.0266507863998413,
+      "learning_rate": 5.704628185127405e-05,
+      "loss": 1.3881,
+      "step": 5500
+    },
+    {
+      "epoch": 2.1645865834633384,
+      "grad_norm": 1.1399188041687012,
+      "learning_rate": 5.574622984919396e-05,
+      "loss": 1.3826,
+      "step": 5550
+    },
+    {
+      "epoch": 2.1840873634945397,
+      "grad_norm": 1.5752311944961548,
+      "learning_rate": 5.444617784711389e-05,
+      "loss": 1.3744,
+      "step": 5600
+    },
+    {
+      "epoch": 2.203588143525741,
+      "grad_norm": 1.1733818054199219,
+      "learning_rate": 5.31461258450338e-05,
+      "loss": 1.3851,
+      "step": 5650
+    },
+    {
+      "epoch": 2.223088923556942,
+      "grad_norm": 2.550503969192505,
+      "learning_rate": 5.184607384295372e-05,
+      "loss": 1.386,
+      "step": 5700
+    },
+    {
+      "epoch": 2.2425897035881435,
+      "grad_norm": 2.088324785232544,
+      "learning_rate": 5.054602184087363e-05,
+      "loss": 1.3871,
+      "step": 5750
+    },
+    {
+      "epoch": 2.2620904836193447,
+      "grad_norm": 1.282410740852356,
+      "learning_rate": 4.924596983879356e-05,
+      "loss": 1.3835,
+      "step": 5800
+    },
+    {
+      "epoch": 2.281591263650546,
+      "grad_norm": 1.6578506231307983,
+      "learning_rate": 4.7945917836713475e-05,
+      "loss": 1.3833,
+      "step": 5850
+    },
+    {
+      "epoch": 2.3010920436817472,
+      "grad_norm": 1.0126018524169922,
+      "learning_rate": 4.664586583463339e-05,
+      "loss": 1.3823,
+      "step": 5900
+    },
+    {
+      "epoch": 2.3205928237129485,
+      "grad_norm": 2.126206159591675,
+      "learning_rate": 4.53458138325533e-05,
+      "loss": 1.3819,
+      "step": 5950
+    },
+    {
+      "epoch": 2.3400936037441498,
+      "grad_norm": 1.1407690048217773,
+      "learning_rate": 4.404576183047322e-05,
+      "loss": 1.3775,
+      "step": 6000
+    },
+    {
+      "epoch": 2.359594383775351,
+      "grad_norm": 0.950842022895813,
+      "learning_rate": 4.274570982839314e-05,
+      "loss": 1.3832,
+      "step": 6050
+    },
+    {
+      "epoch": 2.3790951638065523,
+      "grad_norm": Infinity,
+      "learning_rate": 4.147165886635466e-05,
+      "loss": 1.396,
+      "step": 6100
+    },
+    {
+      "epoch": 2.3985959438377535,
+      "grad_norm": 1.2606666088104248,
+      "learning_rate": 4.017160686427458e-05,
+      "loss": 1.3904,
+      "step": 6150
+    },
+    {
+      "epoch": 2.418096723868955,
+      "grad_norm": 1.041694164276123,
+      "learning_rate": 3.887155486219449e-05,
+      "loss": 1.3736,
+      "step": 6200
+    },
+    {
+      "epoch": 2.437597503900156,
+      "grad_norm": 0.8443174362182617,
+      "learning_rate": 3.7571502860114405e-05,
+      "loss": 1.3889,
+      "step": 6250
+    },
+    {
+      "epoch": 2.4570982839313573,
+      "grad_norm": 1.0350373983383179,
+      "learning_rate": 3.627145085803432e-05,
+      "loss": 1.3852,
+      "step": 6300
+    },
+    {
+      "epoch": 2.4765990639625586,
+      "grad_norm": 1.169245958328247,
+      "learning_rate": 3.497139885595424e-05,
+      "loss": 1.3916,
+      "step": 6350
+    },
+    {
+      "epoch": 2.49609984399376,
+      "grad_norm": 1.7383908033370972,
+      "learning_rate": 3.367134685387416e-05,
+      "loss": 1.3808,
+      "step": 6400
+    },
+    {
+      "epoch": 2.515600624024961,
+      "grad_norm": 1.8447555303573608,
+      "learning_rate": 3.237129485179407e-05,
+      "loss": 1.3826,
+      "step": 6450
+    },
+    {
+      "epoch": 2.5351014040561624,
+      "grad_norm": 1.236523151397705,
+      "learning_rate": 3.1071242849713986e-05,
+      "loss": 1.3744,
+      "step": 6500
+    },
+    {
+      "epoch": 2.5546021840873636,
+      "grad_norm": 1.1100080013275146,
+      "learning_rate": 2.9771190847633907e-05,
+      "loss": 1.3856,
+      "step": 6550
+    },
+    {
+      "epoch": 2.574102964118565,
+      "grad_norm": 2.6339101791381836,
+      "learning_rate": 2.8471138845553824e-05,
+      "loss": 1.4006,
+      "step": 6600
+    },
+    {
+      "epoch": 2.593603744149766,
+      "grad_norm": 1.040090799331665,
+      "learning_rate": 2.7171086843473742e-05,
+      "loss": 1.3847,
+      "step": 6650
+    },
+    {
+      "epoch": 2.6131045241809674,
+      "grad_norm": 1.2017114162445068,
+      "learning_rate": 2.5871034841393656e-05,
+      "loss": 1.3929,
+      "step": 6700
+    },
+    {
+      "epoch": 2.6326053042121687,
+      "grad_norm": 1.175416111946106,
+      "learning_rate": 2.4570982839313573e-05,
+      "loss": 1.3807,
+      "step": 6750
+    },
+    {
+      "epoch": 2.6521060842433695,
+      "grad_norm": 1.074458360671997,
+      "learning_rate": 2.327093083723349e-05,
+      "loss": 1.402,
+      "step": 6800
+    },
+    {
+      "epoch": 2.6716068642745707,
+      "grad_norm": 1.0837610960006714,
+      "learning_rate": 2.1970878835153408e-05,
+      "loss": 1.394,
+      "step": 6850
+    },
+    {
+      "epoch": 2.691107644305772,
+      "grad_norm": 1.3517810106277466,
+      "learning_rate": 2.0670826833073322e-05,
+      "loss": 1.3884,
+      "step": 6900
+    },
+    {
+      "epoch": 2.7106084243369732,
+      "grad_norm": 0.9812959432601929,
+      "learning_rate": 1.937077483099324e-05,
+      "loss": 1.3785,
+      "step": 6950
+    },
+    {
+      "epoch": 2.7301092043681745,
+      "grad_norm": 1.0006356239318848,
+      "learning_rate": 1.8070722828913157e-05,
+      "loss": 1.3773,
+      "step": 7000
+    },
+    {
+      "epoch": 2.7496099843993758,
+      "grad_norm": 1.3178588151931763,
+      "learning_rate": 1.6770670826833075e-05,
+      "loss": 1.3865,
+      "step": 7050
+    },
+    {
+      "epoch": 2.769110764430577,
+      "grad_norm": 1.4253509044647217,
+      "learning_rate": 1.5470618824752992e-05,
+      "loss": 1.3676,
+      "step": 7100
+    },
+    {
+      "epoch": 2.7886115444617783,
+      "grad_norm": 1.2045459747314453,
+      "learning_rate": 1.4170566822672906e-05,
+      "loss": 1.391,
+      "step": 7150
+    },
+    {
+      "epoch": 2.8081123244929795,
+      "grad_norm": 1.6241490840911865,
+      "learning_rate": 1.2870514820592824e-05,
+      "loss": 1.3934,
+      "step": 7200
+    },
+    {
+      "epoch": 2.827613104524181,
+      "grad_norm": 1.3262916803359985,
+      "learning_rate": 1.1570462818512741e-05,
+      "loss": 1.3883,
+      "step": 7250
+    },
+    {
+      "epoch": 2.847113884555382,
+      "grad_norm": 1.2156164646148682,
+      "learning_rate": 1.0270410816432657e-05,
+      "loss": 1.375,
+      "step": 7300
+    },
+    {
+      "epoch": 2.8666146645865833,
+      "grad_norm": 1.203420877456665,
+      "learning_rate": 8.970358814352575e-06,
+      "loss": 1.3872,
+      "step": 7350
+    },
+    {
+      "epoch": 2.8861154446177846,
+      "grad_norm": 1.2399318218231201,
+      "learning_rate": 7.670306812272492e-06,
+      "loss": 1.3845,
+      "step": 7400
+    },
+    {
+      "epoch": 2.905616224648986,
+      "grad_norm": 1.0349050760269165,
+      "learning_rate": 6.370254810192408e-06,
+      "loss": 1.3773,
+      "step": 7450
+    },
+    {
+      "epoch": 2.925117004680187,
+      "grad_norm": 1.736341118812561,
+      "learning_rate": 5.070202808112324e-06,
+      "loss": 1.3739,
+      "step": 7500
+    },
+    {
+      "epoch": 2.9446177847113884,
+      "grad_norm": 1.072678804397583,
+      "learning_rate": 3.770150806032242e-06,
+      "loss": 1.3998,
+      "step": 7550
+    },
+    {
+      "epoch": 2.9641185647425896,
+      "grad_norm": 0.9114436507225037,
+      "learning_rate": 2.470098803952158e-06,
+      "loss": 1.3868,
+      "step": 7600
+    },
+    {
+      "epoch": 2.983619344773791,
+      "grad_norm": 2.6503679752349854,
+      "learning_rate": 1.170046801872075e-06,
+      "loss": 1.3844,
+      "step": 7650
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.3892767429351807,
+      "eval_runtime": 16.8322,
+      "eval_samples_per_second": 76.164,
+      "eval_steps_per_second": 19.071,
+      "step": 7692
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 7692,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.248993509945344e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-7692/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44ca3034cf87097f30254f9aacafe4cc22fa749525bb201546c79db28d1797eb
+size 5240

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}