Update README.md
Browse files
README.md
CHANGED
|
@@ -7,4 +7,48 @@ language:
|
|
| 7 |
base_model:
|
| 8 |
- google/embeddinggemma-300m
|
| 9 |
library_name: transformers
|
| 10 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
base_model:
|
| 8 |
- google/embeddinggemma-300m
|
| 9 |
library_name: transformers
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# EmbeddingGemma-300M Fine Tuned for LLM Prompt Jailbreak Classification
|
| 13 |
+
|
| 14 |
+
The [google/embeddinggemma-300m](https://huggingface.co/google/embeddinggemma-300m) 300M embedding model trained on jailbreak data from [allenai/wildjailbreak](https://huggingface.co/datasets/allenai/wildjailbreak) for classification.
|
| 15 |
+
|
| 16 |
+
# Using the Model
|
| 17 |
+
|
| 18 |
+
```python
|
| 19 |
+
XXX
|
| 20 |
+
```
|
| 21 |
+
|
| 22 |
+
## Training Details
|
| 23 |
+
|
| 24 |
+
Trained for 1 Hour on an A100 with the following parameters
|
| 25 |
+
|
| 26 |
+
| Parameter | Value |
|
| 27 |
+
|-----------|-------|
|
| 28 |
+
| num_train_epochs | 1 |
|
| 29 |
+
| per_device_train_batch_size | 32 |
|
| 30 |
+
| gradient_accumulation_steps | 2 |
|
| 31 |
+
| per_device_eval_batch_size | 64 |
|
| 32 |
+
| learning_rate | 2e-5 |
|
| 33 |
+
| warmup_ratio | 0.1 |
|
| 34 |
+
| weight_decay | 0.01 |
|
| 35 |
+
| fp16 | True |
|
| 36 |
+
| eval_strategy | "steps" |
|
| 37 |
+
| eval_steps | 500 |
|
| 38 |
+
| save_strategy | "steps" |
|
| 39 |
+
| save_steps | 500 |
|
| 40 |
+
| logging_steps | 100 |
|
| 41 |
+
| load_best_model_at_end | True |
|
| 42 |
+
| metric_for_best_model | "eval_loss" |
|
| 43 |
+
|
| 44 |
+
Resulting in the following training metrics:
|
| 45 |
+
|
| 46 |
+
| Step | Training Loss | Validation Loss | Accuracy | F1 | Precision | Recall |
|
| 47 |
+
|------|---------------|-----------------|----------|-------|-----------|--------|
|
| 48 |
+
| 500 | 0.112500 | 0.084654 | 0.980960 | 0.980949 | 0.981595 | 0.980960 |
|
| 49 |
+
| 1000 | 0.071000 | 0.028393 | 0.993501 | 0.993500 | 0.993517 | 0.993501 |
|
| 50 |
+
| 1500 | 0.034400 | 0.022442 | 0.995642 | 0.995641 | 0.995650 | 0.995642 |
|
| 51 |
+
| 2000 | 0.041500 | 0.023433 | 0.994495 | 0.994495 | 0.994543 | 0.994495 |
|
| 52 |
+
| 2500 | 0.015800 | 0.011340 | 0.997859 | 0.997859 | 0.997859 | 0.997859 |
|
| 53 |
+
| 3000 | 0.018700 | 0.007396 | 0.998088 | 0.998088 | 0.998089 | 0.998088 |
|
| 54 |
+
| 3500 | 0.014900 | 0.004368 | 0.999006 | 0.999006 | 0.999006 | 0.999006 |
|