Sam Heutmaker
commited on
Commit
·
b413f49
1
Parent(s):
9a60d2f
update readme
Browse files
README.md
CHANGED
|
@@ -78,7 +78,6 @@ The model was trained on 1 million carefully curated single-frame samples from p
|
|
| 78 |
## Benchmarks
|
| 79 |
|
| 80 |
Performance metrics on our internal evaluation set:
|
| 81 |
-
|
| 82 |
| Model | Avg Judge Score | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU |
|
| 83 |
|-------|-----------------|---------|---------|---------|------|
|
| 84 |
| cliptagger_12b | **3.53** | **0.674** | **0.404** | **0.520** | **0.267** |
|
|
@@ -87,14 +86,16 @@ Performance metrics on our internal evaluation set:
|
|
| 87 |
|
| 88 |
### Benchmark Visualizations
|
| 89 |
|
| 90 |
-
<
|
| 91 |
-
<
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
<
|
| 96 |
-
|
| 97 |
-
|
|
|
|
|
|
|
| 98 |
|
| 99 |
FP8 quantization showed no measurable quality degradation compared to bf16 precision.
|
| 100 |
|
|
|
|
| 78 |
## Benchmarks
|
| 79 |
|
| 80 |
Performance metrics on our internal evaluation set:
|
|
|
|
| 81 |
| Model | Avg Judge Score | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU |
|
| 82 |
|-------|-----------------|---------|---------|---------|------|
|
| 83 |
| cliptagger_12b | **3.53** | **0.674** | **0.404** | **0.520** | **0.267** |
|
|
|
|
| 86 |
|
| 87 |
### Benchmark Visualizations
|
| 88 |
|
| 89 |
+
<table>
|
| 90 |
+
<tr>
|
| 91 |
+
<td><img src="./assets/judge-score.png" alt="Average Judge Score Comparison" width="100%" /></td>
|
| 92 |
+
<td><img src="./assets/rouge-1.png" alt="ROUGE-1 Score Comparison" width="100%" /></td>
|
| 93 |
+
</tr>
|
| 94 |
+
<tr>
|
| 95 |
+
<td><img src="./assets/rouge-L.png" alt="ROUGE-L Score Comparison" width="100%" /></td>
|
| 96 |
+
<td><img src="./assets/bleu.png" alt="BLEU Score Comparison" width="100%" /></td>
|
| 97 |
+
</tr>
|
| 98 |
+
</table>
|
| 99 |
|
| 100 |
FP8 quantization showed no measurable quality degradation compared to bf16 precision.
|
| 101 |
|