joe32140
/

ModernBERT-large-msmarco

Sentence Similarity

sentence-transformers

feature-extraction

Generated from Trainer

dataset_size:11662655

loss:CachedMultipleNegativesRankingLoss

text-embeddings-inference

Model card Files Files and versions

joe32140 commited on Dec 24, 2024

Commit

9c77ae2

·

verified ·

1 Parent(s): fdf5082

Update README.md

Files changed (1) hide show

README.md +7 -10

README.md CHANGED Viewed

@@ -279,16 +279,13 @@ You can finetune this model on your own dataset.
 |:--------------------|:----------|
 | **cosine_accuracy** | **0.994** |
-#### Subset of BEIR
-| dataset   |   ndcg@10 |
-|:----------|----------:|
-| SCIDOCS   |   0.15789 |
-| FiQA2018  |   0.33974 |
-| HotpotQA  |   0.51818 |
-| ArguAna   |   0.47797 |
-| NFCorpus  |   0.28443 |
-| SciFact   |   0.60626 |
-| TRECCOVID |   0.77495 |
 <!--
 ## Bias, Risks and Limitations

 |:--------------------|:----------|
 | **cosine_accuracy** | **0.994** |
+#### Retrieval tasks compared to original numbers in the paper
+|                         	| NFCorpus 	| SciFact 	| TREC-Covid 	|  FiQA 	| ArguAna 	| SciDocs 	| FEVER 	| Climate-FEVER 	| MLDR - OOD 	|
+|:-----------------------:	|:--------:	|:-------:	|:----------:	|:-----:	|:-------:	|:-------:	|-------	|---------------	|:----------:	|
+| ModernBERT-base         	| 23.7     	| 57.0    	| 72.1       	| 28.8  	| 35.7    	| 12.5    	| 59.9  	| 23.6          	| 27.4       	|
+| ModernBERT-base (ours)  	| 26.6    	| 61.6   	| 71.4      	| 30.7 	| 46.3   	| 13.6   	| 65.7  	| 22.6          	| 30.5      	|
+| ModernBERT-large        	| 26.2     	| 60.4    	| 74.1       	| 33.1  	| 38.2    	| 13.8    	| 62.7  	| 20.5          	| 34.3       	|
+| ModernBERT-large (ours) 	| 28.4    	| 63.6   	| 77.4      	| 34.3 	| 47.7   	| 15.7   	| 68.2  	| 22.9          	| 38.9      	|
 <!--
 ## Bias, Risks and Limitations