Add: add README

.ipynb_checkpoints/README-checkpoint.md

@@ -0,0 +1,95 @@
+---
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- feature-extraction
+- sentence-similarity
+- transformers
+- sentence-embedding
+license: apache-2.0
+language:
+- fr
+metrics:
+- pearsonr
+- spearmanr
+---
+
+# [bilingual-embedding-base](https://huggingface.co/Lajavaness/bilingual-embedding-base)
+
+bilingual-embedding is the Embedding Model for bilingual language: french and english. This model is a specialized sentence-embedding trained specifically for the bilingual language, leveraging the robust capabilities of [XLM-RoBERTa](https://huggingface.co/FacebookAI/xlm-roberta-base), a pre-trained language model based on the [XLM-RoBERTa](https://huggingface.co/FacebookAI/xlm-roberta-base) architecture. The model utilizes xlm-roberta to encode english-french sentences into a 1024-dimensional vector space, facilitating a wide range of applications from semantic search to text clustering. The embeddings capture the nuanced meanings of english-french sentences, reflecting both the lexical and contextual layers of the language.
+
+
+## Full Model Architecture
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BilingualModel 
+  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Training and Fine-tuning process
+#### Stage 1: NLI Training
+- Dataset: [(SNLI+XNLI) for english+french]
+- Method: Training using Multi-Negative Ranking Loss. This stage focused on improving the model's ability to discern and rank nuanced differences in sentence semantics.
+### Stage 3: Continued Fine-tuning for Semantic Textual Similarity on STS Benchmark
+- Dataset: [STSB-fr and en]
+- Method: Fine-tuning specifically for the semantic textual similarity benchmark using Siamese BERT-Networks configured with the 'sentence-transformers' library. 
+### Stage 4: Advanced Augmentation Fine-tuning
+- Dataset: STSB-vn with generate [silver sample from gold sample](https://www.sbert.net/examples/training/data_augmentation/README.html)
+- Method: Employed an advanced strategy using [Augmented SBERT](https://arxiv.org/abs/2010.08240) with Pair Sampling Strategies, integrating both Cross-Encoder and Bi-Encoder models. This stage further refined the embeddings by enriching the training data dynamically, enhancing the model's robustness and accuracy.
+
+
+## Usage:
+
+Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
+
+```
+pip install -U sentence-transformers
+pip install -q pyvi
+```
+
+Then you can use the model like this:
+
+```python
+from sentence_transformers import SentenceTransformer
+from pyvi.ViTokenizer import tokenize
+
+sentences = ["Paris est une capitale de la France", "Paris is a capital of France"]
+
+model = SentenceTransformer('Lajavaness/bilingual-embedding-base', trust_remote_code=True)
+print(embeddings)
+
+```
+
+
+
+
+
+## Evaluation
+
+TODO
+
+## Citation
+
+    @article{conneau2019unsupervised,
+      title={Unsupervised cross-lingual representation learning at scale},
+      author={Conneau, Alexis and Khandelwal, Kartikay and Goyal, Naman and Chaudhary, Vishrav and Wenzek, Guillaume and Guzm{\'a}n, Francisco and Grave, Edouard and Ott, Myle and Zettlemoyer, Luke and Stoyanov, Veselin},
+      journal={arXiv preprint arXiv:1911.02116},
+      year={2019}
+    }
+
+	@article{reimers2019sentence,
+	   title={Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks},
+	   author={Nils Reimers, Iryna Gurevych},
+	   journal={https://arxiv.org/abs/1908.10084},
+	   year={2019}
+	}
+
+    @article{thakur2020augmented,
+      title={Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks},
+      author={Thakur, Nandan and Reimers, Nils and Daxenberger, Johannes and Gurevych, Iryna},
+      journal={arXiv e-prints},
+      pages={arXiv--2010},
+      year={2020}

README.md

@@ -5,86 +5,91 @@ tags:
 - sentence-transformers
 - feature-extraction
 - sentence-similarity
-
+- transformers
+- sentence-embedding
+license: apache-2.0
+language:
+- fr
+metrics:
+- pearsonr
+- spearmanr
 ---
 
-# {MODEL_NAME}
+# [bilingual-embedding-base](https://huggingface.co/Lajavaness/bilingual-embedding-base)
+
+bilingual-embedding is the Embedding Model for bilingual language: french and english. This model is a specialized sentence-embedding trained specifically for the bilingual language, leveraging the robust capabilities of [XLM-RoBERTa](https://huggingface.co/FacebookAI/xlm-roberta-base), a pre-trained language model based on the [XLM-RoBERTa](https://huggingface.co/FacebookAI/xlm-roberta-base) architecture. The model utilizes xlm-roberta to encode english-french sentences into a 1024-dimensional vector space, facilitating a wide range of applications from semantic search to text clustering. The embeddings capture the nuanced meanings of english-french sentences, reflecting both the lexical and contextual layers of the language.
+
+
+## Full Model Architecture
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BilingualModel 
+  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
 
-This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
+## Training and Fine-tuning process
+#### Stage 1: NLI Training
+- Dataset: [(SNLI+XNLI) for english+french]
+- Method: Training using Multi-Negative Ranking Loss. This stage focused on improving the model's ability to discern and rank nuanced differences in sentence semantics.
+### Stage 3: Continued Fine-tuning for Semantic Textual Similarity on STS Benchmark
+- Dataset: [STSB-fr and en]
+- Method: Fine-tuning specifically for the semantic textual similarity benchmark using Siamese BERT-Networks configured with the 'sentence-transformers' library. 
+### Stage 4: Advanced Augmentation Fine-tuning
+- Dataset: STSB-vn with generate [silver sample from gold sample](https://www.sbert.net/examples/training/data_augmentation/README.html)
+- Method: Employed an advanced strategy using [Augmented SBERT](https://arxiv.org/abs/2010.08240) with Pair Sampling Strategies, integrating both Cross-Encoder and Bi-Encoder models. This stage further refined the embeddings by enriching the training data dynamically, enhancing the model's robustness and accuracy.
 
-<!--- Describe your model here -->
 
-## Usage (Sentence-Transformers)
+## Usage:
 
 Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
 
 ```
 pip install -U sentence-transformers
+pip install -q pyvi
 ```
 
 Then you can use the model like this:
 
 ```python
 from sentence_transformers import SentenceTransformer
-sentences = ["This is an example sentence", "Each sentence is converted"]
-
-model = SentenceTransformer('{MODEL_NAME}')
-embeddings = model.encode(sentences)
-print(embeddings)
-```
-
+from pyvi.ViTokenizer import tokenize
 
+sentences = ["Paris est une capitale de la France", "Paris is a capital of France"]
 
-## Evaluation Results
+model = SentenceTransformer('Lajavaness/bilingual-embedding-base', trust_remote_code=True)
+print(embeddings)
 
-<!--- Describe how your model was evaluated -->
+```
 
-For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})
 
 
-## Training
-The model was trained with the parameters:
 
-**DataLoader**:
 
-`torch.utils.data.dataloader.DataLoader` of length 1438 with parameters:
-```
-{'batch_size': 8, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
-```
+## Evaluation
 
-**Loss**:
-
-`__main__.CosineSimilarityLoss` 
-
-Parameters of the fit()-Method:
-```
-{
-    "epochs": 10,
-    "evaluation_steps": 1000,
-    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
-    "max_grad_norm": 1,
-    "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
-    "optimizer_params": {
-        "eps": 1e-06,
-        "lr": 5e-07
-    },
-    "scheduler": "WarmupLinear",
-    "steps_per_epoch": null,
-    "warmup_steps": 1438,
-    "weight_decay": 0.01
-}
-```
+TODO
 
+## Citation
 
-## Full Model Architecture
-```
-SentenceTransformer(
-  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
-  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
-  (2): Normalize()
-)
-```
+    @article{conneau2019unsupervised,
+      title={Unsupervised cross-lingual representation learning at scale},
+      author={Conneau, Alexis and Khandelwal, Kartikay and Goyal, Naman and Chaudhary, Vishrav and Wenzek, Guillaume and Guzm{\'a}n, Francisco and Grave, Edouard and Ott, Myle and Zettlemoyer, Luke and Stoyanov, Veselin},
+      journal={arXiv preprint arXiv:1911.02116},
+      year={2019}
+    }
 
-## Citing & Authors
+	@article{reimers2019sentence,
+	   title={Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks},
+	   author={Nils Reimers, Iryna Gurevych},
+	   journal={https://arxiv.org/abs/1908.10084},
+	   year={2019}
+	}
 
-<!--- Describe where people can find more information -->
+    @article{thakur2020augmented,
+      title={Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks},
+      author={Thakur, Nandan and Reimers, Nils and Daxenberger, Johannes and Gurevych, Iryna},
+      journal={arXiv e-prints},
+      pages={arXiv--2010},
+      year={2020}