---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dense
- generated_from_trainer
- dataset_size:139891
- loss:MultipleNegativesRankingLoss
base_model: sentence-transformers/all-mpnet-base-v2
widget:
- source_sentence: The three of us went biking this weekend.
sentences:
- The person surfing emoji is used to represent the activity of surfing or being
at the beach. It can be used in contexts related to water sports, summer time,
vacations, or simply enjoying the beach life.
- This emoji is used to symbolize zombies or Halloween. It can also be used to indicate
someone feeling tired or exhausted.
- This emoji represents a family with a woman, girl, and boy. It can be used to
refer to a mother with her daughter and son, or any other combination of a woman
and children. It conveys a sense of family unity and bond.
- source_sentence: Happy birthday to my favorite Guatemalan cousin!
sentences:
- The flag of Guatemala emoji represents the national flag of Guatemala. It features
two horizontal bands of light blue and white with the Guatemalan coat of arms
in the center. This emoji is often used to show pride in Guatemala or to celebrate
Guatemalan culture.
- The flag of Poland emoji represents the country of Poland. It is often used in
the context of showing patriotism, celebrating Polish holidays, or expressing
pride in Polish heritage.
- The man judge emoji is commonly used to represent a judge or someone who adjudicates
legal matters. It can also symbolize fairness, justice, and authority in various
contexts.
- source_sentence: Passover is coming up soon!
sentences:
- The flag of French Guiana emoji is used to represent the overseas department and
region of France located in South America. It features the French tricolor flag
with a yellow star in the center symbolizing the Guianan people. This emoji is
often used in the context of national pride, cultural identity, and geographic
location.
- The mage emoji is commonly used to represent a magical wizard or sorcerer. It
can be used in fantasy-related conversations, or to express a sense of mystery
or enchantment.
- The monkey emoji is often used to represent mischief, playfulness, or even silliness.
It can also symbolize agility, intelligence, or curiosity. Additionally, it may
be used in reference to the Chinese zodiac sign of the Monkey.
- source_sentence: These beetles seem to be everywhere lately!
sentences:
- The beetle emoji is often used to represent insects, bugs, nature, and perseverance.
It can also symbolize transformation and growth.
- The flag of Cuba emoji represents the country of Cuba. It is commonly used to
show patriotism towards Cuba, celebrate Cuban culture, or during events related
to the country.
- The man facepalming emoji is used to express frustration, exasperation, or embarrassment.
It can also convey disbelief or disappointment. This emoji is often used humorously
to show a facepalm reaction to something dumb or silly.
- source_sentence: Inspired by the history behind Norfolk Island’s flag.
sentences:
- The 🧒 emoji is used to represent a child, typically a young boy. It can be used
in various contexts related to childhood, innocence, and youth. It may also symbolize
playfulness and carefree attitude.
- The flag of Norfolk Island emoji represents the unique flag of Norfolk Island,
which is an external territory of Australia. It is used to symbolize the island's
culture and identity.
- The gear emoji is commonly used to represent machinery, equipment, tools, or mechanics.
It can also symbolize maintenance, repair, or work involving gears and mechanical
parts.
pipeline_tag: sentence-similarity
library_name: sentence-transformers
---
# SentenceTransformer based on sentence-transformers/all-mpnet-base-v2
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
- **Maximum Sequence Length:** 384 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 384, 'do_lower_case': False, 'architecture': 'MPNetModel'})
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("zoharzaig/emoji-prediction-model")
# Run inference
sentences = [
'Inspired by the history behind Norfolk Island’s flag.',
"The flag of Norfolk Island emoji represents the unique flag of Norfolk Island, which is an external territory of Australia. It is used to symbolize the island's culture and identity.",
'The gear emoji is commonly used to represent machinery, equipment, tools, or mechanics. It can also symbolize maintenance, repair, or work involving gears and mechanical parts.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, 0.7065, -0.0235],
# [ 0.7065, 1.0000, -0.0110],
# [-0.0235, -0.0110, 1.0000]])
```
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 139,891 training samples
* Columns: sentence_0 and sentence_1
* Approximate statistics based on the first 1000 samples:
| | sentence_0 | sentence_1 |
|:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string |
| details | - min: 6 tokens
- mean: 12.16 tokens
- max: 29 tokens
| - min: 22 tokens
- mean: 46.23 tokens
- max: 89 tokens
|
* Samples:
| sentence_0 | sentence_1 |
|:-------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Lunch is scheduled for eleven today | The eleven o’clock emoji is used to indicate the time of 11:00 on a clock. It can be used to show that it is late morning, or to signify that an event is happening at this specific time. It can also be used in a more figurative sense to represent the idea of being right on time for something. |
| Just finished reading an inspiring article on trans rights. | The transgender symbol emoji is often used to represent individuals who identify as transgender or non-binary |
| I'm curious about the history behind Lesotho’s flag. | The flag of Lesotho represents the country of Lesotho in southern Africa. It is a tricolor flag of horizontal stripes with a blue triangle on the left side. The colors symbolize different aspects of the country's history and culture. |
* Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
```json
{
"scale": 20.0,
"similarity_fct": "cos_sim"
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `num_train_epochs`: 5
- `multi_dataset_batch_sampler`: round_robin
#### All Hyperparameters
Click to expand
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 5
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `hub_revision`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin
- `router_mapping`: {}
- `learning_rate_mapping`: {}