File size: 3,931 Bytes
13c28f2 240921d 5cdbf21 240921d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
---
title: README
emoji: 🌍
colorFrom: purple
colorTo: red
sdk: static
pinned: false
---
MTEB is a Python framework for evaluating embeddings and retrieval systems for both text and image.
MTEB covers more than 1000 languages and diverse tasks, from classics like classification and clustering to use-case specialized tasks such as legal, code, or healthcare retrieval.
You can get started using [`mteb`](https://github.com/embeddings-benchmark/mteb), check out our [documentation](https://embeddings-benchmark.github.io/mteb/usage/get_started/).
| Overview | |
|--------------------------------|--------------------------------------------------------------------------------------|
| 📈 [Leaderboard] | The interactive leaderboard of the benchmark |
| **Get Started**. | |
| 🏃 [Get Started] | Overview of how to use mteb |
| 🤖 [Defining Models] | How to use existing model and define custom ones |
| 📋 [Selecting tasks] | How to select tasks, benchmarks, splits etc. |
| 🏭 [Running Evaluation] | How to run the evaluations, including cache management, speeding up evaluations etc. |
| 📊 [Loading Results] | How to load and work with existing model results |
| **Overview**. | |
| 📋 [Tasks] | Overview of available tasks |
| 📐 [Benchmarks] | Overview of available benchmarks |
| 🤖 [Models] | Overview of available Models |
| **Contributing** | |
| 🤖 [Adding a model] | How to submit a model to MTEB and to the leaderboard |
| 👩💻 [Adding a dataset] | How to add a new task/dataset to MTEB |
| 👩💻 [Adding a benchmark] | How to add a new benchmark to MTEB and to the leaderboard |
| 🤝 [Contributing] | How to contribute to MTEB and set it up for development |
[Get Started]: https://embeddings-benchmark.github.io/mteb/usage/get_started/
[Defining Models]: https://embeddings-benchmark.github.io/mteb/usage/defining_the_model/
[Selecting tasks]: https://embeddings-benchmark.github.io/mteb/usage/selecting_tasks/
[Running Evaluation]: https://embeddings-benchmark.github.io/mteb/usage/running_the_evaluation/
[Loading Results]: https://embeddings-benchmark.github.io/mteb/usage/loading_results/
[Tasks]: https://embeddings-benchmark.github.io/mteb/overview/available_tasks/any2anymultilingualretrieval/
[Benchmarks]: https://embeddings-benchmark.github.io/mteb/overview/available_benchmarks/
[Models]: https://embeddings-benchmark.github.io/mteb/overview/available_models/text/
[Contributing]: https://embeddings-benchmark.github.io/mteb/CONTRIBUTING/
[Adding a model]: https://embeddings-benchmark.github.io/mteb/contributing/adding_a_model/
[Adding a dataset]: https://embeddings-benchmark.github.io/mteb/contributing/adding_a_dataset/
[Adding a benchmark]: https://embeddings-benchmark.github.io/mteb/contributing/adding_a_benchmark/
[Leaderboard]: https://huggingface.co/spaces/mteb/leaderboard
|