leaderboard not working

#174
by jadenhoch - opened

https://huggingface.co/spaces/mteb/leaderboard

What happen?

untime error
Exit code: 2. Reason: eb/results/benchmark_results.py:375: UserWarning: No scores data available. Returning empty DataFrame.
warnings.warn(msg)
/mteb/mteb/results/benchmark_results.py:375: UserWarning: No scores data available. Returning empty DataFrame.
warnings.warn(msg)
/mteb/mteb/models/model_meta.py:514: UserWarning: Could not get source model: "Model 'https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct' not found in MTEB registry." in MTEB
warnings.warn(msg)
/mteb/mteb/models/model_meta.py:514: UserWarning: Could not get source model: "Model 'Qwen/Qwen2.5-VL-3B-Instruct' not found in MTEB registry. Did you mean: 'Alibaba-NLP/gme-Qwen2-VL-7B-Instruct' or Alibaba-NLP/gme-Qwen2-VL-2B-Instruct?" in MTEB
warnings.warn(msg)
/mteb/mteb/results/benchmark_results.py:375: UserWarning: No scores data available. Returning empty DataFrame.
warnings.warn(msg)
/mteb/mteb/results/benchmark_results.py:375: UserWarning: No scores data available. Returning empty DataFrame.
warnings.warn(msg)
/mteb/mteb/results/benchmark_results.py:375: UserWarning: No scores data available. Returning empty DataFrame.
warnings.warn(msg)
2026-01-01 08:27:01,582 - PID:25 - main - INFO - Prerun complete: Processed 59 benchmarks in 20.39s
2026-01-01 08:27:01,582 - PID:25 - main - INFO - === Leaderboard app initialization complete in 331.25s ===
2026-01-01 08:27:01,959 - PID:25 - httpx - INFO - HTTP Request: GET http://localhost:7860/gradio_api/startup-events "HTTP/1.1 200 OK"

Massive Text Embedding Benchmark org

I've rolled back leaderboard for now and it's working

Sign up or log in to comment