Spaces:
Running
Running
Commit
·
4142d5d
1
Parent(s):
b17e273
Update README.md
Browse files
README.md
CHANGED
|
@@ -7,6 +7,7 @@ sdk: static
|
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
|
|
|
| 10 |
This repository is a demo leaderboard template.
|
| 11 |
You can copy the leaderboard space and the two datasets (results and requests) to your org to get started with your own leaderboard!
|
| 12 |
|
|
@@ -15,10 +16,13 @@ The space does 3 things:
|
|
| 15 |
- reads the submissions depending on their status/date of creation, and launches evaluations through the `main_backend.py` file, using the Eleuther AI Harness. Results of running evaluations are then sent to `results`
|
| 16 |
- reads the results and displays them in a leaderboard.
|
| 17 |
|
|
|
|
|
|
|
| 18 |
To get started on your own leaderboard, you will need to edit 2 files:
|
| 19 |
- `src/envs.py` to define your own environment variable (like the org name in which this has been copied)
|
| 20 |
- `src/about.py` with the tasks and number of few_shots you want for your tasks
|
| 21 |
|
|
|
|
| 22 |
Once this is done, you need to edit the "fake results" file to fit the format of your tasks: in the sub dictionary `results`, replace task_name1 and metric_name by the correct values you defined in Tasks above.
|
| 23 |
```
|
| 24 |
"results": {
|
|
|
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
# What is this?
|
| 11 |
This repository is a demo leaderboard template.
|
| 12 |
You can copy the leaderboard space and the two datasets (results and requests) to your org to get started with your own leaderboard!
|
| 13 |
|
|
|
|
| 16 |
- reads the submissions depending on their status/date of creation, and launches evaluations through the `main_backend.py` file, using the Eleuther AI Harness. Results of running evaluations are then sent to `results`
|
| 17 |
- reads the results and displays them in a leaderboard.
|
| 18 |
|
| 19 |
+
# Getting started
|
| 20 |
+
## Defining environment variables
|
| 21 |
To get started on your own leaderboard, you will need to edit 2 files:
|
| 22 |
- `src/envs.py` to define your own environment variable (like the org name in which this has been copied)
|
| 23 |
- `src/about.py` with the tasks and number of few_shots you want for your tasks
|
| 24 |
|
| 25 |
+
## Setting up fake results to initialize the leaderboard
|
| 26 |
Once this is done, you need to edit the "fake results" file to fit the format of your tasks: in the sub dictionary `results`, replace task_name1 and metric_name by the correct values you defined in Tasks above.
|
| 27 |
```
|
| 28 |
"results": {
|