Spaces:
Running
Running
refine
Browse files
README.md
CHANGED
|
@@ -65,7 +65,7 @@ For each PR reviewed by an agent, we determine its status:
|
|
| 65 |
The leaderboard refreshes automatically every day at 12:00 AM UTC.
|
| 66 |
|
| 67 |
**Community Submissions**
|
| 68 |
-
Anyone can submit a coding agent to track via the leaderboard. We store agent metadata in Hugging Face datasets (`SWE-Arena/
|
| 69 |
|
| 70 |
## Using the Leaderboard
|
| 71 |
|
|
|
|
| 65 |
The leaderboard refreshes automatically every day at 12:00 AM UTC.
|
| 66 |
|
| 67 |
**Community Submissions**
|
| 68 |
+
Anyone can submit a coding agent to track via the leaderboard. We store agent metadata in Hugging Face datasets (`SWE-Arena/agent_metadata`) and review metadata in (`SWE-Arena/review_metadata`). The leaderboard is dynamically constructed from the review metadata. All submissions are automatically validated through GitHub's API to ensure the account exists and has public activity.
|
| 69 |
|
| 70 |
## Using the Leaderboard
|
| 71 |
|
app.py
CHANGED
|
@@ -26,7 +26,7 @@ load_dotenv()
|
|
| 26 |
# CONFIGURATION
|
| 27 |
# =============================================================================
|
| 28 |
|
| 29 |
-
AGENTS_REPO = "SWE-Arena/
|
| 30 |
REVIEW_METADATA_REPO = "SWE-Arena/review_metadata" # HuggingFace dataset for review metadata
|
| 31 |
LEADERBOARD_REPO = "SWE-Arena/leaderboard_metadata" # HuggingFace dataset for leaderboard data
|
| 32 |
LEADERBOARD_TIME_FRAME_DAYS = 180 # Time frame for constructing leaderboard
|
|
|
|
| 26 |
# CONFIGURATION
|
| 27 |
# =============================================================================
|
| 28 |
|
| 29 |
+
AGENTS_REPO = "SWE-Arena/agent_metadata" # HuggingFace dataset for agent metadata
|
| 30 |
REVIEW_METADATA_REPO = "SWE-Arena/review_metadata" # HuggingFace dataset for review metadata
|
| 31 |
LEADERBOARD_REPO = "SWE-Arena/leaderboard_metadata" # HuggingFace dataset for leaderboard data
|
| 32 |
LEADERBOARD_TIME_FRAME_DAYS = 180 # Time frame for constructing leaderboard
|
msr.py
CHANGED
|
@@ -19,7 +19,7 @@ load_dotenv()
|
|
| 19 |
# CONFIGURATION
|
| 20 |
# =============================================================================
|
| 21 |
|
| 22 |
-
AGENTS_REPO = "SWE-Arena/
|
| 23 |
REVIEW_METADATA_REPO = "SWE-Arena/review_metadata"
|
| 24 |
LEADERBOARD_REPO = "SWE-Arena/leaderboard_metadata" # HuggingFace dataset for leaderboard data
|
| 25 |
LEADERBOARD_TIME_FRAME_DAYS = 180 # Time frame for leaderboard
|
|
|
|
| 19 |
# CONFIGURATION
|
| 20 |
# =============================================================================
|
| 21 |
|
| 22 |
+
AGENTS_REPO = "SWE-Arena/agent_metadata"
|
| 23 |
REVIEW_METADATA_REPO = "SWE-Arena/review_metadata"
|
| 24 |
LEADERBOARD_REPO = "SWE-Arena/leaderboard_metadata" # HuggingFace dataset for leaderboard data
|
| 25 |
LEADERBOARD_TIME_FRAME_DAYS = 180 # Time frame for leaderboard
|