refine
Browse files
README.md
CHANGED
|
@@ -59,7 +59,7 @@ We search GitHub using multiple query patterns to catch all issues associated wi
|
|
| 59 |
The leaderboard refreshes automatically every day at 12:00 AM UTC.
|
| 60 |
|
| 61 |
**Community Submissions**
|
| 62 |
-
Anyone can submit a coding agent to track via the leaderboard. We store agent metadata in Hugging Face datasets (`SWE-Arena/
|
| 63 |
|
| 64 |
## Using the Leaderboard
|
| 65 |
|
|
|
|
| 59 |
The leaderboard refreshes automatically every day at 12:00 AM UTC.
|
| 60 |
|
| 61 |
**Community Submissions**
|
| 62 |
+
Anyone can submit a coding agent to track via the leaderboard. We store agent metadata in Hugging Face datasets (`SWE-Arena/bot_metadata`) and issue metadata in (`SWE-Arena/issue_metadata`). The leaderboard is dynamically constructed from the issue metadata. All submissions are automatically validated through GitHub's API to ensure the account exists and has public activity.
|
| 63 |
|
| 64 |
## Using the Leaderboard
|
| 65 |
|
app.py
CHANGED
|
@@ -24,7 +24,7 @@ load_dotenv()
|
|
| 24 |
# CONFIGURATION
|
| 25 |
# =============================================================================
|
| 26 |
|
| 27 |
-
AGENTS_REPO = "SWE-Arena/
|
| 28 |
ISSUE_METADATA_REPO = "SWE-Arena/issue_metadata" # HuggingFace dataset for issue metadata
|
| 29 |
LEADERBOARD_TIME_FRAME_DAYS = 180 # Time frame for leaderboard
|
| 30 |
UPDATE_TIME_FRAME_DAYS = 30 # How often to re-mine data via BigQuery
|
|
|
|
| 24 |
# CONFIGURATION
|
| 25 |
# =============================================================================
|
| 26 |
|
| 27 |
+
AGENTS_REPO = "SWE-Arena/bot_metadata" # HuggingFace dataset for agent metadata
|
| 28 |
ISSUE_METADATA_REPO = "SWE-Arena/issue_metadata" # HuggingFace dataset for issue metadata
|
| 29 |
LEADERBOARD_TIME_FRAME_DAYS = 180 # Time frame for leaderboard
|
| 30 |
UPDATE_TIME_FRAME_DAYS = 30 # How often to re-mine data via BigQuery
|
msr.py
CHANGED
|
@@ -19,7 +19,7 @@ load_dotenv()
|
|
| 19 |
# CONFIGURATION
|
| 20 |
# =============================================================================
|
| 21 |
|
| 22 |
-
AGENTS_REPO = "SWE-Arena/
|
| 23 |
ISSUE_METADATA_REPO = "SWE-Arena/issue_metadata"
|
| 24 |
LEADERBOARD_REPO = "SWE-Arena/leaderboard_metadata"
|
| 25 |
LEADERBOARD_TIME_FRAME_DAYS = 180 # Time frame for leaderboard
|
|
|
|
| 19 |
# CONFIGURATION
|
| 20 |
# =============================================================================
|
| 21 |
|
| 22 |
+
AGENTS_REPO = "SWE-Arena/bot_metadata"
|
| 23 |
ISSUE_METADATA_REPO = "SWE-Arena/issue_metadata"
|
| 24 |
LEADERBOARD_REPO = "SWE-Arena/leaderboard_metadata"
|
| 25 |
LEADERBOARD_TIME_FRAME_DAYS = 180 # Time frame for leaderboard
|