Spaces:
Running
Running
UI changes 11 Nov
Browse files
common.py
CHANGED
|
@@ -1,17 +1,19 @@
|
|
| 1 |
# Page Headers
|
| 2 |
-
MAIN_TITLE = "# Judge Arena
|
| 3 |
|
| 4 |
# How it works section
|
| 5 |
HOW_IT_WORKS = """
|
| 6 |
-
|
| 7 |
-
- **Evaluate anything:** coding, analysis, creative writing, math, or general knowledge
|
| 8 |
"""
|
| 9 |
|
| 10 |
BATTLE_RULES = """
|
| 11 |
-
## 🤺
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
| 15 |
<br>
|
| 16 |
"""
|
| 17 |
|
|
@@ -35,34 +37,25 @@ CSS_STYLES = """
|
|
| 35 |
gap: 8px;
|
| 36 |
}
|
| 37 |
"""
|
| 38 |
-
|
| 39 |
# Default Eval Prompt
|
| 40 |
EVAL_DESCRIPTION = """
|
| 41 |
-
## 📝
|
| 42 |
-
**Precise evaluation criteria
|
| 43 |
- Evaluation criteria
|
| 44 |
- Scoring rubric
|
| 45 |
-
- (Optional)
|
| 46 |
-
|
| 47 |
-
**Any variables you define in your prompt using {{double curly braces}} will automatically map to the corresponding input fields under the "Sample to evaluate" section on the right.**
|
| 48 |
-
|
| 49 |
-
<br><br>
|
| 50 |
"""
|
| 51 |
|
| 52 |
-
DEFAULT_EVAL_PROMPT = """You are assessing a chat bot response to a user's input based on [
|
| 53 |
|
| 54 |
Score:
|
| 55 |
A score of 1 means that the response's answer meets all of the evaluation criteria.
|
| 56 |
A score of 0 means that the response's answer does not meet all of the evaluation criteria.
|
| 57 |
|
| 58 |
-
Here is the data:
|
| 59 |
-
[BEGIN DATA]
|
| 60 |
-
***
|
| 61 |
[User Query]: {{input}}
|
| 62 |
-
|
| 63 |
-
[Response]: {{response}}
|
| 64 |
-
***
|
| 65 |
-
[END DATA]"""
|
| 66 |
|
| 67 |
# Default Variable Values
|
| 68 |
DEFAULT_INPUT = """Which of these animals is least likely to be found in a rainforest?"
|
|
@@ -79,7 +72,7 @@ VOTING_HEADER = """
|
|
| 79 |
|
| 80 |
# Acknowledgements
|
| 81 |
ACKNOWLEDGEMENTS = """
|
| 82 |
-
<br><br
|
| 83 |
# Acknowledgements
|
| 84 |
|
| 85 |
We thank [LMSYS Org](https://lmsys.org/) for their hard work on the Chatbot Arena and fully credit them for the inspiration to build this.
|
|
@@ -152,4 +145,4 @@ Atla currently funds this out of our own pocket. We are looking for API credits
|
|
| 152 |
We are training a general-purpose evaluator that you will soon be able to run in this Judge Arena. Our next step will be to open-source a powerful model that the community can use to run fast and accurate evaluations.
|
| 153 |
<br><br>
|
| 154 |
# Get in touch
|
| 155 |
-
Feel free to email us at [[email protected]](mailto:[email protected]) or leave feedback on our [Github](https://github.com/atla-ai/judge-arena)!"""
|
|
|
|
| 1 |
# Page Headers
|
| 2 |
+
MAIN_TITLE = "# Judge Arena: Benchmarking LLMs as Evaluators"
|
| 3 |
|
| 4 |
# How it works section
|
| 5 |
HOW_IT_WORKS = """
|
| 6 |
+
Vote to help the community find the best LLM-as-a-judge to use!
|
|
|
|
| 7 |
"""
|
| 8 |
|
| 9 |
BATTLE_RULES = """
|
| 10 |
+
## 🤺 Choose the winner
|
| 11 |
+
1. Define your scoring criteria in the **Evaluator Prompt**
|
| 12 |
+
2. Add a test case to the **Sample to evaluate**
|
| 13 |
+
3. Test the evaluators & vote for the model that best aligns with your judgement!
|
| 14 |
+
\n
|
| 15 |
+
Variables defined in your prompt with {{double curly braces}} map to input fields under **Sample to evaluate**.
|
| 16 |
+
|
| 17 |
<br>
|
| 18 |
"""
|
| 19 |
|
|
|
|
| 37 |
gap: 8px;
|
| 38 |
}
|
| 39 |
"""
|
| 40 |
+
|
| 41 |
# Default Eval Prompt
|
| 42 |
EVAL_DESCRIPTION = """
|
| 43 |
+
## 📝 Tips
|
| 44 |
+
**Precise evaluation criteria leads to more consistent and reliable judgments.** A good evaluation prompt should include the following elements:
|
| 45 |
- Evaluation criteria
|
| 46 |
- Scoring rubric
|
| 47 |
+
- Examples (Optional)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
"""
|
| 49 |
|
| 50 |
+
DEFAULT_EVAL_PROMPT = """You are assessing a chat bot response to a user's input based on [WRITE CRITERIA HERE]
|
| 51 |
|
| 52 |
Score:
|
| 53 |
A score of 1 means that the response's answer meets all of the evaluation criteria.
|
| 54 |
A score of 0 means that the response's answer does not meet all of the evaluation criteria.
|
| 55 |
|
|
|
|
|
|
|
|
|
|
| 56 |
[User Query]: {{input}}
|
| 57 |
+
|
| 58 |
+
[Response]: {{response}}"""
|
|
|
|
|
|
|
| 59 |
|
| 60 |
# Default Variable Values
|
| 61 |
DEFAULT_INPUT = """Which of these animals is least likely to be found in a rainforest?"
|
|
|
|
| 72 |
|
| 73 |
# Acknowledgements
|
| 74 |
ACKNOWLEDGEMENTS = """
|
| 75 |
+
<br><br>
|
| 76 |
# Acknowledgements
|
| 77 |
|
| 78 |
We thank [LMSYS Org](https://lmsys.org/) for their hard work on the Chatbot Arena and fully credit them for the inspiration to build this.
|
|
|
|
| 145 |
We are training a general-purpose evaluator that you will soon be able to run in this Judge Arena. Our next step will be to open-source a powerful model that the community can use to run fast and accurate evaluations.
|
| 146 |
<br><br>
|
| 147 |
# Get in touch
|
| 148 |
+
Feel free to email us at [[email protected]](mailto:[email protected]) or leave feedback on our [Github](https://github.com/atla-ai/judge-arena)!"""
|