Qwen
/

Qwen2.5-Coder-7B

@@ -18,13 +18,12 @@ tags:
 ## Introduction
-Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers;  Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:
 - Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o.
 - A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.
 - **Long-context Support** up to 128K tokens.
 **This repo contains the 7B Qwen2.5-Coder model**, which has the following features:
 - Type: Causal Language Models
 - Training Stage: Pretraining
@@ -34,8 +33,8 @@ Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (
 - Number of Layers: 28
 - Number of Attention Heads (GQA): 28 for Q and 4 for KV
 - Context Length: Full 131,072 tokens
-    - Please refer to [this section](#processing-long-texts) for detailed instructions on how to deploy Qwen2.5 for handling long texts.
 **We do not recommend using base language models for conversations.** Instead, you can apply post-training, e.g., SFT, RLHF, continued pretraining, etc., or fill in the middle tasks on this model.
 For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), [Documentation](https://qwen.readthedocs.io/en/latest/), [Arxiv](https://arxiv.org/abs/2409.12186).
@@ -66,9 +65,9 @@ For supported frameworks, you could add the following to `config.json` to enable
 }
 ```
-For deployment, we recommend using vLLM.
 Please refer to our [Documentation](https://qwen.readthedocs.io/en/latest/deployment/vllm.html) for usage if you are not familar with vLLM.
-Presently, vLLM only supports static YARN, which means the scaling factor remains constant regardless of input length, **potentially impacting performance on shorter texts**.
 We advise adding the `rope_scaling` configuration only when processing long contexts is required.
 ## Evaluation & Performance
@@ -83,10 +82,10 @@ If you find our work helpful, feel free to give us a cite.
 ```
 @article{hui2024qwen2,
-  title={Qwen2. 5-Coder Technical Report},
-  author={Hui, Binyuan and Yang, Jian and Cui, Zeyu and Yang, Jiaxi and Liu, Dayiheng and Zhang, Lei and Liu, Tianyu and Zhang, Jiajun and Yu, Bowen and Dang, Kai and others},
-  journal={arXiv preprint arXiv:2409.12186},
-  year={2024}
 }
 @article{qwen2,
       title={Qwen2 Technical Report},

 ## Introduction
+Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:
 - Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o.
 - A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.
 - **Long-context Support** up to 128K tokens.
 **This repo contains the 7B Qwen2.5-Coder model**, which has the following features:
 - Type: Causal Language Models
 - Training Stage: Pretraining
 - Number of Layers: 28
 - Number of Attention Heads (GQA): 28 for Q and 4 for KV
 - Context Length: Full 131,072 tokens
+  - Please refer to [this section](#processing-long-texts) for detailed instructions on how to deploy Qwen2.5 for handling long texts.
 **We do not recommend using base language models for conversations.** Instead, you can apply post-training, e.g., SFT, RLHF, continued pretraining, etc., or fill in the middle tasks on this model.
 For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2.5-coder-family/), [GitHub](https://github.com/QwenLM/Qwen2.5-Coder), [Documentation](https://qwen.readthedocs.io/en/latest/), [Arxiv](https://arxiv.org/abs/2409.12186).
 }
 ```
+For deployment, we recommend using vLLM.
 Please refer to our [Documentation](https://qwen.readthedocs.io/en/latest/deployment/vllm.html) for usage if you are not familar with vLLM.
+Presently, vLLM only supports static YARN, which means the scaling factor remains constant regardless of input length, **potentially impacting performance on shorter texts**.
 We advise adding the `rope_scaling` configuration only when processing long contexts is required.
 ## Evaluation & Performance
 ```
 @article{hui2024qwen2,
+      title={Qwen2. 5-Coder Technical Report},
+      author={Hui, Binyuan and Yang, Jian and Cui, Zeyu and Yang, Jiaxi and Liu, Dayiheng and Zhang, Lei and Liu, Tianyu and Zhang, Jiajun and Yu, Bowen and Dang, Kai and others},
+      journal={arXiv preprint arXiv:2409.12186},
+      year={2024}
 }
 @article{qwen2,
       title={Qwen2 Technical Report},