Beebey commited on
Commit
f0facc0
·
verified ·
1 Parent(s): b2457dd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -27
README.md CHANGED
@@ -1,61 +1,72 @@
1
  ---
2
  language:
3
- - en
4
  license: apache-2.0
5
  base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
6
  tags:
7
- - code
8
- - python
9
- - educational
10
- - lora
11
- - qwen
12
  library_name: peft
13
  ---
14
 
15
  # Qwen2.5-Coder-1.5B-Educational (LoRA)
16
 
17
- LoRA adapter for [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) fine-tuned on educational code generation.
18
 
19
- ## Quick Start
 
 
20
 
 
21
  from transformers import AutoModelForCausalLM, AutoTokenizer
22
  from peft import PeftModel
23
 
24
  # Load base model
25
  base_model = AutoModelForCausalLM.from_pretrained(
26
- "Qwen/Qwen2.5-Coder-1.5B-Instruct",
27
- device_map="auto"
28
  )
29
 
30
  # Load LoRA adapter
31
- model = PeftModel.from_pretrained(base_model, "YOUR_USERNAME/qwen-coder-1.5b-educational")
32
- tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/qwen-coder-1.5b-educational")
33
 
34
  # Generate code
35
- prompt = "Instruction: Write a Python function to reverse a string
36
- Réponse:
37
- "
 
38
  inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
39
  outputs = model.generate(**inputs, max_new_tokens=200)
40
- print(tokenizer.decode(outputs, skip_special_tokens=True))
 
41
 
42
- ## Training Details
43
 
44
- - **Method**: LoRA (r=8, alpha=16, dropout=0.05)
45
- - **Dataset**: OpenCoder-LLM/opc-sft-stage2 (educational_instruct)
46
- - **Steps**: 2000
47
- - **Final Loss**: 0.530
48
- - **Hardware**: TPU v6e-16
49
- - **Training Time**: 43 minutes
50
 
51
- ## Performance
 
 
 
 
 
52
 
53
- Improved over base model on:
 
 
 
 
54
  - Educational Python code generation
55
  - Pythonic idioms and patterns
56
  - Object-oriented architecture
57
  - Code documentation and comments
58
 
59
- ## License
 
 
60
 
61
- Apache 2.0
 
1
  ---
2
  language:
3
+ - en
4
  license: apache-2.0
5
  base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
6
  tags:
7
+ - code
8
+ - python
9
+ - educational
10
+ - lora
11
+ - qwen
12
  library_name: peft
13
  ---
14
 
15
  # Qwen2.5-Coder-1.5B-Educational (LoRA)
16
 
17
+ LoRA adapter for [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct), fine-tuned for educational code generation.
18
 
19
+ ---
20
+
21
+ ## 🚀 Quick Start
22
 
23
+ ```python
24
  from transformers import AutoModelForCausalLM, AutoTokenizer
25
  from peft import PeftModel
26
 
27
  # Load base model
28
  base_model = AutoModelForCausalLM.from_pretrained(
29
+ "Qwen/Qwen2.5-Coder-1.5B-Instruct",
30
+ device_map="auto"
31
  )
32
 
33
  # Load LoRA adapter
34
+ model = PeftModel.from_pretrained(base_model, "Beebey/qwen-coder-1.5b-educational")
35
+ tokenizer = AutoTokenizer.from_pretrained("Beebey/qwen-coder-1.5b-educational")
36
 
37
  # Generate code
38
+ prompt = (
39
+ "Instruction: Write a Python function to reverse a string\n"
40
+ "Réponse:\n"
41
+ )
42
  inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
43
  outputs = model.generate(**inputs, max_new_tokens=200)
44
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
45
+ ```
46
 
47
+ ---
48
 
49
+ ## 🏋️ Training Details
 
 
 
 
 
50
 
51
+ - **Method:** LoRA (r=8, alpha=16, dropout=0.05)
52
+ - **Dataset:** [OpenCoder-LLM/opc-sft-stage2](https://huggingface.co/datasets/OpenCoder-LLM/opc-sft-stage2) (`educational_instruct`)
53
+ - **Steps:** 2000
54
+ - **Final Loss:** 0.530
55
+ - **Hardware:** TPU v6e-16
56
+ - **Training Time:** 43 minutes
57
 
58
+ ---
59
+
60
+ ## 📈 Performance
61
+
62
+ Enhanced capabilities for:
63
  - Educational Python code generation
64
  - Pythonic idioms and patterns
65
  - Object-oriented architecture
66
  - Code documentation and comments
67
 
68
+ ---
69
+
70
+ ## 📄 License
71
 
72
+ Apache 2.0