rahul7star commited on
Commit
f62dcda
·
verified ·
1 Parent(s): ea0a42e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +69 -69
README.md CHANGED
@@ -1,70 +1,70 @@
1
- ---
2
- title: PromptWizard Qwen Training
3
- emoji: 🧙
4
- colorFrom: purple
5
- colorTo: blue
6
- sdk: gradio
7
- sdk_version: 4.14.0
8
- app_file: app.py
9
- pinned: false
10
- license: mit
11
- suggested_hardware: zero-a10g
12
- ---
13
-
14
- # PromptWizard Qwen Fine-tuning
15
-
16
- This Space fine-tunes Qwen models using the GSM8K dataset with PromptWizard optimization methodology.
17
-
18
- ## Features
19
-
20
- - **GPU-Accelerated Training**: Uses HuggingFace's GPU infrastructure for fast training
21
- - **LoRA Fine-tuning**: Efficient parameter-efficient fine-tuning
22
- - **GSM8K Dataset**: High-quality mathematical reasoning dataset
23
- - **PromptWizard Integration**: Uses Microsoft's PromptWizard evaluation methodology
24
- - **Auto Push to Hub**: Trained models are automatically uploaded to HuggingFace Hub
25
-
26
- ## How to Use
27
-
28
- 1. Select your base model (default: Qwen/Qwen2.5-7B)
29
- 2. Configure training parameters:
30
- - Number of epochs (3-5 recommended)
31
- - Batch size (4-8 for T4 GPU)
32
- - Learning rate (2e-5 is a good default)
33
- 3. Click "Start Training" and monitor the output
34
- 4. The trained model will be pushed to HuggingFace Hub
35
-
36
- ## Training Data
37
-
38
- The Space uses the GSM8K dataset, which contains grade school math problems. The data is formatted according to PromptWizard specifications for optimal prompt optimization.
39
-
40
- ## Model Output
41
-
42
- After training, the model will be available at:
43
- - HuggingFace Hub: `your-username/promptwizard-qwen-gsm8k`
44
- - Local download: Available in the Space's output directory
45
-
46
- ## Technical Details
47
-
48
- - **Base Model**: Qwen2.5-7B (or your choice)
49
- - **Training Method**: LoRA with rank 16
50
- - **Quantization**: 8-bit for memory efficiency
51
- - **Mixed Precision**: FP16 for faster training
52
- - **Gradient Checkpointing**: Enabled for memory savings
53
-
54
- ## Resource Requirements
55
-
56
- - **GPU**: T4 or better recommended
57
- - **Memory**: 16GB+ GPU memory
58
- - **Training Time**: ~30-60 minutes on T4
59
-
60
- ## Citation
61
-
62
- If you use this training setup, please cite:
63
-
64
- ```bibtex
65
- @misc{promptwizard2024,
66
- title={PromptWizard: Task-Aware Prompt Optimization},
67
- author={Microsoft Research},
68
- year={2024}
69
- }
70
  ```
 
1
+ ---
2
+ title: PromptWizard Qwen Training
3
+ emoji: 🧙
4
+ colorFrom: purple
5
+ colorTo: blue
6
+ sdk: gradio
7
+ sdk_version: 5.49.1
8
+ app_file: app.py
9
+ pinned: false
10
+ license: mit
11
+ suggested_hardware: zero-a10g
12
+ ---
13
+
14
+ # PromptWizard Qwen Fine-tuning
15
+
16
+ This Space fine-tunes Qwen models using the GSM8K dataset with PromptWizard optimization methodology.
17
+
18
+ ## Features
19
+
20
+ - **GPU-Accelerated Training**: Uses HuggingFace's GPU infrastructure for fast training
21
+ - **LoRA Fine-tuning**: Efficient parameter-efficient fine-tuning
22
+ - **GSM8K Dataset**: High-quality mathematical reasoning dataset
23
+ - **PromptWizard Integration**: Uses Microsoft's PromptWizard evaluation methodology
24
+ - **Auto Push to Hub**: Trained models are automatically uploaded to HuggingFace Hub
25
+
26
+ ## How to Use
27
+
28
+ 1. Select your base model (default: Qwen/Qwen2.5-7B)
29
+ 2. Configure training parameters:
30
+ - Number of epochs (3-5 recommended)
31
+ - Batch size (4-8 for T4 GPU)
32
+ - Learning rate (2e-5 is a good default)
33
+ 3. Click "Start Training" and monitor the output
34
+ 4. The trained model will be pushed to HuggingFace Hub
35
+
36
+ ## Training Data
37
+
38
+ The Space uses the GSM8K dataset, which contains grade school math problems. The data is formatted according to PromptWizard specifications for optimal prompt optimization.
39
+
40
+ ## Model Output
41
+
42
+ After training, the model will be available at:
43
+ - HuggingFace Hub: `your-username/promptwizard-qwen-gsm8k`
44
+ - Local download: Available in the Space's output directory
45
+
46
+ ## Technical Details
47
+
48
+ - **Base Model**: Qwen2.5-7B (or your choice)
49
+ - **Training Method**: LoRA with rank 16
50
+ - **Quantization**: 8-bit for memory efficiency
51
+ - **Mixed Precision**: FP16 for faster training
52
+ - **Gradient Checkpointing**: Enabled for memory savings
53
+
54
+ ## Resource Requirements
55
+
56
+ - **GPU**: T4 or better recommended
57
+ - **Memory**: 16GB+ GPU memory
58
+ - **Training Time**: ~30-60 minutes on T4
59
+
60
+ ## Citation
61
+
62
+ If you use this training setup, please cite:
63
+
64
+ ```bibtex
65
+ @misc{promptwizard2024,
66
+ title={PromptWizard: Task-Aware Prompt Optimization},
67
+ author={Microsoft Research},
68
+ year={2024}
69
+ }
70
  ```