tclf90 commited on
Commit
c8d6023
·
verified ·
1 Parent(s): 850ce88

Update README.md

Browse files

move-up installation instruction

Files changed (1) hide show
  1. README.md +14 -14
README.md CHANGED
@@ -16,6 +16,20 @@ base_model_relation: quantized
16
  # DeepSeek-V3.1-AWQ
17
  Base model: [DeepSeek-V3.1](https://huggingface.co/deepseek-ai/DeepSeek-V3.1)
18
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  ### 【vLLM Single Node with 8 GPUs — Startup Command】
20
  ```
21
  CONTEXT_LENGTH=32768
@@ -35,20 +49,6 @@ vllm serve \
35
  --port 8000
36
  ```
37
 
38
- ### 【Dependencies / Installation】
39
- As of **2025-08-23**, create a fresh Python environment and run:
40
-
41
- ```bash
42
- # ❗there are glitches with vllm 0.10.1.1, still looking for resolutions❗
43
- # ❗downgrade vllm for now ❗
44
- pip install vllm==0.9.0
45
- pip install transformers==4.53
46
-
47
- # ❗patch up AWQ MoE quant config, otherwise some modules cannot be properly loaded❗
48
- SITE_PACKAGES=$(pip -V | awk '{print $4}' | sed 's/\/pip$//')
49
- cp awq_marlin.py "$SITE_PACKAGES/vllm/model_executor/layers/quantization/awq_marlin.py"
50
- ```
51
-
52
  ### 【Logs】
53
  ```
54
  2025-08-23
 
16
  # DeepSeek-V3.1-AWQ
17
  Base model: [DeepSeek-V3.1](https://huggingface.co/deepseek-ai/DeepSeek-V3.1)
18
 
19
+ ### 【Dependencies / Installation】
20
+ As of **2025-08-23**, create a fresh Python environment and run:
21
+
22
+ ```bash
23
+ # ❗there are glitches with vllm 0.10.1.1, still looking for resolutions❗
24
+ # ❗downgrade vllm for now ❗
25
+ pip install vllm==0.9.0
26
+ pip install transformers==4.53
27
+
28
+ # ❗patch up AWQ MoE quant config, otherwise some modules cannot be properly loaded❗
29
+ SITE_PACKAGES=$(pip -V | awk '{print $4}' | sed 's/\/pip$//')
30
+ cp awq_marlin.py "$SITE_PACKAGES/vllm/model_executor/layers/quantization/awq_marlin.py"
31
+ ```
32
+
33
  ### 【vLLM Single Node with 8 GPUs — Startup Command】
34
  ```
35
  CONTEXT_LENGTH=32768
 
49
  --port 8000
50
  ```
51
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
  ### 【Logs】
53
  ```
54
  2025-08-23