Add Q2–Q8_0 quantized models with per-model cards, MODELFILE, CLI examples, and auto-upload

.gitattributes

@@ -33,3 +33,12 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwen3-8B-f16:Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-8B-f16:Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-8B-f16:Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-8B-f16:Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-8B-f16:Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-8B-f16:Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-8B-f16:Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-8B-f16:Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-8B-f16:Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

MODELFILE

@@ -0,0 +1,25 @@
+# MODELFILE for Qwen3-8B-GGUF
+# Used by LM Studio, OpenWebUI, GPT4All, etc.
+
+context_length: 32768
+embedding: false
+f16: cpu
+
+# Chat template using ChatML (used by Qwen)
+prompt_template: >-
+           <|im_start|>system
+        You are a helpful assistant.<|im_end|>
+           <|im_start|>user
+        {prompt}<|im_end|>
+           <|im_start|>assistant
+
+# Stop sequences help end generation cleanly
+stop: "<|im_end|>"
+stop: "<|im_start|>"
+
+# Default sampling (optimized for thinking mode)
+temperature: 0.6
+top_p: 0.95
+top_k: 20
+min_p: 0.0
+repeat_penalty: 1.1

Qwen3-8B-Q2_K/README.md

@@ -0,0 +1,163 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+---
+
+# Qwen3-8B-Q2_K
+
+Quantized version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) at **Q2_K** level, derived from **f16** base weights.
+
+## Model Info
+
+- **Format**: GGUF (for llama.cpp and compatible runtimes)
+- **Size**: 3.1G
+- **Precision**: Q2_K
+- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+- **Conversion Tool**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
+
+## Quality & Performance
+
+| Metric | Value |
+|-------|-------|
+| **Quality** | Very Low |
+| **Speed** | ⚡ Fast |
+| **RAM Required** | ~3.0 GB |
+| **Recommendation** | Only on very weak hardware; poor reasoning. Avoid if possible. Not suitable for thinking mode. |
+
+## Prompt Template (ChatML)
+
+This model uses the **ChatML** format used by Qwen:
+
+```text
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+Set this in your app (LM Studio, OpenWebUI, etc.) for best results.
+
+## Generation Parameters
+
+### Thinking Mode (Recommended for Logic)
+Use when solving math, coding, or logical problems.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.6 |
+| Top-P | 0.95 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+> ❗ DO NOT use greedy decoding — it causes infinite loops.
+
+Enable via:
+- `enable_thinking=True` in tokenizer
+- Or add `/think` in user input during conversation
+
+### Non-Thinking Mode (Fast Dialogue)
+For casual chat and quick replies.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.7 |
+| Top-P | 0.8 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+Enable via:
+- `enable_thinking=False`
+- Or add `/no_think` in prompt
+
+Stop sequences: `<|im_end|>`, `<|im_start|>`
+
+## 💡 Usage Tips
+
+> This model supports two operational modes:
+>
+> ### 🔍 Thinking Mode (Recommended for Logic)
+> Activate with `enable_thinking=True` or append `/think` in prompt.
+>
+> - Ideal for: math, coding, planning, analysis
+> - Use sampling: `temp=0.6`, `top_p=0.95`, `top_k=20`
+> - Avoid greedy decoding
+>
+> ### ⚡ Non-Thinking Mode (Fast Chat)
+> Use `enable_thinking=False` or `/no_think`.
+>
+> - Best for: casual conversation, quick answers
+> - Sampling: `temp=0.7`, `top_p=0.8`
+>
+> ---
+>
+> 🔄 **Switch Dynamically**  
+> In multi-turn chats, the last `/think` or `/no_think` directive takes precedence.
+>
+> 🔁 **Avoid Repetition**  
+> Set `presence_penalty=1.5` if stuck in loops.
+>
+> 📏 **Use Full Context**  
+> Allow up to 32,768 output tokens for complex tasks.
+>
+> 🧰 **Agent Ready**  
+> Works with Qwen-Agent, MCP servers, and custom tools.
+
+## 🖥️ CLI Example Using Ollama or TGI Server
+
+Here’s how you can query this model via API using `curl` and `jq`. Replace the endpoint with your local server (e.g., Ollama, Text Generation Inference).
+
+```bash
+curl http://localhost:11434/api/generate -s -N -d '{
+  "model": "hf.co/geoffmunn/Qwen3-8B:Q2_K;2D",
+  "prompt": "Repeat the following instruction exactly as given: Summarize what a neural network is in one sentence.",
+  "temperature": 0.5,
+  "top_p": 0.95,
+  "top_k": 20,
+  "min_p": 0.0,
+  "repeat_penalty": 1.1,
+  "stream": false
+}' | jq -r '.response'
+```
+
+🎯 **Why this works well**:
+- The prompt is meaningful and demonstrates either **reasoning**, **creativity**, or **clarity** depending on quant level.
+- Temperature is tuned appropriately: lower for factual responses (`0.4`), higher for creative ones (`0.7`).
+- Uses `jq` to extract clean output.
+
+> 💬 Tip: For interactive streaming, set `"stream": true` and process line-by-line.
+
+## Verification
+
+Check integrity:
+
+```bash
+sha256sum -c ../SHA256SUMS.txt
+```
+
+## Usage
+
+Compatible with:
+- [LM Studio](https://lmstudio.ai) – local AI model runner with GPU acceleration
+- [OpenWebUI](https://openwebui.com) – self-hosted AI platform with RAG and tools
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Directly via `llama.cpp`
+
+Supports dynamic switching between thinking modes via `/think` and `/no_think` in multi-turn conversations.
+
+## License
+
+Apache 2.0 – see base model for full terms.

Qwen3-8B-Q3_K_M/README.md

@@ -0,0 +1,163 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+---
+
+# Qwen3-8B-Q3_K_M
+
+Quantized version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) at **Q3_K_M** level, derived from **f16** base weights.
+
+## Model Info
+
+- **Format**: GGUF (for llama.cpp and compatible runtimes)
+- **Size**: 3.9G
+- **Precision**: Q3_K_M
+- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+- **Conversion Tool**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
+
+## Quality & Performance
+
+| Metric | Value |
+|-------|-------|
+| **Quality** | Low-Medium |
+| **Speed** | ⚡ Fast |
+| **RAM Required** | ~3.6 GB |
+| **Recommendation** | Acceptable for basic chat on older CPUs. Do not expect coherent logic. |
+
+## Prompt Template (ChatML)
+
+This model uses the **ChatML** format used by Qwen:
+
+```text
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+Set this in your app (LM Studio, OpenWebUI, etc.) for best results.
+
+## Generation Parameters
+
+### Thinking Mode (Recommended for Logic)
+Use when solving math, coding, or logical problems.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.6 |
+| Top-P | 0.95 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+> ❗ DO NOT use greedy decoding — it causes infinite loops.
+
+Enable via:
+- `enable_thinking=True` in tokenizer
+- Or add `/think` in user input during conversation
+
+### Non-Thinking Mode (Fast Dialogue)
+For casual chat and quick replies.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.7 |
+| Top-P | 0.8 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+Enable via:
+- `enable_thinking=False`
+- Or add `/no_think` in prompt
+
+Stop sequences: `<|im_end|>`, `<|im_start|>`
+
+## 💡 Usage Tips
+
+> This model supports two operational modes:
+>
+> ### 🔍 Thinking Mode (Recommended for Logic)
+> Activate with `enable_thinking=True` or append `/think` in prompt.
+>
+> - Ideal for: math, coding, planning, analysis
+> - Use sampling: `temp=0.6`, `top_p=0.95`, `top_k=20`
+> - Avoid greedy decoding
+>
+> ### ⚡ Non-Thinking Mode (Fast Chat)
+> Use `enable_thinking=False` or `/no_think`.
+>
+> - Best for: casual conversation, quick answers
+> - Sampling: `temp=0.7`, `top_p=0.8`
+>
+> ---
+>
+> 🔄 **Switch Dynamically**  
+> In multi-turn chats, the last `/think` or `/no_think` directive takes precedence.
+>
+> 🔁 **Avoid Repetition**  
+> Set `presence_penalty=1.5` if stuck in loops.
+>
+> 📏 **Use Full Context**  
+> Allow up to 32,768 output tokens for complex tasks.
+>
+> 🧰 **Agent Ready**  
+> Works with Qwen-Agent, MCP servers, and custom tools.
+
+## 🖥️ CLI Example Using Ollama or TGI Server
+
+Here’s how you can query this model via API using `curl` and `jq`. Replace the endpoint with your local server (e.g., Ollama, Text Generation Inference).
+
+```bash
+curl http://localhost:11434/api/generate -s -N -d '{
+  "model": "hf.co/geoffmunn/Qwen3-8B:Q3_K_M;2D",
+  "prompt": "Repeat the following instruction exactly as given: Summarize what a neural network is in one sentence.",
+  "temperature": 0.5,
+  "top_p": 0.95,
+  "top_k": 20,
+  "min_p": 0.0,
+  "repeat_penalty": 1.1,
+  "stream": false
+}' | jq -r '.response'
+```
+
+🎯 **Why this works well**:
+- The prompt is meaningful and demonstrates either **reasoning**, **creativity**, or **clarity** depending on quant level.
+- Temperature is tuned appropriately: lower for factual responses (`0.4`), higher for creative ones (`0.7`).
+- Uses `jq` to extract clean output.
+
+> 💬 Tip: For interactive streaming, set `"stream": true` and process line-by-line.
+
+## Verification
+
+Check integrity:
+
+```bash
+sha256sum -c ../SHA256SUMS.txt
+```
+
+## Usage
+
+Compatible with:
+- [LM Studio](https://lmstudio.ai) – local AI model runner with GPU acceleration
+- [OpenWebUI](https://openwebui.com) – self-hosted AI platform with RAG and tools
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Directly via `llama.cpp`
+
+Supports dynamic switching between thinking modes via `/think` and `/no_think` in multi-turn conversations.
+
+## License
+
+Apache 2.0 – see base model for full terms.

Qwen3-8B-Q3_K_S/README.md

@@ -0,0 +1,163 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+---
+
+# Qwen3-8B-Q3_K_S
+
+Quantized version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) at **Q3_K_S** level, derived from **f16** base weights.
+
+## Model Info
+
+- **Format**: GGUF (for llama.cpp and compatible runtimes)
+- **Size**: 3.6G
+- **Precision**: Q3_K_S
+- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+- **Conversion Tool**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
+
+## Quality & Performance
+
+| Metric | Value |
+|-------|-------|
+| **Quality** | Low |
+| **Speed** | ⚡ Fast |
+| **RAM Required** | ~3.4 GB |
+| **Recommendation** | Minimal viable for simple tasks. Avoid for reasoning or multilingual use. |
+
+## Prompt Template (ChatML)
+
+This model uses the **ChatML** format used by Qwen:
+
+```text
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+Set this in your app (LM Studio, OpenWebUI, etc.) for best results.
+
+## Generation Parameters
+
+### Thinking Mode (Recommended for Logic)
+Use when solving math, coding, or logical problems.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.6 |
+| Top-P | 0.95 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+> ❗ DO NOT use greedy decoding — it causes infinite loops.
+
+Enable via:
+- `enable_thinking=True` in tokenizer
+- Or add `/think` in user input during conversation
+
+### Non-Thinking Mode (Fast Dialogue)
+For casual chat and quick replies.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.7 |
+| Top-P | 0.8 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+Enable via:
+- `enable_thinking=False`
+- Or add `/no_think` in prompt
+
+Stop sequences: `<|im_end|>`, `<|im_start|>`
+
+## 💡 Usage Tips
+
+> This model supports two operational modes:
+>
+> ### 🔍 Thinking Mode (Recommended for Logic)
+> Activate with `enable_thinking=True` or append `/think` in prompt.
+>
+> - Ideal for: math, coding, planning, analysis
+> - Use sampling: `temp=0.6`, `top_p=0.95`, `top_k=20`
+> - Avoid greedy decoding
+>
+> ### ⚡ Non-Thinking Mode (Fast Chat)
+> Use `enable_thinking=False` or `/no_think`.
+>
+> - Best for: casual conversation, quick answers
+> - Sampling: `temp=0.7`, `top_p=0.8`
+>
+> ---
+>
+> 🔄 **Switch Dynamically**  
+> In multi-turn chats, the last `/think` or `/no_think` directive takes precedence.
+>
+> 🔁 **Avoid Repetition**  
+> Set `presence_penalty=1.5` if stuck in loops.
+>
+> 📏 **Use Full Context**  
+> Allow up to 32,768 output tokens for complex tasks.
+>
+> 🧰 **Agent Ready**  
+> Works with Qwen-Agent, MCP servers, and custom tools.
+
+## 🖥️ CLI Example Using Ollama or TGI Server
+
+Here’s how you can query this model via API using `curl` and `jq`. Replace the endpoint with your local server (e.g., Ollama, Text Generation Inference).
+
+```bash
+curl http://localhost:11434/api/generate -s -N -d '{
+  "model": "hf.co/geoffmunn/Qwen3-8B:Q3_K_S;2D",
+  "prompt": "Repeat the following instruction exactly as given: Summarize what a neural network is in one sentence.",
+  "temperature": 0.5,
+  "top_p": 0.95,
+  "top_k": 20,
+  "min_p": 0.0,
+  "repeat_penalty": 1.1,
+  "stream": false
+}' | jq -r '.response'
+```
+
+🎯 **Why this works well**:
+- The prompt is meaningful and demonstrates either **reasoning**, **creativity**, or **clarity** depending on quant level.
+- Temperature is tuned appropriately: lower for factual responses (`0.4`), higher for creative ones (`0.7`).
+- Uses `jq` to extract clean output.
+
+> 💬 Tip: For interactive streaming, set `"stream": true` and process line-by-line.
+
+## Verification
+
+Check integrity:
+
+```bash
+sha256sum -c ../SHA256SUMS.txt
+```
+
+## Usage
+
+Compatible with:
+- [LM Studio](https://lmstudio.ai) – local AI model runner with GPU acceleration
+- [OpenWebUI](https://openwebui.com) – self-hosted AI platform with RAG and tools
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Directly via `llama.cpp`
+
+Supports dynamic switching between thinking modes via `/think` and `/no_think` in multi-turn conversations.
+
+## License
+
+Apache 2.0 – see base model for full terms.

Qwen3-8B-Q4_K_M/README.md

@@ -0,0 +1,163 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+---
+
+# Qwen3-8B-Q4_K_M
+
+Quantized version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) at **Q4_K_M** level, derived from **f16** base weights.
+
+## Model Info
+
+- **Format**: GGUF (for llama.cpp and compatible runtimes)
+- **Size**: 4.7G
+- **Precision**: Q4_K_M
+- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+- **Conversion Tool**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
+
+## Quality & Performance
+
+| Metric | Value |
+|-------|-------|
+| **Quality** | Balanced |
+| **Speed** | 🚀 Fast |
+| **RAM Required** | ~4.3 GB |
+| **Recommendation** | Best speed/quality balance for most users. Ideal for laptops & general use. |
+
+## Prompt Template (ChatML)
+
+This model uses the **ChatML** format used by Qwen:
+
+```text
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+Set this in your app (LM Studio, OpenWebUI, etc.) for best results.
+
+## Generation Parameters
+
+### Thinking Mode (Recommended for Logic)
+Use when solving math, coding, or logical problems.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.6 |
+| Top-P | 0.95 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+> ❗ DO NOT use greedy decoding — it causes infinite loops.
+
+Enable via:
+- `enable_thinking=True` in tokenizer
+- Or add `/think` in user input during conversation
+
+### Non-Thinking Mode (Fast Dialogue)
+For casual chat and quick replies.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.7 |
+| Top-P | 0.8 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+Enable via:
+- `enable_thinking=False`
+- Or add `/no_think` in prompt
+
+Stop sequences: `<|im_end|>`, `<|im_start|>`
+
+## 💡 Usage Tips
+
+> This model supports two operational modes:
+>
+> ### 🔍 Thinking Mode (Recommended for Logic)
+> Activate with `enable_thinking=True` or append `/think` in prompt.
+>
+> - Ideal for: math, coding, planning, analysis
+> - Use sampling: `temp=0.6`, `top_p=0.95`, `top_k=20`
+> - Avoid greedy decoding
+>
+> ### ⚡ Non-Thinking Mode (Fast Chat)
+> Use `enable_thinking=False` or `/no_think`.
+>
+> - Best for: casual conversation, quick answers
+> - Sampling: `temp=0.7`, `top_p=0.8`
+>
+> ---
+>
+> 🔄 **Switch Dynamically**  
+> In multi-turn chats, the last `/think` or `/no_think` directive takes precedence.
+>
+> 🔁 **Avoid Repetition**  
+> Set `presence_penalty=1.5` if stuck in loops.
+>
+> 📏 **Use Full Context**  
+> Allow up to 32,768 output tokens for complex tasks.
+>
+> 🧰 **Agent Ready**  
+> Works with Qwen-Agent, MCP servers, and custom tools.
+
+## 🖥️ CLI Example Using Ollama or TGI Server
+
+Here’s how you can query this model via API using `curl` and `jq`. Replace the endpoint with your local server (e.g., Ollama, Text Generation Inference).
+
+```bash
+curl http://localhost:11434/api/generate -s -N -d '{
+  "model": "hf.co/geoffmunn/Qwen3-8B:Q4_K_M;2D",
+  "prompt": "Repeat the following instruction exactly as given: Write a short haiku about autumn leaves falling gently in a quiet forest.",
+  "temperature": 0.7,
+  "top_p": 0.95,
+  "top_k": 20,
+  "min_p": 0.0,
+  "repeat_penalty": 1.1,
+  "stream": false
+}' | jq -r '.response'
+```
+
+🎯 **Why this works well**:
+- The prompt is meaningful and demonstrates either **reasoning**, **creativity**, or **clarity** depending on quant level.
+- Temperature is tuned appropriately: lower for factual responses (`0.4`), higher for creative ones (`0.7`).
+- Uses `jq` to extract clean output.
+
+> 💬 Tip: For interactive streaming, set `"stream": true` and process line-by-line.
+
+## Verification
+
+Check integrity:
+
+```bash
+sha256sum -c ../SHA256SUMS.txt
+```
+
+## Usage
+
+Compatible with:
+- [LM Studio](https://lmstudio.ai) – local AI model runner with GPU acceleration
+- [OpenWebUI](https://openwebui.com) – self-hosted AI platform with RAG and tools
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Directly via `llama.cpp`
+
+Supports dynamic switching between thinking modes via `/think` and `/no_think` in multi-turn conversations.
+
+## License
+
+Apache 2.0 – see base model for full terms.

Qwen3-8B-Q4_K_S/README.md

@@ -0,0 +1,163 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+---
+
+# Qwen3-8B-Q4_K_S
+
+Quantized version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) at **Q4_K_S** level, derived from **f16** base weights.
+
+## Model Info
+
+- **Format**: GGUF (for llama.cpp and compatible runtimes)
+- **Size**: 4.5G
+- **Precision**: Q4_K_S
+- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+- **Conversion Tool**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
+
+## Quality & Performance
+
+| Metric | Value |
+|-------|-------|
+| **Quality** | Medium |
+| **Speed** | 🚀 Fast |
+| **RAM Required** | ~4.1 GB |
+| **Recommendation** | Good for low-end devices; decent performance. Suitable for mobile/embedded. |
+
+## Prompt Template (ChatML)
+
+This model uses the **ChatML** format used by Qwen:
+
+```text
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+Set this in your app (LM Studio, OpenWebUI, etc.) for best results.
+
+## Generation Parameters
+
+### Thinking Mode (Recommended for Logic)
+Use when solving math, coding, or logical problems.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.6 |
+| Top-P | 0.95 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+> ❗ DO NOT use greedy decoding — it causes infinite loops.
+
+Enable via:
+- `enable_thinking=True` in tokenizer
+- Or add `/think` in user input during conversation
+
+### Non-Thinking Mode (Fast Dialogue)
+For casual chat and quick replies.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.7 |
+| Top-P | 0.8 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+Enable via:
+- `enable_thinking=False`
+- Or add `/no_think` in prompt
+
+Stop sequences: `<|im_end|>`, `<|im_start|>`
+
+## 💡 Usage Tips
+
+> This model supports two operational modes:
+>
+> ### 🔍 Thinking Mode (Recommended for Logic)
+> Activate with `enable_thinking=True` or append `/think` in prompt.
+>
+> - Ideal for: math, coding, planning, analysis
+> - Use sampling: `temp=0.6`, `top_p=0.95`, `top_k=20`
+> - Avoid greedy decoding
+>
+> ### ⚡ Non-Thinking Mode (Fast Chat)
+> Use `enable_thinking=False` or `/no_think`.
+>
+> - Best for: casual conversation, quick answers
+> - Sampling: `temp=0.7`, `top_p=0.8`
+>
+> ---
+>
+> 🔄 **Switch Dynamically**  
+> In multi-turn chats, the last `/think` or `/no_think` directive takes precedence.
+>
+> 🔁 **Avoid Repetition**  
+> Set `presence_penalty=1.5` if stuck in loops.
+>
+> 📏 **Use Full Context**  
+> Allow up to 32,768 output tokens for complex tasks.
+>
+> 🧰 **Agent Ready**  
+> Works with Qwen-Agent, MCP servers, and custom tools.
+
+## 🖥️ CLI Example Using Ollama or TGI Server
+
+Here’s how you can query this model via API using `curl` and `jq`. Replace the endpoint with your local server (e.g., Ollama, Text Generation Inference).
+
+```bash
+curl http://localhost:11434/api/generate -s -N -d '{
+  "model": "hf.co/geoffmunn/Qwen3-8B:Q4_K_S;2D",
+  "prompt": "Repeat the following instruction exactly as given: Summarize what a neural network is in one sentence.",
+  "temperature": 0.5,
+  "top_p": 0.95,
+  "top_k": 20,
+  "min_p": 0.0,
+  "repeat_penalty": 1.1,
+  "stream": false
+}' | jq -r '.response'
+```
+
+🎯 **Why this works well**:
+- The prompt is meaningful and demonstrates either **reasoning**, **creativity**, or **clarity** depending on quant level.
+- Temperature is tuned appropriately: lower for factual responses (`0.4`), higher for creative ones (`0.7`).
+- Uses `jq` to extract clean output.
+
+> 💬 Tip: For interactive streaming, set `"stream": true` and process line-by-line.
+
+## Verification
+
+Check integrity:
+
+```bash
+sha256sum -c ../SHA256SUMS.txt
+```
+
+## Usage
+
+Compatible with:
+- [LM Studio](https://lmstudio.ai) – local AI model runner with GPU acceleration
+- [OpenWebUI](https://openwebui.com) – self-hosted AI platform with RAG and tools
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Directly via `llama.cpp`
+
+Supports dynamic switching between thinking modes via `/think` and `/no_think` in multi-turn conversations.
+
+## License
+
+Apache 2.0 – see base model for full terms.

Qwen3-8B-Q5_K_M/README.md

@@ -0,0 +1,163 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+---
+
+# Qwen3-8B-Q5_K_M
+
+Quantized version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) at **Q5_K_M** level, derived from **f16** base weights.
+
+## Model Info
+
+- **Format**: GGUF (for llama.cpp and compatible runtimes)
+- **Size**: 5.5G
+- **Precision**: Q5_K_M
+- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+- **Conversion Tool**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
+
+## Quality & Performance
+
+| Metric | Value |
+|-------|-------|
+| **Quality** | High+ |
+| **Speed** | 🐢 Medium |
+| **RAM Required** | ~4.9 GB |
+| **Recommendation** | Top choice for reasoning & coding. Recommended for desktops & strong laptops. |
+
+## Prompt Template (ChatML)
+
+This model uses the **ChatML** format used by Qwen:
+
+```text
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+Set this in your app (LM Studio, OpenWebUI, etc.) for best results.
+
+## Generation Parameters
+
+### Thinking Mode (Recommended for Logic)
+Use when solving math, coding, or logical problems.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.6 |
+| Top-P | 0.95 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+> ❗ DO NOT use greedy decoding — it causes infinite loops.
+
+Enable via:
+- `enable_thinking=True` in tokenizer
+- Or add `/think` in user input during conversation
+
+### Non-Thinking Mode (Fast Dialogue)
+For casual chat and quick replies.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.7 |
+| Top-P | 0.8 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+Enable via:
+- `enable_thinking=False`
+- Or add `/no_think` in prompt
+
+Stop sequences: `<|im_end|>`, `<|im_start|>`
+
+## 💡 Usage Tips
+
+> This model supports two operational modes:
+>
+> ### 🔍 Thinking Mode (Recommended for Logic)
+> Activate with `enable_thinking=True` or append `/think` in prompt.
+>
+> - Ideal for: math, coding, planning, analysis
+> - Use sampling: `temp=0.6`, `top_p=0.95`, `top_k=20`
+> - Avoid greedy decoding
+>
+> ### ⚡ Non-Thinking Mode (Fast Chat)
+> Use `enable_thinking=False` or `/no_think`.
+>
+> - Best for: casual conversation, quick answers
+> - Sampling: `temp=0.7`, `top_p=0.8`
+>
+> ---
+>
+> 🔄 **Switch Dynamically**  
+> In multi-turn chats, the last `/think` or `/no_think` directive takes precedence.
+>
+> 🔁 **Avoid Repetition**  
+> Set `presence_penalty=1.5` if stuck in loops.
+>
+> 📏 **Use Full Context**  
+> Allow up to 32,768 output tokens for complex tasks.
+>
+> 🧰 **Agent Ready**  
+> Works with Qwen-Agent, MCP servers, and custom tools.
+
+## 🖥️ CLI Example Using Ollama or TGI Server
+
+Here’s how you can query this model via API using `curl` and `jq`. Replace the endpoint with your local server (e.g., Ollama, Text Generation Inference).
+
+```bash
+curl http://localhost:11434/api/generate -s -N -d '{
+  "model": "hf.co/geoffmunn/Qwen3-8B:Q5_K_M;2D",
+  "prompt": "Repeat the following instruction exactly as given: Explain why the sky appears blue during the day but red at sunrise and sunset, using physics principles like Rayleigh scattering.",
+  "temperature": 0.4,
+  "top_p": 0.95,
+  "top_k": 20,
+  "min_p": 0.0,
+  "repeat_penalty": 1.1,
+  "stream": false
+}' | jq -r '.response'
+```
+
+🎯 **Why this works well**:
+- The prompt is meaningful and demonstrates either **reasoning**, **creativity**, or **clarity** depending on quant level.
+- Temperature is tuned appropriately: lower for factual responses (`0.4`), higher for creative ones (`0.7`).
+- Uses `jq` to extract clean output.
+
+> 💬 Tip: For interactive streaming, set `"stream": true` and process line-by-line.
+
+## Verification
+
+Check integrity:
+
+```bash
+sha256sum -c ../SHA256SUMS.txt
+```
+
+## Usage
+
+Compatible with:
+- [LM Studio](https://lmstudio.ai) – local AI model runner with GPU acceleration
+- [OpenWebUI](https://openwebui.com) – self-hosted AI platform with RAG and tools
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Directly via `llama.cpp`
+
+Supports dynamic switching between thinking modes via `/think` and `/no_think` in multi-turn conversations.
+
+## License
+
+Apache 2.0 – see base model for full terms.

Qwen3-8B-Q5_K_S/README.md

@@ -0,0 +1,163 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+---
+
+# Qwen3-8B-Q5_K_S
+
+Quantized version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) at **Q5_K_S** level, derived from **f16** base weights.
+
+## Model Info
+
+- **Format**: GGUF (for llama.cpp and compatible runtimes)
+- **Size**: 5.4G
+- **Precision**: Q5_K_S
+- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+- **Conversion Tool**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
+
+## Quality & Performance
+
+| Metric | Value |
+|-------|-------|
+| **Quality** | High |
+| **Speed** | 🐢 Medium |
+| **RAM Required** | ~4.8 GB |
+| **Recommendation** | Great for reasoning; slightly faster than Q5_K_M. Recommended for coding. |
+
+## Prompt Template (ChatML)
+
+This model uses the **ChatML** format used by Qwen:
+
+```text
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+Set this in your app (LM Studio, OpenWebUI, etc.) for best results.
+
+## Generation Parameters
+
+### Thinking Mode (Recommended for Logic)
+Use when solving math, coding, or logical problems.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.6 |
+| Top-P | 0.95 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+> ❗ DO NOT use greedy decoding — it causes infinite loops.
+
+Enable via:
+- `enable_thinking=True` in tokenizer
+- Or add `/think` in user input during conversation
+
+### Non-Thinking Mode (Fast Dialogue)
+For casual chat and quick replies.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.7 |
+| Top-P | 0.8 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+Enable via:
+- `enable_thinking=False`
+- Or add `/no_think` in prompt
+
+Stop sequences: `<|im_end|>`, `<|im_start|>`
+
+## 💡 Usage Tips
+
+> This model supports two operational modes:
+>
+> ### 🔍 Thinking Mode (Recommended for Logic)
+> Activate with `enable_thinking=True` or append `/think` in prompt.
+>
+> - Ideal for: math, coding, planning, analysis
+> - Use sampling: `temp=0.6`, `top_p=0.95`, `top_k=20`
+> - Avoid greedy decoding
+>
+> ### ⚡ Non-Thinking Mode (Fast Chat)
+> Use `enable_thinking=False` or `/no_think`.
+>
+> - Best for: casual conversation, quick answers
+> - Sampling: `temp=0.7`, `top_p=0.8`
+>
+> ---
+>
+> 🔄 **Switch Dynamically**  
+> In multi-turn chats, the last `/think` or `/no_think` directive takes precedence.
+>
+> 🔁 **Avoid Repetition**  
+> Set `presence_penalty=1.5` if stuck in loops.
+>
+> 📏 **Use Full Context**  
+> Allow up to 32,768 output tokens for complex tasks.
+>
+> 🧰 **Agent Ready**  
+> Works with Qwen-Agent, MCP servers, and custom tools.
+
+## 🖥️ CLI Example Using Ollama or TGI Server
+
+Here’s how you can query this model via API using `curl` and `jq`. Replace the endpoint with your local server (e.g., Ollama, Text Generation Inference).
+
+```bash
+curl http://localhost:11434/api/generate -s -N -d '{
+  "model": "hf.co/geoffmunn/Qwen3-8B:Q5_K_S;2D",
+  "prompt": "Repeat the following instruction exactly as given: Write a short haiku about autumn leaves falling gently in a quiet forest.",
+  "temperature": 0.7,
+  "top_p": 0.95,
+  "top_k": 20,
+  "min_p": 0.0,
+  "repeat_penalty": 1.1,
+  "stream": false
+}' | jq -r '.response'
+```
+
+🎯 **Why this works well**:
+- The prompt is meaningful and demonstrates either **reasoning**, **creativity**, or **clarity** depending on quant level.
+- Temperature is tuned appropriately: lower for factual responses (`0.4`), higher for creative ones (`0.7`).
+- Uses `jq` to extract clean output.
+
+> 💬 Tip: For interactive streaming, set `"stream": true` and process line-by-line.
+
+## Verification
+
+Check integrity:
+
+```bash
+sha256sum -c ../SHA256SUMS.txt
+```
+
+## Usage
+
+Compatible with:
+- [LM Studio](https://lmstudio.ai) – local AI model runner with GPU acceleration
+- [OpenWebUI](https://openwebui.com) – self-hosted AI platform with RAG and tools
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Directly via `llama.cpp`
+
+Supports dynamic switching between thinking modes via `/think` and `/no_think` in multi-turn conversations.
+
+## License
+
+Apache 2.0 – see base model for full terms.

Qwen3-8B-Q6_K/README.md

@@ -0,0 +1,163 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+---
+
+# Qwen3-8B-Q6_K
+
+Quantized version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) at **Q6_K** level, derived from **f16** base weights.
+
+## Model Info
+
+- **Format**: GGUF (for llama.cpp and compatible runtimes)
+- **Size**: 6.3G
+- **Precision**: Q6_K
+- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+- **Conversion Tool**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
+
+## Quality & Performance
+
+| Metric | Value |
+|-------|-------|
+| **Quality** | Near-FP16 |
+| **Speed** | 🐌 Slow |
+| **RAM Required** | ~5.5 GB |
+| **Recommendation** | Excellent fidelity; ideal for RAG, complex logic. Use if RAM allows. |
+
+## Prompt Template (ChatML)
+
+This model uses the **ChatML** format used by Qwen:
+
+```text
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+Set this in your app (LM Studio, OpenWebUI, etc.) for best results.
+
+## Generation Parameters
+
+### Thinking Mode (Recommended for Logic)
+Use when solving math, coding, or logical problems.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.6 |
+| Top-P | 0.95 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+> ❗ DO NOT use greedy decoding — it causes infinite loops.
+
+Enable via:
+- `enable_thinking=True` in tokenizer
+- Or add `/think` in user input during conversation
+
+### Non-Thinking Mode (Fast Dialogue)
+For casual chat and quick replies.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.7 |
+| Top-P | 0.8 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+Enable via:
+- `enable_thinking=False`
+- Or add `/no_think` in prompt
+
+Stop sequences: `<|im_end|>`, `<|im_start|>`
+
+## 💡 Usage Tips
+
+> This model supports two operational modes:
+>
+> ### 🔍 Thinking Mode (Recommended for Logic)
+> Activate with `enable_thinking=True` or append `/think` in prompt.
+>
+> - Ideal for: math, coding, planning, analysis
+> - Use sampling: `temp=0.6`, `top_p=0.95`, `top_k=20`
+> - Avoid greedy decoding
+>
+> ### ⚡ Non-Thinking Mode (Fast Chat)
+> Use `enable_thinking=False` or `/no_think`.
+>
+> - Best for: casual conversation, quick answers
+> - Sampling: `temp=0.7`, `top_p=0.8`
+>
+> ---
+>
+> 🔄 **Switch Dynamically**  
+> In multi-turn chats, the last `/think` or `/no_think` directive takes precedence.
+>
+> 🔁 **Avoid Repetition**  
+> Set `presence_penalty=1.5` if stuck in loops.
+>
+> 📏 **Use Full Context**  
+> Allow up to 32,768 output tokens for complex tasks.
+>
+> 🧰 **Agent Ready**  
+> Works with Qwen-Agent, MCP servers, and custom tools.
+
+## 🖥️ CLI Example Using Ollama or TGI Server
+
+Here’s how you can query this model via API using `curl` and `jq`. Replace the endpoint with your local server (e.g., Ollama, Text Generation Inference).
+
+```bash
+curl http://localhost:11434/api/generate -s -N -d '{
+  "model": "hf.co/geoffmunn/Qwen3-8B:Q6_K;2D",
+  "prompt": "Repeat the following instruction exactly as given: Explain why the sky appears blue during the day but red at sunrise and sunset, using physics principles like Rayleigh scattering.",
+  "temperature": 0.4,
+  "top_p": 0.95,
+  "top_k": 20,
+  "min_p": 0.0,
+  "repeat_penalty": 1.1,
+  "stream": false
+}' | jq -r '.response'
+```
+
+🎯 **Why this works well**:
+- The prompt is meaningful and demonstrates either **reasoning**, **creativity**, or **clarity** depending on quant level.
+- Temperature is tuned appropriately: lower for factual responses (`0.4`), higher for creative ones (`0.7`).
+- Uses `jq` to extract clean output.
+
+> 💬 Tip: For interactive streaming, set `"stream": true` and process line-by-line.
+
+## Verification
+
+Check integrity:
+
+```bash
+sha256sum -c ../SHA256SUMS.txt
+```
+
+## Usage
+
+Compatible with:
+- [LM Studio](https://lmstudio.ai) – local AI model runner with GPU acceleration
+- [OpenWebUI](https://openwebui.com) – self-hosted AI platform with RAG and tools
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Directly via `llama.cpp`
+
+Supports dynamic switching between thinking modes via `/think` and `/no_think` in multi-turn conversations.
+
+## License
+
+Apache 2.0 – see base model for full terms.

Qwen3-8B-Q8_0/README.md

@@ -0,0 +1,163 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+---
+
+# Qwen3-8B-Q8_0
+
+Quantized version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) at **Q8_0** level, derived from **f16** base weights.
+
+## Model Info
+
+- **Format**: GGUF (for llama.cpp and compatible runtimes)
+- **Size**: 8.2G
+- **Precision**: Q8_0
+- **Base Model**: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)
+- **Conversion Tool**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
+
+## Quality & Performance
+
+| Metric | Value |
+|-------|-------|
+| **Quality** | Lossless* |
+| **Speed** | 🐌 Slow |
+| **RAM Required** | ~7.1 GB |
+| **Recommendation** | Highest quality without FP16; perfect for accuracy-critical tasks, benchmarks. |
+
+## Prompt Template (ChatML)
+
+This model uses the **ChatML** format used by Qwen:
+
+```text
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+Set this in your app (LM Studio, OpenWebUI, etc.) for best results.
+
+## Generation Parameters
+
+### Thinking Mode (Recommended for Logic)
+Use when solving math, coding, or logical problems.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.6 |
+| Top-P | 0.95 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+> ❗ DO NOT use greedy decoding — it causes infinite loops.
+
+Enable via:
+- `enable_thinking=True` in tokenizer
+- Or add `/think` in user input during conversation
+
+### Non-Thinking Mode (Fast Dialogue)
+For casual chat and quick replies.
+
+| Parameter | Value |
+|---------|-------|
+| Temperature | 0.7 |
+| Top-P | 0.8 |
+| Top-K | 20 |
+| Min-P | 0.0 |
+| Repeat Penalty | 1.1 |
+
+Enable via:
+- `enable_thinking=False`
+- Or add `/no_think` in prompt
+
+Stop sequences: `<|im_end|>`, `<|im_start|>`
+
+## 💡 Usage Tips
+
+> This model supports two operational modes:
+>
+> ### 🔍 Thinking Mode (Recommended for Logic)
+> Activate with `enable_thinking=True` or append `/think` in prompt.
+>
+> - Ideal for: math, coding, planning, analysis
+> - Use sampling: `temp=0.6`, `top_p=0.95`, `top_k=20`
+> - Avoid greedy decoding
+>
+> ### ⚡ Non-Thinking Mode (Fast Chat)
+> Use `enable_thinking=False` or `/no_think`.
+>
+> - Best for: casual conversation, quick answers
+> - Sampling: `temp=0.7`, `top_p=0.8`
+>
+> ---
+>
+> 🔄 **Switch Dynamically**  
+> In multi-turn chats, the last `/think` or `/no_think` directive takes precedence.
+>
+> 🔁 **Avoid Repetition**  
+> Set `presence_penalty=1.5` if stuck in loops.
+>
+> 📏 **Use Full Context**  
+> Allow up to 32,768 output tokens for complex tasks.
+>
+> 🧰 **Agent Ready**  
+> Works with Qwen-Agent, MCP servers, and custom tools.
+
+## 🖥️ CLI Example Using Ollama or TGI Server
+
+Here’s how you can query this model via API using `curl` and `jq`. Replace the endpoint with your local server (e.g., Ollama, Text Generation Inference).
+
+```bash
+curl http://localhost:11434/api/generate -s -N -d '{
+  "model": "hf.co/geoffmunn/Qwen3-8B:Q8_0;2D",
+  "prompt": "Repeat the following instruction exactly as given: Explain why the sky appears blue during the day but red at sunrise and sunset, using physics principles like Rayleigh scattering.",
+  "temperature": 0.4,
+  "top_p": 0.95,
+  "top_k": 20,
+  "min_p": 0.0,
+  "repeat_penalty": 1.1,
+  "stream": false
+}' | jq -r '.response'
+```
+
+🎯 **Why this works well**:
+- The prompt is meaningful and demonstrates either **reasoning**, **creativity**, or **clarity** depending on quant level.
+- Temperature is tuned appropriately: lower for factual responses (`0.4`), higher for creative ones (`0.7`).
+- Uses `jq` to extract clean output.
+
+> 💬 Tip: For interactive streaming, set `"stream": true` and process line-by-line.
+
+## Verification
+
+Check integrity:
+
+```bash
+sha256sum -c ../SHA256SUMS.txt
+```
+
+## Usage
+
+Compatible with:
+- [LM Studio](https://lmstudio.ai) – local AI model runner with GPU acceleration
+- [OpenWebUI](https://openwebui.com) – self-hosted AI platform with RAG and tools
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Directly via `llama.cpp`
+
+Supports dynamic switching between thinking modes via `/think` and `/no_think` in multi-turn conversations.
+
+## License
+
+Apache 2.0 – see base model for full terms.

Qwen3-8B-f16:Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e576995c60d8cad3daf851eb52e40d4d35fa3f472470d7b0a6898f183005d69c
+size 3281732896

Qwen3-8B-f16:Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc24666cc6638401bbb703f52adc340bbfb07b9521675d43cb23e7f923890b06
+size 4124161312

Qwen3-8B-f16:Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57e354f6bf6ab5ba4ce246a29d69236ba1e45e542b90eefd080c12389491b569
+size 3769611552

Qwen3-8B-f16:Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c167f66114c97c126c432976b9a081e12baf932126cb90476cac2932deec8b3
+size 5027783968

Qwen3-8B-f16:Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7647177f7a10f06642a6c62285b8547648da4149fcd019f6a99e8e945856e5bf
+size 4802012448

Qwen3-8B-f16:Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cabfa609126bf7250dc89b4d7c637b35a28aa894873d98a4c9e1decde589bc5
+size 5851112736

Qwen3-8B-f16:Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f6637d53d5bccaa0b9c938fd3df78661db6e54a9d4a0d3fb57c99410dc230bb
+size 5720761632

Qwen3-8B-f16:Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:863da90f35b7c22b1cd184a20d04cc0eaf0df67f8f52ab0a6d4f68d192600898
+size 6725899552

Qwen3-8B-f16:Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21962d706e584d0058a3f078ba42a40b7c3c82a1aa1a25588d372d41c99a8b6e
+size 8709518624

README.md

@@ -0,0 +1,84 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning
+  - agent
+  - chat
+  - multilingual
+base_model: Qwen/Qwen3-8B
+author: geoffmunn
+pipeline_tag: text-generation
+language:
+  - en
+  - zh
+  - es
+  - fr
+  - de
+  - ru
+  - ar
+  - ja
+  - ko
+  - hi
+---
+
+# Qwen3-8B-GGUF
+
+This is a **GGUF-quantized version** of the **[Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)** language model — an **8-billion-parameter** LLM from Alibaba's Qwen series, designed for **advanced reasoning, agentic behavior, and multilingual tasks**.
+
+Converted for use with `llama.cpp` and compatible tools like OpenWebUI, LM Studio, GPT4All, and more.
+
+> 💡 **Key Features of Qwen3-8B**:
+> - 🤔 **Thinking Mode**: Use `enable_thinking=True` or `/think` for step-by-step logic, math, and code.
+> - ⚡ **Non-Thinking Mode**: Use `/no_think` for fast, lightweight dialogue.
+> - 🧰 **Agent Capable**: Integrates with tools via MCP, APIs, and plugins.
+> - 🌍 **Multilingual Support**: Fluent in 100+ languages including Chinese, English, Spanish, Arabic, Japanese, etc.
+
+## Available Quantizations (from f16)
+
+These variants were built from a **f16** base model to ensure consistency across quant levels.
+
+| Level     | Quality       | Speed     | Size      | Recommendation |
+|----------|--------------|----------|-----------|----------------|
+| Q2_K     | Very Low     | ⚡ Fastest | 2.7 GB   | Only on severely memory-constrained systems (<6GB RAM). Avoid for reasoning. |
+| Q3_K_S   | Low          | ⚡ Fast    | 3.1 GB   | Minimal viability; basic completion only. Not recommended. |
+| Q3_K_M   | Low-Medium   | ⚡ Fast    | 3.3 GB   | Acceptable for simple chat on older systems. No complex logic. |
+| Q4_K_S   | Medium       | 🚀 Fast    | 3.8 GB   | Good balance for low-end laptops or embedded platforms. |
+| Q4_K_M   | ✅ Balanced   | 🚀 Fast    | 4.0 GB   | Best overall for general use on average hardware. Great speed/quality trade-off. |
+| Q5_K_S   | High         | 🐢 Medium  | 4.5 GB   | Better reasoning; slightly faster than Q5_K_M. Ideal for coding. |
+| Q5_K_M   | ✅✅ High     | 🐢 Medium  | 4.6 GB   | Top pick for deep interactions, logic, and tool use. Recommended for desktops. |
+| Q6_K     | 🔥 Near-FP16 | 🐌 Slow    | 5.2 GB   | Excellent fidelity; ideal for RAG, retrieval, and accuracy-critical tasks. |
+| Q8_0     | 🏆 Lossless*  | 🐌 Slow    | 6.8 GB   | Maximum accuracy; best for research, benchmarking, or archival. |
+
+> 💡 **Recommendations by Use Case**
+>
+> - 💻 **Low-end CPU / Old Laptop**: `Q4_K_M` (best balance under pressure)
+> - 🖥️ **Standard/Mid-tier Laptop (i5/i7/M1/M2)**: `Q5_K_M` (optimal quality)
+> - 🧠 **Reasoning, Coding, Math**: `Q5_K_M` or `Q6_K` (use thinking mode!)
+> - 🤖 **Agent & Tool Integration**: `Q5_K_M` — handles JSON, function calls well
+> - 🔍 **RAG, Retrieval, Precision Tasks**: `Q6_K` or `Q8_0`
+> - 📦 **Storage-Constrained Devices**: `Q4_K_S` or `Q4_K_M`
+> - 🛠️ **Development & Testing**: Test from `Q4_K_M` up to `Q8_0` to assess trade-offs
+
+## Usage
+
+Load this model using:
+- [OpenWebUI](https://openwebui.com) – self-hosted AI interface with RAG & tools
+- [LM Studio](https://lmstudio.ai) – desktop app with GPU support and chat templates
+- [GPT4All](https://gpt4all.io) – private, local AI chatbot (offline-first)
+- Or directly via \`llama.cpp\`
+
+Each quantized model includes its own `README.md` and shares a common `MODELFILE` for optimal configuration.
+
+## Author
+
+👤 Geoff Munn (@geoffmunn)  
+🔗 [Hugging Face Profile](https://huggingface.co/geoffmunn)
+
+## Disclaimer
+
+This is a community conversion for local inference. Not affiliated with Alibaba Cloud or the Qwen team.

SHA256SUMS.txt

@@ -0,0 +1,9 @@
+e576995c60d8cad3daf851eb52e40d4d35fa3f472470d7b0a6898f183005d69c  Qwen3-8B-f16:Q2_K.gguf
+fc24666cc6638401bbb703f52adc340bbfb07b9521675d43cb23e7f923890b06  Qwen3-8B-f16:Q3_K_M.gguf
+57e354f6bf6ab5ba4ce246a29d69236ba1e45e542b90eefd080c12389491b569  Qwen3-8B-f16:Q3_K_S.gguf
+8c167f66114c97c126c432976b9a081e12baf932126cb90476cac2932deec8b3  Qwen3-8B-f16:Q4_K_M.gguf
+7647177f7a10f06642a6c62285b8547648da4149fcd019f6a99e8e945856e5bf  Qwen3-8B-f16:Q4_K_S.gguf
+8cabfa609126bf7250dc89b4d7c637b35a28aa894873d98a4c9e1decde589bc5  Qwen3-8B-f16:Q5_K_M.gguf
+8f6637d53d5bccaa0b9c938fd3df78661db6e54a9d4a0d3fb57c99410dc230bb  Qwen3-8B-f16:Q5_K_S.gguf
+863da90f35b7c22b1cd184a20d04cc0eaf0df67f8f52ab0a6d4f68d192600898  Qwen3-8B-f16:Q6_K.gguf
+21962d706e584d0058a3f078ba42a40b7c3c82a1aa1a25588d372d41c99a8b6e  Qwen3-8B-f16:Q8_0.gguf