Upload gguf/README.md with huggingface_hub

gguf/README.md

@@ -0,0 +1,152 @@
+# ZigNet Qwen2.5-Coder-7B (GGUF Q4_K_M)
+
+Quantized GGUF version of [ZigNet Qwen2.5-Coder-7B](https://huggingface.co/fulgidus/zignet-qwen2.5-coder-7b) for local inference.
+
+## Model Details
+
+- **Base Model**: Qwen/Qwen2.5-Coder-7B-Instruct
+- **Fine-tuning**: LoRA adapters trained on 13,756 Zig examples
+- **Quantization**: Q4_K_M (4-bit, mixed K-quants)
+- **Size**: 4.4GB (73% reduction from F16's 15GB)
+- **Quality**: ~95-98% retention vs full precision
+- **Speed**: 3-4x faster on consumer GPUs
+
+## Quick Start
+
+### With Ollama
+
+```bash
+# Download model
+huggingface-cli download fulgidus/zignet-qwen2.5-coder-7b \
+    gguf/zignet-qwen-7b-q4km.gguf \
+    --local-dir ./models
+
+# Create Modelfile
+cat > Modelfile << 'EOF'
+FROM ./models/gguf/zignet-qwen-7b-q4km.gguf
+
+TEMPLATE """{{ if .System }}<|im_start|>system
+{{ .System }}<|im_end|>
+{{ end }}{{ if .Prompt }}<|im_start|>user
+{{ .Prompt }}<|im_end|>
+{{ end }}<|im_start|>assistant
+{{ .Response }}<|im_end|>
+"""
+
+SYSTEM """You are ZigNet, an AI assistant specialized in Zig programming language (v0.13-0.15).
+
+Your expertise includes:
+- Explaining Zig syntax, features, and idioms
+- Understanding comptime, generics, and error handling
+- Providing code examples and fixes
+- Referencing official Zig documentation"""
+
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER num_ctx 4096
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+EOF
+
+# Import to Ollama
+ollama create zignet:latest -f Modelfile
+
+# Run
+ollama run zignet:latest "Explain comptime in Zig"
+```
+
+### With llama.cpp
+
+```bash
+# Download model
+huggingface-cli download fulgidus/zignet-qwen2.5-coder-7b \
+    gguf/zignet-qwen-7b-q4km.gguf
+
+# Run with llama.cpp
+./llama-cli \
+    -m ~/.cache/huggingface/hub/models--fulgidus--zignet-qwen2.5-coder-7b/gguf/zignet-qwen-7b-q4km.gguf \
+    -p "Explain Zig's error handling system:" \
+    -n 512 \
+    --temp 0.7 \
+    --top-p 0.9 \
+    -ngl 35  # Offload layers to GPU
+```
+
+### With node-llama-cpp (JavaScript/TypeScript)
+
+```typescript
+import { LlamaModel, LlamaContext, LlamaChatSession } from "node-llama-cpp";
+
+const model = new LlamaModel({
+    modelPath: "./models/gguf/zignet-qwen-7b-q4km.gguf"
+});
+
+const context = new LlamaContext({ model });
+const session = new LlamaChatSession({ context });
+
+const response = await session.prompt(
+    "Generate a generic ArrayList in Zig"
+);
+
+console.log(response);
+```
+
+## Hardware Requirements
+
+| Configuration | VRAM | RAM | Speed |
+|--------------|------|-----|-------|
+| CPU only | - | 8GB | ~15-20 tokens/s |
+| NVIDIA GPU | 6GB | 8GB | ~60-80 tokens/s |
+| Apple M1/M2 | - | 8GB | ~40-50 tokens/s |
+| RTX 3090 | 24GB | 16GB | ~100+ tokens/s |
+
+## Use Cases
+
+- **Documentation Lookup**: Explain Zig language features
+- **Code Generation**: Generate idiomatic Zig patterns
+- **Error Fixes**: Suggest corrections for compilation errors
+- **MCP Integration**: Power ZigNet's `get_zig_docs` and `suggest_fix` tools
+
+## Limitations
+
+- Optimized for Zig 0.13-0.15 (may have reduced accuracy on other versions)
+- Not suitable for real-time syntax parsing (use compiler-based tools)
+- Always validate generated code with `zig build` or `zig test`
+
+## Example Outputs
+
+**Prompt**: "Explain error sets in Zig"
+
+**Response**:
+> In Zig, an error set is a tagged union type that allows functions to express failure and return both a value and an error code. By declaring the return type as `error_set_type`, a function communicates which errors it may return, enabling compile-time safety and clarity about how errors are handled.
+
+**Prompt**: "Generate a simple Zig struct with methods"
+
+**Response**:
+```zig
+const Point = struct {
+    x: f32,
+    y: f32,
+
+    pub fn init(x: f32, y: f32) Point {
+        return Point{ .x = x, .y = y };
+    }
+
+    pub fn distance(self: Point, other: Point) f32 {
+        const dx = self.x - other.x;
+        const dy = self.y - other.y;
+        return @sqrt(dx * dx + dy * dy);
+    }
+};
+```
+
+## License
+
+Apache 2.0
+
+## Links
+
+- **Full Model (LoRA)**: [fulgidus/zignet-qwen2.5-coder-7b](https://huggingface.co/fulgidus/zignet-qwen2.5-coder-7b)
+- **Training Dataset**: [fulgidus/zignet-training-dataset](https://huggingface.co/datasets/fulgidus/zignet-training-dataset)
+- **ZigNet MCP Server**: [github.com/fulgidus/zignet](https://github.com/fulgidus/zignet)
+- **Base Model**: [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)