Text Generation
Transformers
Safetensors
qwen2
conversational
text-generation-inference
inikitin commited on
Commit
24ff49f
·
verified ·
1 Parent(s): c69c4eb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -211,6 +211,23 @@ print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))
211
 
212
  ---
213
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
214
  **Fortytwo – An open, networked intelligence shaped collectively by its participants**
215
 
216
  Join the swarm: [fortytwo.network](https://fortytwo.network)
 
211
 
212
  ---
213
 
214
+ ## Quantized Versions
215
+
216
+ Optimized GGUF quantizations of **Strand-Rust-Coder-14B-v1** are available for local and Fortytwo Node deployment, offering reduced memory footprint with minimal performance trade-off.
217
+
218
+ These builds are compatible with **llama.cpp**, **Jan**, **LM Studio**, **Ollama**, and other runtimes supporting the GGUF format.
219
+
220
+ | **Quantization** | **Size** | **Bit Precision** | **Description** |
221
+ |------------------|-----------|------------------|----------------|
222
+ | **Q4_K_M** | 8.99 GB | **4-bit** | Ultra-fast, compact variant for consumer GPUs and laptops |
223
+ | **Q5_K_M** | 10.5 GB | **5-bit** | Lightweight deployment with strong accuracy retention |
224
+ | **Q6_K** | 12.1 GB | **6-bit** | Balanced performance and efficiency |
225
+ | **Q8_0** | 15.7 GB | **8-bit** | Near-full precision, for most demanding local inference |
226
+
227
+ Quant versions: [Fortytwo-Network/Strand-Rust-Coder-14B-v1-GGUF](https://huggingface.co/Fortytwo-Network/Strand-Rust-Coder-14B-v1-GGUF)
228
+
229
+ ---
230
+
231
  **Fortytwo – An open, networked intelligence shaped collectively by its participants**
232
 
233
  Join the swarm: [fortytwo.network](https://fortytwo.network)