How do you convert an SDXL based model from safetensors to GGUF on Linux?
Hello Calcius,
I'm reaching out to get some guidance on converting SDXL models into the GGUF format.
My Goal:
I have been working with SDXL models for image generation and I'm very interested in converting them to GGUF. My primary goal is to run these models on hardware with limited VRAM or in environments where llama.cpp or similar GGUF-compatible inference engines are preferred. I believe this conversion would help with resource management and make the models more accessible.
Specific Questions / Where I'm Stuck:
I'm hoping you or someone can shed light on the following points:
Recommended Tools: Is there a recommended script or tool for converting SDXL checkpoints (.safetensors ) to GGUF?
Process & Steps: Could you outline the general workflow? For example, do I need to convert the model to an intermediate format first before converting to GGUF?
Handling Components: How should the different parts of the SDXL pipeline (the text encoders, the VAE, and the U-Net) be handled? Should they be converted separately and then combined, or is there a way to convert the whole pipeline at once?
Quantization: Are there any best practices for quantization levels (e.g., Q8, Q4_K_M, F16) for SDXL to balance quality and performance?
My Environment:
OS: Bazzite (Fedora)
Python Version: 3.10,3.12
Any guidance, links to resources, or a brief walkthrough would be immensely helpful. Thank you in advance for your time and assistance!
guess you could simply use the convert alpha over gguf node or the tool inside gguf-connect to get the job done