metadata
base_model: Qwen/Qwen-Image
frameworks:
- Pytorch
license: Apache License 2.0
tags:
- LoRA
vision_foundation: QWEN_IMAGE_20_B
Qwen-Image LoRA 蒸馏加速模型
模型介绍
本模型是 Qwen-Image 的蒸馏加速 LoRA，我们沿用了模型 DiffSynth-Studio/Qwen-Image-Distill-Full 的训练流程，将可训练模型参数改为 LoRA，从而更方便地集成到各类图像生成框架中。
训练框架基于 DiffSynth-Studio 构建，训练数据是由原模型根据 DiffusionDB 中随机抽取的提示词生成的 1.6 万张图，训练程序在 8 * MI308X GPU 上运行了约 1 天。
效果展示
| 原版模型 | 原版模型 | 加速模型 | |
|---|---|---|---|
| 推理步数 | 40 | 15 | 15 |
| CFG scale | 4 | 1 | 1 |
| 前向推理次数 | 80 | 15 | 15 |
| 样例1 |  |
 |
 |
| 样例2 |  |
 |
 |
| 样例3 |  |
 |
 |
推理代码
git clone https://github.com/modelscope/DiffSynth-Studio.git
cd DiffSynth-Studio
pip install -e .
from diffsynth.pipelines.qwen_image import QwenImagePipeline, ModelConfig
from modelscope import snapshot_download
import torch
snapshot_download("DiffSynth-Studio/Qwen-Image-Distill-LoRA", local_dir="models/DiffSynth-Studio/Qwen-Image-Distill-LoRA")
pipe = QwenImagePipeline.from_pretrained(
torch_dtype=torch.bfloat16,
device="cuda",
model_configs=[
ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"),
ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="text_encoder/model*.safetensors"),
ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="vae/diffusion_pytorch_model.safetensors"),
],
tokenizer_config=ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="tokenizer/"),
)
pipe.load_lora(pipe.dit, "models/DiffSynth-Studio/Qwen-Image-Distill-LoRA/model.safetensors")
prompt = "精致肖像，水下少女，蓝裙飘逸，发丝轻扬，光影透澈，气泡环绕，面容恬静，细节精致，梦幻唯美。"
image = pipe(prompt, seed=0, num_inference_steps=15, cfg_scale=1)
image.save("image.jpg")