README.md

---
license: apache-2.0
tags:
- text-to-image
- image-editing
- lora
- qwen-image
- diffusion
library_name: diffsynth
base_model:
- Qwen/Qwen-Image-Edit
---

# Eigen-Banana-Qwen-Image-Edit: Fast Image Editing with Qwen-Image-Edit LoRA

⚡ [**Lightning Demo Website**](https://app.eigenai.com/eigen-banana-qwen-image-edit.html) / 📄 [**Blog Post**](https://www.eigenai.com/blog/2025-10-30-eigen-banana.html) /🤗 [**Hugging Face App**](https://huggingface.co/spaces/akhaliq/eigen-banana-qwen-image-edit) 


**Eigen-Banana-Qwen-Image-Edit** is a LoRA (Low-Rank Adaptation) checkpoint for the Qwen-Image-Edit model, optimized for fast, high-quality image editing with text prompts. This model enables efficient text-guided image transformations with reduced inference steps while maintaining excellent quality.

Trained on the **[Pico-Banana-400K](https://github.com/apple/pico-banana-400k)** dataset from Apple—a large-scale collection of ~400K text–image–edit triplets covering 35 edit operations across diverse semantic categories—Eigen-Banana-Qwen-Image-Edit excels at a wide range of editing tasks from object manipulation to stylistic transformations.

## Model Details

- **Base Model**: Qwen/Qwen-Image-Edit
- **Model Type**: LoRA Fine-tuned Diffusion Transformer
- **Training Dataset**: [Pico-Banana-400K](https://github.com/apple/pico-banana-400k)
- **Training Method**: EigenTrain (LoRA fine-tuning)
- **Format**: FP16 SafeTensors
- **License**: Apache 2.0
- **Use Cases**: Text-guided image editing, style transfer, object modification, scene transformation

## Features

✨ **Fast Inference**: Optimized for quick generation with distilled knowledge  
🎨 **High Quality**: Maintains excellent visual quality with fewer steps  
🌐 **Multilingual**: Supports both English and Chinese prompts  
⚡ **Efficient**: Lightweight LoRA weights for easy deployment  

## Training Dataset

This model was trained on **[Pico-Banana-400K](https://github.com/apple/pico-banana-400k)**, a large-scale dataset of ~400K text–image–edit triplets designed for text-guided image editing research.

### Dataset Highlights

- **~257K single-turn text–image–edit triplets** for supervised fine-tuning
- **35 edit operations** across **8 semantic categories**:
  - Object-Level Semantic (35%): Add, remove, replace, or relocate objects
  - Scene Composition & Multi-Subject (20%): Contextual transformations
  - Human-Centric (18%): Clothing, expression, appearance edits
  - Stylistic (10%): Domain and artistic style transfer
  - Text & Symbol (8%): Edits involving visible text or signs
  - Pixel & Photometric (5%): Brightness, contrast, tonal adjustments
  - Scale & Perspective (2%): Zoom, viewpoint changes
  - Spatial/Layout (2%): Outpainting, composition extension

- **Source Images**: [Open Images Dataset](https://storage.googleapis.com/openimages/web/index.html)
- **Instruction Generation**: Gemini-2.5-Flash for natural-language editing prompts
- **Quality Control**: Automated self-evaluation using Gemini-2.5-Pro

### Training Methodology

The model was fine-tuned using **EigenTrain**, a training platform that unifies SFT, offline RL, and online RL for training text LLMs and VLMs, and includes first-class workflows for multimodal image/video generation. Here, we used EigenTrain to do LoRA fine-tuning on Qwen-Image-Edit. Key training parameters:

- **LoRA Rank**: 32
- **Target Modules**: `to_q`, `to_k`, `to_v`, `add_q_proj`, `add_k_proj`, `add_v_proj`, `to_out.0`, `to_add_out`, `img_mlp.net.2`, `img_mod.1`, `txt_mlp.net.2`, `txt_mod.1`
- **Learning Rate**: 1e-4
- **Training Data**: ~257K high-quality text-image-edit triplets
- **Precision**: FP16 for efficient deployment

This combination of high-quality training data and efficient LoRA adaptation enables fast, accurate image editing while maintaining the base model's strong capabilities.

## Demo Images

We present several examples to qualitatively compare the original qwen-image-edit and our eigen-banana-qwen-image-edit. 

### Example 1 – Add a new object to the scene

<!-- Input (moderate width, centered) -->
<p align="center">
  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example1-input.png"
       alt="example1-input"
       width="360">
</p>

**Prompt:** Integrate a minimalist, dark-toned, rectangular gallery bench into the mid-ground, positioned slightly to the right of the central pillar and facing the right wall, ensuring its texture, lighting, and subtle shadows are consistent with the existing black and white aesthetic and diffused ambient light of the art gallery.

**Outputs**

<!-- Two-up grid with captions (works on HF + GitHub) -->
<table>
  <tr>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example1-qwen-image-edit.png"
           alt="example1-qwen-image-edit"
           width="100%"><br/>
      <h5><b>Qwen-Image-Edit</b></h5>
    </td>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example1-eigen-banana.png"
           alt="example1-eigen-banana"
           width="100%"><br/>
      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
    </td>
  </tr>
</table>


### Example 2 – Add a film grain/filter

<!-- Input (moderate width, centered) -->
<p align="center">
  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example2-input.png"
       alt="example2-input"
       width="360">
</p>

**Prompt:** Apply a vintage film aesthetic to the image, featuring a subtle desaturation of colors with a warm, golden-hour tone, introduce a fine and natural-looking film grain across the entire scene, gently reduce overall contrast for a softer appearance, and add a very faint, dark vignette to the edges to mimic an aged photographic print.

**Outputs**

<!-- Two-up grid with captions (works on HF + GitHub) -->
<table>
  <tr>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example2-qwen-image-edit.png"
           alt="example2-qwen-image-edit"
           width="100%"><br/>
      <h5><b>Qwen-Image-Edit</b></h5>
    </td>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example2-eigen-banana.png"
           alt="example2-eigen-banana"
           width="100%"><br/>
      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
    </td>
  </tr>
</table>


### Example 3 – Add a new text

<!-- Input (moderate width, centered) -->
<p align="center">
  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example3-input.png"
       alt="example3-input"
       width="360">
</p>

**Prompt:** Add the text "CHAMPION" in a bold, sans-serif font, horizontally aligned below the existing "GLASGOW" text on the race bib of the runner wearing number 454 (yellow singlet), ensuring the text color, lighting, and subtle fabric distortion match the existing elements on the bib.

**Outputs**

<!-- Two-up grid with captions (works on HF + GitHub) -->
<table>
  <tr>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example3-qwen-image-edit.png"
           alt="example3-qwen-image-edit"
           width="100%"><br/>
      <h5><b>Qwen-Image-Edit</b></h5>
    </td>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example3-eigen-banana.png"
           alt="example3-eigen-banana"
           width="100%"><br/>
      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
    </td>
  </tr>
</table>


### Example 4 – Add a new scene/background 

<!-- Input (moderate width, centered) -->
<p align="center">
  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example4-input.png"
       alt="example4-input"
       width="360">
</p>

**Prompt:** Replace the current plain wall background with a sophisticated, softly lit indoor event space, featuring warm golden ambient lighting, elegant architectural details such as decorative panels or subtle artwork, and a slightly blurred depth of field to keep the focus on the subjects while ensuring the new background's rich, muted tones complement their attire.

**Outputs**

<!-- Two-up grid with captions (works on HF + GitHub) -->
<table>
  <tr>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example4-qwen-image-edit.png"
           alt="example4-qwen-image-edit"
           width="100%"><br/>
      <h5><b>Qwen-Image-Edit</b></h5>
    </td>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example4-eigen-banana.png"
           alt="example4-eigen-banana"
           width="100%"><br/>
      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
    </td>
  </tr>
</table>


### Example 5 – Modify expressions 

<!-- Input (moderate width, centered) -->
<p align="center">
  <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example5-input.png"
       alt="example5-input"
       width="360">
</p>

**Prompt:** Adjust the subject's facial expression to a subtle, closed-mouth smile, ensuring natural skin folds and realistic lighting on the face, while maintaining the existing head posture and integrating seamlessly with the overall image context.

**Outputs**

<!-- Two-up grid with captions (works on HF + GitHub) -->
<table>
  <tr>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example5-qwen-image-edit.png"
           alt="example5-qwen-image-edit"
           width="100%"><br/>
      <h5><b>Qwen-Image-Edit</b></h5>
    </td>
    <td align="center" width="50%">
      <img src="https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit/resolve/main/assets/example5-eigen-banana.png"
           alt="example5-eigen-banana"
           width="100%"><br/>
      <h5><b>Eigen-Banana (⚡Lightning)</b></h5>
    </td>
  </tr>
</table>


## Installation

Install from source (recommended):

```
git clone https://github.com/modelscope/DiffSynth-Studio.git  
cd DiffSynth-Studio
pip install -e .
```

<details>
<summary>Other installation methods</summary>

Install from PyPI (version updates may be delayed; for latest features, install from source)

```
pip install diffsynth
```

If you meet problems during installation, they might be caused by upstream dependencies. Please check the docs of these packages:

* [torch](https://pytorch.org/get-started/locally/)
* [sentencepiece](https://github.com/google/sentencepiece)
* [cmake](https://cmake.org)
* [cupy](https://docs.cupy.dev/en/stable/install.html)

</details>


## Usage

### Basic Image Editing

```python
from diffsynth.pipelines.qwen_image import QwenImagePipeline, ModelConfig
import torch

# Initialize the pipeline
pipe = QwenImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(
            model_id="Qwen/Qwen-Image-Edit",
            origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"
        ),
        ModelConfig(
            model_id="Qwen/Qwen-Image",
            origin_file_pattern="text_encoder/model*.safetensors"
        ),
        ModelConfig(
            model_id="Qwen/Qwen-Image",
            origin_file_pattern="vae/diffusion_pytorch_model.safetensors"
        ),
    ],
    processor_config=ModelConfig(
        model_id="Qwen/Qwen-Image-Edit",
        origin_file_pattern="processor/"
    ),
)

# Load the Eigen-Banana-Qwen-Image-Edit LoRA
pipe.load_lora(pipe.dit, "eigen-ai-labs/eigen-banana-qwen-image-edit/eigen-banana-qwen-image-edit-fp16-lora.safetensors")

# Generate an initial image
prompt = "A beautiful portrait, underwater girl, blue dress flowing, hair drifting, light penetrating, bubbles surrounding, serene face, exquisite details, dreamy and aesthetic."
input_image = pipe(
    prompt=prompt,
    seed=0,
    num_inference_steps=40,
    height=1328,
    width=1024
)
input_image.save("original.jpg")

# Edit the image
edit_prompt = "Change the dress to pink"
edited_image = pipe(
    edit_prompt,
    edit_image=input_image,
    seed=1,
    num_inference_steps=40,
    height=1328,
    width=1024,
    edit_image_auto_resize=True
)
edited_image.save("edited.jpg")
```

### Chinese Prompts Example

```python
# Generate initial image with Chinese prompt
prompt = "精致肖像，水下少女，蓝裙飘逸，发丝轻扬，光影透澈，气泡环绕，面容恬静，细节精致，梦幻唯美。"
input_image = pipe(
    prompt=prompt,
    seed=0,
    num_inference_steps=40,
    height=1328,
    width=1024
)
input_image.save("image1.jpg")

# Edit with Chinese prompt
prompt = "将裙子改为粉色"
edited_image = pipe(
    prompt,
    edit_image=input_image,
    seed=1,
    num_inference_steps=40,
    height=1328,
    width=1024,
    edit_image_auto_resize=True
)
edited_image.save("image2.jpg")
```

### Advanced Usage

#### Auto-Resize Options

The `edit_image_auto_resize` parameter controls how input images are processed:

```python
# Auto-resize: maintains aspect ratio while matching 1024x1024 area
edited_image = pipe(
    prompt,
    edit_image=input_image,
    num_inference_steps=40,
    height=1328,
    width=1024,
    edit_image_auto_resize=True  # Recommended for best results
)

# No resize: use original image size
edited_image = pipe(
    prompt,
    edit_image=input_image,
    num_inference_steps=40,
    height=1328,
    width=1024,
    edit_image_auto_resize=False  # Use when input matches target size
)
```

#### Inference Steps Optimization

The model works well with reduced steps for faster generation:

```python
# High quality (slower)
image = pipe(prompt, edit_image=input_image, num_inference_steps=40)

# Balanced (recommended)
image = pipe(prompt, edit_image=input_image, num_inference_steps=20)

# Fast (may reduce quality slightly)
image = pipe(prompt, edit_image=input_image, num_inference_steps=10)
```

## Example Prompts

### English Prompts

- "Transform this image into a cartoon style"
- "Change the background to a sunset beach"
- "Make it look like a watercolor painting"
- "Add neon lights in cyberpunk style"
- "Convert to black and white photograph"
- "Change the sky to nighttime with stars"

### Chinese Prompts

- "将图片转换为卡通风格"
- "把背景改成夕阳海滩"
- "改成水彩画风格"
- "添加赛博朋克风格的霓虹灯"
- "转换为黑白照片"
- "将天空改为夜晚星空"

## Parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `prompt` | str | - | Text description for image generation/editing |
| `edit_image` | PIL.Image | None | Input image to edit (omit for text-to-image) |
| `num_inference_steps` | int | 40 | Number of denoising steps |
| `height` | int | 1024 | Output image height |
| `width` | int | 1024 | Output image width |
| `seed` | int | 0 | Random seed for reproducibility |
| `edit_image_auto_resize` | bool | True | Auto-resize input to match target area |


## Citation

If you use this model in your research, please cite:

```bibtex
@software{eigen-banana-qwen-image-edit,
  title={Eigen-Banana-Qwen-Image-Edit: Fast Image Editing LoRA for Qwen-Image-Edit},
  author={Eigen AI Labs},
  year={2025},
  url={https://huggingface.co/eigen-ai-labs/eigen-banana-qwen-image-edit}
}
```

If you use the Pico-Banana-400K dataset, please also cite:

```bibtex
@misc{qian2025picobanana400k,
  title={Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editing}, 
  author={Yusu Qian and Eli Bocek-Rivele and Liangchen Song and Jialing Tong and Yinfei Yang and Jiasen Lu and Wenze Hu and Zhe Gan},
  year={2025},
  eprint={2510.19808},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2510.19808}
}
```

## License

This model is a derivative work of the Qwen-Image-Edit model, which is released under the Apache 2.0 License.

The model was trained using the Pico-Banana-400K dataset, which is released under the Creative Commons Attribution–NonCommercial–NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License.

License Details:

- Apache 2.0 License applies to the parts of the model derived from the Qwen-Image-Edit model.

- CC BY-NC-ND 4.0 License applies to the trained model due to the restrictions of the training dataset. Specifically, you may not:

  - Use the model for commercial purposes.

  - Create derivative works of the model.

  - Distribute the model or dataset in a manner that violates the NoDerivatives condition.

Restrictions:

- The model may only be used for non-commercial purposes.

- You may not modify, adapt, or build upon the model.

- You may not distribute the model if the modifications to it would violate the NoDerivatives clause of the dataset’s license.

## Acknowledgements

- Trained on the [Pico-Banana-400K](https://github.com/apple/pico-banana-400k) dataset by Apple
- Built on top of [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio)
- Based on [Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) and [Qwen-Image-Edit](https://huggingface.co/Qwen/Qwen-Image-Edit)
- Special thanks to the Apple ML team for releasing the high-quality Pico-Banana-400K dataset

## Contact

For questions, issues, or collaborations, please contact us at: https://www.eigenai.com/contact.

---

**Note**: This is a LoRA checkpoint and requires the base Qwen-Image-Edit model to function. The base models will be automatically downloaded from HuggingFace when you run the code.