Step1X-Edit: A Practical Framework for General Image Editing
In recent years, image editing models have witnessed remarkable and rapid development. The recent unveiling of cutting-edge multimodal models such as GPT-4o and Gemini2 Flash has introduced highly promising image editing capabilities. These models demonstrate an impressive aptitude for fulfilling a vast majority of user-driven editing requirements, marking a significant advancement in the field of image manipulation. However, there is still a large gap between the open-source algorithm with these closed-source models. Thus, in this paper, we aim to release a state-of-the-art image editing model, called Step1X-Edit, which can provide comparable performance against the closed-source models like GPT-4o and Gemini2 Flash. More specifically, we adopt the Multimodal LLM to process the reference image and the user's editing instruction. A latent embedding has been extracted and integrated with a diffusion image decoder to obtain the target image. To train the model, we build a data generation pipeline to produce a high-quality dataset. For evaluation, we develop the GEdit-Bench, a novel benchmark rooted in real-world user instructions. Experimental results on GEdit-Bench demonstrate that Step1X-Edit outperforms existing open-source baselines by a substantial margin and approaches the performance of leading proprietary models, thereby making significant contributions to the field of image editing.
PDF AbstractCode
Spaces
Datasets
MagicBrush
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Image Editing | ImgEdit | Step1X-Edit | Overall | 3.06 | # 3 | |
| Add | 3.88 | # 1 | ||||
| Adjust | 3.14 | # 4 | ||||
| Extract | 1.76 | # 4 | ||||
| Replace | 3.40 | # 2 | ||||
| Remove | 2.41 | # 4 | ||||
| Background | 3.16 | # 2 | ||||
| Style | 4.63 | # 1 | ||||
| Hybrid | 2.64 | # 2 | ||||
| Action | 2.52 | # 6 |
