-
Notifications
You must be signed in to change notification settings - Fork 6.6k
Add ZImageImg2ImgPipeline #12751
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add ZImageImg2ImgPipeline #12751
Conversation
Updated the pipeline structure to include ZImageImg2ImgPipeline
alongside ZImagePipeline.
Implemented the ZImageImg2ImgPipeline class for image-to-image
transformations, including necessary methods for
encoding prompts, preparing latents, and denoising.
Enhanced the auto_pipeline to map the new ZImageImg2ImgPipeline
for image generation tasks.
Added unit tests for ZImageImg2ImgPipeline to ensure
functionality and performance.
Updated dummy objects to include ZImageImg2ImgPipeline for
testing purposes.
|
For some reason the VAE Tiling couldn't meet the 0.2 diff threshold, my test has upped that to 0.3, whether further investigation is warranted I am not sure. |
asomoza
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks a lot again! for this one we should probably wait for the lora one to be merged. I left a few comments
|
@asomoza I just thought, I have inpainting PR lined up, do you think keeping this one img2img only and inpainting after that, separately, is the better approach, to keep the PR review easier? Or is it less work for you guys if I also merge this in this PR? |
|
I prefer to keep them separated, I'm not really sure the inpainting can be good with this model so I want to test it and maybe we can add something like differential diffusion as a switch for it to be better |
- Add `# Copied from` annotations to encode_prompt and _encode_prompt - Add ZImagePipeline to auto_pipeline.py for AutoPipeline support
|
@CalamitousFelicitousness you need to resolve the conflict before we can proceed |
Resolved conflict in src/diffusers/pipelines/__init__.py by: - Accepting upstream's expanded Kandinsky5 pipelines - Preserving ZImageImg2ImgPipeline addition
|
@asomoza The conflict has been resolved. |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Resolve conflict in _toctree.yml: use upstream's z_image.md naming convention and add ZImageImg2ImgPipeline documentation.
|
@asomoza Gentle reminder if you could review. |



What does this PR do?
This PR adds img2img pipeline for Z-Image. The summary of changes are below
Closes issue #12752
Tested using a simple script:
Testing script
Prompt:
a woman sitting in a dark room, oil painting style, impressionist, vibrant colorsLoRA functionality depends on my other PR #12750, so they will have to be merged sequentially. I did not think there was much point in leaving it out.
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@sayakpaul @asomoza