Skip to content

Conversation

@chopan050
Copy link
Contributor

@chopan050 chopan050 commented Jun 4, 2025

Fixes #2412
Therefore, also fixes its duplicates #2979, #3131, #3559 and part of #3646.

As I commented in the "Image issues Tracking" issue #3482:

The main problem is that the implementation of the Camera.display_image_mobject() method is incomplete:

  • It only resizes and rotates the image.
  • It does not shear the image or apply perspective to it. Therefore, it always looks rectangular and faces the camera, even when you 3D rotate it. Take a look at this TODO.
  • In facing the camera, the image always preserves its 2D orientation (clockwise - counterclockwise). Thus, flipping the image (i.e. rotating it 180° about the Y axis) does not work as expected. A proper flipping would also change the image's orientation, which is currently not happening.

The solution I described, which was also discussed in some of the issues I mentioned before, consists of leveraging PIL.Image.Image.transform() to rewrite Camera.display_image_mobject(). Figuring out how to use PIL.Image.Image.transform() was the actual challenge, but this StackOverflow discussion shed light on this problem (thank you very much, mmgp ❤️).

I'll use the example provided by @hennels on issue #2412 (thanks!). I'm including the generated video after applying these changes.

There are still some issues, though:

  • The image is showing in front of the cube when it shouldn't. This is a known Manim issue which falls outside of this PR's scope.
  • EDIT: this glitch was fixed and is now hidden in the following details section:
    Details
    There is a brief glitch at the beginning of this video when the image is perpendicular to the camera. Maybe floating point error? I couldn't figure out how to fix it, but this video is in medium quality and the glitch doesn't show up when I render the scene in high quality. Key point: high quality = more pixels = bigger pixel coordinates https://github.com/user-attachments/assets/562ef7e7-772d-44e0-9137-2511266576c7
ImageIn3D.mp4

However, this PR must introduce a...

BREAKING CHANGE 💥

PIL.Image.Image.transform() only supports the NEAREST, BILINEAR and BICUBIC resampling algorithms. It does not support BOX, HAMMING or LANCZOS. That is a big issue, since Camera.display_image_mobject() runs for every frame and you can't opt out of using it. Even in 2D scenes where there's no actual need for a perspective transform, one still has to account for possible shears of an image, which seems to require .transform() anyways. There doesn't seem to exist a proper fix to the 3D rotation / flipping / shear issue without using PIL.Image.Image.transform().

If such fix does not exist, then it is not possible to support BOX, HAMMING or LANCZOS. Therefore, I removed support for them in this PR by removing the following constants:

RESAMPLING_ALGORITHM["box"]
RESAMPLING_ALGORITHM["hamming"]
RESAMPLING_ALGORITHM["lanczos"]
RESAMPLING_ALGORITHM["antialias"]  # alias for "lanczos"

I also rewrote tests/test_graphical_units/test_img_and_svg.py::test_ImageInterpolation to remove those now unsupported resampling algorithms and rewrote AbstractImageMobject.set_resampling_algorithm() to better capture invalid values.

Reviewer Checklist

  • The PR title is descriptive enough for the changelog, and the PR is labeled correctly
  • If applicable: newly added non-private functions and classes have a docstring including a short summary and a PARAMETERS section
  • If applicable: newly added functions and classes are tested

@chopan050 chopan050 force-pushed the fix-image-rotation branch from 7b059ef to 6aea7b6 Compare June 5, 2025 18:03
@chopan050 chopan050 added pr:bugfix Bug fix for use in PRs solving a specific issue:bug breaking changes This PR introduces breaking changes needs discussion Things which needs to be discussed before implemented. labels Jul 24, 2025
@chopan050 chopan050 marked this pull request as draft January 16, 2026 04:03
@chopan050
Copy link
Contributor Author

chopan050 commented Jan 17, 2026

I fixed the glitch!

The transform coefficients for Image.transform() had weird values when the image was perpendicular to the camera, when the points of the transformed image were too close to being collinear.

To fix it, I simply skip the rendering of the image if the quadrilateral of the transformed image cannot contain any pixels (when it's perpendicular to the camera, too small, etc). In order to figure it out, I find the longest side of the quadrilateral and calculate the height from that side: if it's less than 0.5 pixels, do not attempt to render the image.

Plus, to help prevent situations where an image is not rendered because two corners are mapped to the same pixel or three corners are mapped to collinear pixels (without the whole image being necessarily invisible, such as when we apply a rather extreme perspective), I defined Camera.points_to_subpixel_coords(), which does almost the exact same thing as Camera.points_to_pixel_coords(), minus the final conversion to integers. The latter method can now simply call the former method and convert its result to integers.

EDIT: a video showing that images can now be properly flipped as well. Example code:

class FlipImage(Scene):
    def construct(self):
        arr = np.zeros((256, 256, 3), dtype=np.uint8)
        arr[:128, :128] = BLUE_E.to_int_rgb()
        arr[:128, 128:] = BLUE_D.to_int_rgb()
        arr[128:, :128] = BLUE_C.to_int_rgb()
        arr[128:, 128:] = DARK_BROWN.to_int_rgb()

        image = ImageMobject(arr)
        self.add(image)
        
        self.play(image.animate.rotate(TAU/2, axis=UP))
        self.wait(0.5)
FlipImage.mp4

@chopan050 chopan050 marked this pull request as ready for review January 17, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking changes This PR introduces breaking changes needs discussion Things which needs to be discussed before implemented. pr:bugfix Bug fix for use in PRs solving a specific issue:bug

Projects

Status: 🆕 New

Development

Successfully merging this pull request may close these issues.

3D rotating and flipping does not work as expected for ImageMobject

1 participant