Image.remap_palette bug?

[TheRealQuantam]

I'm encountering several severe issues with palettized color in Pillow 9.1.1 on Python 3.7 64-bit. I'm posting this in discussions in case I am doing something wrong, but if I'm not, I'll elevate these to issues. Per request I'm splitting off each issue into a separate thread.

Image.remap_palette does not properly handle multiple source palette entries being mapped to the same destination entry (e.g. multiple palette entries being the same color). In such a case, all but the first of those palette entries will be remapped to palette index 0, regardless of whether they are all the same color (i.e. there could be 2 entries each for 2 colors, and 1 of each color will be mapped to index 0).

Partial code (see link at bottom for full code and data):

img = Image.fromarray(bitmap, "P")
img.putpalette(img_pal)
img.save("bad palette indices.gif", palette = img.palette, transparency = -1)

EDIT: Updated the snippet after learning I was setting the palette incorrectly. This does not fix the problem, however.

Correct version with manually remapped palette so the image doesn't use the palette entries that get changed to 0:

Bad remapping of palette indices (multiple green and black palette entries are set to index 0):

This may be caused by the loop at Image.py:1896:

# pick only the used colors from the palette
for i, oldPosition in enumerate(dest_map):
    palette_bytes += source_palette[oldPosition * 3 : oldPosition * 3 + 3]
    new_positions[oldPosition] = i

bad_pillow.zip

[TheRealQuantam]

Hmm. I initially thought it wasn't fixed by Image.putpalette, but I just realized that that was because I forgot to remove the manual specification of the palette on save. Apparently this issue is related to specifying the palette during save. This might still be a bug, though not the bug I thought. Using putpalette and then not specifying a palette on save does not cause this problem.

[radarhere]

remap_palette cannot map multiple palette entries to the same entry. That's not a bug, it's a limitation of the concept.

The docstring states that https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Image.remap_palette

[1,0] would swap a two item palette, and list(range(256)) is the identity transform.

To break it down,
[1,0] - pixels with a palette index of 1 should become 0, and pixels with a palette index of 0 should become 1.
list(range(256)) - pixels with a palette index of 0 should become 0, ..., pixels with a palette index of 255 should become 255.

As for
[1, 1] - pixels with a palette index of 1 should become 0, and... pixels with a palette index of 1 should become 1. So pixels with a palette index of 1 will become 1, and behaviour of other pixels is undefined.

---

So what about the missing colors when you run your code?

import numpy as np
from pathlib import Path

palette = np.frombuffer(Path("palette.bin").read_bytes(), np.uint8).reshape((-1, 3))
print(palette)

gives

[[  0   0   0]
 [236 238 236]
 [152 150 152]
 [  8 124   0]
 [  0   0   0]
 [236 180 176]
 [ 48  50 236]
 [  8 124   0]
 [  0   0   0]
 [212 136  32]
 [ 60  24   0]
 [  8 124   0]
 [  0   0   0]
 [236 174 212]
 [160  20 100]
 [  8 124   0]]

The bug you have found is not in remap_palette, but rather in GifImagePlugin, when you specify a palette that has duplicate entries. I've created PR #6548 to fix it.

[radarhere]

#6548 has now been merged.