Keras issue for loading uint16 (16 bits) images by ImageDataGenerator

[omaghsoudi]

Please make sure that this is a Bug or a Feature Request and provide all applicable information asked by the template.
If your issue is an implementation question, please ask your question on StackOverflow or on the Keras Slack channel instead of opening a GitHub issue.

System information

• Have I written custom code (as opposed to using example directory): no
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux18.04 and Centos7
• TensorFlow backend (yes / no): yes
• TensorFlow version: 1.12.0
• Keras version: 2.2.4
• Python version: 3.7
• CUDA/cuDNN version: 7.5
• GPU model and memory: V100, 16GB

You can obtain the TensorFlow version with:
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
You can obtain the Keras version with:
python -c 'import keras as k; print(k.version)'

Describe the current behavior
The 16 bits images are getting saturated as Keras reads them as 8bits! Anything above 255 will be 255!

Describe the expected behavior
16 bits images should be read properly but it does not!

Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
Save a 16 bits image and load it by ImageDataGenerator; the results will be between 0 to 255.

Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
The issue is straight forward and can be easily resolved. Please let me know if you need more info. Also, thank you for the great API. I hope you resolve the issue soon as I do not want to write the whole ImageDataGenerator just because of this issue!

Thank you!

[kiblee]

This seems like an issue for Pillow and not Keras. Keras uses Pillow to load images and Pillow currently does not support multi-channel images with more than 8 bits. However, it does support higher bit-depth single-channel images.

Please see python-pillow/Pillow#1888

[omaghsoudi]

You are correct about the Pillow issue with multi-channel images in 16 bits. However, this is an issue with Keras too as Keras fails loading images! Keras can provide the ability to load images using OpenCV or Scipy which can resolve this issue. Hopefully either
Keras considers adding the possibility to load images using other packages or Pillow resolve the issue.

[CA4GitHub]

I'm experiencing this issue with Keras too. Keras (& Pillow) should at least provide a warning.

However, Pillow is able to open my uint16 tiff. When the file is opened with Keras the returned object type is PIL.Image.Image, but when the file is opened with Pillow directly the returned object type is PIL.TiffImagePlugin.TiffImageFile.

It's like the Keras version of Pillow is not using the tiff image plugin.

I've asked a question here https://stackoverflow.com/questions/58358169/keras-cant-load-uint16-tiff-images.

Update: It seems in my case the keras_preprocessing/image/utils.py load_img method causes the problem when it calls img.convert method based on color_mode variable. Pre img.convert call, the type(img) is <class 'PIL.TiffImagePlugin.TiffImageFile'> and min,max=268,1435. Post img.convert call the type(img) is <class 'PIL.Image.Image'> and min,max=255,255.

[bersbersbers]

I agree with @CA4GitHub: this is not only a Pillow issue, as it happens also for single-channel 16-bit TIFF images; python-pillow/Pillow#1888 explicitly states that Pillow "is able to open higher bit depth images (e.g. I16, I32, or Float32 images) if they are single channel". In fact, as @CA4GitHub rightly states, PIL loads a 16-bit single-channel TIFF file into an I;16 image successfully, that is then force-converted into either L, rgba or rgb by keras-preprocessing code:
https://github.com/keras-team/keras-preprocessing/blob/0494094a3ba341a67fdb9960e326fe6b9f582708/keras_preprocessing/image/utils.py#L110-L121

And while I am typing this, I am noting keras-team/keras-preprocessing@4e412de
Nice!

If anyone wants to use this before this is released:

pip uninstall keras-preprocessing
git clone --branch 1.1.0 https://github.com/keras-team/keras-preprocessing
cd keras-preprocessing/
git cherry-pick 4e412de
pip install .   

[JoeHRIsaac]

Bumping this since I'm also having the same problem. The 16-bit images are getting truncated at 255

[bersbersbers]

I think this is fixed as of pip install "keras-preprocessing>=1.1.1" and can be closed.

[mmkhan78]

keras-preprocessing version 1.1.2 provides support for 16 bit grayscale images however, there is still no support for 16 bit r,g,b images ? The update https://github.com/keras-team/keras-preprocessing/commit/4e412de9f6dda66309f0e0f0a6ab5acd3691967f also seems to suggest this, right?

[TStein81]

Hello. I have been using keras-preprocessing 1.1.2, so in theory this shoudl be fixed, but the problem continues.

With this code :
train_datagen = ImageDataGenerator(horizontal_flip=True,
vertical_flip=True,
featurewise_center=False,
samplewise_center=False,
brightness_range=[0.8, 1.2])

it = train_datagen.flow_from_directory("/home/tiago/workspace/aquila/test_16_bit/16bit/",
target_size=(256,256),
color_mode='grayscale',
interpolation='bicubic',
#class_mode='binary',
save_to_dir="/tmp/augumented",
batch_size=2)
X = next(it)
for i in range(2):
im = X[0][0][..., 0]
print(np.max(im))
print(np.min(im))
plt.imshow(X[0][0][..., 0])
plt.show()

I get min and max values on the generated image in between 0 and 255, no matter if my original images are 8 bit or 16 bit PNG (both cases grayscale) (the second ones with values up to 4096. If it were working correctly the value range should not have been compressed down to max 255.

Can anyone confirm that has been solved, and if yes if there is something else I should be looking about to fix it?

EDIT: I kind of figured. IF the ImageDataGenerator has the parameter brightness_range set, then it reverts into the problem and all the images are again forcefully squeezed into 8 bit depth. If I leave that parameter out, then it works.