Release Pillow 8.0.0 on October 15, 2020 #4764

hugovk · 2020-07-07T19:08:06Z

hugovk · 2020-07-09T11:46:44Z

Deprecation
`Image.show` command parameter	Deprecated since version 7.2.0 (June 2020). Not long enough to remove.
`Image._showxv`	Deprecated since version 7.2.0 (June 2020). Not long enough to remove.
`ImageFile.raise_ioerror`	Deprecated since version 7.2.0 (June 2020). Not long enough to remove.
`PILLOW_VERSION` constant	Deprecated since version 5.2.0 (July 2018). It was initially removed in Pillow 7.0.0, but brought back in 7.1.0 to give projects more time to upgrade. Deprecation warnings since 7.1.0 (April 2020). Not long enough to remove.
`ImageCms.CmsProfile` attributes	Deprecated since version 3.2.0 (April 2016). Deprecation warnings since 6.0.0 (April 2019). Can be removed: PR #4768.

hugovk · 2020-10-14T15:31:32Z

Will begin the release process a bit later this evening so we're ready to release later tonight or tomorrow morning.

hugovk · 2020-10-14T16:13:52Z

@cgohlke Please could we have Windows binaries for 8.0.0?

cgohlke · 2020-10-14T16:45:50Z

Here you go.

hugovk · 2020-10-14T18:27:02Z

Thanks!

Release is out!

Misc. notes:

The Travis wheel builder took ages, there are meant to be 5 parallel jobs but there were only 2 or 4 running. Other accounts I checked on https://travis-ci.org/ only had 4 parallel.

It took 1 hr 42 min 52 sec (tag). The last master, 3 days ago, took 1 hr 44 min 59 sec, and that ran twice as much (tag + latest).

I downloaded wheels from https://github.com/python-pillow/pillow-wheels/releases/tag/8.0.0 with:

# brew install fetch
mkdir /tmp/assets
fetch --repo https://github.com/python-pillow/pillow-wheels --release-asset="\.whl" --progress --tag 8.0.0 /tmp/assets

nulano · 2020-10-14T18:31:16Z

Travis has been running at almost full capacity the last few days. There was a large uptick in the queue length just before the release started: https://www.traviscistatus.com#system-metrics

hugovk · 2020-10-14T18:39:49Z

Yeah, sure that didn't help. But it's dropped quite a lot, and again only 2 jobs are running.

nulano · 2020-10-14T18:48:29Z

But it's dropped quite a lot, and again only 2 jobs are running.

True, I don't think I've seen 5 concurrent jobs on my repo at all this week.

nulano · 2020-10-14T20:05:21Z

Announce release availability via Twitter e.g. https://twitter.com/PythonPillow/status/1013789184354603010

Can you pin the new tweet (and maybe update the release checklist)? The 7.2.0 release is still pinned on Twitter.

hugovk · 2020-10-14T20:13:10Z

I can't pin via TweetDeck, @aclark4life please could you do it? ^

aclark4life · 2020-10-14T20:34:39Z

Done! Thanks all ❤️

pevogam · 2020-10-15T14:03:48Z

Hi, I see the following in our CI that uses Ubuntu Xenial:

Collecting pillow>=4.1.1 (from torchvision==0.5.0)

  Downloading https://files.pythonhosted.org/packages/74/15/93be74c3124ad183ea3f25251a66d3c7f8641065537973c276e81f703a0b/Pillow-8.0.0.tar.gz (44.6MB)

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/tmp/pip-build-q4jwbs9a/pillow/setup.py", line 42

        f"Pillow {PILLOW_VERSION} does not support Python "

                                                          ^

    SyntaxError: invalid syntax

I guess Xenial is no longer supported?

hugovk · 2020-10-15T14:07:18Z

Xenial is supported but Python 3.5 and lower are not:

https://pillow.readthedocs.io/en/latest/installation.html

What Python do you have? Are you using pip older than version 9? If you update pip to 9+, it should fetch the correct version of Pillow for you.

python3 -m pip install --upgrade pip
python3 -m pip install --upgrade Pillow

hugovk · 2020-10-15T14:20:39Z

I recommend upgrading pip to make sure you have the latest security updates (here's one), and to use the latest features of pip, notably the python_requires metadata that would prevent this problem.

Anyway, you can still use old pip by pinning, for example: python -m pip install "pillow < 8"

pevogam · 2020-10-15T14:29:14Z

I am not insisting on using the old pip and our only goal in the CI is to use as minimal and standard dependency setup as possible. My point above was that default Ubuntu Xenial and every user on it will get the aforementioned error if they run the most standard setup

apt-get -y install python3-pip
pip3 install pillow

but this is a minor problem. I guess these users are a small circle by now and hopefully won't open too many duplicate issues.

nulano · 2020-10-20T12:35:24Z

Looks like the Windows wheels do not have CBDT support enabled: #4998 (comment)

SKIPPED [2] C:\Git\Pillow\Tests\test_imagefont.py:951: freetype compiled without libpng or unsupported

hugovk · 2020-10-21T07:01:54Z

Cross-posting @nulano's #4998 (comment):

From (emphasis mine): sourceforge.net/projects/freetype/files/freetype2/2.10.4

CHANGES BETWEEN 2.10.3 and 2.10.4
I. IMPORTANT BUG FIXES

A heap buffer overflow has been found in the handling of embedded
PNG bitmaps, introduced in FreeType version 2.6.
cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-15999
If you use option FT_CONFIG_OPTION_USE_PNG you should upgrade
immediately.

Does this affect Pillow wheels? FT_CONFIG_OPTION_USE_PNG was added in #4955.

Edit: Windows wheels for 8.0.0 don't use this option, with PyPI wheel I get:
SKIPPED [2] C:\Git\Pillow\Tests\test_imagefont.py:951: freetype compiled without libpng or unsupported
Edit2: Tests on pillow-wheels run without verbose mode, but I don't see the skip message, suggesting that it is enabled there: travis-ci.org/github/python-pillow/pillow-wheels/jobs/736355165
Is this a reason to release 8.0.1?

Does this affect Pillow wheels and do we need an 8.0.1 release?

nulano · 2020-10-22T09:55:48Z

Here is the FreeType discussion about the CVE: https://savannah.nongnu.org/bugs/?59308
The report mentions that the CVE is being actively exploited in Chrome, with more details to be published on 2020-10-26.

The report has an attached test font file; I think the following snippet should trigger the exploit. Running it in a build with address sanitization enabled should report an issue, but I can't test that on Windows. Just enabling heap verification with gflags.exe has no effect in a build from 8.0.0 tag with libpng.

>>> from PIL import ImageFile
# font.ttf from the FreeType report
>>> ImageFile.truetype("font.ttf", 150).getmask("ABC")
# heap corruption happens here
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\git\pillow\src\PIL\ImageFont.py", line 572, in getmask
    ink=ink,
  File "c:\git\pillow\src\PIL\ImageFont.py", line 652, in getmask2
    text, mode, direction, features, language, anchor
OSError: broken file
# I think this exception is raised after the heap corruption

It is not necessary to pass draw.text(..., embedded_color=True) to trigger the exploit, FreeType will load the PNG even with embedded_color=False and just convert it to grayscale. FreeType being compiled with libpng should be enough to trigger it.

As I mentioned in the comment above, the Travis log suggests that Linux and macOS wheels are exploitable (the test_imagefont.py:test_cbdt test is not skipped); Windows was built without libpng. I'm not sure, but this might affect earlier versions as well.

It sounds to me like an 8.0.1 release is needed, ideally before the Chrome exploit details are made public on 2020-10-26.

wiredfool · 2020-10-22T10:12:52Z

I think we should ship a new version, and if we do one binary release, we have to do all of them.

hugovk · 2020-10-22T10:36:18Z

Okay, I'll do 8.0.1.

We've already updated FreeType to 2.10.4 in the Mac/Linux wheel builder.
python-pillow/pillow-wheels#168

It's not need for Windows wheels, but let's include #4998 for consistency.

Will also include #4992 to help with the release process.

Anything else we need (other than release notes etc.)?

wiredfool · 2020-10-22T10:47:47Z

Is there the potential that any older releases are affected, or is this something that got enabled with the color font support?

We should have a list of potentially vulnerable releases, and specifically mention the last python 2 release.

nulano · 2020-10-22T11:04:44Z

Is there the potential that any older releases are affected, or is this something that got enabled with the color font support?

Before #4955 bitmap fonts were disabled with FT_LOAD_NO_BITMAP, but it is not clear to me whether this prevents the exploit. The FreeType documentation states:

FT_LOAD_NO_BITMAP: Ignore bitmap strikes when loading. Bitmap-only fonts ignore this flag.

Other than that, it is sufficient for FreeType to be compiled with libpng enabled. I do not see any changes specific to colour font support in the git blame of config.sh which makes me think this was enabled automatically by the FreeType build based on libpng being present.

nulano · 2020-10-22T11:08:21Z

It's not need for Windows wheels, but let's include #4998 for consistency.

It would be nice to actually add libpng / CBDT font support to the Windows wheels, but it is obviously not necessary. This requires @cgohlke adding the library to the build and maybe release notes.

hugovk · 2020-10-22T12:53:11Z

Feedback welcome on release notes: #5000

hugovk · 2020-10-22T14:24:21Z

hugovk · 2020-10-22T14:49:18Z

Cherry picked these:

0045cca (Clean up dist in release-test #4992 release process)
919fdcc (Updated freetype to 2.10.4 #4998 FreeType update)
558b2e6 & ac348ea (Add release notes for 8.0.1 #5000 release notes)

GHA Window CI failed with "Overwrite D:\a\Pillow\Pillow\Tests\images\string_dimension.tiff (Yes/No/All)?", will be because of #4993 / python-pillow/pillow-depends#34, so will include that too: f886bc9.

Edit: passing now 👍

nulano · 2020-10-22T15:16:15Z

GHA Window CI failed with "Overwrite D:\a\Pillow\Pillow\Tests\images\string_dimension.tiff (Yes/No/All)?", will be because of #4993 / python-pillow/pillow-depends#34, so will include that too: f886bc9.

See #5001 to avoid this in the future.

hugovk · 2020-10-22T15:47:26Z

The Mac and Linux wheel builder is running, currently with one "parallel" job, so it's going to take a long time.

https://travis-ci.org/github/python-pillow/pillow-wheels/builds/738067859

@cgohlke Please could you make Windows binaries for Pillow 8.0.1, making sure to first update to FreeType 2.10.4 to include the security fix? Thank you!

cgohlke · 2020-10-22T15:52:22Z

Here you go.

hugovk · 2020-10-22T19:36:00Z

Thank you!

Travis CI only had 0-3 parallel jobs and took 3.5 hours instead of the usual 5 parallel jobs and half an hour.

Released!

Thanks all!

aclark4life · 2020-10-22T19:40:29Z

https://twitter.com/PythonPillow/status/1319361076463546370

Pinned!

hugovk added the Release label Jul 7, 2020

hugovk pinned this issue Jul 7, 2020

hugovk added this to the 8.0.0 milestone Jul 7, 2020

This was referenced Jul 16, 2020

8.0.0.dev0 version bump #4790

Merged

Drop support for EOL Python 3.5 #4746

Merged

Drop support for EOL Python 3.5 #4794

Merged

CairX mentioned this issue Jul 23, 2020

Drop support for Python 3.5 CairX/extract-colors-py#16

Closed

hugovk mentioned this issue Jul 31, 2020

py -3.9 -m pip install -U pillow fails with pip 20.2 #4827

Closed

jsoriano mentioned this issue Aug 3, 2020

Support Python 3.9 in developer tools elastic/beats#20384

Closed

5 tasks

hugovk self-assigned this Aug 11, 2020

CairX mentioned this issue Oct 6, 2020

Add support for Pyton 3.9 CairX/extract-colors-py#24

Closed

hugovk mentioned this issue Oct 7, 2020

Add support for Python 3.9 #4953

Closed

11 tasks

nulano mentioned this issue Oct 8, 2020

Pillow not installing on Python 3.9 #4956

Closed

boidolr mentioned this issue Oct 10, 2020

Add python 3.9 to tests boidolr/commit-hooks#19

Merged

This was referenced Oct 12, 2020

Drop support for PyPy3 < 7.2.0 #4964

Merged

Release notes for TrueType functions and PyPy support #4969

Merged

hugovk mentioned this issue Oct 13, 2020

Add #4523 ellipse-drawing algorithm changes to release notes #4976

Merged

hugovk closed this as completed Oct 20, 2020

hugovk unpinned this issue Oct 20, 2020

hugovk reopened this Oct 22, 2020

hugovk pinned this issue Oct 22, 2020

hugovk mentioned this issue Oct 22, 2020

Add release notes for 8.0.1 #5000

Merged

nulano mentioned this issue Oct 22, 2020

CI: Replace test images from pillow-depends #5001

Merged

aclark4life unpinned this issue Oct 23, 2020

hugovk mentioned this issue Oct 25, 2020

Releasing: Add a 'git push' to point release, adjust MD formatting #5006

Merged

3 tasks

hugovk closed this as completed Oct 29, 2020

nulano mentioned this issue Nov 17, 2020

Missing 8.0.1 .whl for pypy36 on Windows #5043

Closed

hugovk mentioned this issue Dec 10, 2020

Document when deprecations will be removed #5085

Merged

RohanJoshi28 mentioned this issue Nov 4, 2021

OSError: Broken File related to fonts (ImageFile.truetype) #5810

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release Pillow 8.0.0 on October 15, 2020 #4764

Release Pillow 8.0.0 on October 15, 2020 #4764

hugovk commented Jul 7, 2020 •

edited

Loading

hugovk commented Jul 9, 2020 •

edited

Loading

hugovk commented Oct 14, 2020

hugovk commented Oct 14, 2020

cgohlke commented Oct 14, 2020 •

edited by hugovk

Loading

hugovk commented Oct 14, 2020 •

edited

Loading

nulano commented Oct 14, 2020

hugovk commented Oct 14, 2020

nulano commented Oct 14, 2020

nulano commented Oct 14, 2020

hugovk commented Oct 14, 2020

aclark4life commented Oct 14, 2020

pevogam commented Oct 15, 2020

hugovk commented Oct 15, 2020 •

edited

Loading

hugovk commented Oct 15, 2020 •

edited

Loading

pevogam commented Oct 15, 2020

nulano commented Oct 20, 2020

hugovk commented Oct 21, 2020

nulano commented Oct 22, 2020

wiredfool commented Oct 22, 2020

hugovk commented Oct 22, 2020

wiredfool commented Oct 22, 2020

nulano commented Oct 22, 2020

nulano commented Oct 22, 2020

hugovk commented Oct 22, 2020

hugovk commented Oct 22, 2020 •

edited

Loading

hugovk commented Oct 22, 2020 •

edited

Loading

nulano commented Oct 22, 2020

hugovk commented Oct 22, 2020

cgohlke commented Oct 22, 2020 •

edited by hugovk

Loading

hugovk commented Oct 22, 2020

aclark4life commented Oct 22, 2020

Release Pillow 8.0.0 on October 15, 2020 #4764

Release Pillow 8.0.0 on October 15, 2020 #4764

Comments

hugovk commented Jul 7, 2020 • edited Loading

Release Checklist

Main Release

Binary Distributions

Windows

Mac and Linux

Publicize Release

Documentation

Docker Images

hugovk commented Jul 9, 2020 • edited Loading

hugovk commented Oct 14, 2020

hugovk commented Oct 14, 2020

cgohlke commented Oct 14, 2020 • edited by hugovk Loading

hugovk commented Oct 14, 2020 • edited Loading

nulano commented Oct 14, 2020

hugovk commented Oct 14, 2020

nulano commented Oct 14, 2020

nulano commented Oct 14, 2020

hugovk commented Oct 14, 2020

aclark4life commented Oct 14, 2020

pevogam commented Oct 15, 2020

hugovk commented Oct 15, 2020 • edited Loading

hugovk commented Oct 15, 2020 • edited Loading

pevogam commented Oct 15, 2020

nulano commented Oct 20, 2020

hugovk commented Oct 21, 2020

nulano commented Oct 22, 2020

wiredfool commented Oct 22, 2020

hugovk commented Oct 22, 2020

wiredfool commented Oct 22, 2020

nulano commented Oct 22, 2020

nulano commented Oct 22, 2020

hugovk commented Oct 22, 2020

hugovk commented Oct 22, 2020 • edited Loading

Release Checklist

Point Release

Binary Distributions

Windows

Mac and Linux

Publicize Release

Documentation

Docker Images

hugovk commented Oct 22, 2020 • edited Loading

nulano commented Oct 22, 2020

hugovk commented Oct 22, 2020

cgohlke commented Oct 22, 2020 • edited by hugovk Loading

hugovk commented Oct 22, 2020

aclark4life commented Oct 22, 2020

hugovk commented Jul 7, 2020 •

edited

Loading

hugovk commented Jul 9, 2020 •

edited

Loading

cgohlke commented Oct 14, 2020 •

edited by hugovk

Loading

hugovk commented Oct 14, 2020 •

edited

Loading

hugovk commented Oct 15, 2020 •

edited

Loading

hugovk commented Oct 15, 2020 •

edited

Loading

hugovk commented Oct 22, 2020 •

edited

Loading

hugovk commented Oct 22, 2020 •

edited

Loading

cgohlke commented Oct 22, 2020 •

edited by hugovk

Loading