Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PillowPatch.py #1660

Closed
v-python opened this issue Jan 12, 2016 · 22 comments
Closed

PillowPatch.py #1660

v-python opened this issue Jan 12, 2016 · 22 comments
Labels
Needs Review Packaging Any mention of setup.py; some overlap with Install label

Comments

@v-python
Copy link

The first section of the code below will patch old versions of Pillow (3.0.0 at least) with the newer ImageDraw.text .textsize .multiline_text and .multiline_textsize from the repository. The second section of the code below patches in a new trio of APIs: ImageFont.getBB to get a bounding box for a single line of text, ImageDraw.textInfo to return a bounding box and analysis results for single or multiline text, and ImageDraw.textAtPos to actually draw single or multiline text at a specific position, or at various positions relative to the image on which you are drawing.

I have no experience with anything except Issues on GitHub. I barely know what a Pull request is. I'd be happy to go through any specific incantations that are required if someone tells me how. Meantime, here's the code, which I store in a file named PillowPatch.py (for now), and import after Pillow, so that I can use this functionality.

Markdown compatible version of the code in the next comment.

@v-python
Copy link
Author

Markdown interprets Python code in interesting manners.

from PIL import Image, ImageDraw, ImageFont

def text(self, xy, text, fill=None, font=None, anchor=None):
    if self._multiline_check(text):
        return self.multiline_text(xy, text, fill, font, anchor)

    ink, fill = self._getink(fill)
    if font is None:
        font = self.getfont()
    if ink is None:
        ink = fill
    if ink is not None:
        try:
            mask, offset = font.getmask2(text, self.fontmode)
            xy = xy[0] + offset[0], xy[1] + offset[1]
        except AttributeError:
            try:
                mask = font.getmask(text, self.fontmode)
            except TypeError:
                mask = font.getmask(text)
        self.draw.draw_bitmap(xy, mask, ink)

def multiline_text(self, xy, text, fill=None, font=None, anchor=None,
                   spacing=4, align="left"):
    widths = []
    max_width = 0
    lines = self._multiline_split(text)
    line_spacing = self.textsize('A', font=font)[1] + spacing
    for line in lines:
        line_width, line_height = self.textsize(line, font)
        widths.append(line_width)
        max_width = max(max_width, line_width)
    left, top = xy
    for idx, line in enumerate(lines):
        if align == "left":
            pass  # left = x
        elif align == "center":
            left += (max_width - widths[idx]) / 2.0
        elif align == "right":
            left += (max_width - widths[idx])
        else:
            assert False, 'align must be "left", "center" or "right"'
        self.text((left, top), line, fill, font, anchor)
        top += line_spacing
        left = xy[0]

##
# Get the size of a given string, in pixels.

def textsize(self, text, font=None):
    if self._multiline_check(text):
        return self.multiline_textsize(text, font)

    if font is None:
        font = self.getfont()
    return font.getsize(text)

def multiline_textsize(self, text, font=None, spacing=4):
    max_width = 0
    lines = self._multiline_split(text)
    line_spacing = self.textsize('A', font=font)[1] + spacing
    for line in lines:
        line_width, line_height = self.textsize(line, font)
        max_width = max(max_width, line_width)
    return max_width, len(lines)*line_spacing

ImageDraw.ImageDraw.text = text
ImageDraw.ImageDraw.textsize = textsize
ImageDraw.ImageDraw.multiline_text = multiline_text
ImageDraw.ImageDraw.multiline_textsize = multiline_textsize

def getBB( self, text ):
    size, offset = self.font.getsize( text )
    ascent, descent = self.getmetrics()
    yMax = ascent - offset[ 1 ] # distance from baseline to max horiBearingY
    yMin = yMax - size[ 1 ]
    xMin = offset[ 0 ]
    xMax = size[ 0 ] + xMin
    return ( xMin, yMin, xMax, yMax )

# Note that font metrics assume an origin on a baseline. So xMin is negative
# pixels to the left of the origin, xMax is positive pixels to the right of the
# origin, yMin is negative pixels below the baseline, yMax is positive pixels
# above the baseline.
# if text contains "\n" it is treated as multiple lines of text.
#    yMin becomes really negative.
#    lineHeight and lineHeightPercent become relevant.
#    lineHeightPercent is only used if LineHeight is None, and defaults to 100%.
#    Use of a lineHeight < sum( font.getmetrics ) may result in text overlap.
def textInfo( self, text, font=None, lineHeight=None, lineHeightPercent=None ):
    if font is None:
        font = self.getfont()
    lines = text.split('\n')
    txMax = 0
    txMin = 0
    txWid = 0
    tyMax = None
    tyMin = None
    if len( lines ) > 1:
        if lineHeight is None:
            if lineHeightPercent is None:
                lineHeightPercent = 100
            lineHeight = int( sum( font.getmetrics())
                              * lineHeightPercent / 100 )
    lineBBs = []
    for line in lines:
        lxMin, lyMin, lxMax, lyMax = font.getBB( line )
        lineBBs.append(( lxMin, lyMin, lxMax, lyMax, line ))
        lxWid = lxMax - lxMin
        if txWid < lxWid:
            txWid = lxWid
        if txMax < lxMax:
            txMax = lxMax
        if txMin > lxMin:
            txMin = lxMin
        if tyMax is None:
            tyMax = lyMax # from first line only
        if tyMin is None:
            tyMin = 0 # skips first line (unless it is also last)
        else:
            tyMin -= lineHeight
    tyMin += lyMin # from last line
    return ( txMin, tyMin, txMax, tyMax, txWid, lineHeight, lineBBs )

# Text is drawn as close as possible to the specified alignment edges of the
# image, without truncation on those edges, then adjusted by the origin value.
# alignX or alignY of 'exact' means to use the specified origin point exactly
# in that direction. Otherwise origin is used as an offset from the calculated
# alignment position. alignX can also be 'left', 'center', or 'right'; alignY
# can also be 'top', 'middle', or 'bottom'. justifyX can be 'left', 'center',
# or 'right'. Other parameters like textInfo.
def textAtPos( self, text, font=None, lineHeight=None, lineHeightPercent=None,
               origin=( 0, 0 ), alignX='exact', alignY='exact', justifyX='left',
               fill=None ):
    if font is None:
        font = self.getfont()
    ink, fill = self._getink( fill )
    if ink is None:
        inx = fill
        if ink is None:
            return
    if justifyX not in ('left', 'center', 'right'):
        raise ValueError('Unknown justifyX value "%s".' % justifyX )
    txMin, tyMin, txMax, tyMax, txWid, lineHeight, lineBBs = self.textInfo(
        text, font, lineHeight, lineHeightPercent )
    if alignX == 'exact':
        ox = 0
    elif alignX == 'left':
        ox = -txMin
    elif alignX == 'right':
        ox = self.im.size[ 0 ] - txMax
    elif alignX == 'center':
        ox = self.im.size[ 0 ] // 2 - txMax // 2
    else:
        raise ValueError('Unknown alignX value "%s".' % alignX )
    if alignY == 'exact':
        oy = 0
    elif alignY == 'top':
        oy = tyMax
    elif alignY == 'bottom':
        oy = self.im.size[ 1 ] + tyMin
    elif alignY == 'middle':
        oy = self.im.size[ 1 ] // 2 + ( tyMax + tyMin ) // 2
    else:
        raise ValueError('Unknown alignY value "%s".' % alignY )
    ox += origin[ 0 ]
    oy += origin[ 1 ]
    ascent, descent = font.getmetrics()
    while lineBBs:
        lxMin, lyMin, lxMax, lyMax, line = lineBBs.pop( 0 )
        if justifyX == 'left':
            lox = ox
        elif justifyX == 'right':
            lox = ox + txMax - lxMax
        else:
            lox = ox + txMax // 2 - lxMax // 2

        # finally, draw some text
        lox = lox + lxMin
        loy = oy - lyMax
        im = Image.core.fill("L", ( lxMax - lxMin, lyMax - lyMin ), 0 )
        font.font.render( line, im.id, self.fontmode == "1" )
        self.draw.draw_bitmap(( lox, loy ), im, ink )

        if not lineBBs:
            break
        oy += lineHeight

ImageFont.FreeTypeFont.getBB = getBB
ImageDraw.ImageDraw.textInfo = textInfo
ImageDraw.ImageDraw.textAtPos = textAtPos

@QasimK
Copy link

QasimK commented Jan 14, 2016

I think you've done a lot of great work looking into issue #1646 @v-python. I was experiencing issues with rendering text, particularly with "stylish" fonts. I want to understand your new APIs because I think they might solve the problems that I am having.

Essentially the crux would be, does textAtPos render text like web browser would (which I believe is correct)? Such as dealing with line-height properly, and correctly determining the height of a line when it comes to rendering?

Is the correct way to get the height of a line using getmetrics()? Actually it isn't clear what this height should be - baseline to cap height, or descender to ascender?

In order to create a pull request:

  1. Fork the repository.
  2. Clone the new repository under your github account to your machine, edit the code (new branch?), commit and push to your repository on github.
  3. Create a pull request.

Feel free to ask me for any more specific question.

@v-python
Copy link
Author

@QasimK Thanks for the kind words. I had some false starts and misunderstandings due to limited documentation for some parts of Pillow and FontType, but I think I've worked through them all. At this point, and I think I do understand the processes, parameters, etc., and have solved the problems I was having, some of which were also in #1646. My comments there ranged from misconceived, incorrect, to partially correct, and finally correct (I believe), so it is good to have discussion here which eliminates the misconceptions.

textAtPos has 4 positioning modes in each dimension and 3 horizontal justification modes, but the block of text it renders, which must be all of one size and font, uses the same line height algorithms in all those modes.

A web browser is prepared to deal with interlinear font, font-style, and line-height changes embedded in the markup at any point in a block of text. Neither does a web browser expose pixel-level controls for positioning the block of text. However, within a block of text without such changes, I believe textAtPos can be made to produce text that would very nearly match that of a web browser, by giving both the equivalent parameters for font, fontsize, and lineHeight.

The proper lineheight of a horizontal font, according to its metrics, is yAscender - yDescender + yLineGap. The former two are available in Pillow as font.getmetrics(), except that yDescender is negated, and both are scaled to pixels for the selected pixelsize of the font, rather than being in font units, as inside the font file.

sum( font.getmetrics()) is as close as can be had in Pillow 3.0.0 to proper line height. The font metrics actually include another parameter, yLineGap, which is zero in several fonts I examined with FontTools, which gets included by FontType in its "height" metric, which is mentioned? exposed? in #1540. That wasn't yet available to me in Pillow 3.0, so I used sum( font.getmetrics()), which is numerically the same for fonts which have have yLineGap set to zero. So "correctly determining the height of a line" should use "height", or "sum( font.getmetrics()) + yLineGap", but neither of these were available to me in Pillow 3.0.0, so I approximated it by using sum( font.getmetrics()). However, an external program that can somehow obtain yLineGap from other sources (such as FontTools, or versions of Pillow that include #1540), could pass in to TextAtPos the proper lineheight value, and it would properly work with that. Merging my patch with a Pillow that includes #1540, I would recommend that its font.height would be used instead of summing font.getmetrics(). It should be more accurate in scaling, due to performing the summation in fontunits before the scaling, and would include the yLineGap. As #1540 shows, the sum after scaling is not always the same as the sum before scaling, due to rounding.

@v-python
Copy link
Author

An updated PillowPatch.py that uses font.height in Pillow 3.1.0 and greater (well, any version that implements font.height). With a fallback to using sum( font.getmetrics()) if font.height raises.

from PIL import Image, ImageDraw, ImageFont

def text(self, xy, text, fill=None, font=None, anchor=None):
    if self._multiline_check(text):
        return self.multiline_text(xy, text, fill, font, anchor)

    ink, fill = self._getink(fill)
    if font is None:
        font = self.getfont()
    if ink is None:
        ink = fill
    if ink is not None:
        try:
            mask, offset = font.getmask2(text, self.fontmode)
            xy = xy[0] + offset[0], xy[1] + offset[1]
        except AttributeError:
            try:
                mask = font.getmask(text, self.fontmode)
            except TypeError:
                mask = font.getmask(text)
        self.draw.draw_bitmap(xy, mask, ink)

def multiline_text(self, xy, text, fill=None, font=None, anchor=None,
                   spacing=4, align="left"):
    widths = []
    max_width = 0
    lines = self._multiline_split(text)
    line_spacing = self.textsize('A', font=font)[1] + spacing
    for line in lines:
        line_width, line_height = self.textsize(line, font)
        widths.append(line_width)
        max_width = max(max_width, line_width)
    left, top = xy
    for idx, line in enumerate(lines):
        if align == "left":
            pass  # left = x
        elif align == "center":
            left += (max_width - widths[idx]) / 2.0
        elif align == "right":
            left += (max_width - widths[idx])
        else:
            assert False, 'align must be "left", "center" or "right"'
        self.text((left, top), line, fill, font, anchor)
        top += line_spacing
        left = xy[0]

##
# Get the size of a given string, in pixels.

def textsize(self, text, font=None):
    if self._multiline_check(text):
        return self.multiline_textsize(text, font)

    if font is None:
        font = self.getfont()
    return font.getsize(text)

def multiline_textsize(self, text, font=None, spacing=4):
    max_width = 0
    lines = self._multiline_split(text)
    line_spacing = self.textsize('A', font=font)[1] + spacing
    for line in lines:
        line_width, line_height = self.textsize(line, font)
        max_width = max(max_width, line_width)
    return max_width, len(lines)*line_spacing

ImageDraw.ImageDraw.text = text
ImageDraw.ImageDraw.textsize = textsize
ImageDraw.ImageDraw.multiline_text = multiline_text
ImageDraw.ImageDraw.multiline_textsize = multiline_textsize

def getBB( self, text ):
    size, offset = self.font.getsize( text )
    ascent, descent = self.getmetrics()
    yMax = ascent - offset[ 1 ] # distance from baseline to max horiBearingY
    yMin = yMax - size[ 1 ]
    xMin = offset[ 0 ]
    xMax = size[ 0 ] + xMin
    return ( xMin, yMin, xMax, yMax )

# Note that font metrics assume an origin on a baseline. So xMin is negative
# pixels to the left of the origin, xMax is positive pixels to the right of the
# origin, yMin is negative pixels below the baseline, yMax is positive pixels
# above the baseline.
# if text contains "\n" it is treated as multiple lines of text.
#    yMin becomes really negative.
#    lineHeight and lineHeightPercent become relevant.
#    lineHeightPercent is only used if LineHeight is None, and defaults to 100%.
#    Use of a lineHeight < sum( font.getmetrics ) may result in text overlap.
#    The best lineHeight would be that returned from the font.font.height
#    attribute, but in versions of PIL in which that isn't accessible,
#    sum( font.getmetrics()) is used instead.
def textInfo( self, text, font=None, lineHeight=None, lineHeightPercent=None ):
    if font is None:
        font = self.getfont()
    lines = text.split('\n')
    txMax = 0
    txMin = 0
    txWid = 0
    tyMax = None
    tyMin = None
    if len( lines ) > 1:
        if lineHeight is None:
            if lineHeightPercent is None:
                lineHeightPercent = 100
            try:
                lineHeight = font.font.height
            except Exception as exc:
                lineHeight = sum( font.getmetrics())
            lineHeight = int( lineHeight * lineHeightPercent / 100 )
    lineBBs = []
    for line in lines:
        lxMin, lyMin, lxMax, lyMax = font.getBB( line )
        lineBBs.append(( lxMin, lyMin, lxMax, lyMax, line ))
        lxWid = lxMax - lxMin
        if txWid < lxWid:
            txWid = lxWid
        if txMax < lxMax:
            txMax = lxMax
        if txMin > lxMin:
            txMin = lxMin
        if tyMax is None:
            tyMax = lyMax # from first line only
        if tyMin is None:
            tyMin = 0 # skips first line (unless it is also last)
        else:
            tyMin -= lineHeight
    tyMin += lyMin # from last line
    return ( txMin, tyMin, txMax, tyMax, txWid, lineHeight, lineBBs )

# Text is drawn as close as possible to the specified alignment edges of the
# image, without truncation on those edges, then adjusted by the origin value.
# alignX or alignY of 'exact' means to use the specified origin point exactly
# in that direction. Otherwise origin is used as an offset from the calculated
# alignment position. alignX can also be 'left', 'center', or 'right'; alignY
# can also be 'top', 'middle', or 'bottom'. justifyX can be 'left', 'center',
# or 'right'. Other parameters like textInfo.
def textAtPos( self, text, font=None, lineHeight=None, lineHeightPercent=None,
               origin=( 0, 0 ), alignX='exact', alignY='exact', justifyX='left',
               fill=None ):
    if font is None:
        font = self.getfont()
    ink, fill = self._getink( fill )
    if ink is None:
        inx = fill
        if ink is None:
            return
    if justifyX not in ('left', 'center', 'right'):
        raise ValueError('Unknown justifyX value "%s".' % justifyX )
    txMin, tyMin, txMax, tyMax, txWid, lineHeight, lineBBs = self.textInfo(
        text, font, lineHeight, lineHeightPercent )
    if alignX == 'exact':
        ox = 0
    elif alignX == 'left':
        ox = -txMin
    elif alignX == 'right':
        ox = self.im.size[ 0 ] - txMax
    elif alignX == 'center':
        ox = self.im.size[ 0 ] // 2 - txMax // 2
    else:
        raise ValueError('Unknown alignX value "%s".' % alignX )
    if alignY == 'exact':
        oy = 0
    elif alignY == 'top':
        oy = tyMax
    elif alignY == 'bottom':
        oy = self.im.size[ 1 ] + tyMin
    elif alignY == 'middle':
        oy = self.im.size[ 1 ] // 2 + ( tyMax + tyMin ) // 2
    else:
        raise ValueError('Unknown alignY value "%s".' % alignY )
    ox += origin[ 0 ]
    oy += origin[ 1 ]
    ascent, descent = font.getmetrics()
    while lineBBs:
        lxMin, lyMin, lxMax, lyMax, line = lineBBs.pop( 0 )
        if justifyX == 'left':
            lox = ox
        elif justifyX == 'right':
            lox = ox + txMax - lxMax
        else:
            lox = ox + txMax // 2 - lxMax // 2

        # finally, draw some text
        lox = lox + lxMin
        loy = oy - lyMax
        im = Image.core.fill("L", ( lxMax - lxMin, lyMax - lyMin ), 0 )
        font.font.render( line, im.id, self.fontmode == "1" )
        self.draw.draw_bitmap(( lox, loy ), im, ink )

        if not lineBBs:
            break
        oy += lineHeight

ImageFont.FreeTypeFont.getBB = getBB
ImageDraw.ImageDraw.textInfo = textInfo
ImageDraw.ImageDraw.textAtPos = textAtPos

@aclark4life
Copy link
Member

@v-python Send this in a PR?

@v-python
Copy link
Author

v-python commented Apr 2, 2016

So I'm trying to make my first PR, following the instructions @QasimK supplied above. I got as far as the middle of step 2, have edited the new code into a branch in my local clone of the forked repository.

Is there a way to convince my Python install to use the code in my branch (...GitHub\Pillow...), so I can test it? Or a way to "build" (which I think, with Python, is mostly copying files around) and then install locally? Or should I just copy files around?

Maybe I can figure that out by reading more code or documentation, but I'm sure a novice at GitHub.

@radarhere
Copy link
Member

To install Pillow from it's directory, it should just be a matter of moving to the directory on the command line and then running python setup.py install. If you just wanted to build Pillow in the directory, without installing, then that would be python setup.py build.

If you're like to run tests on TravisCI, like Pillow will do once you have created the PR, you should find it simple to sign up for http://travis-ci.org/. Once you push the new branch to GitHub, Travis will start testing it.

@v-python
Copy link
Author

v-python commented Apr 2, 2016

"jpeg is required unless explicitly disabled..." older PIL (and, presumably, dependencies) already installed. Do I need a C compiler? Or what am I missing?

Happens with build as well as install.

@radarhere
Copy link
Member

What operating system are you using?

@v-python
Copy link
Author

v-python commented Apr 2, 2016

Windows 10

@radarhere
Copy link
Member

Not my operating system, so I might not be as much help as I could have been. I don't think there is a way to easily get the development requirements like on Mac/Linux, so I think you'll have to manually install the various dependencies. For JPEG, it would be either http://www.ijg.org/ or http://www.openjpeg.org/.

@aclark4life
Copy link
Member

We may have some Windows notes somewhere and/or maybe @cgohlke can comment.

@hugovk
Copy link
Member

hugovk commented Apr 3, 2016

If you don't need to make changes to the C layer, and only the Python layer, you generally don't need a C compiler and can skip python setup.py install/python setup.py build run the tests from your dev directory.

A good idea is to enable https://ci.appveyor.com/ for your fork and it'll run the tests on a clean Windows env for your own branches and pushes (and enable Travis CI for Linux builds; see https://github.com/python-pillow/Pillow/blob/master/.github/CONTRIBUTING.md#bug-fixes-feature-additions-etc )

(Side note: we should update CONTRIBUTING.md to mention AppVeyor for Win builds.)

@wiredfool
Copy link
Member

@radarhere OpenJpeg is a Jpeg2000 support library, which is completely different than LibJpeg, which is from Ilg.

The Windows build stuff that we have is in winbuild/, but It's rough and I wouldn't want to point non-experts at it.

@micahcc
Copy link

micahcc commented May 19, 2016

@v-python thanks for the work. I've gone ahead and created a PR. I'll try to add tests etc. Actually kind of needed this for my own stuff. #1915

@v-python
Copy link
Author

Thanks for helping out. Seems that only experts can contribute to Windows. If you can integrate it and test it on Linux, that'd be great. No Linux here at the moment. I added some line comments to hugovk's review, answering some of his questions.

@PanderMusubi
Copy link

In which release will this patch be included?

@aclark4life
Copy link
Member

@PanderMusubi It could go in the next release, after someone fixes #1915
screenshot 2016-09-25 09 24 49

@scsmla
Copy link

scsmla commented Oct 4, 2017

When i run a program using the PIL i got the following errors:
"'ImageFont' object has no attribute 'getmask2'.
Please show me the way to fix it

@wiredfool
Copy link
Member

@scsmla please don't repeat questions on unrelated threads

@aclark4life aclark4life added the Packaging Any mention of setup.py; some overlap with Install label label May 11, 2019
@nulano
Copy link
Contributor

nulano commented Apr 13, 2020

#1915 was closed. Should this be closed as well?

@aclark4life
Copy link
Member

Sure, can always be re-opened if need be

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review Packaging Any mention of setup.py; some overlap with Install label
Projects
None yet
Development

No branches or pull requests

10 participants