Foreground text colours do not work if the font is Source Code Pro after update to latest preview build

[davidchisnall]

Windows Terminal version

1.16.2524.0

Windows build number

10.0.22621.0

Other Software

No response

Steps to reproduce

1. Launch Window Terminal Preview
2. Open a tab with any shell (PowerShell, WSL, cmd.exe)
3. Run a command that should output coloured text

Expected Behavior

See coloured text.

Actual Behavior

White text on a black background.

My colour scheme is the default ('Campbell').

It appears as if background colours work, but foreground do not. Running the following test:

python3 -c "print('\t'.join(f'\u001b[1;38;5;{s}m{s.ljust(4)}' + ('\n' if not int(s) % 8 else '') for s in (str(i) for i in range(256))) + '\u001b[0m')"

In the release version of Windows Terminal gives this output:

In the preview version gives this:

Background colours work fine though. Running this test gives this output:

[237dmitry]

Tested 256 colors palette and it works

[davidchisnall]

Weird. I guess there must be something in my config that's different from yours, but since I've had to switch to the non-preview version until this is resolved I've copied my config over to that and colours are working fine for me there. These are the possibly-relevant bits of my config (defaultProfile and profiles elided):

{
    "$help": "https://aka.ms/terminal-documentation",
    "$schema": "https://aka.ms/terminal-profiles-schema",
    "actions": 
    [
        {
            "command": "find",
            "keys": "ctrl+shift+f"
        },
        {
            "command": 
            {
                "action": "nextTab"
            },
            "keys": "shift+right"
        },
        {
            "command": 
            {
                "action": "splitPane",
                "split": "auto",
                "splitMode": "duplicate"
            },
            "keys": "alt+shift+d"
        },
        {
            "command": 
            {
                "action": "closeTab"
            },
            "keys": "ctrl+w"
        },
        {
            "command": 
            {
                "action": "prevTab"
            },
            "keys": "shift+left"
        },
        {
            "command": 
            {
                "action": "newTab",
                "commandline": "wt.exe"
            },
            "keys": "ctrl+shift+n"
        }
    ],
    "copyFormatting": "all",
    "copyOnSelect": true,
    "schemes": 
    [
        {
            "background": "#0C0C0C",
            "black": "#0C0C0C",
            "blue": "#003CED",
            "brightBlack": "#767676",
            "brightBlue": "#3B78FF",
            "brightCyan": "#61D6D6",
            "brightGreen": "#16C60C",
            "brightPurple": "#ED00D0",
            "brightRed": "#F24C5A",
            "brightWhite": "#F2F2F2",
            "brightYellow": "#F9F1A5",
            "cursorColor": "#FFFFFF",
            "cyan": "#3A96DD",
            "foreground": "#FFFFFF",
            "green": "#13A10E",
            "name": "Campbell",
            "purple": "#AD1DC2",
            "red": "#E60523",
            "selectionBackground": "#FFFFFF",
            "white": "#CCCCCC",
            "yellow": "#C19C00"
        },
        {
            "background": "#012456",
            "black": "#0C0C0C",
            "blue": "#0037DA",
            "brightBlack": "#767676",
            "brightBlue": "#3B78FF",
            "brightCyan": "#61D6D6",
            "brightGreen": "#16C60C",
            "brightPurple": "#B4009E",
            "brightRed": "#E74856",
            "brightWhite": "#F2F2F2",
            "brightYellow": "#F9F1A5",
            "cursorColor": "#FFFFFF",
            "cyan": "#3A96DD",
            "foreground": "#CCCCCC",
            "green": "#13A10E",
            "name": "Campbell Powershell",
            "purple": "#881798",
            "red": "#C50F1F",
            "selectionBackground": "#FFFFFF",
            "white": "#CCCCCC",
            "yellow": "#C19C00"
        },
        {
            "background": "#282C34",
            "black": "#282C34",
            "blue": "#61AFEF",
            "brightBlack": "#5A6374",
            "brightBlue": "#61AFEF",
            "brightCyan": "#56B6C2",
            "brightGreen": "#98C379",
            "brightPurple": "#C678DD",
            "brightRed": "#E06C75",
            "brightWhite": "#DCDFE4",
            "brightYellow": "#E5C07B",
            "cursorColor": "#FFFFFF",
            "cyan": "#56B6C2",
            "foreground": "#DCDFE4",
            "green": "#98C379",
            "name": "One Half Dark",
            "purple": "#C678DD",
            "red": "#E06C75",
            "selectionBackground": "#FFFFFF",
            "white": "#DCDFE4",
            "yellow": "#E5C07B"
        },
        {
            "background": "#FAFAFA",
            "black": "#383A42",
            "blue": "#0184BC",
            "brightBlack": "#4F525D",
            "brightBlue": "#61AFEF",
            "brightCyan": "#56B5C1",
            "brightGreen": "#98C379",
            "brightPurple": "#C577DD",
            "brightRed": "#DF6C75",
            "brightWhite": "#FFFFFF",
            "brightYellow": "#E4C07A",
            "cursorColor": "#4F525D",
            "cyan": "#0997B3",
            "foreground": "#383A42",
            "green": "#50A14F",
            "name": "One Half Light",
            "purple": "#A626A4",
            "red": "#E45649",
            "selectionBackground": "#FFFFFF",
            "white": "#FAFAFA",
            "yellow": "#C18301"
        },
        {
            "background": "#002B36",
            "black": "#002B36",
            "blue": "#268BD2",
            "brightBlack": "#073642",
            "brightBlue": "#839496",
            "brightCyan": "#93A1A1",
            "brightGreen": "#586E75",
            "brightPurple": "#6C71C4",
            "brightRed": "#CB4B16",
            "brightWhite": "#FDF6E3",
            "brightYellow": "#657B83",
            "cursorColor": "#FFFFFF",
            "cyan": "#2AA198",
            "foreground": "#839496",
            "green": "#859900",
            "name": "Solarized Dark",
            "purple": "#D33682",
            "red": "#DC322F",
            "selectionBackground": "#FFFFFF",
            "white": "#EEE8D5",
            "yellow": "#B58900"
        },
        {
            "background": "#FDF6E3",
            "black": "#002B36",
            "blue": "#268BD2",
            "brightBlack": "#073642",
            "brightBlue": "#839496",
            "brightCyan": "#93A1A1",
            "brightGreen": "#586E75",
            "brightPurple": "#6C71C4",
            "brightRed": "#CB4B16",
            "brightWhite": "#FDF6E3",
            "brightYellow": "#657B83",
            "cursorColor": "#002B36",
            "cyan": "#2AA198",
            "foreground": "#657B83",
            "green": "#859900",
            "name": "Solarized Light",
            "purple": "#D33682",
            "red": "#DC322F",
            "selectionBackground": "#FFFFFF",
            "white": "#EEE8D5",
            "yellow": "#B58900"
        },
        {
            "background": "#000000",
            "black": "#000000",
            "blue": "#3465A4",
            "brightBlack": "#555753",
            "brightBlue": "#729FCF",
            "brightCyan": "#34E2E2",
            "brightGreen": "#8AE234",
            "brightPurple": "#AD7FA8",
            "brightRed": "#EF2929",
            "brightWhite": "#EEEEEC",
            "brightYellow": "#FCE94F",
            "cursorColor": "#FFFFFF",
            "cyan": "#06989A",
            "foreground": "#D3D7CF",
            "green": "#4E9A06",
            "name": "Tango Dark",
            "purple": "#75507B",
            "red": "#CC0000",
            "selectionBackground": "#FFFFFF",
            "white": "#D3D7CF",
            "yellow": "#C4A000"
        },
        {
            "background": "#FFFFFF",
            "black": "#000000",
            "blue": "#3465A4",
            "brightBlack": "#555753",
            "brightBlue": "#729FCF",
            "brightCyan": "#34E2E2",
            "brightGreen": "#8AE234",
            "brightPurple": "#AD7FA8",
            "brightRed": "#EF2929",
            "brightWhite": "#EEEEEC",
            "brightYellow": "#FCE94F",
            "cursorColor": "#000000",
            "cyan": "#06989A",
            "foreground": "#555753",
            "green": "#4E9A06",
            "name": "Tango Light",
            "purple": "#75507B",
            "red": "#CC0000",
            "selectionBackground": "#FFFFFF",
            "white": "#D3D7CF",
            "yellow": "#C4A000"
        },
        {
            "background": "#000000",
            "black": "#000000",
            "blue": "#000080",
            "brightBlack": "#808080",
            "brightBlue": "#0000FF",
            "brightCyan": "#00FFFF",
            "brightGreen": "#00FF00",
            "brightPurple": "#FF00FF",
            "brightRed": "#FF0000",
            "brightWhite": "#FFFFFF",
            "brightYellow": "#FFFF00",
            "cursorColor": "#FFFFFF",
            "cyan": "#008080",
            "foreground": "#C0C0C0",
            "green": "#008000",
            "name": "Vintage",
            "purple": "#800080",
            "red": "#800000",
            "selectionBackground": "#FFFFFF",
            "white": "#C0C0C0",
            "yellow": "#808000"
        }
    ],
    "tabSwitcherMode": "disabled",
    "tabWidthMode": "equal",
    "themes": []
}

[237dmitry]

A part of config without profiles and color schemas:

  "$help": "https://aka.ms/terminal-documentation",
  "$schema": "https://aka.ms/terminal-profiles-schema",
  "actions": [
    {
      "command": {
        "action": "copy",
        "singleLine": false
      },
      "keys": "ctrl+c"
    },
    {
      "command": {
        "action": "moveTab",
        "direction": "forward"
      },
      "keys": "ctrl+f"
    },
    {
      "command": "find",
      "keys": "ctrl+shift+f"
    },
    {
      "command": {
        "action": "splitPane",
        "split": "auto",
        "splitMode": "duplicate"
      },
      "keys": "alt+shift+d"
    },
    {
      "command": {
        "action": "openSettings",
        "target": "settingsUI"
      },
      "keys": "ctrl+shift+comma"
    },
    {
      "command": {
        "action": "moveTab",
        "direction": "backward"
      },
      "keys": "ctrl+b"
    },
    {
      "command": "toggleFocusMode",
      "keys": "alt+t"
    }
  ],
  "alwaysOnTop": false,
  "alwaysShowTabs": false,
  "centerOnLaunch": true,
  "copyFormatting": "none",
  "copyOnSelect": false,
  "defaultProfile": "{574e775e-4f2a-5b96-ac1e-a2962a402336}",
  "experimental.input.forceVT": false,
  "experimental.rendering.forceFullRepaint": false,
  "experimental.rendering.software": false,
  "experimental.useAtlasEngine": false,
  "experimental.useBackgroundImageForWindow": false,
  "focusFollowMouse": true,
  "initialCols": 140,
  "initialRows": 38,
  "multiLinePasteWarning": false,
  "profiles": {
    "defaults": {
      "closeOnExit": "always",
      "font": {
        "face": "FiraCode Nerd Font",
        "size": 13,
        "weight": "normal"
      },
      "intenseTextStyle": "all",
      "opacity": 100,
      "padding": "6",
      "scrollbarState": "hidden",
      "useAcrylic": false
    },
   "list": [
    ......
  ],
  "showTabsInTitlebar": true,
  "snapToGridOnResize": true,
  "tabSwitcherMode": "inOrder",
  "theme": "system",
  "trimBlockSelection": true,
  "useAcrylicInTabRow": true
}

[zadjii-msft]

These are the possibly-relevant bits of my config (defaultProfile and profiles elided):

Actually, the profile settings probably are relevant here, especially for the atlas engine setting. Dang I couldn't repro this. I bet it's an atlas bug, but we need to figure out the whole repro here.

[davidchisnall]

I'm afraid I don't know what an atlas engine is. I have this for the default:

        "defaults": 
        {
            "antialiasingMode": "cleartype",
            "colorScheme": "Campbell",
            "font": 
            {
                "face": "Source Code Pro",
                "size": 16,
                "weight": "medium"
            },
            "historySize": 100000
        },

None of my other profiles have anything that looks relevant. Some override the colorScheme but specify "Campbell" (which is also the default).

[davidchisnall]

Deleting profiles.defaults.font.face makes colours work again. I don't know if it's specific to Source Code Pro, or if it's a problem with any non-default font.

[davidchisnall]

Changing to any other font appears to make colours work again. I have no idea what makes Source Code Pro special, but since it's available under the SIL Open Font License, hopefully you can reproduce it.

[DHowett]

Wow. Now @lhecker is going to "love" this one!

Thanks @davidchisnall. If you pull up the Rendering settings and disable the "new text rendering engine," do colors once again work? (You'll need to open a new tab, we can't hot-reload that one yet...)

[lhecker]

Hmm... 🤔

@davidchisnall The only way that a font might cause the issue you're seeing is if it's a color font, but the official Source Code Pro font shouldn't be one. Are you perhaps using a special variant? Mine is from here and is:

> Get-FileHash -Algorithm SHA256 *.ttf

Algorithm  Hash                                                              Path
---------  ----                                                              ----
SHA256     5A32FAFB68B4F7E1D05844943E3AEED2429F927DCD232D46BDCAAFE2CD2C6F81  SourceCodePro-Italic-VariableFont_wght.ttf
SHA256     CF5851D657AFBE3DB00539A8AD73467784723CAC03B189072316D6377DD703F7  SourceCodePro-VariableFont_wght.ttf

[davidchisnall]

@DHowett, yup disabling the new renderer fixes things.

@lhecker, I installed the SourceCodePro fonts from their GitHub release page: https://github.com/adobe-fonts/source-code-pro/releases. However, looking at the releases page, I see that there was a release in 2020 with 'Removes SVG table to allow for color-coded text in text editing apps. (issue adobe-fonts/source-code-pro#250)' in the release notes.

Sure enough, upgrading to a newer version of the font fixes the problem. I think we can probably close this bug, since it appears to be a bug in the font, rather than the terminal.

That said, Source Code Pro is a fairly popular font for terminals and I suspect that I am not alone in assuming that fonts are not in the category of things that I need to upgrade to pick up bug fixes (I think this might be the first time that I've ever encountered a bug in a font other than in ligature rendering, which is a best-effort totally-fine-to-get-a-bit-wrong thing for most uses). It would be great if the terminal could detect if your font has an SVG table and, the first time you launch a terminal with such a font, pop up a dialog / notification / something saying 'Your chosen font has a colour table. If this table is incorrect then colours will be rendered incorrectly. If you can no longer see coloured text in the terminal, check to see if a newer version of your font is available'. I don't know anything about Atlas, but if it's possible for it to expose an option to ignore the font's colours then that would also be nice (some folks may not have admin rights necessary to upgrade their font).

[lhecker]

Sure enough, upgrading to a newer version of the font fixes the problem. I think we can probably close this bug, since it appears to be a bug in the font, rather than the terminal.

It is a bug in the terminal, but I do believe we can close this issue, because...

It would be great if the terminal could detect if your font has an SVG table and, the first time you launch a terminal with such a font, pop up a dialog / notification / something saying 'Your chosen font has a colour table.

The problem is that the new text renderer assumes a font is an Emoji font if it contains any colored glyphs. This doesn't just lead to your issue, but rather causes a series of problems. That's why I'd prefer a complete solution that addresses the issue fundamentally, by correctly detecting colored glyphs on a glyph-by-glyph basis.

In fact I had considered this to be an issue ever since I wrote the code last year. 😅

terminal/src/renderer/atlas/AtlasEngine.cpp

Lines 1506 to 1530 in 71fc4b1

	// Do fonts exist *in practice* which contain both colored and uncolored glyphs? I'm pretty sure...
	// However doing it properly means using either of:
	// * IDWriteFactory2::TranslateColorGlyphRun
	// * IDWriteFactory4::TranslateColorGlyphRun
	// * IDWriteFontFace4::GetGlyphImageData
	//
	// For the first two I wonder how one is supposed to restitch the 27 required parameters from a
	// partial (!) text range returned by GetGlyphs(). Our caller breaks the GetGlyphs() result up
	// into runs of characters up until the first canBreakShapingAfter == true after all.
	// There's no documentation for it and I'm sure I'd mess it up.
	// For very obvious reasons I didn't want to write this code.
	//
	// IDWriteFontFace4::GetGlyphImageData seems like the best approach and DirectWrite uses the
	// same code that GetGlyphImageData uses to implement TranslateColorGlyphRun (well partially).
	// However, it's a heavy operation and parses the font file on disk on every call (TranslateColorGlyphRun doesn't).
	// For less obvious reasons I didn't want to write this code either.
	//
	// So this is a job for future me/someone.
	// Bonus points for doing it without impacting performance.
	auto flags = CellFlags::None;
	if (fontFace)
	{
	const auto fontFace2 = wil::try_com_query<IDWriteFontFace2>(fontFace);
	WI_SetFlagIf(flags, CellFlags::ColoredGlyph, fontFace2 && fontFace2->IsColorFont());
	}

[airosB]

Same issue.
Set "experimental.rendering.software": true makes colors work again, with Source Code Pro.

"new text rendering engine" option makes no difference on my environment.

[lhecker]

@airosB I'm working on fixing this bug. But in either case I can only suggest downloading and installing the latest version of Source Code Pro from here if you haven't done so yet: https://github.com/adobe-fonts/source-code-pro/releases
In theory you should be able to remove "experimental.rendering.software": true then.

[airosB]

@lhecker Thanks. The latest version of font works perfectly with colored texts.

[lhecker]

I'm removing Severity-Blocking and the milestone because there's a workaround for this particular font at least that is somewhat easy to do and I'll unfortunately need longer than 1.17 to implement a fix. I think 1.18 is most likely.