Flattening the window loses scrollback lines

[egmontkob]

Environment

Windows build number: 10.0.18362.0
Windows Terminal version (if applicable): 0.6.2951.0

Steps to reproduce

• ssh to a Linux box
• seq 100
• shrink the window vertically (fewer lines)
• scroll back

Expected behavior

See the numbers from 1 to 100

Actual behavior

Lines get permanently lost as the window is made smaller, e.g. the sequence goes like ... 70 71 72 73 78 79 80 81 ...

[zadjii-msft]

Wow this might be crazy hard to fix with conpty. I've put together a prototype branch to fix this, and it just keeps getting more and more ridiculous. Closer and closer to the expected behavior, but the edge cases might not be solvable here.

From a PR description that I typed up before realizing my fixes weren't quite right:

• Conpty was not emitting the correct thing when you'd decrease the height of the viewport. When you increase the height of the window, we'd both invalidate the entire buffer, and insert new lines to the bottom of the buffer. What this meant is that on every height-only resize, we'd re-emit the entire viewport.
  • If you resized down twice (From R to R-1 to R-2), once during the frame being emitted by conpty, the Terminal would try to process the R-1 lines emitted by conpty when there are only R-2 available rows in the terminal buffer. This would tlead to duplicated lines.
• This buggy conpty behavior was causing us to correctly believe that the terminal viewport should remain "stuck" to the top position. That was what was causing us to apparently erase lines from the scrollback as we'd resize down. So this also changes the terminal to stick to the bottom when the viewport height changes.

What I've gathered from other terminal is that when you increase the height of the buffer, one of two things happens:

1. If there are still lines in the scrollback, those scrollback lines get pushed back into the viewport.
2. If there aren't, then blank lines are inserted at the bottom.

The trick for us here is that conpty isn't a dumb pipe - there's fundamentally a console buffer that we're trying to sync with the state of the terminal connected to it. Additionally, conpty doesn't have a scrollback at all, and it doesn't know anything about the terminal. It doesn't know if the terminal has a scrollback at all, or how many lines are in the terminal's scrollback.

I had a good amount of success with "padding the top" of the conpty buffer when we increase in height, trying to cover case 1. By "padding the top", conpty can create space in it's buffer where it thinks that lines from the terminal will be. However, things break down at that case 2 scenario. Conpty doesn't know when there aren't more lines in the scrollback. Conpty can know how many lines it's theoretically moved to scrollback, but it doesn't actually know if all of those were kept by the terminal - consider emitting 100 lines to something with only 10 lines of scrollback. Conpty would mistakenly think that it would need to "pad the top" 90 more times than the terminal actually would.

Peculiarly enough, this all works well enough until a client application tries to get the cursor position. Until then, the text we're emitting from conpty just so happens to work well enough relative to previous output that the terminal has the expected text in it. As soon as a client app requests the cursor position, that's when things start to break down. The case I'm thinking of particularly is powershell with PSReadline. PSReadline will get the cursor position, and re-move the cursor to that position while typing at the prompt. When PSR tries this after we've incorrectly padded the top, then the cursor in the terminal seems to "jump down" to where the conpty thinks the cursor should be.

I could maybe solve this by having conpty immediately ask the terminal for its cursor position on a ResizePseudoconsole (with a DSR). With that response, we could know where the terminal thinks the cursor is - if it's on the same line, then the terminal inserted blank lines to the bottom of the buffer. If the cursor has moved down, then lines were moved from scrollback.

However, the problem would be that the response would come in asynchronously to that thread. The ResizePseudoconsole handler is on a separate thread from the input handler. The input handler might already be blocked on a ReadFile from the input handle, so if the PtySignalThread tried to also ReadFile the input handle, there'd be no way to know who would handle the read first. The PtySignalThread would need to release the global lock before the read, but one of the input thread or signal thread would complete the read, but maybe after the renderer thread has another go at it. Having the signal thread do the read is a bad idea. But there would be no way to ensure the input thread is the next thread scheduled. We might paint a bad frame to the terminal after the terminal requested a resize, but before we've actually handled the resize.

This last paragraph altogether feels hacky to me anyways. I'll have to keep thinking about this, and maybe prototyping more.

[zadjii-msft]

I'm getting the sinking suspicion that #4354 is wrong, but without more time to complete it, I'm leaving a note for here, so that I'll remember.

There's probably benefit in improving the conpty invalidating story - I've long thought that just InvalidateAlling is the wrong solution. I'm not sure 4354 is right either.

Maybe it's possible that just the viewport moving code is sufficient to get this right. Maybe pull the rest and just move the viewport a little better. I came at this a really strange angle, from first thinking the terminal was doing it wrong, because other terminal behaved a certain way, then changing conpty to try and act like other terminals, realizing it really really doesn't, then changing Terminal to try and behave in accordance with conpty, without reverting everything and starting fresh. I made too many wrong assumptions in a row, and I think I maybe accumulated some of those bad assumptions when I shouldn't have. I'm thinking now that I should scope this smaller - resizing down quickly might still be broken, resizing diagonally might still be broken, but it'll be no worse than it currently is in both those cases