Implement format on type for proper indentation

[brettcannon]

microsoft/vscode-python#481 is the most requested feature in PVSC. Solving it via format-on-type is probably the easiest and most consistent way. Since Pylance knows what the indentation level is, when things would only be syntactically appropriate if it was dedented, etc., Pylance is probably in a great position to be able to support this via https://microsoft.github.io/language-server-protocol/specifications/specification-3-17/#textDocument_onTypeFormatting.

[savannahostrowski]

I was under the impression that indentation was something that the Python extension in VS Code was going to handle (I'd previously brought this up). I would also think we'd want this to work regardless of the language server the user has selected?

[brettcannon]

I think it's something to discuss since fixing someone's indentation to make the code valid is "formatting", but also somewhat of an implicit linting fix.

[erictraut]

Is auto-indent something that can be controlled through the LSP interface? I'm not aware of any facility for this. Maybe @jakebailey is aware of something here.

I'll also point out that since that feature request was filed, auto-formatters have become much more popular and prevalent in the Python world. The black auto-formatter, in particular, is really popular. I wouldn't want to code without it. When an auto-formatter is installed, the value of auto-indent drops to close to zero. I wonder if we have data about how many VS Code Python users have an auto-formatter installed and configured?

[brettcannon]

We do know the percentage of machines that do have formatting turned on and it isn't very high (depending your definition of "very high" 🙂).

[jakebailey]

VS Code has a thing called "on enter rules" which are intended to do indention when you enter a new block. These rules are statically declared by a language provider in a config with regexes that say "if you see this, then insert one indention", and so on. https://github.com/microsoft/vscode-python/blob/3698950c97982f31bb9dbfc19c4cd8308acda284/src/client/language/languageConfiguration.ts#L13

The problem is that it's really difficult to get this right with regexes alone; it works better in brace languages, but not so much in Python. There's a really long discussion about this, and I had tried to get the API expanded out to be a little more powerful so we could better see the context and offer indentions that aren't multiples-of-indent. microsoft/vscode#66235

On enter rules are a VS Code thing, and are not a part of the LSP spec.

As a workaround for not having a better on enter rule, on-type formatting could be used with a \n trigger character, such that an extension or LS is called instead, and can then send the correct text edits to perform the same operation that an on-enter rule might.

There are downsides to this; on-type formatting is not enabled by default in VS Code, so we'd need to force enable it for all Python users, which then means ripping out any other on-type formatters that don't simply do on-enter fixups. It's also slower (done via APIs/out of proc versus static rules), and may mismatch with whole-file formatting done by formatters like black or autopep8 (none of which support formatting at a finer grain than the entire file at once). Some users' issue with the current rule (as microsoft/vscode-python#481 shows) is not that it's "invalid", but that it's not their particular formatting choice, which adds more complexity.

[gramster]

Do we really have to force this and disable others? We could make it a setting and leave it to users to enable it and make sure there are no conflicting handlers. That seems like an MVP that we could use to evaluate if more is needed.

[jakebailey]

We could ask them to set:

{
    "[python]": {
        "editor.formatOnType": true
    }
}

Manually. It's just not super discoverable.

[debonte]

This issue has been fixed as an experiment in prerelease version 2022.9.21, which we've just released. You can find the changelog here: CHANGELOG.md

The experiment will be enabled for a percentage of prerelease users in the near future.