[Perf] VS becomes sluggish/locks up after opening very large C++ files

[duncanp-sonar]

Description

Users have reported that VS slows down after opening very large C++ files (10k+ lines).
See the following community thread for more information : https://community.sonarsource.com/t/sonarlint-vs-extension-performance/54423

Repro

TODO - find a project with C++ with files of 10K+ lines
Candidate: https://github.com/llvm/llvm-project

Notes:

• the issue does not appear if SonarLint is not installed
• "I start scrolling then the UI thread is completely blocked"
• "It seems to happen when i switch tabs"
• log extract:

Finished analyzing C:\WIP\App\Loadboard2\LB\BusLogic\QGWrQMgt\QGMgrApplyInstrConcrete.cpp, analysis time: 47.475s
Found 672 issue(s) for C:\WIP\App\Loadboard2\LB\BusLogic\QGWrQMgt\QGMgrApplyInstrConcrete.cpp

Areas to investigate - internal

• tagging - what is VS asking us to recalculate when scrolling? Are we recalculating more tags than we need to?
• streaming issues back from the subprocess to the Error List. Is VS trying to update the UI too often?
• PCH - how often is PCH analysis being triggered? How often cancelled? How much does it cost?
• calculating rule configuration: how long does it take? How much disc access (note: does not appear to be an issue in the community thread cited above - the log shows it using a cached configuration)

Areas to investigate - external

• effect of anti-virus software: how many temporary files are we writing? How large are they? (e.g. PCH output)

Sample open source project

https://github.com/llvm/llvm-project

Instructions for generating a solution file:

Notes:
- Dependencies: cmake, python3, VS2022
- Run from VS 2022 x64 naitivetools command prompt

Instructions:
git clone https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build
cd build
cmake -G "Visual Studio 17 2022" -Thost=x64 -DLLVM_ENABLE_PROJECTS=clang -DLLVM_TARGETS_TO_BUILD=X86 ..\llvm

// the last command should generate llvm-project\build\llvm.sln
// open the solution and build the project once. The build generate some file that are used in the project. It takes a lot of time to build.
// the freeze for me happen on switching between large files: type.h decl.h type.cpp.

 Directory of c:\llvm\llvm-project\clang\lib\Sema
09/12/2021  10:32           902,515 SemaOpenMP.cpp
09/12/2021  10:32           814,335 SemaExpr.cpp
09/12/2021  10:32           765,089 SemaDecl.cpp

 Directory of c:\llvm\llvm-project\clang\lib\CodeGen
09/12/2021  10:32           814,164 CGBuiltin.cpp
10/12/2021  10:50           577,532 CGOpenMPRuntime.cpp
09/12/2021  10:32           440,493 TargetInfo.cpp

 Directory of c:\llvm\llvm-project\clang\lib\AST
09/12/2021  10:32           592,157 ExprConstant.cpp
09/12/2021  10:32           453,845 ASTContext.cpp
09/12/2021  10:32           371,874 ASTImporter.cpp
09/12/2021  10:32           230,161 ItaniumMangle.cpp

[garethyr]

Hi all, I'm working on an open-source revitalization of an old game, and there are some large, unrefactored cpp files in it that freeze VS like clockwork when sonarlint is running, even with a good cpu and ram.

Please feel free to use it for reproducing the issue if it's helpful, here's the link:
https://github.com/cortex-command-community/Cortex-Command-Community-Project-Source

A couple offending files we haven't yet refactored and trimmed down are:
Entities/AHuman.cpp
Menus/MetagameGUI.cpp - this one is particularly large and prone to freezing

Edit - I was talking about this with one of my project contributors (we've both used and very much appreciated sonarlint for a few years now), and we both recall these performance issues being relatively recent, starting perhaps 1 - 1.5 years ago. He believes they began around the time you all added linting support for cpp header files, though I can't promise that's correct. Still, perhaps this information could be helpful.

[jdlanning]

My team is facing this issue with our large legacy C++ code base. Is there any workaround suggested other than disabling the extension when working with large files (e.g. maybe UI settings, etc) ?

We are using VS2019 as our IDE and this occurs on PCs with both corporate and default Windows antivirus. So from our perspective it is independent of antivirus configuration.

[jdlanning]

This issue is still impacting my team and we are still very interested in a solution. Even something like that proposed in #1348 would be very welcomed as today we have to disable the plugin often.

[duncanp-sonar]

Notes: scrolling in the file generates a lot of tooltip items:

Changing the tooltips to be lazily-created seems to prevent the IDE from freezing when scrolling.
I think this is is a mitigation rather than a complete solution, as we are still creating a very large number of tags (there are "only" around 3000 issues in the file). However, it is worth implementing as an interim solution.

[jdlanning]

@duncanp-sonar just wanted to say thank for you picking up this investigation. I was excited to see the progress in the draft PR #3690 and downloaded the VSIX artifacts to give it a try in our code base.

Happy to report that VS2019 did not face any crippling lack of responsiveness on some of our files which were known to exacerbate the issue. Once released this fix will have a great impact on the 100+ software engineers using sonarlint on a daily basis.

[duncanp-sonar]

@jdlanning thanks for the feedback and for trying out the fix - much appreciated!

We expect this fix to be in the next release, planned for late this week/early next week.

[totalgee]

Looking forward to trying this on some large/slow files of our own! Thanks.

[duncanp-sonar]

@totalgee FYI if you want to try a pre-release build one is available here.

[totalgee]

@totalgee FYI if you want to try a pre-release build one is available here.

Thank you, I'm now running (what I assume is) a release version of SonarLint that includes this fix (v6.13.0.62767). The problems I had with VS locking up on certain of my large C++ files seems to have been resolved. Thanks!

[duncanp-sonar]

@totalgee yes, that's the current release version. Thanks for confirming.

[MaximDude]

No more slowdown and freezing in the files mentioned by @garethyr with the latest release on VS22.
Thanks for the fix!