Support UI Automation custom annotations such as notes in MS Excel

[michaelDCurran]

Link to issue number:

None.

Summary of the issue:

The Microsoft UI Automation accessibility API has a concept of annotations, which are essentially a way of attaching extra meta information (or annotations) to content. E.g. comments. An annotation is made up of both a known type ID E.g. AnnotationType_Comment, and an object (an extra UI automation element containing properties such as the author, date etc). NVDA already supports standard UI Automation annotations.
In Windows 11, UI Automation has been extended to support custom annotations. these are annotations with an application-defined type ID. E.g. an Excel note, or an MS word bookmark.
In order for these type IDs to be agreed upon by both the application and assistive technology at runtime, a mechanism very similar to UI Automation custom property registration was introduced to UI automation for registering custom annotation types, exposed via the Windows.UI.UIAutomation.Core.CoreAutomationRegistrar winRT interface.

Description of how this pull request fixes the issue:

• implemented a new registerUIAAnnotationType function in nvdaHelperLocal that uses the Windows.UI.UIAutomation.Core.CoreAutomationRegistrar winRT interface to register an annotation type GUID, resulting in a new annotation type ID that can be used like any other standard UI automation annotation type ID.
• Implemented infrastructure in NVDA to aide in registering UI Automation annotation types, pretty much identical to the UI Automation custom property registration infrastructure.
• Added support for detecting notes in MS Excel, which are exposed as a custom annotation on cells.
• Added support for detecting bookmarks in Microsoft Word documents in both speech and braille.
• Added support for detecting draft comments and resolved comments in Microsoft Word in both speech and braille.

Testing strategy:

On a Windows 11 machine with MS Excel build 16.0.14509.20000:

• Opened a new workbook in Excel
• Enabled Use UI automation in MS Excel option in NVDA's advanced settings
• Inserted a note in the spreadsheet by pressing shift+f2, typing the note and pressing enter.
• Navigated away from and back to the cell with the arrow keys, and confirmed that NVDA reported "has note".
• Pressed NVDA+alt+c when on the cell with the note, and confirmed that NVDA reported the content of the note.

Known issues with pull request:

None known.

Change log entries:

New features

• Support for reporting notes in MS Excel with UI Automation enabled on Windows 11.
• In recent builds of Microsoft Word with UIA enabled on Windows 11, the existence of bookmarks are now reported in both speech and braille.
• In recent builds of Microsoft Word with UIA enabled on Windows 11, the existence of both draft and resolved comments are now reported in both speech and braille.

Changes for Developers

• Building NVDA now requires Windows 11 SDK 10.0.22000.

Code Review Checklist:

• Pull Request description:
  • description is up to date
  • change log entries
• Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• API is compatible with existing add-ons.
• Documentation:
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English

[codeofdusk]

Nice to see! Guessing custom patterns are on the roadmap as well (and perhaps better leveraging remote ops)?

[AppVeyorBot]

• Build (for testing PR): https://ci.appveyor.com/api/buildjobs/4i6bfwhg3se05nfj/artifacts/output/

See test results for failed build of commit f3a31caf02

[michaelDCurran]

This cannot yet build on Appveyor as the Windows SDK is not new enough.

[AppVeyorBot]

• Build (for testing PR): https://ci.appveyor.com/api/buildjobs/y81kxv1w9c32f5ia/artifacts/output/

See test results for failed build of commit c72e5e740f

[AppVeyorBot]

• Build (for testing PR): https://ci.appveyor.com/api/buildjobs/yp6w726f3h9avorx/artifacts/output/

See test results for failed build of commit 8e181946b5

[AppVeyorBot]

• Build (for testing PR): https://ci.appveyor.com/api/buildjobs/rftayflmm3ld6t0i/artifacts/output/

See test results for failed build of commit 8f398739cf

[michaelDCurran]

Looks like the needed files are in Windows SDK 10.0.22000, but not in 10.0.20348, which seems to be the latest public release that comes with VS 2019.

[seanbudd]

I've triggered a rebuild for this now that the SDK has been updated on appveyor

[lukaszgo1]

@michaelDCurran Don't forget to update the readme to mention that the never version of SDK is now required.

[LeonarddeR]

@michaelDCurran could you please update the changes for developers section that the Windows 11 SDK is now required? I think this one's easy to miss.

[feerrenrut]

Looks good.

If there are more features to add, please make a new PR with this as base.

[michaelDCurran]

This pr now covers all the existing MS Office custom UIA annotation types.