NVDA does not recognize QtWebEngine wrapper of Chromium rendering engine as being web contents

[jooyoungseo]

Background

RStudio Desktop is basically a web application made of Ace text editor and some other JavaScripts wrapped with Qt widgets.

From RStudio Desktop v1.3, they are going to provide screen reader users with accessibility options that toggle QT accessibility switch and Chromium accessibility flag like "--force-renderer-accessibility".

However, NVDA does not recognize QtWebEngine wrapper of Chromium rendering engine as being web contents. Thus, NVDA users cannot employ browse mode inside RStudio Desktop.

VoiceOver on Mac system does not have this issue, meaning that VoiceOver recognizes RStudio Desktop as being web browsers.

Steps to reproduce:

1. Download and install R for Windows if you have not had an R on your system.
2. Download the latest RStudio Desktop.
3. Install RStudio on your system with all the default options.
4. Run "C:\Program Files\RStudio\bin\rstudio.exe"
5. Press Alt+H followed by E, then enter on "Screen Reader Support (disabled)." If "Confirm Toggle Screen Reader Support" dialogue window pops up, enter on "Yes" button.
6. After restarting RStudio Desktop with accessibility option enabled as described in number 5, please see if NVDA recognizes the application as being web browsers like Electron apps, such as VSCode.

Actual behavior:

NVDA does not recognize QtWebEngine wrapper of Chromium rendering engine as being web contents.

Expected behavior:

NVDA should treat QtWebEngine as being web browsers so that users can toggle between focus and browse modes like Electron apps, such as VSCode and Slack.

System configuration

Using the default NVDA configuration.

NVDA installed/portable/running from source:

Installed on the system.

NVDA version:

NVDA 2019.3.1

Windows version:

Microsoft Windows 10 Pro x64 10.0.18363 Build 18363

Name and version of other software in use when reproducing the issue:

RStudio Desktop for Windows 1.3.883
Qt 5.12.6

This issue has also been mentioned in rstudio/rstudio#5912.

Other information about your system:

Nothing special.

Other questions

I was wondering if there would be any way for NVDA to recognize QtWebEngine as being web browsers.

Does the issue still occur after restarting your PC?

Yes.

Have you tried any other versions of NVDA? If so, please report their behaviors.

They are all the same.

[JulienCochuyt]

Dear @jooyoungseo,
Could you please consider providing a test build of RStudio with accessibility feature enabled by default? Or maybe a command line parameter to turn it on?
The very first dialog shown by RStudio upon installation - prompting to choose an R engine - does not expose to screen readers which of its controls has focus.

For the record, I've recently faced a similar situation with wxPython's wx.html2.WebView control.
NVDA's browse mode did not trigger when focusing the WebView control until the focus was forced to the proper child document control using object navigation commands and NVDA+shift+numPadMinus (I could finally come up with a proper handling of the focus on wx's side.)

[jooyoungseo]

cc @gtritchie

[gtritchie]

Ah sorry about that. This dialog you are seeing indicates that you have not installed the R language. Unfortunately RStudio itself does not install R.

That said, I will check this dialog's accessibility today; it is not a QtWebEngine-based dialog, but a native dialog, so I need to see what's going on (suspect I missed testing this dialog via screen reader).

Also, I like the idea of a command-line switch as another way to enable accessibility and will get that in, probably today.

Sorry we can't currently auto-detect screen reader and enable support, that's something I do hope to do in the future.

[jooyoungseo]

Thanks for your help, @JulienCochuyt.

I have added one more preliminary step to the reproducible procedure above to clarify. As @gtritchie described, please install R for Windows on your system, you will then be able to interact with RStudio Desktop.

[gtritchie]

Also, we are aware of and have reported several QtWebEngine accessibility issues. Some have been fixed by Qt, others are in-process. Eventually we (RStudio) will incorporate these fixes either by backporting them to the stable version of Qt that we are using, or by moving to a newer version. For reference:

• QtWebEngine not announcing live regions on Windows/NVDA (https://bugreports.qt.io/browse/QTBUG-80832)
• Checked menu items not read correctly on Windows (https://bugreports.qt.io/browse/QTBUG-80551)
• Menus that bring up submenus not read correctly on Windows (https://bugreports.qt.io/browse/QTBUG-80550)

There is an additional issue, that someone from JAWS identified, which I still need to log to Qt. Specifically, "QTWebEngine needs to implement QAccessibleInterface::focusChild to drill down to the inner most child. As things stand it always reports the body as being the inner most child."

Qt has its own accessibility abstraction layer based on QAccessibleInterface (which was modeled after MSAA). My understanding is they convert the accessibility tree produced by Chromium into a Qt model, then that layer is responsible for communicating to the OS APIs. So a couple translations with the opportunity to introduce errors.

[gtritchie]

FYI, I've yet to nail down what's happening in this particular dialog (the Choose R Engine dialog). Sometimes it does report what's got focus, sometimes not. JAWS 2020 seems ok with it, Windows-10 Narrator never works.

Tracking issue for RStudio: rstudio/rstudio#6407

Possibly related (or at least educational) issue with Qt and NVDA: https://bugreports.qt.io/browse/QTBUG-69472

Also: #8604

RStudio currently uses Qt 5.12.6; this particular dialog actually uses Qt widgets, versus most of RStudio which is hosted inside QtWebEngine.

[LeonarddeR]

I think it might even be better to have a very simple app that has a very simple html page wrapped in it with a focusable control (e.g. a link). That would make testing a lot easier.

[gtritchie]

Sure, I can build a simple example for you, will provide a link once I've got it for you.

[gtritchie]

I have created a standalone app that uses QtWebEngine and loads a simple page.

The page it loads is: https://gtritchie.github.io/focustest.html

The app is a stripped down version of a Qt example program. The source code is available to view, if desired, at: https://github.com/gtritchie/focusbrowser

I have compiled it for 64-bit Windows, then bundled it all up into a zip file (includes all the dependencies needed to run).

Download and extract: https://github.com/gtritchie/focusbrowser/blob/master/focusbrowser.zip

Then run simplebrowser.exe. This is not a signed executable so you will receive warnings from Windows about running an untrusted application.

[LeonarddeR]

@gtritchie Many thanks for your example.

I'm afraid the problem is stubborn. All items on the web page are exposed using the Accessibility implementation of Qt itself, so with UIA instead of IAccessible2 (as default in Chromium). Even when forcefully disabling UIA, there's no access to the IAccessible2 implementation of Chromium itself. May be there is a flag to expose this as yet?

Therefore, when there is a fix for this, it's not a simple one unfortunately.

[gtritchie]

Thanks, that is useful to know. I suspect there is not a way to get at IAccessible2 of the Chromium instance hosted by QtWebEngine, but I will do some digging to be sure.

[Adriani90]

cc: @dglee42

[gtritchie]

For what it's worth, the RStudio application has since been moved to Electron, largely to achieve better accessibility support.

[Adriani90]

cc: @michaelDCurran I think there was a PR been recently merged to improve how NVDA works with embedded Chromium objects but unfortunately I cannot find it anymore. I guess this issue is not only related to this specific application though.
@gtritchie many thanks for your motivation and work done so far to make the application as accessible as possible.

[Adriani90]

@jooyoungseo do you still have this issue with NVDA 2023.2?

[Adriani90]

No updates from the initial author upon request since many years, I am closing as abandoned. If you are having this issue, please comment and we can reopen.