[🐛 Bug]: Python CDP library crashes on events from Chrome Beta

[teqwve]

What happened?

When using CDB connection from Python and subscribing to network events in Chrome Beta the library fails on parsing messages with KeyError: 'Network.policyUpdated'.

AFAIU it's so because Network.policyUpdated event is not present in devtools protocol used by current Chrome Stable and as a result is not included in the latest version of generated devtools protocol library included in Selenium.

I'm not sure if it's really a bug since the documentation states that only the last 3 versions are supported, but I'd like to ask if you know any way to workaround this, or if it would be possible to ignore unknown events instead of crashing.

It should be sufficient to wrap event parsing with try-except (somewhere here) and log in case of an exception. To avoid hiding issues with stable versions this could be enabled only when matching devtools library was not found and the latest was used as a fallback (somewhere here). If like this idea I can prepare a PR.

How can we reproduce the issue?

import selenium.webdriver
import selenium.webdriver.chrome.service
import trio

async def main():
    options = selenium.webdriver.ChromeOptions()
    options.binary_location = "./chrome-linux64/chrome"

    driver = selenium.webdriver.Chrome(
        service=selenium.webdriver.chrome.service.Service(executable_path="./chromedriver-linux64/chromedriver"),
        options=options,
    )

    async with driver.bidi_connection() as connection:
        await connection.session.execute(connection.devtools.network.enable())
        driver.get("https://www.google.com")
        await trio.sleep(60)

trio.run(main)

Relevant log output

+ Exception Group Traceback (most recent call last):
  |   File "/home/teqwve/repro/repro.py", line 21, in <module>
  |     trio.run(main)
  |   File "/home/teqwve/venv/lib/python3.10/site-packages/trio/_core/_run.py", line 2306, in run
  |     raise runner.main_task_outcome.error
  |   File "/home/teqwve/repro/repro.py", line 16, in main
  |     async with driver.bidi_connection() as connection:
  |   File "/usr/lib/python3.10/contextlib.py", line 217, in __aexit__
  |     await self.gen.athrow(typ, value, traceback)
  |   File "/home/teqwve/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 1068, in bidi_connection
  |     async with cdp.open_cdp(ws_url) as conn:
  |   File "/usr/lib/python3.10/contextlib.py", line 217, in __aexit__
  |     await self.gen.athrow(typ, value, traceback)
  |   File "/home/teqwve/venv/lib/python3.10/site-packages/selenium/webdriver/common/bidi/cdp.py", line 477, in open_cdp
  |     async with trio.open_nursery() as nursery:
  |   File "/home/teqwve/venv/lib/python3.10/site-packages/trio/_core/_run.py", line 959, in __aexit__
  |     raise combined_error_from_nursery
  | exceptiongroup.ExceptionGroup: Exceptions from Trio nursery (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/home/teqwve/venv/lib/python3.10/site-packages/selenium/webdriver/common/bidi/cdp.py", line 454, in _reader_task
    |     session._handle_data(data)
    |   File "/home/teqwve/venv/lib/python3.10/site-packages/selenium/webdriver/common/bidi/cdp.py", line 258, in _handle_data
    |     self._handle_event(data)
    |   File "/home/teqwve/venv/lib/python3.10/site-packages/selenium/webdriver/common/bidi/cdp.py", line 293, in _handle_event
    |     event = devtools.util.parse_json_event(data)
    |   File "/home/teqwve/venv/lib/python3.10/site-packages/selenium/webdriver/common/devtools/v127/util.py", line 20, in parse_json_event
    |     return _event_parsers[json['method']].from_json(json['params'])
    | KeyError: 'Network.policyUpdated'
    +------------------------------------

Operating System

Ubuntu 22.04

Selenium version

4.23.1

What are the browser(s) and version(s) where you see this issue?

Google Chrome for Testing 128.0.6613.36

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 128.0.6613.36

Are you using Selenium Grid?

No response

[github-actions]

@teqwve, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[diemol]

We only package the DevTools protocol for the stable versions. In addition, we do not plan to invest more in CDP integration features because the time we have available is to integrate Selenium with WebDriver BiDi.

[teqwve]

Understood, thanks!

[winty56]

I'm experiencing the same problem. If you don't plan to support CDP in the future, I think exception handling is necessary to at least skip non-existent event parser to maintain backward compatibility.

In order to temporarily resolve this issue, I share how to force an empty parser to be added to connection.devtools.util.
just add code before driver.get().

class Dummy:
    def from_json(cls, json):
        pass

connection.devtools.util._event_parsers["Network.policyUpdated"] = Dummy()