uBlock Origin blocks local files opening in a new window under certain conditions

[abhishek-shanthkumar]

Prerequisites

• I verified that this is not a filter list issue. Report any issues with filter lists or broken website functionality in the uAssets issue tracker.
• This is NOT an issue with YouTube, Facebook or Twitch.
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue.
• The issue is not present after disabling uBO in the browser.
• I checked the documentation to understand that the issue I am reporting is not normal behavior.

I tried to reproduce the issue when...

• uBO is the only extension.
• using a new, unmodified browser profile.

Description

On Chromium-based browsers including Google Chrome and Microsoft Edge, when uBlock Origin has an active "selfie" of the filters, local files opening in a new window are blocked by uBlock Origin in certain scenarios where the selfie is not loaded correctly.
The logger message when the file open is blocked:

It seems that the selfie is not correctly loaded when the browser is launched without a web URL in general, because of which some default rule is probably kicking in and closing the new window. When the filters are correctly loaded, the blocking does not happen. If there is no selfie currently, the blocking does not happen. See repro steps below.

A specific URL where the issue occurs.

Any local file URL, such as file:///D:/Testing/StudyInDMinor.pdf

Steps to Reproduce

1. Install Edge or Chrome (or create a new profile) and install uBlock Origin.
2. Load the latest filters and get uBlock Origin to take a selfie.
3. Close the browser and ensure all browser processes are dead.
4. From a command prompt, run <path to browser executable> <path to local file> --new-window. Example: "C:\Program Files\Google\Chrome\Application\chrome.exe" file:///D:/Testing/StudyInDMinor.pdf --new-window​.
5. The browser opens the PDF. Optionally open the uBlock Origin logger, but don't navigate to any other website.
6. Now run the same command from step (4) above in the command prompt.

Expected behavior

The file opens in a new window and remains open.

Actual behavior

The file opens in a new window which gets closed immediately by uBlock Origin, with this call stack:

At this point, if you navigate to any website with potential ads in the first open window and then run the command again, the window does not get closed by uBlock Origin. But if you close all the browser processes again and repeat from step (4) in the repro steps above, the issue reoccurs.

Configuration

uBlock Origin: 1.52.2
Chromium: 119
filterset (summary):
 network: 104704
 cosmetic: 40319
 scriptlet: 18833
 html: 0
listset (total-discarded, last-updated):
 default:
  user-filters: 0-0, never
  easylist: 72268-15, 20h.23m
  easyprivacy: 32851-64, 20h.23m
  plowe-0: 3721-1072, 20h.23m
  ublock-badware: 7412-138, 20h.23m
  ublock-filters: 35158-145, 20h.23m
  ublock-privacy: 668-6, 20h.23m
  ublock-quick-fixes: 212-128, 7h.55m
  ublock-unbreak: 2135-32, 20h.23m
  urlhaus-1: 11180-0, 20h.23m
filterset (user): [empty]
userSettings: [none]
hiddenSettings: [none]
supportStats:
 allReadyAfter: 648 ms (selfie)
 maxAssetCacheWait: 266 ms

[gwarser]

Seems to be Windows only. Or specific to Chrome build. Can repro in Win11 in VM, cannot in Manjaro with google-chrome from AUR.

---

Windows: 118.0.5993.118 or 119.0.6045.106
Linux: 119.0.6045.105

[gorhill]

Windows only

So I won't be able to investigate. Unfortunate because I find this issue mind-binding. Where is that ||^popup filter coming from? Loading a selfie in memory just load the arrays of numbers as is for the most part -- well that is what I expect, so the issue has to be elsewhere than the selfie. Wasn't there other reports of opening PDF files crashing the browser when uBO is enabled? Could be related.

Also, I notice that the part of the URL which is selected is :, this does not match the filter.

[abhishek-shanthkumar]

Yes, gorhill, I suspect that the issue reported earlier - #2882 - has the same underlying root cause as this one. There is no actual browser crash there; the reporter perceived it as a crash because the window appeared and immediately disappeared.
In the repro above, the window that was initially opened remains open when the second window gets closed. In case of a crash, all windows would go away.

[gorhill]

When uBO launches, the network request listener is disabled until all filters are loaded, so that the filtering engines are not being used until fully loaded.

However, looking at the selfie-loading code, which is asynchronous, I do not see any internal barrier to prevent evaluation until a fully loaded selfie.

The code which checks popup filters does not go through the network request listener, so it could start to try to use the static network filtering engine while it is not yet fully loaded. It is difficult to predict what exactly would happen in this case, and maybe the issue here is such case.

I need a barrier raised in both fromSelfie and fromCompiled to ensure that any attempt at trying to evaluate whether there are matching filters immediately returns as "no match".

[abulgatz]

uBlock Origin has been preventing me from opening local PDFs and PWAs (I utilize Outlook Webmail PWAs opened in Window mode) on multiple Windows 10 and Windows 11 machines, with fully up-to-date Windows and Edge. With Edge running, a msedge process starts and immediately exits (visible in task manager, but no visible window).

With uBlock Origin uninstall in Edge, both local PDFs and my Outlook PWAs open.

[gwarser]

Looks fixed when using local build.

[abhishek-shanthkumar]

Thanks for the fix, @gorhill! When will a new non-dev release get created with this fix and published to the Chrome and Edge Stores?

[gorhill]

We are discussing preparing the current dev build to be a release candidate given we want differential update capabilities to be out there asap, so maybe between 1-2 week if no big changes occurs to the code.

[gwarser]

The fix should be in 1.53.1b3, right? New window is still closed for me. This is a fresh installation of uBO dev build + wait for selfie.

---

The local build is fine.

[gwarser]

The local build had access to file URLs allowed, I changed the setting few times in both - now on both window is closed, no matter the "file URL access" setting.

---

Clean reinstall of the local build with few browser restarts in between, and it's again fine.

[gorhill]

If I could reproduce I would add console.trace where uBO closes tabs. Unfortunately, I can't reproduce with Chromium when I ask to open a PDF with Chromium. If the issue is in uBO it should also occur with any Chromium-based browser (and maybe even Firefox), but timing-sensitive issues are difficult to reproduce.

[abulgatz]

Do you have instructions for how to log this? The uBlock log doesn’t seem to be capturing anything because the browser has to start closed and the bug obviously closes the browser right after opening it?

[gwarser]

In Edge this affects the first window opened, but in Chrome you can open the first one, keep it open, and the subsequent ones will close. You can then enable dev mode in chrome://extensions, click the "backgorund.html" in uBO tile and set a breakpoint. I'm trying to step in code, but cannot figure it out.

[gwarser]

Put file-scheme into "Trusted sites" as a workaround?

[gorhill]

in Chrome you can open the first one, keep it open, and the subsequent ones will close

I don't understand that part. Subsequent windows? Opened how? uBO is already fully loaded with the first one, this means if you open another windows and it closes, the logger would show what uBO did if it did close it.

[gwarser]

From OP

From a command prompt, run <path to browser executable> <path to local file> --new-window. Example: "C:\Program Files\Google\Chrome\Application\chrome.exe" file:///D:/Testing/StudyInDMinor.pdf --new-window​.

And you get this:

(also from OP)

---

In realmMatchString something is found in bucket01 .

---

Looks like broken trie, because it only blocks the popup when all default filters are selected.

[gorhill]

realmMatchString something is found in bucket01

Probably popup filter, but what is the stack at that point? I would like to know the entry point.

[gwarser]

[gorhill]

Can you look at the content of the "register" variables around line 250? I am guessing that $requestHostname is empty string?

I can re-create the symptom by disabling all filter lists and adding the filter $popup, but then the logger is reporting *$popup. The fact that ||^$popup is being reported suggest the trie matches the empty hostname being passed to it. I have to understand how this can happen.

---

This is where the trie is being tested:

[gwarser]

[gwarser]

[gorhill]

I think there might be a problem if ever the trie is unserialized and first tested with an empty hostname. The length of the last hostname tested is always stored at byte offset 255 in the trie. It's supposed to always match the length of cached needle property. This needle property is initialized to '' when the trie is unserialized, but the byte at offset 255 is not cleared, it could contain whatever was there when the trie was serialized. Not sure what could happen as a result, but I don't think that is good. So I will fix this by properly resetting the array at offset 255. Since you can reproduce at will, this will tell us if this was the issue.

[gwarser]

Since you can reproduce at will, this will tell us if this was the issue.

Right now, only in build from CWS.

---

Was able to reproduce on 1.53.1rc0, will try master.

---

Maybe it works, or maybe I cannot reproduce again. We'll see in next rc.

[gorhill]

If this was indeed the source of the issue (I am pretty sure it was), this means that for the issue to occurs, the last trie test right before serialization had to be a match, and the first trie test right after deserialization had to be an empty hostname. Might explain why it's difficult to reproduce.

[gorhill]

Found a reliable way to reproduce, as per my above comment:

• Ensure the browser is set to "Open the New Tab page" on startup
• Ensure there are no selfie, just force a manual update of a list, say "Quick fixes"
• Open a new tab, navigate to doubleclick.net, this will trigger strict blocking
  • This is to ensure the last test in the trie container is a match
• Wait for the selfie to be serialized, this occurs after ~2 minutes
• Quit the browser
• Launch the browser at the command line as described in top post
  • On linux I used: microsoft-edge --new-window [path-to-pdf-file]
• Repeat, launch another browser at the command line as in previous step

Result: second window closes.

Here I used Microsoft Edge in a virtual machine, but should work for any Chromium-based browser really, and probably for Firefox if I looked up how to do the same with it.

[abhishek-shanthkumar]

Thanks for the deeper investigation and fix, @gorhill! I validated that the issue doesn’t occur on a fresh install of the release candidate version of the extension. But what will the behaviour be for users of the extension who are currently in the bad state (where file opens are getting blocked) and get an update with the fix? Will the fix start working immediately or will it apply only after the selfie is recreated on their browser? In other words, is the above fix applicable at the time of reading the selfie or writing to it?

[gwarser]

Selfie version number changed in this version, so it will be invalidated on startup after uBO update.

gorhill/uBlock@7c3e060

[gorhill]

will the behaviour be for users of the extension who are currently in the bad state (where file opens are getting blocked) and get an update with the fix?

Yes, this will be fixed immediately, the selfie itself was not bad, it's the proper initialization of a single byte that was missing after the selfie was loaded.

[gorhill]

@gwarser That will happen for 1.54.0 release.

[gorhill]

Err sorry, I must be tired. Of course the fix is in the current dev build, not in 1.53.2. I will create a stable release just for Microsoft Edge with the fix above.

[gorhill]

Fix is in 1.53.4 which is now live at https://microsoftedge.microsoft.com/addons/detail/ublock-origin/odfafepnkmbhccpbejgmiehpchacaeak

[abhishek-shanthkumar]

Thanks for the hotfix release, @gorhill! Is the earlier commit - gorhill/uBlock@89b2727 - not necessary to resolve this issue in all cases?

[gorhill]

not necessary to resolve this issue in all cases?

I am just guessing the byte one was the cause. The other commit has more changes and I would not be comfortable for a quick release.

[gwarser]

Dev build is stuck on 1.53.1rc0 and Edge add-ons store has 1.53.2.

[gorhill]

I saw 1.53.4 yesterday in MSFT store, don't know what happened. As for Chrome store, 1.53.1rc3 has been pending review since Nov. 8.

[abhishek-shanthkumar]

@gorhill, the version on the Microsoft Edge Addons Store is still 1.53.2, and there doesn't seem to be a pending submission for the update to 1.53.4. Can you please double-check if the release got submitted to the store correctly?

[gorhill]

Can you please double-check if the release got submitted to the store correctly?

It's out of my control since I am not the owner of extension in MSFT store. I can tell however that I did see 1.53.4 in the store not too long after I created the release. So this was reversed somehow afterward.

[SampeiNihira]

As of a few minutes ago, in the Microsoft Store, the version of UBO available is 1.53.4.
Thank you Nik R.

[gwarser]

On Edge, 1.53.4 is fine, 1.53.0 from CWS is broken.

On Chrome, 1.53.1.103 is fine, 1.53.0 is broken.

[PaulCoddington]

Fix is in 1.53.4 which is now live at https://microsoftedge.microsoft.com/addons/detail/ublock-origin/odfafepnkmbhccpbejgmiehpchacaeak

Opening MD and SVG files from Explorer (associated with Edge), the problem is now mostly resolved, but unfortunately still crops up again between 1/6 to 1/3 of attempts (again, when Edge is not already open to begin with).

[Gurjeet-msft]

Hello All,

I am Gurjeet from the Microsoft Edge Add-ons team.

Thank you for reporting this issue.

Our team was able to review this and has now resolved the reported issue, request you to please check and confirm in case you are still facing the same issue.

Also, uBlock Origin has released a new Version 1.53.4 on November 14, 2023, for the extension to help resolve this issue.

In case the issue persists, please let us know and our support team will investigate this further.

Thank you.

[gwarser]

*$popup still works. Users may have custom filters like this. Maybe @@|file:///$popup should be added uBO lists? Limited to Chromium perhaps?

I'm not sure, but Edge can be a specific case, because it keeps a process open in the background.

On the other hand, we have window-close-if because $popup was not enough in some local-system cases - such exception can be exploited.

[Yuki2718]

Maybe @@|file:///$popup should be added uBO lists?

Despite you said

1.53.1.103 is fine

??

[gwarser]

The bug is fixed - popup is no longer closed by invalid ||^$popup filter, but users may still create very generic or just poor $popup filters that will match when opening pdf documents from local system.

[Yuki2718]

Isn't that user's own choice? We only fix issues by default lists, or at most built-in lists.

[gorhill]

Could also be the no-popup switch, @@|file:///$popup won't help in that case.

[gwarser]

I cannot reproduce with no-popup switch.