Implement URL standard and URLSearchParams spec compliant

[charpeni]

Fixes #25717
Fixes #24428
Fixes #16434

Summary

Currently, our implementation of URL is not spec-compliant and this is causing headaches to RN users.

Tests were added to URL, to be sure we're spec compliant with https://url.spec.whatwg.org/.

As a solution to those issues, we could do one of the following:

• Implement whatwg-url instead of our custom implementation. ⚠️ This would add significant weight to React Native.
• Fork whatwg-url and remove the support of unicode from there, because tr46 dependency is like ~80% of whatwg-url size.
• Implement a lightweight polyfill. Those available on npm are not really trustworthy or depend heavily on the DOM.
• Fix RN's URL issues, case by case. IMO, this is not a viable solution. 🤷‍♂

I'll tackle an implementation of whatwg-url and compare what will be the increase of RN size with and without unicode support.

Let me know if you have any thoughts, concerns or ideas.

---

After looking at this, our best approach is to use whatwg-url to cover all the spec. But we can't use whatwg-url directly, because it's coming with a dependency to tr46 to handle Unicode cases. I think we can live without Unicode, so I forked whatwg-url, and I removed the unicode support, edited tests, and published this as whatwg-url-without-unicode.

Here are some stats:

📦whatwg-url is taking 352.5 KB.
📦whatwg-url-without-unicode is taking 53.5 KB.

Then, whatwg-url-without-unicode requires Buffer, which is 142 B.

As URLSearchParams is bundled in whatwg-url, I replaced our custom implementation with the spec-compliant one and some tests to cover this.

Changelog

[General] [Changed] - Implement spec compliant URL and URLSearchParams

Test Plan

Added tests, ran RNTester, and called new URL().

[pull-bot]

	Warnings
⚠️	🔒 package.json - Changes were made to package.json. This will require a manual import by a Facebook employee.

Messages

📖

📋 Verify Changelog Format - A changelog entry has the following format: [CATEGORY] [TYPE] - Message. CATEGORY may be: General iOS Android JavaScript Internal (for changes that do not need to be called out in the release notes) TYPE may be: Added, for new features. Changed, for changes in existing functionality. Deprecated, for soon-to-be removed features. Removed, for now removed features. Fixed, for any bug fixes. Security, in case of vulnerabilities. MESSAGE may answer "what and why" on a feature level. Use this to briefly tell React Native users about notable changes.

Generated by 🚫 dangerJS against 690ec42

[cpojer]

@charpeni did you make any progress on this?

[charpeni]

@cpojer Yes! Jungling with other priorities at the moment, but I should be able to push changes soon.

[charpeni]

After looking at this, our best approach is to use whatwg-url to cover all the spec. But we can't use whatwg-url directly, because it's coming with a dependency to tr46 to handle Unicode cases. I think we can live without Unicode, so I forked whatwg-url, and I removed the unicode support, edited tests, and published this as whatwg-url-without-unicode.

Here are some stats:

📦whatwg-url is taking 352.5 KB.
📦whatwg-url-without-unicode is taking 53.5 KB.

Then, whatwg-url-without-unicode requires Buffer, which is 142 B.

As URLSearchParams is bundled in whatwg-url, I replaced our custom implementation with the spec-compliant one and some tests to cover this.

This PR is ready to review!

[cpojer]

Thanks for doing this exploration. Adding 60k to bundle size for something that not everyone needs seems excessive. At Facebook we are trying to reduce the size of our apps and it will be hard to justify adding this much code at this time.

What if we made the URL implementation injectable? That way people could modify the URL class that is used, like require('react-native').setURLImplementation(…). What do you think?

[charpeni]

Thanks, @cpojer. I would have loved getting that concern during the draft PR 😅.

I totally agree that adding something to the bundle size for something that not everyone needs is excessive.

Adding the initial 352 KB is excessive, but adding the forked version of 53 KB doesn't seem that excessive considering the fact that URL may not be as trivial as we think. We can't really opt-in or opt-out of URL, because it's part of the core of JavaScript, at least in the Web Standard and Node spec, right? Sure, we can limit our usage of URL within our app (let's say from a require), but can we control what's going on in third-party dependencies?

I'd be afraid that React Native consumers will still have weird behavior without acknowledging that it could come from our URL implementation. I'd prefer to not provide an URL implementation by default at all than providing a minimal one that doesn't respect the spec.

⚠️ Note that RN requires an URL implementation that implement static createObjectURL(blob) to support blob.

If we keep our current implementation with a possibility to inject one at the choice of users, it would be the same frustration as now:

1. Try to use new URL()
2. Something is unexpected in the behavior
3. Doubt on ourself
4. Doubt on React Native
5. Be sad about React Native
6. Implement it ourself in our app
7. Unexpected behaviors with blob?

I'd be more prompt to fix our implementation even regarding the additional 53 KB to the bundle size than shipping something that is not polished/usable to React Native consumers. Or, do a breaking change by removing it. The initial 352 KB was excessive, but 53 KB to solve those issues seems to worth a discussion.

If we want to focus on reducing the bundle size, I think that focusing our efforts on tree shaking within Metro would be beneficial for every dependency that not everyone needs. Like by removing whatwg-fetch and abort-controller for people that are not using fetch, or even URL, or any other dependencies.

Otherwise, making them all injectable could be helpful? Not sure. 🤷‍♂

What do y'all think? Would love to hear other feedback on the bundle size in general and URL usage.

[charpeni]

My previous numbers were the minified size from bundlephobia.

I did some tests while playing with 📦 RNTester's bundle, here are my observations:

• fetch is adding 10 KB
• Legacy implementation of URL and URLSearchParams are adding 4 KB
• Blob and File are adding 3 KB
• Websocket is adding 4 KB
• XMLHTTPRequest is adding 21 KB
• AbortController is adding 2 KB
• whatwg-url would add 455 KB
• whatwg-url-without-unicode is adding 63 KB (NEW)
• Buffer is adding 23 KB (NEW)

[cpojer]

@charpeni sorry for not raising it earlier. I didn't realize that after my comment you would immediately get back to iterate on this and I wasn't sure which solution you were picking. That's on me.

I am pointing out that adding 50k of compressed app size to fix hypothetical use cases is a very very hard sell at Facebook right now. The reason I suggested the approach with injection is because we could document it and only the people who need this will have to deal with a bigger bundle size. In that case we can recommend people to use the official whatwg-url solution as well.

[charpeni]

@cpojer if we make it injectable, what do you think about doing a breaking change by completely removing URL and URLSearchParams? 🤔

[cpojer]

Interesting question, I think that could work. I unfortunately don't have a good idea on how much URL is used within RN itself and how many users depend on it. How can we assess the impact of such a breaking change?

[leethree]

@cpojer I'm sorry but I cannot agree with your assessment that proper URL handling is "hypothetical use case" when you cannot even handle the most basic use case like new URL('http://facebook.com') (see #16434)

The React Native team should either make Blob compatible with the standard URL, or rename the broken URL to something else.

[charpeni]

What we all want is a way to have polyfill for missing classes, right?

If we split the apple in half, we could say that we all want to configure the polyfills we actually need without adding unnecessary code to the bundle. But, we won't have to deal with installing and applying polyfills ourself.

What do y'all think if we expose something like this from React Native:

configurePolyfills({
  URL: true,
  URLSearchParams: true,
  fetch: false,
});

Then, the corresponding polyfill will be added to global or not, based on this or on the RN's default value.

React Native will provide expected polyfills within its dependencies`, but may not include them by default, which won't affect the bundle size, right?

react-native/Libraries/Core/setUpXHR.js

Lines 31 to 32 in a5ea720

	polyfillGlobal('URL', () => require('../Blob/URL').URL); // flowlint-line untyped-import:off
	polyfillGlobal('URLSearchParams', () => require('../Blob/URL').URLSearchParams); // flowlint-line untyped-import:off

---

How can we assess the impact of such a breaking change?

We could let the legacy implementation of URL and URLSearchParams for the next release, but show a warning message saying that implementation is deprecated, thus consumers while have to enable the polyfill with configurePolyfills or move away of URL and URLSearchParams.

[charpeni]

Closing in favor of https://github.com/charpeni/react-native-url-polyfill.