Zlib: Add some protections to the allocator used by zlib

[GrabYourPitchforks]

This adds defense-in-depth protections to the allocator used by zlib to help mitigate the risk posed by potential future CVEs against this library. Basic defenses against use-of-uninitialized-memory bugs, local buffer overruns, and double-free bugs are provided. The overall goal is that should a future CVE be found that fits one of those categories, the CVE's nominal severity can drop from Critical -> Important or from Important -> Moderate because of the difficulty of successful exploit.

The .c files included in this PR describe the defenses in more detail.

Windows benchmark results:

Method	Toolchain	level	file	Mean	Error	StdDev	Ratio	RatioSD
Compress	compare	Optimal	C:\pe(...)t.pdf [93]	3,729.7 μs	715.06 μs	39.19 μs	1.03	0.03
Compress	main	Optimal	C:\pe(...)t.pdf [93]	3,606.6 μs	1,178.33 μs	64.59 μs	1.00	0.00
Decompress	compare	Optimal	C:\pe(...)t.pdf [93]	404.5 μs	41.28 μs	2.26 μs	0.95	0.03
Decompress	main	Optimal	C:\pe(...)t.pdf [93]	427.2 μs	264.41 μs	14.49 μs	1.00	0.00
Compress	compare	Optimal	C:\pe(...)9.txt [88]	5,210.0 μs	1,589.48 μs	87.12 μs	0.96	0.11
Compress	main	Optimal	C:\pe(...)9.txt [88]	5,504.3 μs	12,272.10 μs	672.68 μs	1.00	0.00
Decompress	compare	Optimal	C:\pe(...)9.txt [88]	521.7 μs	280.13 μs	15.35 μs	1.01	0.03
Decompress	main	Optimal	C:\pe(...)9.txt [88]	517.0 μs	201.16 μs	11.03 μs	1.00	0.00
Compress	compare	Optimal	C:\pe(...)a\sum [80]	777.8 μs	466.43 μs	25.57 μs	1.01	0.02
Compress	main	Optimal	C:\pe(...)a\sum [80]	772.3 μs	413.65 μs	22.67 μs	1.00	0.00
Decompress	compare	Optimal	C:\pe(...)a\sum [80]	157.2 μs	98.73 μs	5.41 μs	1.00	0.03
Decompress	main	Optimal	C:\pe(...)a\sum [80]	157.4 μs	66.97 μs	3.67 μs	1.00	0.00
Compress	compare	Fastest	C:\pe(...)t.pdf [93]	2,658.7 μs	1,499.52 μs	82.19 μs	1.03	0.04
Compress	main	Fastest	C:\pe(...)t.pdf [93]	2,579.9 μs	2,456.89 μs	134.67 μs	1.00	0.00
Decompress	compare	Fastest	C:\pe(...)t.pdf [93]	382.7 μs	117.18 μs	6.42 μs	0.96	0.04
Decompress	main	Fastest	C:\pe(...)t.pdf [93]	398.6 μs	173.96 μs	9.54 μs	1.00	0.00
Compress	compare	Fastest	C:\pe(...)9.txt [88]	1,697.4 μs	420.08 μs	23.03 μs	1.00	0.01
Compress	main	Fastest	C:\pe(...)9.txt [88]	1,701.4 μs	215.07 μs	11.79 μs	1.00	0.00
Decompress	compare	Fastest	C:\pe(...)9.txt [88]	591.7 μs	346.91 μs	19.02 μs	1.04	0.04
Decompress	main	Fastest	C:\pe(...)9.txt [88]	568.1 μs	233.40 μs	12.79 μs	1.00	0.00
Compress	compare	Fastest	C:\pe(...)a\sum [80]	347.1 μs	166.74 μs	9.14 μs	0.98	0.08
Compress	main	Fastest	C:\pe(...)a\sum [80]	355.5 μs	457.27 μs	25.06 μs	1.00	0.00
Decompress	compare	Fastest	C:\pe(...)a\sum [80]	151.5 μs	133.82 μs	7.34 μs	0.97	0.05
Decompress	main	Fastest	C:\pe(...)a\sum [80]	155.9 μs	121.91 μs	6.68 μs	1.00	0.00

Though the three PRs #84602, #84603, and #84604 are all related to zlib; they're each fully standalone and can be reviewed as isolated units.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This adds defense-in-depth protections to the allocator used by zlib to help mitigate the risk posed by potential future CVEs against this library. Basic defenses against use-of-uninitialized-memory bugs, local buffer overruns, and double-free bugs are provided. The overall goal is that should a future CVE be found that fits one of those categories, the CVE's nominal severity can drop from Critical -> Important or from Important -> Moderate because of the difficulty of successful exploit.

The .c files included in this PR describe the defenses in more detail.

Windows benchmark results:

Method	Toolchain	level	file	Mean	Error	StdDev	Ratio	RatioSD
Compress	compare	Optimal	C:\pe(...)t.pdf [93]	3,729.7 μs	715.06 μs	39.19 μs	1.03	0.03
Compress	main	Optimal	C:\pe(...)t.pdf [93]	3,606.6 μs	1,178.33 μs	64.59 μs	1.00	0.00
Decompress	compare	Optimal	C:\pe(...)t.pdf [93]	404.5 μs	41.28 μs	2.26 μs	0.95	0.03
Decompress	main	Optimal	C:\pe(...)t.pdf [93]	427.2 μs	264.41 μs	14.49 μs	1.00	0.00
Compress	compare	Optimal	C:\pe(...)9.txt [88]	5,210.0 μs	1,589.48 μs	87.12 μs	0.96	0.11
Compress	main	Optimal	C:\pe(...)9.txt [88]	5,504.3 μs	12,272.10 μs	672.68 μs	1.00	0.00
Decompress	compare	Optimal	C:\pe(...)9.txt [88]	521.7 μs	280.13 μs	15.35 μs	1.01	0.03
Decompress	main	Optimal	C:\pe(...)9.txt [88]	517.0 μs	201.16 μs	11.03 μs	1.00	0.00
Compress	compare	Optimal	C:\pe(...)a\sum [80]	777.8 μs	466.43 μs	25.57 μs	1.01	0.02
Compress	main	Optimal	C:\pe(...)a\sum [80]	772.3 μs	413.65 μs	22.67 μs	1.00	0.00
Decompress	compare	Optimal	C:\pe(...)a\sum [80]	157.2 μs	98.73 μs	5.41 μs	1.00	0.03
Decompress	main	Optimal	C:\pe(...)a\sum [80]	157.4 μs	66.97 μs	3.67 μs	1.00	0.00
Compress	compare	Fastest	C:\pe(...)t.pdf [93]	2,658.7 μs	1,499.52 μs	82.19 μs	1.03	0.04
Compress	main	Fastest	C:\pe(...)t.pdf [93]	2,579.9 μs	2,456.89 μs	134.67 μs	1.00	0.00
Decompress	compare	Fastest	C:\pe(...)t.pdf [93]	382.7 μs	117.18 μs	6.42 μs	0.96	0.04
Decompress	main	Fastest	C:\pe(...)t.pdf [93]	398.6 μs	173.96 μs	9.54 μs	1.00	0.00
Compress	compare	Fastest	C:\pe(...)9.txt [88]	1,697.4 μs	420.08 μs	23.03 μs	1.00	0.01
Compress	main	Fastest	C:\pe(...)9.txt [88]	1,701.4 μs	215.07 μs	11.79 μs	1.00	0.00
Decompress	compare	Fastest	C:\pe(...)9.txt [88]	591.7 μs	346.91 μs	19.02 μs	1.04	0.04
Decompress	main	Fastest	C:\pe(...)9.txt [88]	568.1 μs	233.40 μs	12.79 μs	1.00	0.00
Compress	compare	Fastest	C:\pe(...)a\sum [80]	347.1 μs	166.74 μs	9.14 μs	0.98	0.08
Compress	main	Fastest	C:\pe(...)a\sum [80]	355.5 μs	457.27 μs	25.06 μs	1.00	0.00
Decompress	compare	Fastest	C:\pe(...)a\sum [80]	151.5 μs	133.82 μs	7.34 μs	0.97	0.05
Decompress	main	Fastest	C:\pe(...)a\sum [80]	155.9 μs	121.91 μs	6.68 μs	1.00	0.00

Author:	GrabYourPitchforks
Assignees:	-
Labels:	area-System.IO.Compression
Milestone:	-

[carlossanlop]

Do you think we can have tests that verify this new code?

[GrabYourPitchforks]

Do you think we can have tests that verify this new code?

@carlossanlop I'm open to suggestions as to how to test this! The most straightforward way that occurs to me is to have a callback mechanism, but now we're leaking abstractions through the DLL's exports and inserting complexity. :(

[danmoseley]

do you plan to offer these to zlib upstream?

[GrabYourPitchforks]

do you plan to offer these to zlib upstream?

No. Per our other conversation, I don't think it's necessary.

[GrabYourPitchforks]

I also reran the aspnet benchmarks after ripping out the LFH code and didn't observe any difference before + after the change.

[GrabYourPitchforks]

High level notes from the latest iteration:

• Moved the custom allocators out of zlib and zlib-intel and into the Compression.Native folder, where the PAL already exists.
• Removed custom typedefs, as requested.
• Added a flag so that we know when we're using zlib vs zlib-intel, which allows both the allocator and the PAL to pull the correct header. Previously it was always pulling the normal (non-intel) zlib header, which seems to have worked, but probably was not intended behavior.

I smoke tested the Unix version on my Ubuntu box by making a change to cmakelists.txt and forcing us to use the internal version, even though the distro already had a shared zlib library.

New test run since the other one went stale: https://dev.azure.com/dnceng-public/public/_build/results?buildId=246796

[carlossanlop]

Thanks for the change. And thanks for all the comments, they were quite helpful.

[GrabYourPitchforks]

/backport to release/7.0-staging

[github-actions]

Started backporting to release/7.0-staging: https://github.com/dotnet/runtime/actions/runs/5674150369