Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade zlib-ng to 2.2.1 #105771

Merged
merged 5 commits into from
Sep 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions THIRD-PARTY-NOTICES.TXT
Original file line number Diff line number Diff line change
Expand Up @@ -69,19 +69,27 @@ written authorization of the copyright holder.
License notice for zlib-ng
-----------------------

https://github.com/zlib-ng/zlib-ng/blob/develop/LICENSE.md
https://github.com/zlib-ng/zlib-ng/blob/d54e3769be0c522015b784eca2af258b1c026107/LICENSE.md

(C) 1995-2024 Jean-loup Gailly and Mark Adler

This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software.
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.

Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions:
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:

The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.

Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.

This notice may not be removed or altered from any source distribution.
3. This notice may not be removed or altered from any source distribution.

License notice for LinuxTracepoints
-----------------------------------
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

namespace System.IO.Compression;

internal static partial class ZLibNative
{
/// <summary>
/// <p>ZLib can accept any integer value between 0 and 9 (inclusive) as a valid compression level parameter:
/// 1 gives best speed, 9 gives best compression, 0 gives no compression at all (the input data is simply copied a block at a time).
/// <code>CompressionLevel.DefaultCompression</code> = -1 requests a default compromise between speed and compression
/// (currently equivalent to level 6).</p>
///
/// <p><strong>How to choose a compression level:</strong></p>
///
/// <p>The names <code>NoCompression</code>, <code>BestSpeed</code>, <code>DefaultCompression</code>, <code>BestCompression</code> are taken over from
/// the corresponding ZLib definitions, which map to our public NoCompression, Fastest, Optimal, and SmallestSize respectively.</p>
/// <p><em>Optimal Compression:</em></p>
/// <p><code>ZLibNative.CompressionLevel compressionLevel = ZLibNative.CompressionLevel.DefaultCompression;</code> <br />
/// <code>int windowBits = 15; // or -15 if no headers required</code> <br />
/// <code>int memLevel = 8;</code> <br />
/// <code>ZLibNative.CompressionStrategy strategy = ZLibNative.CompressionStrategy.DefaultStrategy;</code> </p>
///
///<p><em>Fastest compression:</em></p>
///<p><code>ZLibNative.CompressionLevel compressionLevel = ZLibNative.CompressionLevel.BestSpeed;</code> <br />
/// <code>int windowBits = 15; // or -15 if no headers required</code> <br />
/// <code>int memLevel = 8; </code> <br />
/// <code>ZLibNative.CompressionStrategy strategy = ZLibNative.CompressionStrategy.DefaultStrategy;</code> </p>
///
/// <p><em>No compression (even faster, useful for data that cannot be compressed such some image formats):</em></p>
/// <p><code>ZLibNative.CompressionLevel compressionLevel = ZLibNative.CompressionLevel.NoCompression;</code> <br />
/// <code>int windowBits = 15; // or -15 if no headers required</code> <br />
/// <code>int memLevel = 7;</code> <br />
/// <code>ZLibNative.CompressionStrategy strategy = ZLibNative.CompressionStrategy.DefaultStrategy;</code> </p>
///
/// <p><em>Smallest Size Compression:</em></p>
/// <p><code>ZLibNative.CompressionLevel compressionLevel = ZLibNative.CompressionLevel.BestCompression;</code> <br />
/// <code>int windowBits = 15; // or -15 if no headers required</code> <br />
/// <code>int memLevel = 8;</code> <br />
/// <code>ZLibNative.CompressionStrategy strategy = ZLibNative.CompressionStrategy.DefaultStrategy;</code> </p>
/// </summary>
public enum CompressionLevel : int
{
NoCompression = 0,
BestSpeed = 1,
DefaultCompression = -1,
BestCompression = 9
}
}
42 changes: 0 additions & 42 deletions src/libraries/Common/src/System/IO/Compression/ZLibNative.cs
Original file line number Diff line number Diff line change
Expand Up @@ -37,48 +37,6 @@ public enum ErrorCode : int
VersionError = -6
}

/// <summary>
/// <p>ZLib can accept any integer value between 0 and 9 (inclusive) as a valid compression level parameter:
/// 1 gives best speed, 9 gives best compression, 0 gives no compression at all (the input data is simply copied a block at a time).
/// <code>CompressionLevel.DefaultCompression</code> = -1 requests a default compromise between speed and compression
/// (currently equivalent to level 6).</p>
///
/// <p><strong>How to choose a compression level:</strong></p>
///
/// <p>The names <code>NoCompression</code>, <code>BestSpeed</code>, <code>DefaultCompression</code>, <code>BestCompression</code> are taken over from
/// the corresponding ZLib definitions, which map to our public NoCompression, Fastest, Optimal, and SmallestSize respectively.</p>
/// <p><em>Optimal Compression:</em></p>
/// <p><code>ZLibNative.CompressionLevel compressionLevel = ZLibNative.CompressionLevel.DefaultCompression;</code> <br />
/// <code>int windowBits = 15; // or -15 if no headers required</code> <br />
/// <code>int memLevel = 8;</code> <br />
/// <code>ZLibNative.CompressionStrategy strategy = ZLibNative.CompressionStrategy.DefaultStrategy;</code> </p>
///
///<p><em>Fastest compression:</em></p>
///<p><code>ZLibNative.CompressionLevel compressionLevel = ZLibNative.CompressionLevel.BestSpeed;</code> <br />
/// <code>int windowBits = 15; // or -15 if no headers required</code> <br />
/// <code>int memLevel = 8; </code> <br />
/// <code>ZLibNative.CompressionStrategy strategy = ZLibNative.CompressionStrategy.DefaultStrategy;</code> </p>
///
/// <p><em>No compression (even faster, useful for data that cannot be compressed such some image formats):</em></p>
/// <p><code>ZLibNative.CompressionLevel compressionLevel = ZLibNative.CompressionLevel.NoCompression;</code> <br />
/// <code>int windowBits = 15; // or -15 if no headers required</code> <br />
/// <code>int memLevel = 7;</code> <br />
/// <code>ZLibNative.CompressionStrategy strategy = ZLibNative.CompressionStrategy.DefaultStrategy;</code> </p>
///
/// <p><em>Smallest Size Compression:</em></p>
/// <p><code>ZLibNative.CompressionLevel compressionLevel = ZLibNative.CompressionLevel.BestCompression;</code> <br />
/// <code>int windowBits = 15; // or -15 if no headers required</code> <br />
/// <code>int memLevel = 8;</code> <br />
/// <code>ZLibNative.CompressionStrategy strategy = ZLibNative.CompressionStrategy.DefaultStrategy;</code> </p>
/// </summary>
public enum CompressionLevel : int
{
NoCompression = 0,
BestSpeed = 1,
DefaultCompression = -1,
BestCompression = 9
}

/// <summary>
/// <p><strong>From the ZLib manual:</strong></p>
/// <p><code>CompressionStrategy</code> is used to tune the compression algorithm.<br />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -497,6 +497,30 @@ async Task<long> GetLengthAsync(CompressionLevel compressionLevel)
Assert.True(optimalLength >= smallestLength);
}

[Theory]
[MemberData(nameof(UncompressedTestFilesZLib))]
public async Task ZLibCompressionOptions_SizeInOrder(string testFile)
{
using var uncompressedStream = await LocalMemoryStream.readAppFileAsync(testFile);

async Task<long> GetLengthAsync(int compressionLevel)
{
uncompressedStream.Position = 0;
using var mms = new MemoryStream();
using var compressor = CreateStream(mms, new ZLibCompressionOptions() { CompressionLevel = compressionLevel, CompressionStrategy = ZLibCompressionStrategy.Default }, leaveOpen: false);
await uncompressedStream.CopyToAsync(compressor);
await compressor.FlushAsync();
return mms.Length;
}

long fastestLength = await GetLengthAsync(1);
long optimalLength = await GetLengthAsync(5);
long smallestLength = await GetLengthAsync(9);

Assert.True(fastestLength >= optimalLength);
Assert.True(optimalLength >= smallestLength);
}

[Theory]
[MemberData(nameof(ZLibOptionsRoundTripTestData))]
public async Task RoundTripWithZLibCompressionOptions(string testFile, ZLibCompressionOptions options)
Expand Down Expand Up @@ -537,28 +561,6 @@ private async Task<MemoryStream> CompressTestFile(LocalMemoryStream testStream,
return compressorOutput;
}

protected async Task CompressionLevel_SizeInOrderBase(string testFile)
{
using var uncompressedStream = await LocalMemoryStream.readAppFileAsync(testFile);

async Task<long> GetLengthAsync(int compressionLevel)
{
uncompressedStream.Position = 0;
using var mms = new MemoryStream();
using var compressor = CreateStream(mms, new ZLibCompressionOptions() { CompressionLevel = compressionLevel, CompressionStrategy = ZLibCompressionStrategy.Default }, leaveOpen: false);
await uncompressedStream.CopyToAsync(compressor);
await compressor.FlushAsync();
return mms.Length;
}

long prev = await GetLengthAsync(0);
for (int i = 1; i < 10; i++)
{
long cur = await GetLengthAsync(i);
Assert.True(cur <= prev, $"Expected {cur} <= {prev} for quality {i}");
prev = cur;
}
}
}

public enum TestScenario
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@
<Compile Include="$(CommonTestPath)System\IO\ConnectedStreams.cs" Link="Common\System\IO\ConnectedStreams.cs" />
<Compile Include="$(CommonPath)System\Net\MultiArrayBuffer.cs" Link="ProductionCode\Common\System\Net\MultiArrayBuffer.cs" />
<Compile Include="$(CommonPath)System\Net\StreamBuffer.cs" Link="ProductionCode\Common\System\Net\StreamBuffer.cs" />
<Compile Include="$(CommonPath)System\IO\Compression\ZLibNative.CompressionLevel.cs"
Link="Common\System\IO\Compression\ZLibNative.CompressionLevel.cs" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="$(CommonTestPath)StreamConformanceTests\StreamConformanceTests.csproj" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
<Compile Include="System\IO\Compression\DeflateZLib\ZLibException.cs" />
<Compile Include="$(CommonPath)System\IO\Compression\ZLibNative.cs"
Link="Common\System\IO\Compression\ZLibNative.cs" />
<Compile Include="$(CommonPath)System\IO\Compression\ZLibNative.CompressionLevel.cs"
Link="Common\System\IO\Compression\ZLibNative.CompressionLevel.cs" />
<Compile Include="$(CommonPath)System\IO\Compression\ZLibNative.ZStream.cs"
Link="Common\System\IO\Compression\ZLibNative.ZStream.cs" />
<Compile Include="System\IO\Compression\CompressionLevel.cs" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -219,12 +219,5 @@ public override Task WriteAsync(byte[] buffer, int offset, int count, Cancellati
return base.WriteAsync(buffer, offset, count, cancellationToken);
}
}

[Theory]
[MemberData(nameof(UncompressedTestFilesZLib))]
public async Task ZLibCompressionLevel_SizeInOrder(string testFile)
{
await CompressionLevel_SizeInOrderBase(testFile);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -441,12 +441,5 @@ public override Task WriteAsync(byte[] buffer, int offset, int count, Cancellati
return base.WriteAsync(buffer, offset, count, cancellationToken);
}
}

[Theory]
[MemberData(nameof(UncompressedTestFilesZLib))]
public async Task ZLibCompressionLevel_SizeInOrder(string testFile)
{
await CompressionLevel_SizeInOrderBase(testFile);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -150,12 +150,5 @@ public void StreamTruncation_IsDetected(TestScenario testScenario)
}
}, testScenario.ToString()).Dispose();
}

[Theory]
[MemberData(nameof(UncompressedTestFilesZLib))]
public async Task ZLibCompressionLevel_SizeInOrder(string testFile)
{
await CompressionLevel_SizeInOrderBase(testFile);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
<Compile Include="$(CommonTestPath)System\IO\Compression\FileData.cs" Link="Common\System\IO\Compression\FileData.cs" />
<Compile Include="$(CommonTestPath)System\IO\Compression\LocalMemoryStream.cs" Link="Common\System\IO\Compression\LocalMemoryStream.cs" />
<Compile Include="$(CommonTestPath)System\IO\Compression\StreamHelpers.cs" Link="Common\System\IO\Compression\StreamHelpers.cs" />
<Compile Include="$(CommonPath)System\IO\Compression\ZLibNative.CompressionLevel.cs"
Link="Common\System\IO\Compression\ZLibNative.CompressionLevel.cs" />
<Compile Include="$(CommonTestPath)System\IO\TempFile.cs" Link="Common\System\IO\TempFile.cs" />
<Compile Include="$(CommonTestPath)System\IO\WrappedStream.cs" Link="Common\System\IO\WrappedStream.cs" />
<Compile Include="$(CommonTestPath)System\IO\Compression\ZipTestHelper.cs" Link="Common\System\IO\Compression\ZipTestHelper.cs" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@
Link="Common\System\Net\WebSockets\WebSocketValidate.cs" />
<Compile Include="$(CommonPath)System\IO\Compression\ZLibNative.cs"
Link="Common\System\IO\Compression\ZLibNative.cs" />
<Compile Include="$(CommonPath)System\IO\Compression\ZLibNative.CompressionLevel.cs"
Link="Common\System\IO\Compression\ZLibNative.CompressionLevel.cs" />
<Compile Include="$(CommonPath)System\IO\Compression\ZLibNative.ZStream.cs"
Link="Common\System\IO\Compression\ZLibNative.ZStream.cs" />
<Compile Include="$(CommonPath)Interop\Interop.zlib.cs"
Expand Down
2 changes: 1 addition & 1 deletion src/native/external/cgmanifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
"Type": "git",
"Git": {
"RepositoryUrl": "https://github.com/zlib-ng/zlib-ng",
carlossanlop marked this conversation as resolved.
Show resolved Hide resolved
"CommitHash": "74253725f884e2424a0dd8ae3f69896d5377f325"
"CommitHash": "d54e3769be0c522015b784eca2af258b1c026107"
}
},
"DevelopmentDependency": false
Expand Down
10 changes: 5 additions & 5 deletions src/native/external/zlib-ng-version.txt
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
v2.1.6
74253725f884e2424a0dd8ae3f69896d5377f325
v2.2.1
d54e3769be0c522015b784eca2af258b1c026107

https://github.com/zlib-ng/zlib-ng/releases/tag/2.1.6
https://github.com/zlib-ng/zlib-ng/releases/tag/2.2.1

We have removed the following folders from our local copy as these files are not needed for our compilation:

- zlib-ng/docs/
- zlib-ng/doc/
- zlib-ng/test/
- zlib-ng/arch/s390/self-hosted-builder/

Apply https://github.com/dotnet/runtime/pull/105433.patch
Also, if the next version does not yet contain the fixes included in 12bc7edc73308f017ec40c6b2db694a6e3490ac2, cherry-pick it as a patch.
Loading
Loading