System.ArgumentOutOfRangeException in fresh solution

[domai86]

First things first: Great seeing you put work into this package, thanks a lot for that.

Describe the bug
When building my solution I encounter a System.ArgumentOutOfRangeException: Length cannot be less than zero.

To Reproduce
Steps to reproduce the behavior:

1. create empty solution
2. add .NET Core 3.1 Console Application
3. add NugetDefense and some other package (in my case Newtonsoft JSON)
4. rebuild project
5. NugetDefense.json is generated:
   { "WarnOnly": false, "Log": null, "VulnerabilityReports": {}, "Logs": null, "CheckTransitiveDependencies": true, "ErrorSettings": { "ErrorSeverityThreshold": 5, "Cvss3Threshold": -1, "IgnoredPackages": [ { "Id": "NugetDefense", "Version": "1.0.8.0" } ], "IgnoredCvEs": [], "AllowedPackages": [], "WhiteListedPackages": null, "BlockedPackages": [], "BlacklistedPackages": null }, "OssIndex": { "Enabled": true, "BreakIfCannotRun": true }, "NVD": { "SelfUpdate": false, "TimeoutInSeconds": 15, "Enabled": true, "BreakIfCannotRun": true } }
6. build failed with:
   error : Encountered a fatal exception while checking for Dependencies in C:\Users\***\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj. Exception: System.ArgumentOutOfRangeException: Length cannot be less than zero. (Parameter 'length') 1> at System.String.Substring(Int32 startIndex, Int32 length) 1> at NuGetDefense.Core.NuGetFile.<>c__DisplayClass7_0.<ParseListPackages>b__2(String l) 1> at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext()
   1> at System.Linq.Enumerable.ToDictionary[TSource,TKey](IEnumerable1 source, Func2 keySelector, IEqualityComparer1 comparer) 1> at NuGetDefense.Core.NuGetFile.ParseListPackages(String dotnetListOutput) 1> at NuGetDefense.Core.NuGetFile.dotnetListPackages(String projectFile, String targetFramework) 1> at NuGetDefense.Core.NuGetFile.LoadPackages(String targetFramework, Boolean checkTransitiveDependencies) 1> at NuGetDefense.Program.Main(String[] args)

Expected behavior
I expected a successful build with positive / negative NugetDefense messages.

Screenshots

Tools (please complete the following information):

• IDE: VS2019 16.7.3
• OS: Windows 10 pro 64 bit Build 19041.508

Additional context

[digitalcoyote]

Thanks for reporting this. I found that dotnet list had a lot more variation than I'd expected when I started using GitExtensions for testing.

Could you provide the output of dotnet list package --include-transitive for that project? I'll try to look into this after work today.

[domai86]

Thant's for the quick response, sure:

`

dotnet list package --include-transitive
Das Projekt "ConsoleApp1" enthält die folgenden Paketverweise.
[netcoreapp3.1]:
Paket oberster Ebene Angefordert Aufgelöst
Newtonsoft.Json 12.0.3 12.0.3
NuGetDefense 1.0.8 1.0.8`

[digitalcoyote]

It's the second time I've made this kind of mistake. The problem is that I'm looking for the headers in English. There is an environment variable I can feed into the command that should correct this without requiring any changes in your environment.

I'm on lunch, but I don't feel comfortable trying to throw something together without time to do some additional testing first. I'll let you know when I can get something you can try.

[digitalcoyote]

I'm attaching a test package that should force en-US as the culture for the dotnet list command when it runs inside NuGetDefense. This should allow it to parse the output. Let me know how it goes when you get a chance to run it

NuGetDefense Test Release.zip

[digitalcoyote]

That build failed the ci test. I'll see what I can do about it.

[domai86]

Thanks for working this fast on this problem, really appreciated!

[digitalcoyote]

Found the issue, I made a small deserialization change right after this one that broke it. And I inverted a check in that original version. This should work for you. If it does, I'll push it up to NuGet.Org as 1.0.9
NuGetDefense.1.0.9-pre0002.nupkg.zip

[domai86]

Hey thanks for the quick update, unfortunately I'm still getting the same error:

1> at System.String.Substring(Int32 startIndex, Int32 length) 1> at NuGetDefense.Core.NuGetFile.<>c__DisplayClass7_0.<ParseListPackages>b__2(String l) 1> at System.Linq.Enumerable.WhereSelectEnumerableIterator2.MoveNext()
1> at System.Linq.Enumerable.ToDictionary[TSource,TKey](IEnumerable1 source, Func2 keySelector, IEqualityComparer1 comparer) 1> at NuGetDefense.Core.NuGetFile.ParseListPackages(String dotnetListOutput) 1> at NuGetDefense.Core.NuGetFile.dotnetListPackages(String projectFile, String targetFramework) 1> at NuGetDefense.Core.NuGetFile.LoadPackages(String targetFramework, Boolean checkTransitiveDependencies) 1> at NuGetDefense.Program.Main(String[] args)

After checking a bit I realized set DOTNET_CLI_UI_LANGUAGE doesn't work on my machine, it's still printing in german. I will have a closer look to this and get back to you.

[domai86]

Update:
I got it running, after deleting the de folder within my dotnet installation. When changing the environment variable my dotnet help is indeed in english, but dotnet list remains german. I randomly installed some older versions of NuGet packages and now my output is:

1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj : error : NuGetDefense : NVD scan failed with exception: System.ArgumentException: An item with the same key has already been added. Key: CVE-2019-8331 1> at System.Collections.Generic.Dictionary2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
1> at NuGetDefense.NVD.Scanner.GetVulnerabilitiesForPackages(NuGetPackage[] pkgs, Dictionary`2 vulnDict)

1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : 8 vulnerabilities found for bootstrap @ 3.0.3
1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2016-10735 : In Bootstrap 3.x before 3.4.0 and 4.x-beta before 4.0.0-beta.2, XSS is possible in the data-target attribute, a different vulnerability than CVE-2018-14041.
1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2018-20677 : In Bootstrap before 3.4.0, XSS is possible in the affix configuration target property.
1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2018-20676 : In Bootstrap before 3.4.0, XSS is possible in the tooltip data-viewport attribute.
1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2018-14040 : In Bootstrap before 4.1.2, XSS is possible in the collapse data-parent attribute.
1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2019-8331 : In Bootstrap before 3.4.1 and 4.3.x before 4.3.1, XSS is possible in the tooltip or popover data-template attribute.
1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : OSS Index ID: 6dd9e321-93cd-4d79-b33a-ff7e01b15ad9 : The software does not neutralize or incorrectly neutralizes user-controllable input before it is placed in output that is used as a web page that is served to other users.
1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2018-14042 : In Bootstrap before 4.1.2, XSS is possible in the data-container property of tooltip.
1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : OSS Index ID: 3e831af5-428b-4712-874f-8f6ff932e2b2 : The software does not neutralize or incorrectly neutralizes user-controllable input before it is placed in output that is used as a web page that is served to other users.
1>C:\Users\xxx.nuget\packages\nugetdefense\1.0.9-pre0002\build\nugetdefense.targets(10,5): error MSB3073: The command "dotnet "C:\Users\xxx.nuget\packages\nugetdefense\1.0.9-pre0002\build..\tools\netcoreapp3.1\NuGetDefense.dll" "C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj" netcoreapp3.1" exited with code -1.
1>Done building project "ConsoleApp1.csproj" -- FAILED.`

[digitalcoyote]

I'm about to go to sleep and it will probably be a good 16 hours before I can look at it again. If worst comes to worse, I'll try to dig into the dotnet code and pull out the localized strings that may end up in the output.

I'm guessing the NVD scan error is a separate issue. I'll see what I can do to reproduce that while I'm at it.

[domai86]

Thanks a lot for your effort, really appreciated.

[digitalcoyote]

I believe I have a working method that should be language independant as long as no cultures use something other than the ">" for the package lines.

            var transitivelines = lines
            // Skip the informational Text at the beginning
            .SkipWhile(l => l.IndexOf('>') == -1)
            // Skip the Direct Dependencies
            .SkipWhile(l => l.IndexOf(">") != -1)
            // Skip the Header(s)
            .SkipWhile(l => l.IndexOf('>') == -1)
            // Only Take lines that still reference packages
            .TakeWhile(l => l.IndexOf(">") != -1);

[digitalcoyote]

I added a partial test that used the output you provided earlier to ensure I don't accidentally do something like that again. I hope to have a version up we can test for that soon. I'm just now looking into reproducing the second exception.

If you have a project with transitive dependencies, I'd love to add the de Transitive headers to the test

[digitalcoyote]

I believe I have the issue with NVD fixed (writing tests now). Expect a test build sometime tonight.

[digitalcoyote]

NuGetDefense.1.0.9-pre0003.nupkg.zip (Should also be available from the GitHub Package Repository as v1.0.9-ci0005, but I haven't had much luck with adding it as a source outside of GitHub Actions)

Hopefully this takes care of it, I added a check for the key before adding it to the vulnerablity dictionary that NVD uses.

[domai86]

Great job, thank you! Seems to work now:

1>------ Rebuild All started: Project: ConsoleApp1, Configuration: Debug Any CPU ------ 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj : warning NU1701: Package 'EPPlus 4.1.0' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8' instead of the project target framework '.NETCoreApp,Version=v3.1'. This package may not be fully compatible with your project. 1>ConsoleApp1 -> C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\bin\Debug\netcoreapp3.1\ConsoleApp1.dll 1>dotnet list Errors: 1> 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : 8 vulnerabilities found for bootstrap @ 3.0.3 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2018-20676 : In Bootstrap before 3.4.0, XSS is possible in the tooltip data-viewport attribute. 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2018-14040 : In Bootstrap before 4.1.2, XSS is possible in the collapse data-parent attribute. 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : OSS Index ID: 6dd9e321-93cd-4d79-b33a-ff7e01b15ad9 : The software does not neutralize or incorrectly neutralizes user-controllable input before it is placed in output that is used as a web page that is served to other users. 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : OSS Index ID: 3e831af5-428b-4712-874f-8f6ff932e2b2 : The software does not neutralize or incorrectly neutralizes user-controllable input before it is placed in output that is used as a web page that is served to other users. 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2016-10735 : In Bootstrap 3.x before 3.4.0 and 4.x-beta before 4.0.0-beta.2, XSS is possible in the data-target attribute, a different vulnerability than CVE-2018-14041. 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2018-20677 : In Bootstrap before 3.4.0, XSS is possible in the affix configuration target property. 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2018-14042 : In Bootstrap before 4.1.2, XSS is possible in the data-container property of tooltip. 1>C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj(9,6): error : CVE-2019-8331 : In Bootstrap before 3.4.1 and 4.3.x before 4.3.1, XSS is possible in the tooltip or popover data-template attribute. 1>C:\Users\xxx\.nuget\packages\nugetdefense\1.0.9-pre0003\build\nugetdefense.targets(10,5): error MSB3073: The command "dotnet "C:\Users\xxx\.nuget\packages\nugetdefense\1.0.9-pre0003\build\..\tools\netcoreapp3.1\NuGetDefense.dll" "C:\Users\xxx\source\repos\NugetDefenseTest\ConsoleApp1\ConsoleApp1.csproj" netcoreapp3.1" exited with code -1. 1>Done building project "ConsoleApp1.csproj" -- FAILED. ========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========

[digitalcoyote]

96bca90 should fix the issue and has been released as v1.0.9