Report a diagnostic for return types with HResult-like named structures and provide a code-fix to do the correct marshalling

[jkoritzinsky]

Many developers use struct based HRESULT types to provide a more helpful API interface on top of HRESULT-based errors. This is technically incorrect per the ABI, but due to prexisting behavior, we couldn't fix this in bulit-in COM. In source-generated COM, we fixed this from the outset, but we didn't provide a good way for developers to determine that this change is the cause of runtime failures.

This PR implements a feature where MarshalAs(UnmanagedType.Error) on a value type in a COM interface marshals the type as an int at the ABI boundary. We will provide a diagnostic when this may be useful and a code fix to add the attribute. Additionally, we also enlighten the "Convert to source-generated COM" fixer to add the attribute when it is likely to be helpful.

Fixes #89603

[ghost]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Many developers use struct based HRESULT types to provide a more helpful API interface on top of HRESULT-based errors. This is technically incorrect per the ABI, but due to prexisting behavior, we couldn't fix this in bulit-in COM. In source-generated COM, we fixed this from the outset, but we didn't provide a good way for developers to determine that this change is the cause of runtime failures.

This PR implements a feature where MarshalAs(UnmanagedType.Error) on a value type in a COM interface marshals the type as an int at the ABI boundary. We will provide a diagnostic when this may be useful and a code fix to add the attribute. Additionally, we also enlighten the "Convert to source-generated COM" fixer to add the attribute when it is likely to be helpful.

Fixes #89603

Author:	jkoritzinsky
Assignees:	-
Labels:	area-System.Runtime.InteropServices
Milestone:	8.0.0

[jtschuster]

LGTM, my only other thought is if we want to add a warning that the struct should be 4 bytes for the bitcast.

[jkoritzinsky]

LGTM, my only other thought is if we want to add a warning that the struct should be 4 bytes for the bitcast

I don't think we need to add a warning here as likely everyone that will use this will have a struct that just has one int field in it. I'd be shocked if anyone used any other HRESULT type shape. A runtime error is fine here I think.