Consider supporting varargs calling conv in function pointers to unblock interop scenarios

[john-h-k]

Currently, function pointers don't support the vararg calling convention (analogous to a final param of __arglist in a normal method), which blocks any native interop in scenarios you need to dynamically load pointers to vararg functions (such as invoking the python API where are large proportion of methods are varargs, or in objective-C interop for Metal).

I could see 3 possible syntaxes for this - explicit __arglist, varargs specifier, or both

 // native sig 'float Method(int, ...)'

delegate* <int, __arglist, float> // explicit __arglist
delegate* varargs<int, float> // varargs specifier,
delegate* varargs<int, __arglist, float>// both

Invocation sytnax could then be implicit or explicit, again

 // native sig 'float Method(int, ...)'
// assuming the last syntax above
delegate* varargs<int, __arglist, float> func = ...;

float a = func(1, __arglist(2.0, 'a', null, 66)); // explicit, as is currently done
float b = func(1, 2.0, 'a', null, 66); // implicit

I think this is a reasonable request as there is effectively no workaround for this without language support and so it is a pretty unfixable blocker for some of these scenarios

The runtime support is already there, as far as I know (on windows).

[john-h-k]

mean to put this in the lang repo thanks fred 🙂

[sos-dll]

The runtime support is already there, as far as I know (on windows).

Just a brief note.
If you try to call vararg method (__arglist) at non-Windows, it will throw:
Unhandled exception. System.InvalidProgramException: Vararg calling convention not supported.
Probably due __arglist being there for very long ago before .NET Core were introduced, it only works for Windows OS.
For what its worth, it would make sense to support it for interop. 👍

(I like to think of __arglist as "Microsoft extension to C#", similarly to this, but not really a part of the language which explains their lack of docs ever since.)

[msedi]

See also discussions here:

dotnet/runtime#82081
dotnet/runtime#48796

Currently this blocks us from supporting linux for a single library that has varargs.

[JackHarckness]

This prevents us from interoping with code that expects caller to make some kind of vprintf call. Example: AVCodec has export C function av_log_set_callback that expects a function pointer to a callback with va_list in arguments.

Currently it is done like this:

[UnmanagedCallersOnly]
[SkipLocalsInit]
static void LogEntryPoint(void* avcl, int level, byte* fmt, void* varargs)
{
    // Some code requiring manual varargs parsing...
}

delegate* unmanaged<void*, int, byte*, void*, void> callback = &LogEntryPoint;
av_log_set_callback(callback);

I personally would prefer having __arglist as a type parameter like this:

delegate* unmanaged<void*, int, byte*, __arglist, void> 

Or maybe just use ArgIterator:

delegate* unmanaged<void*, int, byte*, ArgIterator, void>