.Net Kernel and ISKFunction Streaming Option #1 (Phase 1)

[RogerBarreto]

Context and Problem Statement

Resolves #1649
Resolves #1298

It is quite common in co-pilot implementations to have a streamlined output of messages from the LLM (large language models) and currently that is not possible while using ISKFunctions.InvokeAsync or Kernel.RunAsync methods, which enforces users to work around the Kernel and Functions to use ITextCompletion and IChatCompletion services directly as the only interfaces that currently support streaming.

Currently streaming is a capability that not all providers do support and this as part of our design we try to ensure the services will have the proper abstractions to support streaming not only of text but be open to other types of data like images, audio, video, etc.

Needs to be clear for the sk developer when he is attempting to get streaming data.

Decision Drivers

1. The sk developer should be able to get streaming data from the Kernel and Functions using Kernel.RunAsync or ISKFunctions.InvokeAsync methods

2. The sk developer should be able to get the data in a generic way, so the Kernel and Functions can be able to stream data of any type, not limited to text.

3. The sk developer when using streaming from a model that does not support streaming should still be able to use it with only one streaming update representing the whole data.

User Experience Goal

//(providing the type at as generic parameter)

// Getting a Raw Streaming data from Kernel
await foreach(string update in kernel.StreamingRunAsync<byte[]>(function, variables))

// Getting a String as Streaming data from Kernel
await foreach(string update in kernel.StreamingRunAsync<string>(function, variables))

// Getting a StreamingResultUpdate as Streaming data from Kernel
await foreach(StreamingResultUpdate update in kernel.StreamingRunAsync<StreamingResultUpdate>(variables, function))
// OR
await foreach(StreamingResultUpdate update in kernel.StreamingRunAsync(function, variables)) // defaults to Generic above)
{
    Console.WriteLine(update);
}

Out of Scope

• Streaming with plans will not be supported in this phase. Attempting to do so will throw an exception.
• Kernel streaming will not support multiple functions (pipeline).

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[markwallace-microsoft]

What is the plan to add unit and integration tests for this?