diff --git a/.github/_typos.toml b/.github/_typos.toml index 37bf426d04b6..917745e1ae83 100644 --- a/.github/_typos.toml +++ b/.github/_typos.toml @@ -15,6 +15,7 @@ extend-exclude = [ "CodeTokenizerTests.cs", "test_code_tokenizer.py", "*response.json", + "test_content.txt", ] [default.extend-words] diff --git a/docs/decisions/0046-azure-model-as-a-service.md b/docs/decisions/0046-azure-model-as-a-service.md new file mode 100644 index 000000000000..a91468e253b0 --- /dev/null +++ b/docs/decisions/0046-azure-model-as-a-service.md @@ -0,0 +1,44 @@ +--- +# These are optional elements. Feel free to remove any of them. +status: { accepted } +contact: { rogerbarreto, taochen } +date: { 2024-06-20 } +deciders: { alliscode, moonbox3, eavanvalkenburg } +consulted: {} +informed: {} +--- + +# Support for Azure Model-as-a-Service in SK + +## Context and Problem Statement + +There has been a demand from customers for the implementation of Model-as-a-Service (MaaS) in SK. MaaS, which is also referred to as [serverless API](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/model-catalog-overview#model-deployment-managed-compute-and-serverless-api-pay-as-you-go), is available in [Azure AI Studio](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio). This mode of consumption operates on a pay-as-you-go basis, typically using tokens for billing purposes. Clients can access the service via the [Azure AI Model Inference API](https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-api?tabs=azure-studio) or client SDKs. + +At present, there is no official support for MaaS in SK. The purpose of this ADR is to examine the constraints of the service and explore potential solutions to enable support for the service in SK via the development of a new AI connector. + +## Client SDK + +The Azure team will be providing a new client library, namely `Azure.AI.Inference` in .Net and `azure-ai-inference` in Python, for effectively interacting with the service. While the service API is OpenAI-compatible, it is not permissible to use the OpenAI and the Azure OpenAI client libraries for interacting with the service as they are not independent with respect to both the models and their providers. This is because Azure AI Studio features a diverse range of open-source models, other than OpenAI models. + +### Limitations + +The initial release of the client SDK will only support chat completion and text/image embedding generation, with image generation to be added later. + +Plans to support for text completion are currently unclear, and it is highly unlikely that the SDK will ever include support for text completion. As a result, the new AI connector will **NOT** support text completions in the initial version until we get more customer signals or the client SDK adds support. + +## AI Connector + +### Naming options + +- Azure +- AzureAI +- AzureAIInference +- AzureAIModelInference + + Decision: `AzureAIInference` + +### Support for model-specific parameters + +Models can possess supplementary parameters that are not part of the default API. The service API and the client SDK enable the provision of model-specific parameters. Users can provide model-specific settings via a dedicated argument along with other settings, such as `temperature` and `top_p`, among others. + +In the context of SK, execution parameters are categorized under `PromptExecutionSettings`, which is inherited by all connector-specific setting classes. The settings of the new connector will contain a member of type `dictionary`, which will group together the model-specific parameters. diff --git a/docs/decisions/0045-kernel-content-graduation.md b/docs/decisions/0046-kernel-content-graduation.md similarity index 100% rename from docs/decisions/0045-kernel-content-graduation.md rename to docs/decisions/0046-kernel-content-graduation.md diff --git a/docs/decisions/0047-azure-open-ai-connectors.md b/docs/decisions/0047-azure-open-ai-connectors.md new file mode 100644 index 000000000000..c909574b9563 --- /dev/null +++ b/docs/decisions/0047-azure-open-ai-connectors.md @@ -0,0 +1,210 @@ +--- +# These are optional elements. Feel free to remove any of them. +status: approved +contact: rogerbarreto +date: 2024-06-24 +deciders: rogerbarreto, matthewbolanos, markwallace-microsoft, sergeymenshykh +consulted: stephentoub, dmytrostruk +--- + +# OpenAI and Azure Connectors Naming and Structuring + +## Context and Problem Statement + +It has recently been announced that OpenAI and Azure will each have their own dedicated SDKs for accessing their services. Previously, there was no official SDK for OpenAI, and our OpenAI Connector relied solely on the Azure SDK client for access. + +With the introduction of the official OpenAI SDK, we now have access to more up-to-date features provided by OpenAI, making it advantageous to use this SDK instead of the Azure SDK. + +Additionally, it has become clear that we need to separate the OpenAI connector into two distinct targets: one for OpenAI and another for Azure OpenAI. This separation will enhance code clarity and facilitate a better understanding of the usage of each target. + +## Decision Drivers + +- Update our connectors to use latest versions of OpenAI and Azure SDKs. +- Minimize or eliminate any breaking changes for developers currently using the existing OpenAI connector. +- Changes made should be be future proof. + +## Versioning + +Although current `Azure.AI.OpenAI` and `OpenAI` SDK packages have its major versions updated (2.0.0), that change does not represent a `SemanticKernel` major breaking change. Any of the alternative options provided below take in consideration the that the new updated version of `SemanticKernel.Connectors.OpenAI` and `SemanticKernel.Connectors.AzureOpenAI` will be a minor version bump `1.N+1.0` for all SemanticKernel packages. + +### Meta Package Strategy + +Currently the `Microsoft.SemanticKernel` package is a meta package that includes both `SemanticKernel.Core` and `SemanticKernel.Connectors.OpenAI`, with the new changes a new project will be added to the meta package `SemanticKernel.Connectors.AzureOpenAI` that will include the new Azure OpenAI connector. + +## Documentation (Upgrade Path) + +A documentation guidance and samples/examples will be created to guide on how to upgrade from the current OpenAI connector to the new when needed. + +## OpenAI SDK limitations + +The new OpenAI SDK introduce some limitations that need to be considered and pontentially can introduce breaking changes if not remediated by our internal implementation. + +- #### ⚠️ No support for multiple results (Choices) per request. + + **Remediation**: Internally make the multiple requests and combine them. + **No remediation**: Breaking change removing `ResultsPerPrompt` from `OpenAIPromptExecutionSettings`. + +- #### ⚠️ Text Generation modality is not supported. + + **Remediation**: Internally provide a HttpClient to be used against `gpt-3.5-turbo-instruct` for text generation modality. Same way was done for `TextToImage`, `AudioToText` service modalities. + **No remediation**: Breaking change removing any specific `TextGeneration` service implementations, this change don't impact `ChatCompletion` services that may still being used as `ITextGenerationService` implementations. + +## Improvements + +This also represents an opportunity to improve the current OpenAI connector by introducing the `Configuration` pattern to allow more flexibility and control over the services and their configurations. + +```csharp +// Before +builder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey, httpClient); +// After +builder.AddAzureOpenAIChatCompletion(new +{ + DeploymentName = modelId; + Endpoint = endpoint; + ApiKey = apiKey; +}); +``` + +```csharp +// Before +builder.AddAzureOpenAIChatCompletion(deploymentName, openAIClient, serviceId, modelId) +// After +builder.AddAzureOpenAIChatCompletion(new +{ + DeploymentName = deploymentName; + ServiceId = serviceId; + ModelId = modelId; +}, openAIClient); +``` + +## Potential Dependency Conflicts + +Since `SemanticKernel.Connectors.AzureOpenAI` and `SemanticKernel.Connectors.OpenAI` share same `OpenAI 2.0.0` dependency, if the vestion of `OpenAI 2.0.0` differ on each, that may create conflict when both connector packages are used together in a project. + +If this happens: + +1. Before updating our OpenAI connector package we will get in touch with `Azure.AI.OpenAI` team to align on the ETAs for their update. + +2. Investigate if the most recent `OpenAI` package when used with a `Azure.AI.OpenAI` that initially was targeting an older version of `OpenAI` SDK will not cause any breaking changes or conflicts. + +3. If There are conflicts and their ETA is small we may keep the `OpenAI` dependency on our `SemanticKernel.Connectors.OpenAI` similar to Azure's for a short period of time, otherwise we will evaluate moving forward with the `OpenAI` dependency version upgrade. + +## Considered Options + +- Option 1 - Merge New and Legacy (Slow transition for independent connectors). +- Option 2 - Independent Connectors from Start. +- Option 3 - Keep OpenAI and Azure in the same connector (As is). + +## Option 1 - Merge New and Legacy (Slow transition for independent connectors). + +This is the least breaking approach where we keep the current legacy OpenAI and AzureOpenAI APIs temporarily in the connector using last Azure SDK `Azure.AI.OpenAI 1.0.0-beta.17` and add new OpenAI specific APIs using the new `OpenAI 2.0.0-beta.*` SDK package. + +This approach also implies that a new connector will be created on a second moment for Azure OpenAI services specifically fully dependent on the latest `Azure.AI.OpenAI 2.0.0-beta.*` SDK package. + +In a later stage we will deprecate all the OpenAI and Azure legacy APIs in the `SemanticKernel.Connectors.OpenAI` namespace and remove Azure SDK `Azure.AI.OpenAI 1.0.0-beta.17` and those APIs in a future release, making the OpenAI Connector fully dedicated for OpenAI services only depending on with the `OpenAI 2.0.0-beta.*` dependency. + +```mermaid +graph TD + A[SemanticKernel.Connectors.OpenAI] --> B[OpenAI 2.0.0-beta.*] + A --> C[Azure.OpenAI 1.0.0-beta.17] + D[SemanticKernel.Connectors.AzureOpenAI] --> E[Azure.AI.OpenAI 2.0.0-beta.*] +``` + +The new `Options` pattern we be used as an improvement as well as a measure to avoid breaking changes with the legacy APIs. + +Following this change the `SemanticKernel.Connectors.OpenAI` and a new `SemanticKernel.Connectors.AzureOpenAI` connector will be created for Azure specific services, using the new Azure SDK `Azure.AI.OpenAI 2.0.0-beta.*` with all new APIs using the options approach. + +### Phases of the transition + +- **Phase 1**: Add new OpenAI SDK APIs to the current OpenAI connector and keep the Azure OpenAI APIs using the last Azure SDK. +- **Phase 2**: + - Create a new connector for Azure OpenAI services using the new Azure SDK + - Deprecate all Azure OpenAI APIs in the `OpenAI` connector pointing to new `AzureOpenAI` connector + - Remove Azure SDK dependency from the OpenAI connector. + - Add `AzureOpenAI` connector to the `Microsoft.SemanticKernel` meta package. +- **Phase 3**: Deprecate all legacy `OpenAI APIs` in the `OpenAI` connector pointing to new `Options` APIs. +- **Phase 4**: Remove all legacy APIs from the OpenAI connector. + +### Impact + +Pros: + +- Minimal breaking changes for developers using the current OpenAI connector. +- Clear separation of concerns between OpenAI and Azure OpenAI connectors. + +Cons: + +- Since `SemanticKernel.Connectors.AzureOpenAI` and `SemanticKernel.Connectors.OpenAI` share a same dependency of different versions, both packages cannot be used in the same project and a strategy will be needed when deploying both connectors. +- Added dependency for both `Azure OpenAI 1.0-beta17` and `OpenAI 2.0-beta1`. + +### Dependency Management Strategies + +1. Use only one of the connectors in the same project, some modifications will be needed to accommodate `Concepts` and other projects that shares OpenAI and AzureOpenAI examples. +2. Hold AzureOpenAI connector implementation until we are ready to break (exclude) all Azure APIs in OpenAI connector. +3. Deploy a new project with a new namespace for `Azure.AI.OpenAI.Legacy 1.0.0-beta.17` and update our `SemanticKernel.Connectors.OpenAI` to use this new namespace to avoid version clashing on the `Azure.AI.OpenAI` namespace. + +## Option 2 - Independent Connectors from Start. + +This option is focused on creating fully independent connectors for OpenAI and Azure OpenAI services since the start with all breaking changes needed to achieve that. + +```mermaid +graph TD + D[SemanticKernel.Connectors.AzureOpenAI] --> E[Azure.AI.OpenAI 2.0.0-beta.*] + E --> B[OpenAI 2.0.0-beta.*] + A[SemanticKernel.Connectors.OpenAI] --> B[OpenAI 2.0.0-beta.*] +``` + +Impact: + +- All `Azure` related logic will be removed from `SemanticKernel.Connectors.OpenAI` to avoid any clashes with same names introduced in the new `SemanticKernel.Connectors.AzureOpenAI` as well as sending a congruent message to developers that the OpenAI connector is focused on OpenAI services only moving forward. + +### Impact + +Pros: + +- Clear separation of concerns between OpenAI and Azure OpenAI connectors. +- Small breaking changes for developers focused on OpenAI specific APIs. +- Faster transition to the new OpenAI SDK and Azure OpenAI SDK. + +Cons: + +- Large breaking changes for developers using the current OpenAI connector for Azure. +- [Potential Dependency Conflicts](#potential-dependency-conflicts) may arise if the `Azure.AI.OpenAI` team does not update their package. + +## Option 3 - Keep OpenAI and Azure in the same connector (As is). + +This option is fully focused in the least impact possible, combining both Azure and OpenAI SDK dependencies in one single connector following the same approach as the current connector. + +Changes: + +1. Update all current OpenAI specific services and client to use new OpenAI SDK +2. Update Azure specific services and client to use the latest Azure OpenAI SDK. +3. Optionally add `Options` pattern new APIs to the connector services and deprecate old ones. + +### Impact + +Pros: + +- Minimal breaking changes for developers using the current OpenAI connector. +- The breaking changes will be limited on how we tackle the points mentioned in the [OpenAI SDK limitations](#openai-sdk-limitations) above. +- Will not have a dependency conflict between `Azure.AI.OpenAI` and `OpenAI` SDKs. + +Cons: + +- We will be limited on the OpenAI SDK version that is used by the latest `Azure.AI.OpenAI` package, which may not be the latest version available. +- When using direct Azure or OpenAI specific services developers don't expect to see other provider specific services in their pool of options and dependencies. + +## Decision Outcome + +### Option 2 - Independent Connectors from Start. + +This option is the faster approach on transitioning to a potential 1.0 general availability of `OpenAI` SDK. + +This also option provides a clear separation of concerns between OpenAI and Azure OpenAI connectors from the start. + +Prevents any confusion sending a clear message on our intentions on splitting `OpenAI` and `AzureOpenAI` components away. + +#### OpenAI SDK limitations: + +- [Multiple results](#openai-sdk-limitations): **Do not remediate**. +- [Text Generation modality is not supported](#openai-sdk-limitations): **Do not remediate**. diff --git a/dotnet/Directory.Packages.props b/dotnet/Directory.Packages.props index 146311afca6f..bb4233ad6ba9 100644 --- a/dotnet/Directory.Packages.props +++ b/dotnet/Directory.Packages.props @@ -9,9 +9,9 @@ - + - + @@ -20,7 +20,7 @@ - + @@ -53,14 +53,14 @@ - + - + @@ -73,8 +73,8 @@ - - + + @@ -85,7 +85,7 @@ - + @@ -97,7 +97,7 @@ - + all runtime; build; native; contentfiles; analyzers; buildtransitive diff --git a/dotnet/nuget/nuget-package.props b/dotnet/nuget/nuget-package.props index 6a48e76f58fc..d91b4c61c640 100644 --- a/dotnet/nuget/nuget-package.props +++ b/dotnet/nuget/nuget-package.props @@ -1,7 +1,7 @@ - 1.15.0 + 1.15.1 $(VersionPrefix)-$(VersionSuffix) $(VersionPrefix) diff --git a/dotnet/samples/Concepts/Agents/ChatCompletion_Streaming.cs b/dotnet/samples/Concepts/Agents/ChatCompletion_Streaming.cs new file mode 100644 index 000000000000..ee6fb9b38f2a --- /dev/null +++ b/dotnet/samples/Concepts/Agents/ChatCompletion_Streaming.cs @@ -0,0 +1,69 @@ +// Copyright (c) Microsoft. All rights reserved. +using System.Text; +using Microsoft.SemanticKernel; +using Microsoft.SemanticKernel.Agents; +using Microsoft.SemanticKernel.ChatCompletion; + +namespace Agents; + +/// +/// Demonstrate creation of and +/// eliciting its response to three explicit user messages. +/// +public class ChatCompletion_Streaming(ITestOutputHelper output) : BaseTest(output) +{ + private const string ParrotName = "Parrot"; + private const string ParrotInstructions = "Repeat the user message in the voice of a pirate and then end with a parrot sound."; + + [Fact] + public async Task UseStreamingChatCompletionAgentAsync() + { + // Define the agent + ChatCompletionAgent agent = + new() + { + Name = ParrotName, + Instructions = ParrotInstructions, + Kernel = this.CreateKernelWithChatCompletion(), + }; + + ChatHistory chat = []; + + // Respond to user input + await InvokeAgentAsync("Fortune favors the bold."); + await InvokeAgentAsync("I came, I saw, I conquered."); + await InvokeAgentAsync("Practice makes perfect."); + + // Local function to invoke agent and display the conversation messages. + async Task InvokeAgentAsync(string input) + { + chat.Add(new ChatMessageContent(AuthorRole.User, input)); + + Console.WriteLine($"# {AuthorRole.User}: '{input}'"); + + StringBuilder builder = new(); + await foreach (StreamingChatMessageContent message in agent.InvokeStreamingAsync(chat)) + { + if (string.IsNullOrEmpty(message.Content)) + { + continue; + } + + if (builder.Length == 0) + { + Console.WriteLine($"# {message.Role} - {message.AuthorName ?? "*"}:"); + } + + Console.WriteLine($"\t > streamed: '{message.Content}'"); + builder.Append(message.Content); + } + + if (builder.Length > 0) + { + // Display full response and capture in chat history + Console.WriteLine($"\t > complete: '{builder}'"); + chat.Add(new ChatMessageContent(AuthorRole.Assistant, builder.ToString()) { AuthorName = agent.Name }); + } + } + } +} diff --git a/dotnet/samples/Concepts/Agents/OpenAIAssistant_FileService.cs b/dotnet/samples/Concepts/Agents/OpenAIAssistant_FileService.cs index ac3c70a750b3..7537f53da726 100644 --- a/dotnet/samples/Concepts/Agents/OpenAIAssistant_FileService.cs +++ b/dotnet/samples/Concepts/Agents/OpenAIAssistant_FileService.cs @@ -1,5 +1,4 @@ // Copyright (c) Microsoft. All rights reserved. -using Azure.AI.OpenAI.Assistants; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.Connectors.OpenAI; using Resources; @@ -7,7 +6,7 @@ namespace Agents; /// -/// Demonstrate uploading and retrieving files with . +/// Demonstrate using . /// public class OpenAIAssistant_FileService(ITestOutputHelper output) : BaseTest(output) { @@ -19,7 +18,6 @@ public class OpenAIAssistant_FileService(ITestOutputHelper output) : BaseTest(ou [Fact] public async Task UploadAndRetrieveFilesAsync() { - var openAIClient = new AssistantsClient(TestConfiguration.OpenAI.ApiKey); OpenAIFileService fileService = new(TestConfiguration.OpenAI.ApiKey); BinaryContent[] files = [ @@ -29,41 +27,40 @@ public async Task UploadAndRetrieveFilesAsync() new BinaryContent(data: await EmbeddedResource.ReadAllAsync("travelinfo.txt"), mimeType: "text/plain") { InnerContent = "travelinfo.txt" } ]; - var fileIds = new Dictionary(); - foreach (var file in files) + var fileContents = new Dictionary(); + foreach (BinaryContent file in files) { - var result = await openAIClient.UploadFileAsync(new BinaryData(file.Data), Azure.AI.OpenAI.Assistants.OpenAIFilePurpose.FineTune); - fileIds.Add(result.Value.Id, file); + OpenAIFileReference result = await fileService.UploadContentAsync(file, new(file.InnerContent!.ToString()!, OpenAIFilePurpose.FineTune)); + fileContents.Add(result.Id, file); } - foreach (var file in (await openAIClient.GetFilesAsync(Azure.AI.OpenAI.Assistants.OpenAIFilePurpose.FineTune)).Value) + foreach (OpenAIFileReference fileReference in await fileService.GetFilesAsync(OpenAIFilePurpose.FineTune)) { - if (!fileIds.ContainsKey(file.Id)) + // Only interested in the files we uploaded + if (!fileContents.ContainsKey(fileReference.Id)) { continue; } - var data = (await openAIClient.GetFileContentAsync(file.Id)).Value; + BinaryContent content = await fileService.GetFileContentAsync(fileReference.Id); - var mimeType = fileIds[file.Id].MimeType; - var fileName = fileIds[file.Id].InnerContent!.ToString(); - var metadata = new Dictionary { ["id"] = file.Id }; - var uri = new Uri($"https://api.openai.com/v1/files/{file.Id}/content"); - var content = mimeType switch + string? mimeType = fileContents[fileReference.Id].MimeType; + string? fileName = fileContents[fileReference.Id].InnerContent!.ToString(); + ReadOnlyMemory data = content.Data ?? new(); + + var typedContent = mimeType switch { - "image/jpeg" => new ImageContent(data, mimeType) { Uri = uri, InnerContent = fileName, Metadata = metadata }, - "audio/wav" => new AudioContent(data, mimeType) { Uri = uri, InnerContent = fileName, Metadata = metadata }, - _ => new BinaryContent(data, mimeType) { Uri = uri, InnerContent = fileName, Metadata = metadata } + "image/jpeg" => new ImageContent(data, mimeType) { Uri = content.Uri, InnerContent = fileName, Metadata = content.Metadata }, + "audio/wav" => new AudioContent(data, mimeType) { Uri = content.Uri, InnerContent = fileName, Metadata = content.Metadata }, + _ => new BinaryContent(data, mimeType) { Uri = content.Uri, InnerContent = fileName, Metadata = content.Metadata } }; - // Display the the file-name and mime-tyupe for each content type. - Console.WriteLine($"File: {fileName} - {mimeType}"); - - // Display the each content type-name. - Console.WriteLine($"Type: {content}"); + Console.WriteLine($"\nFile: {fileName} - {mimeType}"); + Console.WriteLine($"Type: {typedContent}"); + Console.WriteLine($"Uri: {typedContent.Uri}"); // Delete the test file remotely - await openAIClient.DeleteFileAsync(file.Id); + await fileService.DeleteFileAsync(fileReference.Id); } } } diff --git a/dotnet/samples/Concepts/ChatCompletion/Google_GeminiChatCompletion.cs b/dotnet/samples/Concepts/ChatCompletion/Google_GeminiChatCompletion.cs index de2e996dc2fc..2e8f750e5476 100644 --- a/dotnet/samples/Concepts/ChatCompletion/Google_GeminiChatCompletion.cs +++ b/dotnet/samples/Concepts/ChatCompletion/Google_GeminiChatCompletion.cs @@ -89,7 +89,7 @@ private async Task SimpleChatAsync(Kernel kernel) { Console.WriteLine("======== Simple Chat ========"); - var chatHistory = new ChatHistory(); + var chatHistory = new ChatHistory("You are an expert in the tool shop."); var chat = kernel.GetRequiredService(); // First user message diff --git a/dotnet/samples/Concepts/ChatCompletion/Google_GeminiChatCompletionStreaming.cs b/dotnet/samples/Concepts/ChatCompletion/Google_GeminiChatCompletionStreaming.cs index 97f4873cfd52..803a6b6fafcd 100644 --- a/dotnet/samples/Concepts/ChatCompletion/Google_GeminiChatCompletionStreaming.cs +++ b/dotnet/samples/Concepts/ChatCompletion/Google_GeminiChatCompletionStreaming.cs @@ -90,7 +90,7 @@ private async Task StreamingChatAsync(Kernel kernel) { Console.WriteLine("======== Streaming Chat ========"); - var chatHistory = new ChatHistory(); + var chatHistory = new ChatHistory("You are an expert in the tool shop."); var chat = kernel.GetRequiredService(); // First user message diff --git a/dotnet/samples/Concepts/ChatCompletion/Google_GeminiVision.cs b/dotnet/samples/Concepts/ChatCompletion/Google_GeminiVision.cs index 1bf70ca28f5b..179b2b40937d 100644 --- a/dotnet/samples/Concepts/ChatCompletion/Google_GeminiVision.cs +++ b/dotnet/samples/Concepts/ChatCompletion/Google_GeminiVision.cs @@ -14,7 +14,7 @@ public async Task GoogleAIAsync() Console.WriteLine("============= Google AI - Gemini Chat Completion with vision ============="); string geminiApiKey = TestConfiguration.GoogleAI.ApiKey; - string geminiModelId = "gemini-pro-vision"; + string geminiModelId = TestConfiguration.GoogleAI.Gemini.ModelId; if (geminiApiKey is null) { @@ -28,7 +28,7 @@ public async Task GoogleAIAsync() apiKey: geminiApiKey) .Build(); - var chatHistory = new ChatHistory(); + var chatHistory = new ChatHistory("Your job is describing images."); var chatCompletionService = kernel.GetRequiredService(); // Load the image from the resources @@ -55,7 +55,7 @@ public async Task VertexAIAsync() Console.WriteLine("============= Vertex AI - Gemini Chat Completion with vision ============="); string geminiBearerKey = TestConfiguration.VertexAI.BearerKey; - string geminiModelId = "gemini-pro-vision"; + string geminiModelId = TestConfiguration.VertexAI.Gemini.ModelId; string geminiLocation = TestConfiguration.VertexAI.Location; string geminiProject = TestConfiguration.VertexAI.ProjectId; @@ -96,7 +96,7 @@ public async Task VertexAIAsync() // location: TestConfiguration.VertexAI.Location, // projectId: TestConfiguration.VertexAI.ProjectId); - var chatHistory = new ChatHistory(); + var chatHistory = new ChatHistory("Your job is describing images."); var chatCompletionService = kernel.GetRequiredService(); // Load the image from the resources diff --git a/dotnet/samples/Concepts/Filtering/TelemetryWithFilters.cs b/dotnet/samples/Concepts/Filtering/TelemetryWithFilters.cs new file mode 100644 index 000000000000..a2edd8948e51 --- /dev/null +++ b/dotnet/samples/Concepts/Filtering/TelemetryWithFilters.cs @@ -0,0 +1,207 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Diagnostics; +using System.Text.Json; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; +using Microsoft.SemanticKernel; +using Microsoft.SemanticKernel.Connectors.OpenAI; + +namespace Filtering; + +/// +/// Kernel and connectors have out-of-the-box telemetry to capture key information, which is available during requests. +/// In most cases this telemetry should be enough to understand how the application behaves. +/// This example contains the same telemetry recreated using Filters. +/// This should allow to extend existing telemetry if needed with additional information and have the same set of logging messages for custom connectors. +/// +public class TelemetryWithFilters(ITestOutputHelper output) : BaseTest(output) +{ + [Fact] + public async Task LoggingAsync() + { + // Initialize kernel with chat completion service. + var builder = Kernel + .CreateBuilder() + .AddOpenAIChatCompletion("gpt-4", TestConfiguration.OpenAI.ApiKey); + + // Create and add logger, which will output messages to test detail summary window. + var logger = this.LoggerFactory.CreateLogger(); + builder.Services.AddSingleton(logger); + + // Add filters with logging. + builder.Services.AddSingleton(); + builder.Services.AddSingleton(); + builder.Services.AddSingleton(); + + var kernel = builder.Build(); + + // Import sample functions. + kernel.ImportPluginFromFunctions("HelperFunctions", + [ + kernel.CreateFunctionFromMethod(() => DateTime.UtcNow.ToString("R"), "GetCurrentUtcTime", "Retrieves the current time in UTC."), + kernel.CreateFunctionFromMethod((string cityName) => + cityName switch + { + "Boston" => "61 and rainy", + "London" => "55 and cloudy", + "Miami" => "80 and sunny", + "Paris" => "60 and rainy", + "Tokyo" => "50 and sunny", + "Sydney" => "75 and sunny", + "Tel Aviv" => "80 and sunny", + _ => "31 and snowing", + }, "GetWeatherForCity", "Gets the current weather for the specified city"), + ]); + + // Enable automatic function calling. + var executionSettings = new OpenAIPromptExecutionSettings + { + ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions, + ModelId = "gpt-4" + }; + + // Define custom transaction ID to group set of operations related to the request. + var transactionId = new Guid("2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2"); + + // Note: logging scopes are available for out-of-the-box SK telemetry as well. + using (logger.BeginScope($"Transaction ID: [{transactionId}]")) + { + // Invoke prompt with arguments. + const string Prompt = "Given the current time of day and weather, what is the likely color of the sky in {{$city}}?"; + var result = await kernel.InvokePromptAsync(Prompt, new(executionSettings) { ["city"] = "Boston" }); + + Console.WriteLine(result); + } + + // Output: + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function InvokePromptAsync_Id invoking. + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function arguments: {"city":"Boston"} + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Execution settings: {"default":{"service_id":null,"model_id":"gpt-4"}} + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Rendered prompt: Given the current time of day and weather, what is the likely color of the sky in Boston? + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] ChatHistory: [{"Role":{"Label":"user"},... + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function count: 1 + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function call requests: HelperFunctions-GetCurrentUtcTime({}) + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function GetCurrentUtcTime invoking. + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function GetCurrentUtcTime succeeded. + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function result: Tue, 25 Jun 2024 15:30:16 GMT + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function completed. Duration: 0.0011554s + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] ChatHistory: [{"Role":{"Label":"user"},... + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function count: 1 + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function call requests: HelperFunctions-GetWeatherForCity({"cityName":"Boston"}) + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function GetWeatherForCity invoking. + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function arguments: {"cityName":"Boston"} + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function GetWeatherForCity succeeded. + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function result: 61 and rainy + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function completed. Duration: 0.0020878s + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function InvokePromptAsync_Id succeeded. + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function result: The sky in Boston would likely be gray due to the rain and current time of day. + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Usage: {"CompletionTokens":19,"PromptTokens":169,"TotalTokens":188} + // Transaction ID: [2d9ca2ce-8bf7-4d43-9f90-05eda7122aa2] Function completed. Duration: 5.397173s + } + + /// + /// Filter which logs an information available during function invocation such as: + /// Function name, arguments, execution settings, result, duration, token usage. + /// + private sealed class FunctionInvocationLoggingFilter(ILogger logger) : IFunctionInvocationFilter + { + public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func next) + { + long startingTimestamp = Stopwatch.GetTimestamp(); + + logger.LogInformation("Function {FunctionName} invoking.", context.Function.Name); + + if (context.Arguments.Count > 0) + { + logger.LogTrace("Function arguments: {Arguments}", JsonSerializer.Serialize(context.Arguments)); + } + + if (logger.IsEnabled(LogLevel.Information) && context.Arguments.ExecutionSettings is not null) + { + logger.LogInformation("Execution settings: {Settings}", JsonSerializer.Serialize(context.Arguments.ExecutionSettings)); + } + + try + { + await next(context); + + logger.LogInformation("Function {FunctionName} succeeded.", context.Function.Name); + logger.LogTrace("Function result: {Result}", context.Result.ToString()); + + if (logger.IsEnabled(LogLevel.Information)) + { + var usage = context.Result.Metadata?["Usage"]; + + if (usage is not null) + { + logger.LogInformation("Usage: {Usage}", JsonSerializer.Serialize(usage)); + } + } + } + catch (Exception exception) + { + logger.LogError(exception, "Function failed. Error: {Message}", exception.Message); + throw; + } + finally + { + if (logger.IsEnabled(LogLevel.Information)) + { + TimeSpan duration = new((long)((Stopwatch.GetTimestamp() - startingTimestamp) * (10_000_000.0 / Stopwatch.Frequency))); + + // Capturing the duration in seconds as per OpenTelemetry convention for instrument units: + // More information here: https://opentelemetry.io/docs/specs/semconv/general/metrics/#instrument-units + logger.LogInformation("Function completed. Duration: {Duration}s", duration.TotalSeconds); + } + } + } + } + + /// + /// Filter which logs an information available during prompt rendering such as rendered prompt. + /// + private sealed class PromptRenderLoggingFilter(ILogger logger) : IPromptRenderFilter + { + public async Task OnPromptRenderAsync(PromptRenderContext context, Func next) + { + await next(context); + + logger.LogTrace("Rendered prompt: {Prompt}", context.RenderedPrompt); + } + } + + /// + /// Filter which logs an information available during automatic function calling such as: + /// Chat history, number of functions to call, which functions to call and their arguments. + /// + private sealed class AutoFunctionInvocationLoggingFilter(ILogger logger) : IAutoFunctionInvocationFilter + { + public async Task OnAutoFunctionInvocationAsync(AutoFunctionInvocationContext context, Func next) + { + if (logger.IsEnabled(LogLevel.Trace)) + { + logger.LogTrace("ChatHistory: {ChatHistory}", JsonSerializer.Serialize(context.ChatHistory)); + } + + if (logger.IsEnabled(LogLevel.Debug)) + { + logger.LogDebug("Function count: {FunctionCount}", context.FunctionCount); + } + + var functionCalls = FunctionCallContent.GetFunctionCalls(context.ChatHistory.Last()).ToList(); + + if (logger.IsEnabled(LogLevel.Trace)) + { + functionCalls.ForEach(functionCall + => logger.LogTrace( + "Function call requests: {PluginName}-{FunctionName}({Arguments})", + functionCall.PluginName, + functionCall.FunctionName, + JsonSerializer.Serialize(functionCall.Arguments))); + } + + await next(context); + } + } +} diff --git a/dotnet/samples/Concepts/Functions/MethodFunctions_Yaml.cs b/dotnet/samples/Concepts/Functions/MethodFunctions_Yaml.cs new file mode 100644 index 000000000000..d02bc2ff311c --- /dev/null +++ b/dotnet/samples/Concepts/Functions/MethodFunctions_Yaml.cs @@ -0,0 +1,73 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Reflection; +using Microsoft.SemanticKernel; + +namespace Functions; + +public class MethodFunctions_Yaml(ITestOutputHelper output) : BaseTest(output) +{ + private const string FunctionConfig = """ + name: ValidateTaskId + description: Validate a task id. + input_variables: + - name: kernel + description: Kernel instance. + - name: taskId + description: Task identifier. + is_required: true + output_variable: + description: String indicating whether or not the task id is valid. + """; + + /// + /// This example create a plugin and uses a separate configuration file for the function metadata. + /// + /// + /// Some reasons you would want to do this: + /// 1. It's not possible to modify the existing code to add the KernelFunction attribute. + /// 2. You want to keep the function metadata separate from the function implementation. + /// + [Fact] + public async Task CreateFunctionFromMethodWithYamlConfigAsync() + { + var kernel = new Kernel(); + + var config = KernelFunctionYaml.ToPromptTemplateConfig(FunctionConfig); + + var target = new ValidatorPlugin(); + MethodInfo method = target.GetType().GetMethod(config.Name!)!; + var functions = new List(); + var functionName = config.Name; + var description = config.Description; + var parameters = config.InputVariables; + functions.Add(KernelFunctionFactory.CreateFromMethod(method, target, new() + { + FunctionName = functionName, + Description = description, + Parameters = parameters.Select(p => new KernelParameterMetadata(p.Name) { Description = p.Description, IsRequired = p.IsRequired }).ToList(), + })); + + var plugin = kernel.ImportPluginFromFunctions("ValidatorPlugin", functions); + + var function = plugin["ValidateTaskId"]; + var result = await kernel.InvokeAsync(function, new() { { "taskId", "1234" } }); + Console.WriteLine(result.GetValue()); + + Console.WriteLine("Function Metadata:"); + Console.WriteLine(function.Metadata.Description); + Console.WriteLine(function.Metadata.Parameters[0].Description); + Console.WriteLine(function.Metadata.Parameters[1].Description); + } + + /// + /// Plugin example with no KernelFunction or Description attributes. + /// + private sealed class ValidatorPlugin + { + public string ValidateTaskId(Kernel kernel, string taskId) + { + return taskId.Equals("1234", StringComparison.Ordinal) ? "Valid task id" : "Invalid task id"; + } + } +} diff --git a/dotnet/samples/Concepts/Optimization/PluginSelection.cs b/dotnet/samples/Concepts/Optimization/PluginSelection.cs new file mode 100644 index 000000000000..70c55456e72d --- /dev/null +++ b/dotnet/samples/Concepts/Optimization/PluginSelection.cs @@ -0,0 +1,414 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.ComponentModel; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; +using Microsoft.SemanticKernel; +using Microsoft.SemanticKernel.ChatCompletion; +using Microsoft.SemanticKernel.Connectors.OpenAI; +using Microsoft.SemanticKernel.Embeddings; +using Microsoft.SemanticKernel.Memory; + +namespace Optimization; + +/// +/// Single kernel instance may have multiple imported plugins/functions. It's possible to enable automatic function calling, +/// so AI model will decide which functions to call for specific request. +/// In case there are a lot of plugins/functions in application, some of them (or all of them) need to be shared with the model. +/// This example shows how to use different plugin/function selection strategies, to share with AI only those functions, +/// which are related to specific request. +/// This technique should decrease token usage, as fewer functions will be shared with AI. +/// It also helps to handle the scenario with a general purpose chat experience for a large enterprise, +/// where there are so many plugins, that it's impossible to share all of them with AI model in a single request. +/// +public sealed class PluginSelection(ITestOutputHelper output) : BaseTest(output) +{ + /// + /// This method shows how to select best functions to share with AI using vector similarity search. + /// + [Fact] + public async Task UsingVectorSearchWithKernelAsync() + { + // Initialize kernel with chat completion and embedding generation services. + // It's possible to combine different models from different AI providers to achieve the lowest token usage. + var builder = Kernel + .CreateBuilder() + .AddOpenAIChatCompletion("gpt-4", TestConfiguration.OpenAI.ApiKey) + .AddOpenAITextEmbeddingGeneration("text-embedding-3-small", TestConfiguration.OpenAI.ApiKey); + + // Add logging. + var logger = this.LoggerFactory.CreateLogger(); + builder.Services.AddSingleton(logger); + + // Add memory store to keep functions and search for the most relevant ones for specific request. + builder.Services.AddSingleton(); + + // Add helper components defined in this example. + builder.Services.AddSingleton(); + builder.Services.AddSingleton(); + builder.Services.AddSingleton(); + + var kernel = builder.Build(); + + // Import plugins with different features. + kernel.ImportPluginFromType(); + kernel.ImportPluginFromType(); + kernel.ImportPluginFromType(); + kernel.ImportPluginFromType(); + kernel.ImportPluginFromType(); + + // Get registered plugin store to save information about plugins. + var pluginStore = kernel.GetRequiredService(); + + // Save information about kernel plugins in plugin store. + const string CollectionName = "functions"; + + await pluginStore.SaveAsync(CollectionName, kernel.Plugins); + + // Enable automatic function calling by default. + var executionSettings = new OpenAIPromptExecutionSettings { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions }; + + // Define kernel arguments with specific request. + var kernelArguments = new KernelArguments(executionSettings) { ["Request"] = "Provide latest headlines" }; + + // Invoke the request without plugin selection filter first for comparison purposes. + Console.WriteLine("Run without filter:"); + var result = await kernel.InvokePromptAsync("{{$Request}}", kernelArguments); + + Console.WriteLine(result); + Console.WriteLine(result.Metadata?["Usage"]?.AsJson()); // All functions were shared with AI. Total tokens: ~250 + + // Define plugin selection filter. + var filter = new PluginSelectionFilter( + functionProvider: kernel.GetRequiredService(), + logger: kernel.GetRequiredService(), + collectionName: CollectionName, + numberOfBestFunctions: 1); + + // Add filter to kernel. + kernel.FunctionInvocationFilters.Add(filter); + + // Invoke the request with plugin selection filter. + Console.WriteLine("\nRun with filter:"); + + // ToolCallBehavior.AutoInvokeKernelFunctions is used here as well as defined above. + // In case there will be related functions found for specific request, the ToolCallBehavior will be updated in filter to + // ToolCallBehavior.EnableFunctions(functions, autoInvoke: true) - this will allow to share only related set of functions with AI. + result = await kernel.InvokePromptAsync("{{$Request}}", kernelArguments); + + Console.WriteLine(result); + Console.WriteLine(result.Metadata?["Usage"]?.AsJson()); // Just one function was shared with AI. Total tokens: ~150 + } + + [Fact] + public async Task UsingVectorSearchWithChatCompletionAsync() + { + // Initialize kernel with chat completion and embedding generation services. + // It's possible to combine different models from different AI providers to achieve the lowest token usage. + var builder = Kernel + .CreateBuilder() + .AddOpenAIChatCompletion("gpt-4", TestConfiguration.OpenAI.ApiKey) + .AddOpenAITextEmbeddingGeneration("text-embedding-3-small", TestConfiguration.OpenAI.ApiKey); + + // Add logging. + var logger = this.LoggerFactory.CreateLogger(); + builder.Services.AddSingleton(logger); + + // Add memory store to keep functions and search for the most relevant ones for specific request. + builder.Services.AddSingleton(); + + // Add helper components defined in this example. + builder.Services.AddSingleton(); + builder.Services.AddSingleton(); + builder.Services.AddSingleton(); + + var kernel = builder.Build(); + + // Import plugins with different features. + kernel.ImportPluginFromType(); + kernel.ImportPluginFromType(); + kernel.ImportPluginFromType(); + kernel.ImportPluginFromType(); + kernel.ImportPluginFromType(); + + // Get registered plugin store to save information about plugins. + var pluginStore = kernel.GetRequiredService(); + + // Store information about kernel plugins in plugin store. + const string CollectionName = "functions"; + + await pluginStore.SaveAsync(CollectionName, kernel.Plugins); + + // Enable automatic function calling by default. + var executionSettings = new OpenAIPromptExecutionSettings { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions }; + + // Get function provider and find best functions for specified prompt. + var functionProvider = kernel.GetRequiredService(); + + const string Prompt = "Provide latest headlines"; + + var bestFunctions = await functionProvider.GetBestFunctionsAsync(CollectionName, Prompt, kernel.Plugins, numberOfBestFunctions: 1); + + // If any found, update execution settings to share only selected functions. + if (bestFunctions.Count > 0) + { + bestFunctions.ForEach(function + => logger.LogInformation("Best function found: {PluginName}-{FunctionName}", function.PluginName, function.Name)); + + // Convert selected functions to OpenAI functions. + var openAIFunctions = bestFunctions.Select(function => function.Metadata.ToOpenAIFunction()); + + // Share only selected functions with AI. + executionSettings.ToolCallBehavior = ToolCallBehavior.EnableFunctions(openAIFunctions, autoInvoke: true); + } + + // Get chat completion service and execute a request. + var chatCompletionService = kernel.GetRequiredService(); + + var chatHistory = new ChatHistory(); + chatHistory.AddUserMessage(Prompt); + + var result = await chatCompletionService.GetChatMessageContentAsync(chatHistory, executionSettings, kernel); + + Console.WriteLine(result); + Console.WriteLine(result.Metadata?["Usage"]?.AsJson()); // Just one function was shared with AI. Total tokens: ~150 + } + + /// + /// Filter which performs vector similarity search on imported functions in + /// to select the best ones to share with AI. + /// + private sealed class PluginSelectionFilter( + IFunctionProvider functionProvider, + ILogger logger, + string collectionName, + int numberOfBestFunctions) : IFunctionInvocationFilter + { + public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func next) + { + var request = GetRequestArgument(context.Arguments); + + // Execute plugin selection logic for "InvokePrompt" function only, as main entry point. + if (context.Function.Name.Contains(nameof(KernelExtensions.InvokePromptAsync)) && !string.IsNullOrWhiteSpace(request)) + { + // Get imported plugins in kernel. + var plugins = context.Kernel.Plugins; + + // Find best functions for original request. + var bestFunctions = await functionProvider.GetBestFunctionsAsync(collectionName, request, plugins, numberOfBestFunctions); + + // If any found, update execution settings and execute the request. + if (bestFunctions.Count > 0) + { + bestFunctions.ForEach(function + => logger.LogInformation("Best function found: {PluginName}-{FunctionName}", function.PluginName, function.Name)); + + var updatedExecutionSettings = GetExecutionSettings(context.Arguments, bestFunctions); + + if (updatedExecutionSettings is not null) + { + // Update execution settings. + context.Arguments.ExecutionSettings = updatedExecutionSettings; + + // Execute the request. + await next(context); + + return; + } + } + } + + // Otherwise, execute a request with default logic, where all plugins will be shared. + await next(context); + } + + private static Dictionary? GetExecutionSettings(KernelArguments arguments, List functions) + { + var promptExecutionSettings = arguments.ExecutionSettings?[PromptExecutionSettings.DefaultServiceId]; + + if (promptExecutionSettings is not null && promptExecutionSettings is OpenAIPromptExecutionSettings openAIPromptExecutionSettings) + { + // Convert selected functions to OpenAI functions. + var openAIFunctions = functions.Select(function => function.Metadata.ToOpenAIFunction()); + + // Share only selected functions with AI. + openAIPromptExecutionSettings.ToolCallBehavior = ToolCallBehavior.EnableFunctions(openAIFunctions, autoInvoke: true); + + return new() { [PromptExecutionSettings.DefaultServiceId] = openAIPromptExecutionSettings }; + } + + return null; + } + + private static string? GetRequestArgument(KernelArguments arguments) + => arguments.TryGetValue("Request", out var requestObj) && requestObj is string request ? request : null; + } + + #region Helper components + + /// + /// Helper function key provider. + /// + public interface IFunctionKeyProvider + { + string GetFunctionKey(KernelFunction kernelFunction); + } + + /// + /// Helper function provider to get best functions for specific request. + /// + public interface IFunctionProvider + { + Task> GetBestFunctionsAsync( + string collectionName, + string request, + KernelPluginCollection plugins, + int numberOfBestFunctions); + } + + /// + /// Helper plugin store to save information about imported plugins in vector database. + /// + public interface IPluginStore + { + Task SaveAsync(string collectionName, KernelPluginCollection plugins); + } + + public class FunctionKeyProvider : IFunctionKeyProvider + { + public string GetFunctionKey(KernelFunction kernelFunction) + { + return !string.IsNullOrWhiteSpace(kernelFunction.PluginName) ? + $"{kernelFunction.PluginName}-{kernelFunction.Name}" : + kernelFunction.Name; + } + } + + public class FunctionProvider( + ITextEmbeddingGenerationService textEmbeddingGenerationService, + IMemoryStore memoryStore, + IFunctionKeyProvider functionKeyProvider) : IFunctionProvider + { + public async Task> GetBestFunctionsAsync( + string collectionName, + string request, + KernelPluginCollection plugins, + int numberOfBestFunctions) + { + // Generate embedding for original request. + var requestEmbedding = await textEmbeddingGenerationService.GenerateEmbeddingAsync(request); + + // Find best functions to call for original request. + var memoryRecordKeys = await memoryStore + .GetNearestMatchesAsync(collectionName, requestEmbedding, limit: numberOfBestFunctions) + .Select(l => l.Item1.Key) + .ToListAsync(); + + return plugins + .SelectMany(plugin => plugin) + .Where(function => memoryRecordKeys.Contains(functionKeyProvider.GetFunctionKey(function))) + .ToList(); + } + } + + public class PluginStore( + ITextEmbeddingGenerationService textEmbeddingGenerationService, + IMemoryStore memoryStore, + IFunctionKeyProvider functionKeyProvider) : IPluginStore + { + public async Task SaveAsync(string collectionName, KernelPluginCollection plugins) + { + // Collect data about imported functions in kernel. + var memoryRecords = new List(); + var functionsData = GetFunctionsData(plugins); + + // Generate embedding for each function. + var embeddings = await textEmbeddingGenerationService + .GenerateEmbeddingsAsync(functionsData.Select(l => l.TextToVectorize).ToArray()); + + // Create memory record instances with function information and embedding. + for (var i = 0; i < functionsData.Count; i++) + { + var (function, textToVectorize) = functionsData[i]; + + memoryRecords.Add(MemoryRecord.LocalRecord( + id: functionKeyProvider.GetFunctionKey(function), + text: textToVectorize, + description: null, + embedding: embeddings[i])); + } + + // Create collection and upsert all memory records for search. + // It's possible to do it only once and re-use the same functions for future requests. + await memoryStore.CreateCollectionAsync(collectionName); + await memoryStore.UpsertBatchAsync(collectionName, memoryRecords).ToListAsync(); + } + + private static List<(KernelFunction Function, string TextToVectorize)> GetFunctionsData(KernelPluginCollection plugins) + => plugins + .SelectMany(plugin => plugin) + .Select(function => (function, $"Plugin name: {function.PluginName}. Function name: {function.Name}. Description: {function.Description}")) + .ToList(); + } + + #endregion + + #region Sample Plugins + + private sealed class TimePlugin + { + [KernelFunction, Description("Provides the current date and time.")] + public string GetCurrentTime() => DateTime.Now.ToString("R"); + } + + private sealed class WeatherPlugin + { + [KernelFunction, Description("Provides weather information for various cities.")] + public string GetWeather(string cityName) => cityName switch + { + "Boston" => "61 and rainy", + "London" => "55 and cloudy", + "Miami" => "80 and sunny", + "Paris" => "60 and rainy", + "Tokyo" => "50 and sunny", + "Sydney" => "75 and sunny", + "Tel Aviv" => "80 and sunny", + _ => "No information", + }; + } + + private sealed class EmailPlugin(ILogger logger) + { + [KernelFunction, Description("Sends email to recipient with subject and body.")] + public void SendEmail(string from, string to, string subject, string body) + { + logger.LogInformation("Email has been sent successfully."); + } + } + + private sealed class NewsPlugin + { + [KernelFunction, Description("Provides the latest news headlines.")] + public List GetLatestHeadlines() => new() + { + "Tourism Industry Sees Record Growth", + "Tech Company Releases New Product", + "Sports Team Wins Championship", + "New Study Reveals Health Benefits of Walking" + }; + } + + private sealed class CalendarPlugin + { + [KernelFunction, Description("Provides a list of upcoming events.")] + public List GetUpcomingEvents() => new() + { + "Meeting with Bob on June 22", + "Project deadline on June 30", + "Dentist appointment on July 5", + "Vacation starts on July 12" + }; + } + + #endregion +} diff --git a/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenAI_AzureKeyVault.cs b/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenAI_AzureKeyVault.cs index f351f9af2636..e6c94622ddd6 100644 --- a/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenAI_AzureKeyVault.cs +++ b/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenAI_AzureKeyVault.cs @@ -11,6 +11,7 @@ namespace Plugins; +[Obsolete("OpenAI plugins are deprecated and will be removed in a future version.")] public class CreatePluginFromOpenAI_AzureKeyVault(ITestOutputHelper output) : BaseTest(output) { private const string SecretName = "Foo"; @@ -118,6 +119,7 @@ private async Task GetSecretFromAzureKeyVaultWithRetryAsync(Kernel kernel, Kerne /// /// Provides authentication for HTTP requests to OpenAI using OAuth or verification tokens. /// +[Obsolete("OpenAI plugins are deprecated and will be removed in a future version.")] internal sealed class OpenAIAuthenticationProvider(Dictionary>? oAuthValues = null, Dictionary? credentials = null) { private readonly Dictionary> _oAuthValues = oAuthValues ?? []; diff --git a/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Github.cs b/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Github.cs index 5445f52b16c4..7d17658ad214 100644 --- a/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Github.cs +++ b/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Github.cs @@ -53,7 +53,7 @@ public async Task RunOpenAIPluginWithMetadataAsync() WriteStringToStream(schemaStream, schema); // Import an Open API plugin from a stream. - var plugin = await kernel.CreatePluginFromOpenApiAsync("GithubVersionsApi", schemaStream, new OpenAIFunctionExecutionParameters(httpClient)); + var plugin = await kernel.CreatePluginFromOpenApiAsync("GithubVersionsApi", schemaStream, new OpenApiFunctionExecutionParameters(httpClient)); // Get the function to be invoked and its metadata and extension properties. var function = plugin["getVersions"]; diff --git a/dotnet/samples/Concepts/Plugins/OpenAIPlugins.cs b/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Klarna.cs similarity index 84% rename from dotnet/samples/Concepts/Plugins/OpenAIPlugins.cs rename to dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Klarna.cs index 77846b0d5290..40975f0df6ef 100644 --- a/dotnet/samples/Concepts/Plugins/OpenAIPlugins.cs +++ b/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Klarna.cs @@ -5,16 +5,16 @@ namespace Plugins; -public class OpenAIPlugins(ITestOutputHelper output) : BaseTest(output) +public class CreatePluginFromOpenApiSpec_Klarna(ITestOutputHelper output) : BaseTest(output) { /// - /// This sample shows how to invoke an OpenAI plugin. + /// This sample shows how to invoke an OpenApi plugin. /// /// /// You must provide the plugin name and a URI to the Open API manifest before running this sample. /// [Fact(Skip = "Run it only after filling the template below")] - public async Task InvokeOpenAIPluginAsync() + public async Task InvokeOpenApiPluginAsync() { Kernel kernel = new(); @@ -22,7 +22,7 @@ public async Task InvokeOpenAIPluginAsync() using HttpClient httpClient = new(); // Import an Open AI plugin via URI - var plugin = await kernel.ImportPluginFromOpenAIAsync("", new Uri(""), new OpenAIFunctionExecutionParameters(httpClient)); + var plugin = await kernel.ImportPluginFromOpenApiAsync("", new Uri(""), new OpenApiFunctionExecutionParameters(httpClient)); // Add arguments for required parameters, arguments for optional ones can be skipped. var arguments = new KernelArguments { [""] = "" }; @@ -39,11 +39,11 @@ public async Task InvokeOpenAIPluginAsync() /// This sample shows how to invoke the Klarna Get Products function as an OpenAPI plugin. /// [Fact] - public async Task InvokeKlarnaGetProductsAsOpenAPIPluginAsync() + public async Task InvokeKlarnaGetProductsAsOpenApiPluginAsync() { Kernel kernel = new(); - var plugin = await kernel.ImportPluginFromOpenAIAsync("Klarna", new Uri("https://www.klarna.com/.well-known/ai-plugin.json")); + var plugin = await kernel.ImportPluginFromOpenApiAsync("Klarna", new Uri("https://www.klarna.com/us/shopping/public/openai/v0/api-docs/")); var arguments = new KernelArguments { @@ -70,7 +70,7 @@ public async Task InvokeKlarnaGetProductsAsOpenAPIPluginAsync() /// The contains the , and . /// [Fact] - public async Task UseDelegatingHandlerWhenInvokingAnOpenAPIFunctionAsync() + public async Task UseDelegatingHandlerWhenInvokingAnOpenApiFunctionAsync() { using var httpHandler = new HttpClientHandler(); using var customHandler = new CustomHandler(httpHandler); @@ -78,7 +78,7 @@ public async Task UseDelegatingHandlerWhenInvokingAnOpenAPIFunctionAsync() Kernel kernel = new(); - var plugin = await kernel.ImportPluginFromOpenAIAsync("Klarna", new Uri("https://www.klarna.com/.well-known/ai-plugin.json"), new OpenAIFunctionExecutionParameters(httpClient)); + var plugin = await kernel.ImportPluginFromOpenApiAsync("Klarna", new Uri("https://www.klarna.com/us/shopping/public/openai/v0/api-docs/"), new OpenApiFunctionExecutionParameters(httpClient)); var arguments = new KernelArguments { diff --git a/dotnet/samples/Concepts/README.md b/dotnet/samples/Concepts/README.md index 7eaa2a8a7ae6..fea33c88822e 100644 --- a/dotnet/samples/Concepts/README.md +++ b/dotnet/samples/Concepts/README.md @@ -69,6 +69,7 @@ Down below you can find the code snippets that demonstrate the usage of many Sem - [PromptRenderFiltering](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Filtering/PromptRenderFiltering.cs) - [RetryWithFilters](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Filtering/RetryWithFilters.cs) - [PIIDetectionWithFilters](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Filtering/PIIDetectionWithFilters.cs) +- [TelemetryWithFilters](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Filtering/TelemetryWithFilters.cs) ## Functions - Invoking [`Method`](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromMethod.cs) or [`Prompt`](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromPrompt.cs) functions with [`Kernel`](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Abstractions/Kernel.cs) @@ -78,6 +79,7 @@ Down below you can find the code snippets that demonstrate the usage of many Sem - [MethodFunctions](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Functions/MethodFunctions.cs) - [MethodFunctions_Advanced](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Functions/MethodFunctions_Advanced.cs) - [MethodFunctions_Types](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Functions/MethodFunctions_Types.cs) +- [MethodFunctions_Yaml](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Functions/MethodFunctions_Yaml.cs) - [PromptFunctions_Inline](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Functions/PromptFunctions_Inline.cs) - [PromptFunctions_MultipleArguments](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Functions/PromptFunctions_MultipleArguments.cs) @@ -103,6 +105,7 @@ Down below you can find the code snippets that demonstrate the usage of many Sem ## Optimization - Examples of different cost and performance optimization techniques - [FrugalGPT](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Optimization/FrugalGPT.cs) +- [PluginSelection](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Optimization/PluginSelection.cs) ## Planners - Examples on using `Planners` @@ -113,14 +116,14 @@ Down below you can find the code snippets that demonstrate the usage of many Sem - [ApiManifestBasedPlugins](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/ApiManifestBasedPlugins.cs) - [ConversationSummaryPlugin](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/ConversationSummaryPlugin.cs) -- [CreatePluginFromOpenAI_AzureKeyVault](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenAI_AzureKeyVault.cs) +- [CreatePluginFromOpenAI_AzureKeyVault](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenAI_AzureKeyVault.cs)(Deprecated) - [CreatePluginFromOpenApiSpec_Github](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Github.cs) - [CreatePluginFromOpenApiSpec_Jira](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Jira.cs) +- [CreatePluginFromOpenApiSpec_Klarna](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/CreatePluginFromOpenApiSpec_Klarna.cs) - [CustomMutablePlugin](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/CustomMutablePlugin.cs) - [DescribeAllPluginsAndFunctions](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/DescribeAllPluginsAndFunctions.cs) - [GroundednessChecks](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/GroundednessChecks.cs) - [ImportPluginFromGrpc](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/ImportPluginFromGrpc.cs) -- [OpenAIPlugins](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Plugins/OpenAIPlugins.cs) ## PromptTemplates - Using [`Templates`](https://github.com/microsoft/semantic-kernel/blob/main/dotnet/src/SemanticKernel.Abstractions/PromptTemplate/IPromptTemplate.cs) with parametrization for `Prompt` rendering diff --git a/dotnet/samples/Demos/FunctionInvocationApproval/README.md b/dotnet/samples/Demos/FunctionInvocationApproval/README.md new file mode 100644 index 000000000000..99ff202e45fd --- /dev/null +++ b/dotnet/samples/Demos/FunctionInvocationApproval/README.md @@ -0,0 +1,44 @@ +# Function Invocation Approval + +This console application shows how to use function invocation filter (`IFunctionInvocationFilter`) to invoke a Kernel Function only if such operation was approved. +If function invocation was rejected, the result will contain the reason why, so the LLM can respond appropriately. + +The application uses a sample plugin which builds software by following these development stages: collection of requirements, design, implementation, testing and deployment. + +Each step can be approved or rejected. Based on that, the LLM will decide how to proceed. + +## Configuring Secrets + +The example requires credentials to access OpenAI or Azure OpenAI. + +If you have set up those credentials as secrets within Secret Manager or through environment variables for other samples from the solution in which this project is found, they will be re-used. + +### To set your secrets with Secret Manager: + +``` +cd dotnet/samples/Demos/FunctionInvocationApproval + +dotnet user-secrets init + +dotnet user-secrets set "OpenAI:ChatModelId" "..." +dotnet user-secrets set "OpenAI:ApiKey" "..." + +dotnet user-secrets set "AzureOpenAI:ChatDeploymentName" "..." +dotnet user-secrets set "AzureOpenAI:Endpoint" "https://... .openai.azure.com/" +dotnet user-secrets set "AzureOpenAI:ApiKey" "..." +``` + +### To set your secrets with environment variables + +Use these names: + +``` +# OpenAI +OpenAI__ChatModelId +OpenAI__ApiKey + +# Azure OpenAI +AzureOpenAI__ChatDeploymentName +AzureOpenAI__Endpoint +AzureOpenAI__ApiKey +``` diff --git a/dotnet/samples/Demos/HomeAutomation/README.md b/dotnet/samples/Demos/HomeAutomation/README.md index 09907e5363e5..aa5c33cec248 100644 --- a/dotnet/samples/Demos/HomeAutomation/README.md +++ b/dotnet/samples/Demos/HomeAutomation/README.md @@ -12,7 +12,7 @@ If you have set up those credentials as secrets within Secret Manager or through ### To set your secrets with Secret Manager: ``` -cd dotnet/samples/HouseAutomation +cd dotnet/samples/Demos/HouseAutomation dotnet user-secrets init diff --git a/dotnet/samples/GettingStartedWithAgents/Step1_Agent.cs b/dotnet/samples/GettingStartedWithAgents/Step1_Agent.cs index 7ecfb2c5348f..ddab79f032b0 100644 --- a/dotnet/samples/GettingStartedWithAgents/Step1_Agent.cs +++ b/dotnet/samples/GettingStartedWithAgents/Step1_Agent.cs @@ -15,7 +15,7 @@ public class Step1_Agent(ITestOutputHelper output) : BaseTest(output) private const string ParrotInstructions = "Repeat the user message in the voice of a pirate and then end with a parrot sound."; [Fact] - public async Task RunAsync() + public async Task UseSingleChatCompletionAgentAsync() { // Define the agent ChatCompletionAgent agent = @@ -27,7 +27,7 @@ public async Task RunAsync() }; /// Create a chat for agent interaction. For more, . - AgentGroupChat chat = new(); + ChatHistory chat = []; // Respond to user input await InvokeAgentAsync("Fortune favors the bold."); @@ -37,11 +37,11 @@ public async Task RunAsync() // Local function to invoke agent and display the conversation messages. async Task InvokeAgentAsync(string input) { - chat.AddChatMessage(new ChatMessageContent(AuthorRole.User, input)); + chat.Add(new ChatMessageContent(AuthorRole.User, input)); Console.WriteLine($"# {AuthorRole.User}: '{input}'"); - await foreach (var content in chat.InvokeAsync(agent)) + await foreach (ChatMessageContent content in agent.InvokeAsync(chat)) { Console.WriteLine($"# {content.Role} - {content.AuthorName ?? "*"}: '{content.Content}'"); } diff --git a/dotnet/samples/GettingStartedWithAgents/Step2_Plugins.cs b/dotnet/samples/GettingStartedWithAgents/Step2_Plugins.cs index 708fab321f04..61737de498be 100644 --- a/dotnet/samples/GettingStartedWithAgents/Step2_Plugins.cs +++ b/dotnet/samples/GettingStartedWithAgents/Step2_Plugins.cs @@ -17,7 +17,7 @@ public class Step2_Plugins(ITestOutputHelper output) : BaseTest(output) private const string HostInstructions = "Answer questions about the menu."; [Fact] - public async Task RunAsync() + public async Task UseChatCompletionWithPluginAgentAsync() { // Define the agent ChatCompletionAgent agent = @@ -34,7 +34,7 @@ public async Task RunAsync() agent.Kernel.Plugins.Add(plugin); /// Create a chat for agent interaction. For more, . - AgentGroupChat chat = new(); + ChatHistory chat = []; // Respond to user input, invoking functions where appropriate. await InvokeAgentAsync("Hello"); @@ -45,10 +45,10 @@ public async Task RunAsync() // Local function to invoke agent and display the conversation messages. async Task InvokeAgentAsync(string input) { - chat.AddChatMessage(new ChatMessageContent(AuthorRole.User, input)); + chat.Add(new ChatMessageContent(AuthorRole.User, input)); Console.WriteLine($"# {AuthorRole.User}: '{input}'"); - await foreach (var content in chat.InvokeAsync(agent)) + await foreach (var content in agent.InvokeAsync(chat)) { Console.WriteLine($"# {content.Role} - {content.AuthorName ?? "*"}: '{content.Content}'"); } diff --git a/dotnet/samples/GettingStartedWithAgents/Step3_Chat.cs b/dotnet/samples/GettingStartedWithAgents/Step3_Chat.cs index c539532ef52c..0c9c60f870a7 100644 --- a/dotnet/samples/GettingStartedWithAgents/Step3_Chat.cs +++ b/dotnet/samples/GettingStartedWithAgents/Step3_Chat.cs @@ -34,7 +34,7 @@ Consider suggestions when refining an idea. """; [Fact] - public async Task RunAsync() + public async Task UseAgentGroupChatWithTwoAgentsAsync() { // Define the agents ChatCompletionAgent agentReviewer = diff --git a/dotnet/samples/GettingStartedWithAgents/Step4_KernelFunctionStrategies.cs b/dotnet/samples/GettingStartedWithAgents/Step4_KernelFunctionStrategies.cs index 06dfe0fcc4ed..cd99531ec27b 100644 --- a/dotnet/samples/GettingStartedWithAgents/Step4_KernelFunctionStrategies.cs +++ b/dotnet/samples/GettingStartedWithAgents/Step4_KernelFunctionStrategies.cs @@ -33,7 +33,7 @@ Consider suggestions when refining an idea. """; [Fact] - public async Task RunAsync() + public async Task UseKernelFunctionStrategiesWithAgentGroupChatAsync() { // Define the agents ChatCompletionAgent agentReviewer = diff --git a/dotnet/samples/GettingStartedWithAgents/Step5_JsonResult.cs b/dotnet/samples/GettingStartedWithAgents/Step5_JsonResult.cs index e5ec480f8773..b1e83a202505 100644 --- a/dotnet/samples/GettingStartedWithAgents/Step5_JsonResult.cs +++ b/dotnet/samples/GettingStartedWithAgents/Step5_JsonResult.cs @@ -28,7 +28,7 @@ Think step-by-step and rate the user input on creativity and expressivness from """; [Fact] - public async Task RunAsync() + public async Task UseKernelFunctionStrategiesWithJsonResultAsync() { // Define the agents ChatCompletionAgent agent = diff --git a/dotnet/samples/GettingStartedWithAgents/Step6_DependencyInjection.cs b/dotnet/samples/GettingStartedWithAgents/Step6_DependencyInjection.cs index c759053dbe1c..a7e3b9b41450 100644 --- a/dotnet/samples/GettingStartedWithAgents/Step6_DependencyInjection.cs +++ b/dotnet/samples/GettingStartedWithAgents/Step6_DependencyInjection.cs @@ -30,7 +30,7 @@ Think step-by-step and rate the user input on creativity and expressivness from """; [Fact] - public async Task RunAsync() + public async Task UseDependencyInjectionToCreateAgentAsync() { ServiceCollection serviceContainer = new(); diff --git a/dotnet/samples/GettingStartedWithAgents/Step7_Logging.cs b/dotnet/samples/GettingStartedWithAgents/Step7_Logging.cs index 4b8b48c5ef87..4372d71e37f8 100644 --- a/dotnet/samples/GettingStartedWithAgents/Step7_Logging.cs +++ b/dotnet/samples/GettingStartedWithAgents/Step7_Logging.cs @@ -37,7 +37,7 @@ Consider suggestions when refining an idea. """; [Fact] - public async Task RunAsync() + public async Task UseLoggerFactoryWithAgentGroupChatAsync() { // Define the agents ChatCompletionAgent agentReviewer = @@ -46,6 +46,7 @@ public async Task RunAsync() Instructions = ReviewerInstructions, Name = ReviewerName, Kernel = this.CreateKernelWithChatCompletion(), + LoggerFactory = this.LoggerFactory, }; ChatCompletionAgent agentWriter = @@ -54,6 +55,7 @@ public async Task RunAsync() Instructions = CopyWriterInstructions, Name = CopyWriterName, Kernel = this.CreateKernelWithChatCompletion(), + LoggerFactory = this.LoggerFactory, }; // Create a chat for agent interaction. diff --git a/dotnet/samples/GettingStartedWithAgents/Step8_OpenAIAssistant.cs b/dotnet/samples/GettingStartedWithAgents/Step8_OpenAIAssistant.cs index 32ce38da8b2f..09afcfc44826 100644 --- a/dotnet/samples/GettingStartedWithAgents/Step8_OpenAIAssistant.cs +++ b/dotnet/samples/GettingStartedWithAgents/Step8_OpenAIAssistant.cs @@ -18,7 +18,7 @@ public class Step8_OpenAIAssistant(ITestOutputHelper output) : BaseTest(output) private const string HostInstructions = "Answer questions about the menu."; [Fact] - public async Task RunAsync() + public async Task UseSingleOpenAIAssistantAgentAsync() { // Define the agent OpenAIAssistantAgent agent = @@ -37,7 +37,7 @@ await OpenAIAssistantAgent.CreateAsync( agent.Kernel.Plugins.Add(plugin); // Create a chat for agent interaction. - var chat = new AgentGroupChat(); + string threadId = await agent.CreateThreadAsync(); // Respond to user input try @@ -49,19 +49,23 @@ await OpenAIAssistantAgent.CreateAsync( } finally { + await agent.DeleteThreadAsync(threadId); await agent.DeleteAsync(); } // Local function to invoke agent and display the conversation messages. async Task InvokeAgentAsync(string input) { - chat.AddChatMessage(new ChatMessageContent(AuthorRole.User, input)); + await agent.AddChatMessageAsync(threadId, new ChatMessageContent(AuthorRole.User, input)); Console.WriteLine($"# {AuthorRole.User}: '{input}'"); - await foreach (var content in chat.InvokeAsync(agent)) + await foreach (var content in agent.InvokeAsync(threadId)) { - Console.WriteLine($"# {content.Role} - {content.AuthorName ?? "*"}: '{content.Content}'"); + if (content.Role != AuthorRole.Tool) + { + Console.WriteLine($"# {content.Role} - {content.AuthorName ?? "*"}: '{content.Content}'"); + } } } } diff --git a/dotnet/src/Agents/Abstractions/Agent.cs b/dotnet/src/Agents/Abstractions/Agent.cs index 4ebe3d1416cf..8af2de3b0869 100644 --- a/dotnet/src/Agents/Abstractions/Agent.cs +++ b/dotnet/src/Agents/Abstractions/Agent.cs @@ -4,6 +4,7 @@ using System.Threading; using System.Threading.Tasks; using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; namespace Microsoft.SemanticKernel.Agents; @@ -36,6 +37,16 @@ public abstract class Agent /// public string? Name { get; init; } + /// + /// A for this . + /// + public ILoggerFactory LoggerFactory { get; init; } = NullLoggerFactory.Instance; + + /// + /// The associated with this . + /// + protected ILogger Logger => this._logger ??= this.LoggerFactory.CreateLogger(this.GetType()); + /// /// Set of keys to establish channel affinity. Minimum expected key-set: /// @@ -53,12 +64,13 @@ public abstract class Agent /// /// Produce the an appropriate for the agent type. /// - /// An agent specific logger. /// The to monitor for cancellation requests. The default is . /// An appropriate for the agent type. /// /// Every agent conversation, or , will establish one or more /// objects according to the specific type. /// - protected internal abstract Task CreateChannelAsync(ILogger logger, CancellationToken cancellationToken); + protected internal abstract Task CreateChannelAsync(CancellationToken cancellationToken); + + private ILogger? _logger; } diff --git a/dotnet/src/Agents/Abstractions/AgentChat.cs b/dotnet/src/Agents/Abstractions/AgentChat.cs index 26b51928c362..7e7dea00a805 100644 --- a/dotnet/src/Agents/Abstractions/AgentChat.cs +++ b/dotnet/src/Agents/Abstractions/AgentChat.cs @@ -256,10 +256,7 @@ async Task GetOrCreateChannelAsync() { this.Logger.LogDebug("[{MethodName}] Creating channel for {AgentType}: {AgentId}", nameof(InvokeAgentAsync), agent.GetType(), agent.Id); - // Creating an agent-typed logger for CreateChannelAsync - channel = await agent.CreateChannelAsync(this.LoggerFactory.CreateLogger(agent.GetType()), cancellationToken).ConfigureAwait(false); - // Creating an channel-typed logger for the channel - channel.Logger = this.LoggerFactory.CreateLogger(channel.GetType()); + channel = await agent.CreateChannelAsync(cancellationToken).ConfigureAwait(false); this._agentChannels.Add(channelKey, channel); diff --git a/dotnet/src/Agents/Abstractions/AggregatorAgent.cs b/dotnet/src/Agents/Abstractions/AggregatorAgent.cs index c236cd7a565a..00964fdc9e57 100644 --- a/dotnet/src/Agents/Abstractions/AggregatorAgent.cs +++ b/dotnet/src/Agents/Abstractions/AggregatorAgent.cs @@ -44,14 +44,14 @@ protected internal override IEnumerable GetChannelKeys() } /// - protected internal override Task CreateChannelAsync(ILogger logger, CancellationToken cancellationToken) + protected internal override Task CreateChannelAsync(CancellationToken cancellationToken) { - logger.LogDebug("[{MethodName}] Creating channel {ChannelType}", nameof(CreateChannelAsync), nameof(AggregatorChannel)); + this.Logger.LogDebug("[{MethodName}] Creating channel {ChannelType}", nameof(CreateChannelAsync), nameof(AggregatorChannel)); AgentChat chat = chatProvider.Invoke(); AggregatorChannel channel = new(chat); - logger.LogInformation("[{MethodName}] Created channel {ChannelType} ({ChannelMode}) with: {AgentChatType}", nameof(CreateChannelAsync), nameof(AggregatorChannel), this.Mode, chat.GetType()); + this.Logger.LogInformation("[{MethodName}] Created channel {ChannelType} ({ChannelMode}) with: {AgentChatType}", nameof(CreateChannelAsync), nameof(AggregatorChannel), this.Mode, chat.GetType()); return Task.FromResult(channel); } diff --git a/dotnet/src/Agents/Abstractions/ChatHistoryChannel.cs b/dotnet/src/Agents/Abstractions/ChatHistoryChannel.cs index 281529bffd8e..2bb5616ff959 100644 --- a/dotnet/src/Agents/Abstractions/ChatHistoryChannel.cs +++ b/dotnet/src/Agents/Abstractions/ChatHistoryChannel.cs @@ -25,7 +25,7 @@ protected internal sealed override async IAsyncEnumerable In throw new KernelException($"Invalid channel binding for agent: {agent.Id} ({agent.GetType().FullName})"); } - await foreach (var message in historyHandler.InvokeAsync(this._history, this.Logger, cancellationToken).ConfigureAwait(false)) + await foreach (ChatMessageContent message in historyHandler.InvokeAsync(this._history, cancellationToken).ConfigureAwait(false)) { this._history.Add(message); diff --git a/dotnet/src/Agents/Abstractions/ChatHistoryKernelAgent.cs b/dotnet/src/Agents/Abstractions/ChatHistoryKernelAgent.cs index ee86a7af770e..3de87da3de06 100644 --- a/dotnet/src/Agents/Abstractions/ChatHistoryKernelAgent.cs +++ b/dotnet/src/Agents/Abstractions/ChatHistoryKernelAgent.cs @@ -3,6 +3,7 @@ using System.Threading; using System.Threading.Tasks; using Microsoft.Extensions.Logging; +using Microsoft.SemanticKernel.ChatCompletion; namespace Microsoft.SemanticKernel.Agents; @@ -18,14 +19,24 @@ protected internal sealed override IEnumerable GetChannelKeys() } /// - protected internal sealed override Task CreateChannelAsync(ILogger logger, CancellationToken cancellationToken) + protected internal sealed override Task CreateChannelAsync(CancellationToken cancellationToken) { - return Task.FromResult(new ChatHistoryChannel()); + ChatHistoryChannel channel = + new() + { + Logger = this.LoggerFactory.CreateLogger() + }; + + return Task.FromResult(channel); } /// public abstract IAsyncEnumerable InvokeAsync( - IReadOnlyList history, - ILogger logger, + ChatHistory history, + CancellationToken cancellationToken = default); + + /// + public abstract IAsyncEnumerable InvokeStreamingAsync( + ChatHistory history, CancellationToken cancellationToken = default); } diff --git a/dotnet/src/Agents/Abstractions/IChatHistoryHandler.cs b/dotnet/src/Agents/Abstractions/IChatHistoryHandler.cs index f377d38ba58e..8b7dab748c81 100644 --- a/dotnet/src/Agents/Abstractions/IChatHistoryHandler.cs +++ b/dotnet/src/Agents/Abstractions/IChatHistoryHandler.cs @@ -1,7 +1,7 @@ // Copyright (c) Microsoft. All rights reserved. using System.Collections.Generic; using System.Threading; -using Microsoft.Extensions.Logging; +using Microsoft.SemanticKernel.ChatCompletion; namespace Microsoft.SemanticKernel.Agents; @@ -11,14 +11,22 @@ namespace Microsoft.SemanticKernel.Agents; public interface IChatHistoryHandler { /// - /// Entry point for calling into an agent from a a . + /// Entry point for calling into an agent from a . /// /// The chat history at the point the channel is created. - /// The logger associated with the /// The to monitor for cancellation requests. The default is . /// Asynchronous enumeration of messages. IAsyncEnumerable InvokeAsync( - IReadOnlyList history, - ILogger logger, + ChatHistory history, + CancellationToken cancellationToken = default); + + /// + /// Entry point for calling into an agent from a for streaming content. + /// + /// The chat history at the point the channel is created. + /// The to monitor for cancellation requests. The default is . + /// Asynchronous enumeration of streaming content. + public abstract IAsyncEnumerable InvokeStreamingAsync( + ChatHistory history, CancellationToken cancellationToken = default); } diff --git a/dotnet/src/Agents/Core/ChatCompletionAgent.cs b/dotnet/src/Agents/Core/ChatCompletionAgent.cs index e8f9378e8a39..b84d29494b8e 100644 --- a/dotnet/src/Agents/Core/ChatCompletionAgent.cs +++ b/dotnet/src/Agents/Core/ChatCompletionAgent.cs @@ -2,6 +2,7 @@ using System.Collections.Generic; using System.Runtime.CompilerServices; using System.Threading; +using System.Threading.Tasks; using Microsoft.Extensions.Logging; using Microsoft.SemanticKernel.ChatCompletion; @@ -23,22 +24,16 @@ public sealed class ChatCompletionAgent : ChatHistoryKernelAgent /// public override async IAsyncEnumerable InvokeAsync( - IReadOnlyList history, - ILogger logger, + ChatHistory history, [EnumeratorCancellation] CancellationToken cancellationToken = default) { - var chatCompletionService = this.Kernel.GetRequiredService(); + IChatCompletionService chatCompletionService = this.Kernel.GetRequiredService(); - ChatHistory chat = []; - if (!string.IsNullOrWhiteSpace(this.Instructions)) - { - chat.Add(new ChatMessageContent(AuthorRole.System, this.Instructions) { AuthorName = this.Name }); - } - chat.AddRange(history); + ChatHistory chat = this.SetupAgentChatHistory(history); int messageCount = chat.Count; - logger.LogDebug("[{MethodName}] Invoking {ServiceType}.", nameof(InvokeAsync), chatCompletionService.GetType()); + this.Logger.LogDebug("[{MethodName}] Invoking {ServiceType}.", nameof(InvokeAsync), chatCompletionService.GetType()); IReadOnlyList messages = await chatCompletionService.GetChatMessageContentsAsync( @@ -47,9 +42,9 @@ await chatCompletionService.GetChatMessageContentsAsync( this.Kernel, cancellationToken).ConfigureAwait(false); - if (logger.IsEnabled(LogLevel.Information)) // Avoid boxing if not enabled + if (this.Logger.IsEnabled(LogLevel.Information)) // Avoid boxing if not enabled { - logger.LogInformation("[{MethodName}] Invoked {ServiceType} with message count: {MessageCount}.", nameof(InvokeAsync), chatCompletionService.GetType(), messages.Count); + this.Logger.LogInformation("[{MethodName}] Invoked {ServiceType} with message count: {MessageCount}.", nameof(InvokeAsync), chatCompletionService.GetType(), messages.Count); } // Capture mutated messages related function calling / tools @@ -59,7 +54,7 @@ await chatCompletionService.GetChatMessageContentsAsync( message.AuthorName = this.Name; - yield return message; + history.Add(message); } foreach (ChatMessageContent message in messages ?? []) @@ -70,4 +65,62 @@ await chatCompletionService.GetChatMessageContentsAsync( yield return message; } } + + /// + public override async IAsyncEnumerable InvokeStreamingAsync( + ChatHistory history, + [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + IChatCompletionService chatCompletionService = this.Kernel.GetRequiredService(); + + ChatHistory chat = this.SetupAgentChatHistory(history); + + int messageCount = chat.Count; + + this.Logger.LogDebug("[{MethodName}] Invoking {ServiceType}.", nameof(InvokeAsync), chatCompletionService.GetType()); + + IAsyncEnumerable messages = + chatCompletionService.GetStreamingChatMessageContentsAsync( + chat, + this.ExecutionSettings, + this.Kernel, + cancellationToken); + + if (this.Logger.IsEnabled(LogLevel.Information)) + { + this.Logger.LogInformation("[{MethodName}] Invoked {ServiceType} with streaming messages.", nameof(InvokeAsync), chatCompletionService.GetType()); + } + + // Capture mutated messages related function calling / tools + for (int messageIndex = messageCount; messageIndex < chat.Count; messageIndex++) + { + ChatMessageContent message = chat[messageIndex]; + + message.AuthorName = this.Name; + + history.Add(message); + } + + await foreach (StreamingChatMessageContent message in messages.ConfigureAwait(false)) + { + // TODO: MESSAGE SOURCE - ISSUE #5731 + message.AuthorName = this.Name; + + yield return message; + } + } + + private ChatHistory SetupAgentChatHistory(IReadOnlyList history) + { + ChatHistory chat = []; + + if (!string.IsNullOrWhiteSpace(this.Instructions)) + { + chat.Add(new ChatMessageContent(AuthorRole.System, this.Instructions) { AuthorName = this.Name }); + } + + chat.AddRange(history); + + return chat; + } } diff --git a/dotnet/src/Agents/OpenAI/AssistantThreadActions.cs b/dotnet/src/Agents/OpenAI/AssistantThreadActions.cs new file mode 100644 index 000000000000..37649844a230 --- /dev/null +++ b/dotnet/src/Agents/OpenAI/AssistantThreadActions.cs @@ -0,0 +1,525 @@ +// Copyright (c) Microsoft. All rights reserved. +using System.Collections.Generic; +using System.Linq; +using System.Net; +using System.Runtime.CompilerServices; +using System.Text.Json; +using System.Threading; +using System.Threading.Tasks; +using Azure; +using Azure.AI.OpenAI.Assistants; +using Microsoft.Extensions.Logging; +using Microsoft.SemanticKernel.ChatCompletion; + +namespace Microsoft.SemanticKernel.Agents.OpenAI; + +/// +/// Actions associated with an Open Assistant thread. +/// +internal static class AssistantThreadActions +{ + /*AssistantsClient client, string threadId, OpenAIAssistantConfiguration.PollingConfiguration pollingConfiguration*/ + private const string FunctionDelimiter = "-"; + + private static readonly HashSet s_messageRoles = + [ + AuthorRole.User, + AuthorRole.Assistant, + ]; + + private static readonly HashSet s_pollingStatuses = + [ + RunStatus.Queued, + RunStatus.InProgress, + RunStatus.Cancelling, + ]; + + private static readonly HashSet s_terminalStatuses = + [ + RunStatus.Expired, + RunStatus.Failed, + RunStatus.Cancelled, + ]; + + /// + /// Create a message in the specified thread. + /// + /// The assistant client + /// The thread identifier + /// The message to add + /// The to monitor for cancellation requests. The default is . + /// if a system message is present, without taking any other action + public static async Task CreateMessageAsync(AssistantsClient client, string threadId, ChatMessageContent message, CancellationToken cancellationToken) + { + if (!s_messageRoles.Contains(message.Role)) + { + throw new KernelException($"Invalid message role: {message.Role}"); + } + + if (string.IsNullOrWhiteSpace(message.Content)) + { + return; + } + + await client.CreateMessageAsync( + threadId, + message.Role.ToMessageRole(), + message.Content, + cancellationToken: cancellationToken).ConfigureAwait(false); + } + + /// + /// Retrieves the thread messages. + /// + /// The assistant client + /// The thread identifier + /// The to monitor for cancellation requests. The default is . + /// Asynchronous enumeration of messages. + public static async IAsyncEnumerable GetMessagesAsync(AssistantsClient client, string threadId, [EnumeratorCancellation] CancellationToken cancellationToken) + { + Dictionary agentNames = []; // Cache agent names by their identifier + + PageableList messages; + + string? lastId = null; + do + { + messages = await client.GetMessagesAsync(threadId, limit: 100, ListSortOrder.Descending, after: lastId, null, cancellationToken).ConfigureAwait(false); + foreach (ThreadMessage message in messages) + { + AuthorRole role = new(message.Role.ToString()); + + string? assistantName = null; + if (!string.IsNullOrWhiteSpace(message.AssistantId) && + !agentNames.TryGetValue(message.AssistantId, out assistantName)) + { + Assistant assistant = await client.GetAssistantAsync(message.AssistantId, cancellationToken).ConfigureAwait(false); + if (!string.IsNullOrWhiteSpace(assistant.Name)) + { + agentNames.Add(assistant.Id, assistant.Name); + } + } + + assistantName ??= message.AssistantId; + + foreach (MessageContent item in message.ContentItems) + { + ChatMessageContent? content = null; + + if (item is MessageTextContent contentMessage) + { + content = GenerateTextMessageContent(assistantName, role, contentMessage); + } + else if (item is MessageImageFileContent contentImage) + { + content = GenerateImageFileContent(assistantName, role, contentImage); + } + + if (content is not null) + { + yield return content; + } + } + + lastId = message.Id; + } + } + while (messages.HasMore); + } + + /// + /// Invoke the assistant on the specified thread. + /// + /// The assistant agent to interact with the thread. + /// The assistant client + /// The thread identifier + /// Config to utilize when polling for run state. + /// The logger to utilize (might be agent or channel scoped) + /// The to monitor for cancellation requests. The default is . + /// Asynchronous enumeration of messages. + public static async IAsyncEnumerable InvokeAsync( + OpenAIAssistantAgent agent, + AssistantsClient client, + string threadId, + OpenAIAssistantConfiguration.PollingConfiguration pollingConfiguration, + ILogger logger, + [EnumeratorCancellation] CancellationToken cancellationToken) + { + if (agent.IsDeleted) + { + throw new KernelException($"Agent Failure - {nameof(OpenAIAssistantAgent)} agent is deleted: {agent.Id}."); + } + + ToolDefinition[]? tools = [.. agent.Tools, .. agent.Kernel.Plugins.SelectMany(p => p.Select(f => f.ToToolDefinition(p.Name, FunctionDelimiter)))]; + + logger.LogDebug("[{MethodName}] Creating run for agent/thrad: {AgentId}/{ThreadId}", nameof(InvokeAsync), agent.Id, threadId); + + CreateRunOptions options = + new(agent.Id) + { + OverrideInstructions = agent.Instructions, + OverrideTools = tools, + }; + + // Create run + ThreadRun run = await client.CreateRunAsync(threadId, options, cancellationToken).ConfigureAwait(false); + + logger.LogInformation("[{MethodName}] Created run: {RunId}", nameof(InvokeAsync), run.Id); + + // Evaluate status and process steps and messages, as encountered. + HashSet processedStepIds = []; + Dictionary functionSteps = []; + + do + { + // Poll run and steps until actionable + PageableList steps = await PollRunStatusAsync().ConfigureAwait(false); + + // Is in terminal state? + if (s_terminalStatuses.Contains(run.Status)) + { + throw new KernelException($"Agent Failure - Run terminated: {run.Status} [{run.Id}]: {run.LastError?.Message ?? "Unknown"}"); + } + + // Is tool action required? + if (run.Status == RunStatus.RequiresAction) + { + logger.LogDebug("[{MethodName}] Processing run steps: {RunId}", nameof(InvokeAsync), run.Id); + + // Execute functions in parallel and post results at once. + FunctionCallContent[] activeFunctionSteps = steps.Data.SelectMany(step => ParseFunctionStep(agent, step)).ToArray(); + if (activeFunctionSteps.Length > 0) + { + // Emit function-call content + yield return GenerateFunctionCallContent(agent.GetName(), activeFunctionSteps); + + // Invoke functions for each tool-step + IEnumerable> functionResultTasks = ExecuteFunctionSteps(agent, activeFunctionSteps, cancellationToken); + + // Block for function results + FunctionResultContent[] functionResults = await Task.WhenAll(functionResultTasks).ConfigureAwait(false); + + // Process tool output + ToolOutput[] toolOutputs = GenerateToolOutputs(functionResults); + + await client.SubmitToolOutputsToRunAsync(run, toolOutputs, cancellationToken).ConfigureAwait(false); + } + + if (logger.IsEnabled(LogLevel.Information)) // Avoid boxing if not enabled + { + logger.LogInformation("[{MethodName}] Processed #{MessageCount} run steps: {RunId}", nameof(InvokeAsync), activeFunctionSteps.Length, run.Id); + } + } + + // Enumerate completed messages + logger.LogDebug("[{MethodName}] Processing run messages: {RunId}", nameof(InvokeAsync), run.Id); + + IEnumerable completedStepsToProcess = + steps + .Where(s => s.CompletedAt.HasValue && !processedStepIds.Contains(s.Id)) + .OrderBy(s => s.CreatedAt); + + int messageCount = 0; + foreach (RunStep completedStep in completedStepsToProcess) + { + if (completedStep.Type.Equals(RunStepType.ToolCalls)) + { + RunStepToolCallDetails toolCallDetails = (RunStepToolCallDetails)completedStep.StepDetails; + + foreach (RunStepToolCall toolCall in toolCallDetails.ToolCalls) + { + ChatMessageContent? content = null; + + // Process code-interpreter content + if (toolCall is RunStepCodeInterpreterToolCall toolCodeInterpreter) + { + content = GenerateCodeInterpreterContent(agent.GetName(), toolCodeInterpreter); + } + // Process function result content + else if (toolCall is RunStepFunctionToolCall toolFunction) + { + FunctionCallContent functionStep = functionSteps[toolFunction.Id]; // Function step always captured on invocation + content = GenerateFunctionResultContent(agent.GetName(), functionStep, toolFunction.Output); + } + + if (content is not null) + { + ++messageCount; + + yield return content; + } + } + } + else if (completedStep.Type.Equals(RunStepType.MessageCreation)) + { + RunStepMessageCreationDetails messageCreationDetails = (RunStepMessageCreationDetails)completedStep.StepDetails; + + // Retrieve the message + ThreadMessage? message = await RetrieveMessageAsync(messageCreationDetails, cancellationToken).ConfigureAwait(false); + + if (message is not null) + { + AuthorRole role = new(message.Role.ToString()); + + foreach (MessageContent itemContent in message.ContentItems) + { + ChatMessageContent? content = null; + + // Process text content + if (itemContent is MessageTextContent contentMessage) + { + content = GenerateTextMessageContent(agent.GetName(), role, contentMessage); + } + // Process image content + else if (itemContent is MessageImageFileContent contentImage) + { + content = GenerateImageFileContent(agent.GetName(), role, contentImage); + } + + if (content is not null) + { + ++messageCount; + + yield return content; + } + } + } + } + + processedStepIds.Add(completedStep.Id); + } + + if (logger.IsEnabled(LogLevel.Information)) // Avoid boxing if not enabled + { + logger.LogInformation("[{MethodName}] Processed #{MessageCount} run messages: {RunId}", nameof(InvokeAsync), messageCount, run.Id); + } + } + while (RunStatus.Completed != run.Status); + + logger.LogInformation("[{MethodName}] Completed run: {RunId}", nameof(InvokeAsync), run.Id); + + // Local function to assist in run polling (participates in method closure). + async Task> PollRunStatusAsync() + { + logger.LogInformation("[{MethodName}] Polling run status: {RunId}", nameof(PollRunStatusAsync), run.Id); + + int count = 0; + + do + { + // Reduce polling frequency after a couple attempts + await Task.Delay(count >= 2 ? pollingConfiguration.RunPollingInterval : pollingConfiguration.RunPollingBackoff, cancellationToken).ConfigureAwait(false); + ++count; + +#pragma warning disable CA1031 // Do not catch general exception types + try + { + run = await client.GetRunAsync(threadId, run.Id, cancellationToken).ConfigureAwait(false); + } + catch + { + // Retry anyway.. + } +#pragma warning restore CA1031 // Do not catch general exception types + } + while (s_pollingStatuses.Contains(run.Status)); + + logger.LogInformation("[{MethodName}] Run status is {RunStatus}: {RunId}", nameof(PollRunStatusAsync), run.Status, run.Id); + + return await client.GetRunStepsAsync(run, cancellationToken: cancellationToken).ConfigureAwait(false); + } + + // Local function to capture kernel function state for further processing (participates in method closure). + IEnumerable ParseFunctionStep(OpenAIAssistantAgent agent, RunStep step) + { + if (step.Status == RunStepStatus.InProgress && step.StepDetails is RunStepToolCallDetails callDetails) + { + foreach (RunStepFunctionToolCall toolCall in callDetails.ToolCalls.OfType()) + { + var nameParts = FunctionName.Parse(toolCall.Name, FunctionDelimiter); + + KernelArguments functionArguments = []; + if (!string.IsNullOrWhiteSpace(toolCall.Arguments)) + { + Dictionary arguments = JsonSerializer.Deserialize>(toolCall.Arguments)!; + foreach (var argumentKvp in arguments) + { + functionArguments[argumentKvp.Key] = argumentKvp.Value.ToString(); + } + } + + var content = new FunctionCallContent(nameParts.Name, nameParts.PluginName, toolCall.Id, functionArguments); + + functionSteps.Add(toolCall.Id, content); + + yield return content; + } + } + } + + async Task RetrieveMessageAsync(RunStepMessageCreationDetails detail, CancellationToken cancellationToken) + { + ThreadMessage? message = null; + + bool retry = false; + int count = 0; + do + { + try + { + message = await client.GetMessageAsync(threadId, detail.MessageCreation.MessageId, cancellationToken).ConfigureAwait(false); + } + catch (RequestFailedException exception) + { + // Step has provided the message-id. Retry on of NotFound/404 exists. + // Extremely rarely there might be a synchronization issue between the + // assistant response and message-service. + retry = exception.Status == (int)HttpStatusCode.NotFound && count < 3; + } + + if (retry) + { + await Task.Delay(pollingConfiguration.MessageSynchronizationDelay, cancellationToken).ConfigureAwait(false); + } + + ++count; + } + while (retry); + + return message; + } + } + + private static AnnotationContent GenerateAnnotationContent(MessageTextAnnotation annotation) + { + string? fileId = null; + if (annotation is MessageTextFileCitationAnnotation citationAnnotation) + { + fileId = citationAnnotation.FileId; + } + else if (annotation is MessageTextFilePathAnnotation pathAnnotation) + { + fileId = pathAnnotation.FileId; + } + + return + new() + { + Quote = annotation.Text, + StartIndex = annotation.StartIndex, + EndIndex = annotation.EndIndex, + FileId = fileId, + }; + } + + private static ChatMessageContent GenerateImageFileContent(string agentName, AuthorRole role, MessageImageFileContent contentImage) + { + return + new ChatMessageContent( + role, + [ + new FileReferenceContent(contentImage.FileId) + ]) + { + AuthorName = agentName, + }; + } + + private static ChatMessageContent? GenerateTextMessageContent(string agentName, AuthorRole role, MessageTextContent contentMessage) + { + ChatMessageContent? messageContent = null; + + string textContent = contentMessage.Text.Trim(); + + if (!string.IsNullOrWhiteSpace(textContent)) + { + messageContent = + new(role, textContent) + { + AuthorName = agentName + }; + + foreach (MessageTextAnnotation annotation in contentMessage.Annotations) + { + messageContent.Items.Add(GenerateAnnotationContent(annotation)); + } + } + + return messageContent; + } + + private static ChatMessageContent GenerateCodeInterpreterContent(string agentName, RunStepCodeInterpreterToolCall contentCodeInterpreter) + { + return + new ChatMessageContent( + AuthorRole.Tool, + [ + new TextContent(contentCodeInterpreter.Input) + ]) + { + AuthorName = agentName, + }; + } + + private static ChatMessageContent GenerateFunctionCallContent(string agentName, FunctionCallContent[] functionSteps) + { + ChatMessageContent functionCallContent = new(AuthorRole.Tool, content: null) + { + AuthorName = agentName + }; + + functionCallContent.Items.AddRange(functionSteps); + + return functionCallContent; + } + + private static ChatMessageContent GenerateFunctionResultContent(string agentName, FunctionCallContent functionStep, string result) + { + ChatMessageContent functionCallContent = new(AuthorRole.Tool, content: null) + { + AuthorName = agentName + }; + + functionCallContent.Items.Add( + new FunctionResultContent( + functionStep.FunctionName, + functionStep.PluginName, + functionStep.Id, + result)); + + return functionCallContent; + } + + private static Task[] ExecuteFunctionSteps(OpenAIAssistantAgent agent, FunctionCallContent[] functionSteps, CancellationToken cancellationToken) + { + Task[] functionTasks = new Task[functionSteps.Length]; + + for (int index = 0; index < functionSteps.Length; ++index) + { + functionTasks[index] = functionSteps[index].InvokeAsync(agent.Kernel, cancellationToken); + } + + return functionTasks; + } + + private static ToolOutput[] GenerateToolOutputs(FunctionResultContent[] functionResults) + { + ToolOutput[] toolOutputs = new ToolOutput[functionResults.Length]; + + for (int index = 0; index < functionResults.Length; ++index) + { + FunctionResultContent functionResult = functionResults[index]; + + object resultValue = functionResult.Result ?? string.Empty; + + if (resultValue is not string textResult) + { + textResult = JsonSerializer.Serialize(resultValue); + } + + toolOutputs[index] = new ToolOutput(functionResult.CallId, textResult!); + } + + return toolOutputs; + } +} diff --git a/dotnet/src/Agents/OpenAI/OpenAIAssistantAgent.cs b/dotnet/src/Agents/OpenAI/OpenAIAssistantAgent.cs index ca016a5d97cb..b46cdb013c18 100644 --- a/dotnet/src/Agents/OpenAI/OpenAIAssistantAgent.cs +++ b/dotnet/src/Agents/OpenAI/OpenAIAssistantAgent.cs @@ -162,15 +162,91 @@ public static async Task RetrieveAsync( }; } - /// - public async Task DeleteAsync(CancellationToken cancellationToken = default) + /// + /// Create a new assistant thread. + /// + /// The to monitor for cancellation requests. The default is . + /// The thread identifier + public async Task CreateThreadAsync(CancellationToken cancellationToken = default) { - if (this.IsDeleted) + AssistantThread thread = await this._client.CreateThreadAsync(cancellationToken).ConfigureAwait(false); + + return thread.Id; + } + + /// + /// Create a new assistant thread. + /// + /// The thread identifier + /// The to monitor for cancellation requests. The default is . + /// The thread identifier + public async Task DeleteThreadAsync( + string threadId, + CancellationToken cancellationToken = default) + { + // Validate input + Verify.NotNullOrWhiteSpace(threadId, nameof(threadId)); + + return await this._client.DeleteThreadAsync(threadId, cancellationToken).ConfigureAwait(false); + } + + /// + /// Adds a message to the specified thread. + /// + /// The thread identifier + /// A non-system message with which to append to the conversation. + /// The to monitor for cancellation requests. The default is . + public Task AddChatMessageAsync(string threadId, ChatMessageContent message, CancellationToken cancellationToken = default) + { + this.ThrowIfDeleted(); + + return AssistantThreadActions.CreateMessageAsync(this._client, threadId, message, cancellationToken); + } + + /// + /// Gets messages for a specified thread. + /// + /// The thread identifier + /// The to monitor for cancellation requests. The default is . + /// Asynchronous enumeration of messages. + public IAsyncEnumerable GetThreadMessagesAsync(string threadId, CancellationToken cancellationToken = default) + { + this.ThrowIfDeleted(); + + return AssistantThreadActions.GetMessagesAsync(this._client, threadId, cancellationToken); + } + + /// + /// Delete the assistant definition. + /// + /// + /// True if assistant definition has been deleted + /// + /// Assistant based agent will not be useable after deletion. + /// + public async Task DeleteAsync(CancellationToken cancellationToken = default) + { + if (!this.IsDeleted) { - return; + this.IsDeleted = (await this._client.DeleteAssistantAsync(this.Id, cancellationToken).ConfigureAwait(false)).Value; } - this.IsDeleted = (await this._client.DeleteAssistantAsync(this.Id, cancellationToken).ConfigureAwait(false)).Value; + return this.IsDeleted; + } + + /// + /// Invoke the assistant on the specified thread. + /// + /// The thread identifier + /// The to monitor for cancellation requests. The default is . + /// Asynchronous enumeration of messages. + public IAsyncEnumerable InvokeAsync( + string threadId, + CancellationToken cancellationToken = default) + { + this.ThrowIfDeleted(); + + return AssistantThreadActions.InvokeAsync(this, this._client, threadId, this._config.Polling, this.Logger, cancellationToken); } /// @@ -204,15 +280,27 @@ protected override IEnumerable GetChannelKeys() } /// - protected override async Task CreateChannelAsync(ILogger logger, CancellationToken cancellationToken) + protected override async Task CreateChannelAsync(CancellationToken cancellationToken) { - logger.LogDebug("[{MethodName}] Creating assistant thread", nameof(CreateChannelAsync)); + this.Logger.LogDebug("[{MethodName}] Creating assistant thread", nameof(CreateChannelAsync)); AssistantThread thread = await this._client.CreateThreadAsync(cancellationToken).ConfigureAwait(false); - logger.LogInformation("[{MethodName}] Created assistant thread: {ThreadId}", nameof(CreateChannelAsync), thread.Id); + this.Logger.LogInformation("[{MethodName}] Created assistant thread: {ThreadId}", nameof(CreateChannelAsync), thread.Id); - return new OpenAIAssistantChannel(this._client, thread.Id, this._config.Polling); + return + new OpenAIAssistantChannel(this._client, thread.Id, this._config.Polling) + { + Logger = this.LoggerFactory.CreateLogger() + }; + } + + internal void ThrowIfDeleted() + { + if (this.IsDeleted) + { + throw new KernelException($"Agent Failure - {nameof(OpenAIAssistantAgent)} agent is deleted: {this.Id}."); + } } /// diff --git a/dotnet/src/Agents/OpenAI/OpenAIAssistantChannel.cs b/dotnet/src/Agents/OpenAI/OpenAIAssistantChannel.cs index 0d8b20b5b931..b84ef800ebd4 100644 --- a/dotnet/src/Agents/OpenAI/OpenAIAssistantChannel.cs +++ b/dotnet/src/Agents/OpenAI/OpenAIAssistantChannel.cs @@ -1,15 +1,8 @@ // Copyright (c) Microsoft. All rights reserved. using System.Collections.Generic; -using System.Linq; -using System.Net; -using System.Runtime.CompilerServices; -using System.Text.Json; using System.Threading; using System.Threading.Tasks; -using Azure; using Azure.AI.OpenAI.Assistants; -using Microsoft.Extensions.Logging; -using Microsoft.SemanticKernel.ChatCompletion; namespace Microsoft.SemanticKernel.Agents.OpenAI; @@ -19,485 +12,31 @@ namespace Microsoft.SemanticKernel.Agents.OpenAI; internal sealed class OpenAIAssistantChannel(AssistantsClient client, string threadId, OpenAIAssistantConfiguration.PollingConfiguration pollingConfiguration) : AgentChannel { - private const string FunctionDelimiter = "-"; - - private static readonly HashSet s_pollingStatuses = - [ - RunStatus.Queued, - RunStatus.InProgress, - RunStatus.Cancelling, - ]; - - private static readonly HashSet s_terminalStatuses = - [ - RunStatus.Expired, - RunStatus.Failed, - RunStatus.Cancelled, - ]; - private readonly AssistantsClient _client = client; private readonly string _threadId = threadId; - private readonly Dictionary _agentTools = []; - private readonly Dictionary _agentNames = []; // Cache agent names by their identifier for GetHistoryAsync() /// protected override async Task ReceiveAsync(IReadOnlyList history, CancellationToken cancellationToken) { foreach (ChatMessageContent message in history) { - if (string.IsNullOrWhiteSpace(message.Content)) - { - continue; - } - - await this._client.CreateMessageAsync( - this._threadId, - message.Role.ToMessageRole(), - message.Content, - cancellationToken: cancellationToken).ConfigureAwait(false); + await AssistantThreadActions.CreateMessageAsync(this._client, this._threadId, message, cancellationToken).ConfigureAwait(false); } } /// - protected override async IAsyncEnumerable InvokeAsync( + protected override IAsyncEnumerable InvokeAsync( OpenAIAssistantAgent agent, - [EnumeratorCancellation] CancellationToken cancellationToken) + CancellationToken cancellationToken) { - if (agent.IsDeleted) - { - throw new KernelException($"Agent Failure - {nameof(OpenAIAssistantAgent)} agent is deleted: {agent.Id}."); - } - - if (!this._agentTools.TryGetValue(agent.Id, out ToolDefinition[]? tools)) - { - tools = [.. agent.Tools, .. agent.Kernel.Plugins.SelectMany(p => p.Select(f => f.ToToolDefinition(p.Name, FunctionDelimiter)))]; - this._agentTools.Add(agent.Id, tools); - } - - if (!this._agentNames.ContainsKey(agent.Id) && !string.IsNullOrWhiteSpace(agent.Name)) - { - this._agentNames.Add(agent.Id, agent.Name); - } - - this.Logger.LogDebug("[{MethodName}] Creating run for agent/thrad: {AgentId}/{ThreadId}", nameof(InvokeAsync), agent.Id, this._threadId); - - CreateRunOptions options = - new(agent.Id) - { - OverrideInstructions = agent.Instructions, - OverrideTools = tools, - }; - - // Create run - ThreadRun run = await this._client.CreateRunAsync(this._threadId, options, cancellationToken).ConfigureAwait(false); - - this.Logger.LogInformation("[{MethodName}] Created run: {RunId}", nameof(InvokeAsync), run.Id); - - // Evaluate status and process steps and messages, as encountered. - HashSet processedStepIds = []; - Dictionary functionSteps = []; - - do - { - // Poll run and steps until actionable - PageableList steps = await PollRunStatusAsync().ConfigureAwait(false); - - // Is in terminal state? - if (s_terminalStatuses.Contains(run.Status)) - { - throw new KernelException($"Agent Failure - Run terminated: {run.Status} [{run.Id}]: {run.LastError?.Message ?? "Unknown"}"); - } - - // Is tool action required? - if (run.Status == RunStatus.RequiresAction) - { - this.Logger.LogDebug("[{MethodName}] Processing run steps: {RunId}", nameof(InvokeAsync), run.Id); - - // Execute functions in parallel and post results at once. - FunctionCallContent[] activeFunctionSteps = steps.Data.SelectMany(step => ParseFunctionStep(agent, step)).ToArray(); - if (activeFunctionSteps.Length > 0) - { - // Emit function-call content - yield return GenerateFunctionCallContent(agent.GetName(), activeFunctionSteps); - - // Invoke functions for each tool-step - IEnumerable> functionResultTasks = ExecuteFunctionSteps(agent, activeFunctionSteps, cancellationToken); - - // Block for function results - FunctionResultContent[] functionResults = await Task.WhenAll(functionResultTasks).ConfigureAwait(false); - - // Process tool output - ToolOutput[] toolOutputs = GenerateToolOutputs(functionResults); - - await this._client.SubmitToolOutputsToRunAsync(run, toolOutputs, cancellationToken).ConfigureAwait(false); - } - - if (this.Logger.IsEnabled(LogLevel.Information)) // Avoid boxing if not enabled - { - this.Logger.LogInformation("[{MethodName}] Processed #{MessageCount} run steps: {RunId}", nameof(InvokeAsync), activeFunctionSteps.Length, run.Id); - } - } - - // Enumerate completed messages - this.Logger.LogDebug("[{MethodName}] Processing run messages: {RunId}", nameof(InvokeAsync), run.Id); - - IEnumerable completedStepsToProcess = - steps - .Where(s => s.CompletedAt.HasValue && !processedStepIds.Contains(s.Id)) - .OrderBy(s => s.CreatedAt); - - int messageCount = 0; - foreach (RunStep completedStep in completedStepsToProcess) - { - if (completedStep.Type.Equals(RunStepType.ToolCalls)) - { - RunStepToolCallDetails toolCallDetails = (RunStepToolCallDetails)completedStep.StepDetails; - - foreach (RunStepToolCall toolCall in toolCallDetails.ToolCalls) - { - ChatMessageContent? content = null; - - // Process code-interpreter content - if (toolCall is RunStepCodeInterpreterToolCall toolCodeInterpreter) - { - content = GenerateCodeInterpreterContent(agent.GetName(), toolCodeInterpreter); - } - // Process function result content - else if (toolCall is RunStepFunctionToolCall toolFunction) - { - FunctionCallContent functionStep = functionSteps[toolFunction.Id]; // Function step always captured on invocation - content = GenerateFunctionResultContent(agent.GetName(), functionStep, toolFunction.Output); - } + agent.ThrowIfDeleted(); - if (content is not null) - { - ++messageCount; - - yield return content; - } - } - } - else if (completedStep.Type.Equals(RunStepType.MessageCreation)) - { - RunStepMessageCreationDetails messageCreationDetails = (RunStepMessageCreationDetails)completedStep.StepDetails; - - // Retrieve the message - ThreadMessage? message = await this.RetrieveMessageAsync(messageCreationDetails, cancellationToken).ConfigureAwait(false); - - if (message is not null) - { - AuthorRole role = new(message.Role.ToString()); - - foreach (MessageContent itemContent in message.ContentItems) - { - ChatMessageContent? content = null; - - // Process text content - if (itemContent is MessageTextContent contentMessage) - { - content = GenerateTextMessageContent(agent.GetName(), role, contentMessage); - } - // Process image content - else if (itemContent is MessageImageFileContent contentImage) - { - content = GenerateImageFileContent(agent.GetName(), role, contentImage); - } - - if (content is not null) - { - ++messageCount; - - yield return content; - } - } - } - } - - processedStepIds.Add(completedStep.Id); - } - - if (this.Logger.IsEnabled(LogLevel.Information)) // Avoid boxing if not enabled - { - this.Logger.LogInformation("[{MethodName}] Processed #{MessageCount} run messages: {RunId}", nameof(InvokeAsync), messageCount, run.Id); - } - } - while (RunStatus.Completed != run.Status); - - this.Logger.LogInformation("[{MethodName}] Completed run: {RunId}", nameof(InvokeAsync), run.Id); - - // Local function to assist in run polling (participates in method closure). - async Task> PollRunStatusAsync() - { - this.Logger.LogInformation("[{MethodName}] Polling run status: {RunId}", nameof(PollRunStatusAsync), run.Id); - - int count = 0; - - do - { - // Reduce polling frequency after a couple attempts - await Task.Delay(count >= 2 ? pollingConfiguration.RunPollingInterval : pollingConfiguration.RunPollingBackoff, cancellationToken).ConfigureAwait(false); - ++count; - -#pragma warning disable CA1031 // Do not catch general exception types - try - { - run = await this._client.GetRunAsync(this._threadId, run.Id, cancellationToken).ConfigureAwait(false); - } - catch - { - // Retry anyway.. - } -#pragma warning restore CA1031 // Do not catch general exception types - } - while (s_pollingStatuses.Contains(run.Status)); - - this.Logger.LogInformation("[{MethodName}] Run status is {RunStatus}: {RunId}", nameof(PollRunStatusAsync), run.Status, run.Id); - - return await this._client.GetRunStepsAsync(run, cancellationToken: cancellationToken).ConfigureAwait(false); - } - - // Local function to capture kernel function state for further processing (participates in method closure). - IEnumerable ParseFunctionStep(OpenAIAssistantAgent agent, RunStep step) - { - if (step.Status == RunStepStatus.InProgress && step.StepDetails is RunStepToolCallDetails callDetails) - { - foreach (RunStepFunctionToolCall toolCall in callDetails.ToolCalls.OfType()) - { - var nameParts = FunctionName.Parse(toolCall.Name, FunctionDelimiter); - - KernelArguments functionArguments = []; - if (!string.IsNullOrWhiteSpace(toolCall.Arguments)) - { - Dictionary arguments = JsonSerializer.Deserialize>(toolCall.Arguments)!; - foreach (var argumentKvp in arguments) - { - functionArguments[argumentKvp.Key] = argumentKvp.Value.ToString(); - } - } - - var content = new FunctionCallContent(nameParts.Name, nameParts.PluginName, toolCall.Id, functionArguments); - - functionSteps.Add(toolCall.Id, content); - - yield return content; - } - } - } + return AssistantThreadActions.InvokeAsync(agent, this._client, this._threadId, pollingConfiguration, this.Logger, cancellationToken); } /// - protected override async IAsyncEnumerable GetHistoryAsync([EnumeratorCancellation] CancellationToken cancellationToken) - { - PageableList messages; - - string? lastId = null; - do - { - messages = await this._client.GetMessagesAsync(this._threadId, limit: 100, ListSortOrder.Descending, after: lastId, null, cancellationToken).ConfigureAwait(false); - foreach (ThreadMessage message in messages) - { - AuthorRole role = new(message.Role.ToString()); - - string? assistantName = null; - if (!string.IsNullOrWhiteSpace(message.AssistantId) && - !this._agentNames.TryGetValue(message.AssistantId, out assistantName)) - { - Assistant assistant = await this._client.GetAssistantAsync(message.AssistantId, cancellationToken).ConfigureAwait(false); - if (!string.IsNullOrWhiteSpace(assistant.Name)) - { - this._agentNames.Add(assistant.Id, assistant.Name); - } - } - - assistantName ??= message.AssistantId; - - foreach (MessageContent item in message.ContentItems) - { - ChatMessageContent? content = null; - - if (item is MessageTextContent contentMessage) - { - content = GenerateTextMessageContent(assistantName, role, contentMessage); - } - else if (item is MessageImageFileContent contentImage) - { - content = GenerateImageFileContent(assistantName, role, contentImage); - } - - if (content is not null) - { - yield return content; - } - } - - lastId = message.Id; - } - } - while (messages.HasMore); - } - - private static AnnotationContent GenerateAnnotationContent(MessageTextAnnotation annotation) - { - string? fileId = null; - if (annotation is MessageTextFileCitationAnnotation citationAnnotation) - { - fileId = citationAnnotation.FileId; - } - else if (annotation is MessageTextFilePathAnnotation pathAnnotation) - { - fileId = pathAnnotation.FileId; - } - - return - new() - { - Quote = annotation.Text, - StartIndex = annotation.StartIndex, - EndIndex = annotation.EndIndex, - FileId = fileId, - }; - } - - private static ChatMessageContent GenerateImageFileContent(string agentName, AuthorRole role, MessageImageFileContent contentImage) - { - return - new ChatMessageContent( - role, - [ - new FileReferenceContent(contentImage.FileId) - ]) - { - AuthorName = agentName, - }; - } - - private static ChatMessageContent? GenerateTextMessageContent(string agentName, AuthorRole role, MessageTextContent contentMessage) - { - ChatMessageContent? messageContent = null; - - string textContent = contentMessage.Text.Trim(); - - if (!string.IsNullOrWhiteSpace(textContent)) - { - messageContent = - new(role, textContent) - { - AuthorName = agentName - }; - - foreach (MessageTextAnnotation annotation in contentMessage.Annotations) - { - messageContent.Items.Add(GenerateAnnotationContent(annotation)); - } - } - - return messageContent; - } - - private static ChatMessageContent GenerateCodeInterpreterContent(string agentName, RunStepCodeInterpreterToolCall contentCodeInterpreter) - { - return - new ChatMessageContent( - AuthorRole.Tool, - [ - new TextContent(contentCodeInterpreter.Input) - ]) - { - AuthorName = agentName, - }; - } - - private static ChatMessageContent GenerateFunctionCallContent(string agentName, FunctionCallContent[] functionSteps) - { - ChatMessageContent functionCallContent = new(AuthorRole.Tool, content: null) - { - AuthorName = agentName - }; - - functionCallContent.Items.AddRange(functionSteps); - - return functionCallContent; - } - - private static ChatMessageContent GenerateFunctionResultContent(string agentName, FunctionCallContent functionStep, string result) - { - ChatMessageContent functionCallContent = new(AuthorRole.Tool, content: null) - { - AuthorName = agentName - }; - - functionCallContent.Items.Add( - new FunctionResultContent( - functionStep.FunctionName, - functionStep.PluginName, - functionStep.Id, - result)); - - return functionCallContent; - } - - private static Task[] ExecuteFunctionSteps(OpenAIAssistantAgent agent, FunctionCallContent[] functionSteps, CancellationToken cancellationToken) - { - Task[] functionTasks = new Task[functionSteps.Length]; - - for (int index = 0; index < functionSteps.Length; ++index) - { - functionTasks[index] = functionSteps[index].InvokeAsync(agent.Kernel, cancellationToken); - } - - return functionTasks; - } - - private static ToolOutput[] GenerateToolOutputs(FunctionResultContent[] functionResults) + protected override IAsyncEnumerable GetHistoryAsync(CancellationToken cancellationToken) { - ToolOutput[] toolOutputs = new ToolOutput[functionResults.Length]; - - for (int index = 0; index < functionResults.Length; ++index) - { - FunctionResultContent functionResult = functionResults[index]; - - object resultValue = (functionResult.Result as FunctionResult)?.GetValue() ?? string.Empty; - - if (resultValue is not string textResult) - { - textResult = JsonSerializer.Serialize(resultValue); - } - - toolOutputs[index] = new ToolOutput(functionResult.CallId, textResult!); - } - - return toolOutputs; - } - - private async Task RetrieveMessageAsync(RunStepMessageCreationDetails detail, CancellationToken cancellationToken) - { - ThreadMessage? message = null; - - bool retry = false; - int count = 0; - do - { - try - { - message = await this._client.GetMessageAsync(this._threadId, detail.MessageCreation.MessageId, cancellationToken).ConfigureAwait(false); - } - catch (RequestFailedException exception) - { - // Step has provided the message-id. Retry on of NotFound/404 exists. - // Extremely rarely there might be a synchronization issue between the - // assistant response and message-service. - retry = exception.Status == (int)HttpStatusCode.NotFound && count < 3; - } - - if (retry) - { - await Task.Delay(pollingConfiguration.MessageSynchronizationDelay, cancellationToken).ConfigureAwait(false); - } - - ++count; - } - while (retry); - - return message; + return AssistantThreadActions.GetMessagesAsync(this._client, this._threadId, cancellationToken); } } diff --git a/dotnet/src/Agents/UnitTests/AgentChannelTests.cs b/dotnet/src/Agents/UnitTests/AgentChannelTests.cs index 544bf946c332..7223b8d46805 100644 --- a/dotnet/src/Agents/UnitTests/AgentChannelTests.cs +++ b/dotnet/src/Agents/UnitTests/AgentChannelTests.cs @@ -5,7 +5,6 @@ using System.Runtime.CompilerServices; using System.Threading; using System.Threading.Tasks; -using Microsoft.Extensions.Logging; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.Agents; using Xunit; @@ -68,7 +67,7 @@ private sealed class NextAgent : TestAgent; private class TestAgent : KernelAgent { - protected internal override Task CreateChannelAsync(ILogger logger, CancellationToken cancellationToken) + protected internal override Task CreateChannelAsync(CancellationToken cancellationToken) { throw new NotImplementedException(); } diff --git a/dotnet/src/Agents/UnitTests/AgentChatTests.cs b/dotnet/src/Agents/UnitTests/AgentChatTests.cs index d3c61e4c0a85..89ff7f02cff2 100644 --- a/dotnet/src/Agents/UnitTests/AgentChatTests.cs +++ b/dotnet/src/Agents/UnitTests/AgentChatTests.cs @@ -4,7 +4,6 @@ using System.Runtime.CompilerServices; using System.Threading; using System.Threading.Tasks; -using Microsoft.Extensions.Logging; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.Agents; using Microsoft.SemanticKernel.ChatCompletion; @@ -136,8 +135,7 @@ private sealed class TestAgent : ChatHistoryKernelAgent public int InvokeCount { get; private set; } public override async IAsyncEnumerable InvokeAsync( - IReadOnlyList history, - ILogger logger, + ChatHistory history, [EnumeratorCancellation] CancellationToken cancellationToken = default) { await Task.Delay(0, cancellationToken); @@ -146,5 +144,16 @@ public override async IAsyncEnumerable InvokeAsync( yield return new ChatMessageContent(AuthorRole.Assistant, "sup"); } + + public override IAsyncEnumerable InvokeStreamingAsync( + ChatHistory history, + CancellationToken cancellationToken = default) + { + this.InvokeCount++; + + StreamingChatMessageContent[] contents = [new(AuthorRole.Assistant, "sup")]; + + return contents.ToAsyncEnumerable(); + } } } diff --git a/dotnet/src/Agents/UnitTests/AggregatorAgentTests.cs b/dotnet/src/Agents/UnitTests/AggregatorAgentTests.cs index f544c1426526..c4a974cbadc9 100644 --- a/dotnet/src/Agents/UnitTests/AggregatorAgentTests.cs +++ b/dotnet/src/Agents/UnitTests/AggregatorAgentTests.cs @@ -1,9 +1,7 @@ // Copyright (c) Microsoft. All rights reserved. -using System.Collections.Generic; using System.Linq; using System.Threading; using System.Threading.Tasks; -using Microsoft.Extensions.Logging; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.Agents; using Microsoft.SemanticKernel.ChatCompletion; @@ -88,7 +86,7 @@ private static Mock CreateMockAgent() Mock agent = new(); ChatMessageContent[] messages = [new ChatMessageContent(AuthorRole.Assistant, "test agent")]; - agent.Setup(a => a.InvokeAsync(It.IsAny>(), It.IsAny(), It.IsAny())).Returns(() => messages.ToAsyncEnumerable()); + agent.Setup(a => a.InvokeAsync(It.IsAny(), It.IsAny())).Returns(() => messages.ToAsyncEnumerable()); return agent; } diff --git a/dotnet/src/Agents/UnitTests/ChatHistoryChannelTests.cs b/dotnet/src/Agents/UnitTests/ChatHistoryChannelTests.cs index 40a83d739312..7ef624c61ab9 100644 --- a/dotnet/src/Agents/UnitTests/ChatHistoryChannelTests.cs +++ b/dotnet/src/Agents/UnitTests/ChatHistoryChannelTests.cs @@ -4,7 +4,6 @@ using System.Linq; using System.Threading; using System.Threading.Tasks; -using Microsoft.Extensions.Logging; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.Agents; using Xunit; @@ -30,7 +29,7 @@ public async Task VerifyAgentWithoutIChatHistoryHandlerAsync() private sealed class TestAgent : KernelAgent { - protected internal override Task CreateChannelAsync(ILogger logger, CancellationToken cancellationToken) + protected internal override Task CreateChannelAsync(CancellationToken cancellationToken) { throw new NotImplementedException(); } diff --git a/dotnet/src/Agents/UnitTests/Core/AgentGroupChatTests.cs b/dotnet/src/Agents/UnitTests/Core/AgentGroupChatTests.cs index 3948f4b46836..921e0acce016 100644 --- a/dotnet/src/Agents/UnitTests/Core/AgentGroupChatTests.cs +++ b/dotnet/src/Agents/UnitTests/Core/AgentGroupChatTests.cs @@ -4,7 +4,6 @@ using System.Linq; using System.Threading; using System.Threading.Tasks; -using Microsoft.Extensions.Logging; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.Agents; using Microsoft.SemanticKernel.Agents.Chat; @@ -199,7 +198,7 @@ private static Mock CreateMockAgent() Mock agent = new(); ChatMessageContent[] messages = [new ChatMessageContent(AuthorRole.Assistant, "test")]; - agent.Setup(a => a.InvokeAsync(It.IsAny>(), It.IsAny(), It.IsAny())).Returns(() => messages.ToAsyncEnumerable()); + agent.Setup(a => a.InvokeAsync(It.IsAny(), It.IsAny())).Returns(() => messages.ToAsyncEnumerable()); return agent; } diff --git a/dotnet/src/Agents/UnitTests/Core/ChatCompletionAgentTests.cs b/dotnet/src/Agents/UnitTests/Core/ChatCompletionAgentTests.cs index e1c873598951..ae7657c8189c 100644 --- a/dotnet/src/Agents/UnitTests/Core/ChatCompletionAgentTests.cs +++ b/dotnet/src/Agents/UnitTests/Core/ChatCompletionAgentTests.cs @@ -3,7 +3,6 @@ using System.Threading; using System.Threading.Tasks; using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging.Abstractions; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.Agents; using Microsoft.SemanticKernel.ChatCompletion; @@ -60,7 +59,7 @@ public async Task VerifyChatCompletionAgentInvocationAsync() ExecutionSettings = new(), }; - var result = await agent.InvokeAsync([], NullLogger.Instance).ToArrayAsync(); + var result = await agent.InvokeAsync([]).ToArrayAsync(); Assert.Single(result); @@ -74,6 +73,48 @@ public async Task VerifyChatCompletionAgentInvocationAsync() Times.Once); } + /// + /// Verify the streaming invocation and response of . + /// + [Fact] + public async Task VerifyChatCompletionAgentStreamingAsync() + { + StreamingChatMessageContent[] returnContent = + [ + new(AuthorRole.Assistant, "wh"), + new(AuthorRole.Assistant, "at?"), + ]; + + var mockService = new Mock(); + mockService.Setup( + s => s.GetStreamingChatMessageContentsAsync( + It.IsAny(), + It.IsAny(), + It.IsAny(), + It.IsAny())).Returns(returnContent.ToAsyncEnumerable()); + + var agent = + new ChatCompletionAgent() + { + Instructions = "test instructions", + Kernel = CreateKernel(mockService.Object), + ExecutionSettings = new(), + }; + + var result = await agent.InvokeStreamingAsync([]).ToArrayAsync(); + + Assert.Equal(2, result.Length); + + mockService.Verify( + x => + x.GetStreamingChatMessageContentsAsync( + It.IsAny(), + It.IsAny(), + It.IsAny(), + It.IsAny()), + Times.Once); + } + private static Kernel CreateKernel(IChatCompletionService chatCompletionService) { var builder = Kernel.CreateBuilder(); diff --git a/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/Clients/GeminiChatGenerationTests.cs b/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/Clients/GeminiChatGenerationTests.cs index 6b5bda155483..5232c40b005d 100644 --- a/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/Clients/GeminiChatGenerationTests.cs +++ b/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/Clients/GeminiChatGenerationTests.cs @@ -259,21 +259,7 @@ await Assert.ThrowsAsync( } [Fact] - public async Task ShouldThrowInvalidOperationExceptionIfChatHistoryContainsMoreThanOneSystemMessageAsync() - { - var client = this.CreateChatCompletionClient(); - var chatHistory = new ChatHistory("System message"); - chatHistory.AddSystemMessage("System message 2"); - chatHistory.AddSystemMessage("System message 3"); - chatHistory.AddUserMessage("hello"); - - // Act & Assert - await Assert.ThrowsAsync( - () => client.GenerateChatMessageAsync(chatHistory)); - } - - [Fact] - public async Task ShouldPassConvertedSystemMessageToUserMessageToRequestAsync() + public async Task ShouldPassSystemMessageToRequestAsync() { // Arrange var client = this.CreateChatCompletionClient(); @@ -287,40 +273,35 @@ public async Task ShouldPassConvertedSystemMessageToUserMessageToRequestAsync() // Assert GeminiRequest? request = JsonSerializer.Deserialize(this._messageHandlerStub.RequestContent); Assert.NotNull(request); - var systemMessage = request.Contents[0].Parts![0].Text; - var messageRole = request.Contents[0].Role; - Assert.Equal(AuthorRole.User, messageRole); + Assert.NotNull(request.SystemInstruction); + var systemMessage = request.SystemInstruction.Parts![0].Text; + Assert.Null(request.SystemInstruction.Role); Assert.Equal(message, systemMessage); } [Fact] - public async Task ShouldThrowNotSupportedIfChatHistoryHaveIncorrectOrderAsync() + public async Task ShouldPassMultipleSystemMessagesToRequestAsync() { // Arrange + string[] messages = ["System message 1", "System message 2", "System message 3"]; var client = this.CreateChatCompletionClient(); - var chatHistory = new ChatHistory(); + var chatHistory = new ChatHistory(messages[0]); + chatHistory.AddSystemMessage(messages[1]); + chatHistory.AddSystemMessage(messages[2]); chatHistory.AddUserMessage("Hello"); - chatHistory.AddAssistantMessage("Hi"); - chatHistory.AddAssistantMessage("Hi me again"); - chatHistory.AddUserMessage("How are you?"); - // Act & Assert - await Assert.ThrowsAsync( - () => client.GenerateChatMessageAsync(chatHistory)); - } - - [Fact] - public async Task ShouldThrowNotSupportedIfChatHistoryNotEndWithUserMessageAsync() - { - // Arrange - var client = this.CreateChatCompletionClient(); - var chatHistory = new ChatHistory(); - chatHistory.AddUserMessage("Hello"); - chatHistory.AddAssistantMessage("Hi"); + // Act + await client.GenerateChatMessageAsync(chatHistory); - // Act & Assert - await Assert.ThrowsAsync( - () => client.GenerateChatMessageAsync(chatHistory)); + // Assert + GeminiRequest? request = JsonSerializer.Deserialize(this._messageHandlerStub.RequestContent); + Assert.NotNull(request); + Assert.NotNull(request.SystemInstruction); + Assert.Null(request.SystemInstruction.Role); + Assert.Collection(request.SystemInstruction.Parts!, + item => Assert.Equal(messages[0], item.Text), + item => Assert.Equal(messages[1], item.Text), + item => Assert.Equal(messages[2], item.Text)); } [Fact] diff --git a/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/Clients/GeminiChatStreamingTests.cs b/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/Clients/GeminiChatStreamingTests.cs index 73b647429297..d47115fe4ebc 100644 --- a/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/Clients/GeminiChatStreamingTests.cs +++ b/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/Clients/GeminiChatStreamingTests.cs @@ -248,7 +248,7 @@ public async Task ShouldUsePromptExecutionSettingsAsync() } [Fact] - public async Task ShouldPassConvertedSystemMessageToUserMessageToRequestAsync() + public async Task ShouldPassSystemMessageToRequestAsync() { // Arrange var client = this.CreateChatCompletionClient(); @@ -262,12 +262,37 @@ public async Task ShouldPassConvertedSystemMessageToUserMessageToRequestAsync() // Assert GeminiRequest? request = JsonSerializer.Deserialize(this._messageHandlerStub.RequestContent); Assert.NotNull(request); - var systemMessage = request.Contents[0].Parts![0].Text; - var messageRole = request.Contents[0].Role; - Assert.Equal(AuthorRole.User, messageRole); + Assert.NotNull(request.SystemInstruction); + var systemMessage = request.SystemInstruction.Parts![0].Text; + Assert.Null(request.SystemInstruction.Role); Assert.Equal(message, systemMessage); } + [Fact] + public async Task ShouldPassMultipleSystemMessagesToRequestAsync() + { + // Arrange + string[] messages = ["System message 1", "System message 2", "System message 3"]; + var client = this.CreateChatCompletionClient(); + var chatHistory = new ChatHistory(messages[0]); + chatHistory.AddSystemMessage(messages[1]); + chatHistory.AddSystemMessage(messages[2]); + chatHistory.AddUserMessage("Hello"); + + // Act + await client.StreamGenerateChatMessageAsync(chatHistory).ToListAsync(); + + // Assert + GeminiRequest? request = JsonSerializer.Deserialize(this._messageHandlerStub.RequestContent); + Assert.NotNull(request); + Assert.NotNull(request.SystemInstruction); + Assert.Null(request.SystemInstruction.Role); + Assert.Collection(request.SystemInstruction.Parts!, + item => Assert.Equal(messages[0], item.Text), + item => Assert.Equal(messages[1], item.Text), + item => Assert.Equal(messages[2], item.Text)); + } + [Theory] [InlineData(0)] [InlineData(-15)] diff --git a/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/GeminiRequestTests.cs b/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/GeminiRequestTests.cs index 4053fb8ee79f..e74ce51d4463 100644 --- a/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/GeminiRequestTests.cs +++ b/dotnet/src/Connectors/Connectors.Google.UnitTests/Core/Gemini/GeminiRequestTests.cs @@ -15,7 +15,7 @@ namespace SemanticKernel.Connectors.Google.UnitTests.Core.Gemini; public sealed class GeminiRequestTests { [Fact] - public void FromPromptItReturnsGeminiRequestWithConfiguration() + public void FromPromptItReturnsWithConfiguration() { // Arrange var prompt = "prompt-example"; @@ -37,7 +37,7 @@ public void FromPromptItReturnsGeminiRequestWithConfiguration() } [Fact] - public void FromPromptItReturnsGeminiRequestWithSafetySettings() + public void FromPromptItReturnsWithSafetySettings() { // Arrange var prompt = "prompt-example"; @@ -59,7 +59,7 @@ public void FromPromptItReturnsGeminiRequestWithSafetySettings() } [Fact] - public void FromPromptItReturnsGeminiRequestWithPrompt() + public void FromPromptItReturnsWithPrompt() { // Arrange var prompt = "prompt-example"; @@ -73,7 +73,7 @@ public void FromPromptItReturnsGeminiRequestWithPrompt() } [Fact] - public void FromChatHistoryItReturnsGeminiRequestWithConfiguration() + public void FromChatHistoryItReturnsWithConfiguration() { // Arrange ChatHistory chatHistory = []; @@ -98,7 +98,7 @@ public void FromChatHistoryItReturnsGeminiRequestWithConfiguration() } [Fact] - public void FromChatHistoryItReturnsGeminiRequestWithSafetySettings() + public void FromChatHistoryItReturnsWithSafetySettings() { // Arrange ChatHistory chatHistory = []; @@ -123,10 +123,11 @@ public void FromChatHistoryItReturnsGeminiRequestWithSafetySettings() } [Fact] - public void FromChatHistoryItReturnsGeminiRequestWithChatHistory() + public void FromChatHistoryItReturnsWithChatHistory() { // Arrange - ChatHistory chatHistory = []; + string systemMessage = "system-message"; + var chatHistory = new ChatHistory(systemMessage); chatHistory.AddUserMessage("user-message"); chatHistory.AddAssistantMessage("assist-message"); chatHistory.AddUserMessage("user-message2"); @@ -136,18 +137,41 @@ public void FromChatHistoryItReturnsGeminiRequestWithChatHistory() var request = GeminiRequest.FromChatHistoryAndExecutionSettings(chatHistory, executionSettings); // Assert + Assert.NotNull(request.SystemInstruction?.Parts); + Assert.Single(request.SystemInstruction.Parts); + Assert.Equal(request.SystemInstruction.Parts[0].Text, systemMessage); Assert.Collection(request.Contents, - c => Assert.Equal(chatHistory[0].Content, c.Parts![0].Text), c => Assert.Equal(chatHistory[1].Content, c.Parts![0].Text), - c => Assert.Equal(chatHistory[2].Content, c.Parts![0].Text)); + c => Assert.Equal(chatHistory[2].Content, c.Parts![0].Text), + c => Assert.Equal(chatHistory[3].Content, c.Parts![0].Text)); Assert.Collection(request.Contents, - c => Assert.Equal(chatHistory[0].Role, c.Role), c => Assert.Equal(chatHistory[1].Role, c.Role), - c => Assert.Equal(chatHistory[2].Role, c.Role)); + c => Assert.Equal(chatHistory[2].Role, c.Role), + c => Assert.Equal(chatHistory[3].Role, c.Role)); + } + + [Fact] + public void FromChatHistoryMultipleSystemMessagesItReturnsWithSystemMessages() + { + // Arrange + string[] systemMessages = ["system-message", "system-message2", "system-message3", "system-message4"]; + var chatHistory = new ChatHistory(systemMessages[0]); + chatHistory.AddUserMessage("user-message"); + chatHistory.AddSystemMessage(systemMessages[1]); + chatHistory.AddMessage(AuthorRole.System, + [new TextContent(systemMessages[2]), new TextContent(systemMessages[3])]); + var executionSettings = new GeminiPromptExecutionSettings(); + + // Act + var request = GeminiRequest.FromChatHistoryAndExecutionSettings(chatHistory, executionSettings); + + // Assert + Assert.NotNull(request.SystemInstruction?.Parts); + Assert.All(systemMessages, msg => Assert.Contains(request.SystemInstruction.Parts, p => p.Text == msg)); } [Fact] - public void FromChatHistoryTextAsTextContentItReturnsGeminiRequestWithChatHistory() + public void FromChatHistoryTextAsTextContentItReturnsWithChatHistory() { // Arrange ChatHistory chatHistory = []; @@ -163,11 +187,11 @@ public void FromChatHistoryTextAsTextContentItReturnsGeminiRequestWithChatHistor Assert.Collection(request.Contents, c => Assert.Equal(chatHistory[0].Content, c.Parts![0].Text), c => Assert.Equal(chatHistory[1].Content, c.Parts![0].Text), - c => Assert.Equal(chatHistory[2].Items!.Cast().Single().Text, c.Parts![0].Text)); + c => Assert.Equal(chatHistory[2].Items.Cast().Single().Text, c.Parts![0].Text)); } [Fact] - public void FromChatHistoryImageAsImageContentItReturnsGeminiRequestWithChatHistory() + public void FromChatHistoryImageAsImageContentItReturnsWithChatHistory() { // Arrange ReadOnlyMemory imageAsBytes = new byte[] { 0x00, 0x01, 0x02, 0x03 }; @@ -187,7 +211,7 @@ public void FromChatHistoryImageAsImageContentItReturnsGeminiRequestWithChatHist Assert.Collection(request.Contents, c => Assert.Equal(chatHistory[0].Content, c.Parts![0].Text), c => Assert.Equal(chatHistory[1].Content, c.Parts![0].Text), - c => Assert.Equal(chatHistory[2].Items!.Cast().Single().Uri, + c => Assert.Equal(chatHistory[2].Items.Cast().Single().Uri, c.Parts![0].FileData!.FileUri), c => Assert.True(imageAsBytes.ToArray() .SequenceEqual(Convert.FromBase64String(c.Parts![0].InlineData!.InlineData)))); @@ -272,7 +296,7 @@ public void FromChatHistoryToolCallsNotNullAddsFunctionCalls() } [Fact] - public void AddFunctionItAddsFunctionToGeminiRequest() + public void AddFunctionToGeminiRequest() { // Arrange var request = new GeminiRequest(); @@ -287,7 +311,7 @@ public void AddFunctionItAddsFunctionToGeminiRequest() } [Fact] - public void AddMultipleFunctionsItAddsFunctionsToGeminiRequest() + public void AddMultipleFunctionsToGeminiRequest() { // Arrange var request = new GeminiRequest(); @@ -308,7 +332,7 @@ public void AddMultipleFunctionsItAddsFunctionsToGeminiRequest() } [Fact] - public void AddChatMessageToRequestItAddsChatMessageToGeminiRequest() + public void AddChatMessageToRequest() { // Arrange ChatHistory chat = []; diff --git a/dotnet/src/Connectors/Connectors.Google/Core/Gemini/Clients/GeminiChatCompletionClient.cs b/dotnet/src/Connectors/Connectors.Google/Core/Gemini/Clients/GeminiChatCompletionClient.cs index e52b5f4e6bd6..9750af44c0c7 100644 --- a/dotnet/src/Connectors/Connectors.Google/Core/Gemini/Clients/GeminiChatCompletionClient.cs +++ b/dotnet/src/Connectors/Connectors.Google/Core/Gemini/Clients/GeminiChatCompletionClient.cs @@ -164,11 +164,11 @@ public async Task> GenerateChatMessageAsync( for (state.Iteration = 1; ; state.Iteration++) { - GeminiResponse geminiResponse; List chatResponses; using (var activity = ModelDiagnostics.StartCompletionActivity( this._chatGenerationEndpoint, this._modelId, ModelProvider, chatHistory, state.ExecutionSettings)) { + GeminiResponse geminiResponse; try { geminiResponse = await this.SendRequestAndReturnValidGeminiResponseAsync( @@ -297,8 +297,7 @@ private ChatCompletionState ValidateInputAndCreateChatCompletionState( Kernel? kernel, PromptExecutionSettings? executionSettings) { - var chatHistoryCopy = new ChatHistory(chatHistory); - ValidateAndPrepareChatHistory(chatHistoryCopy); + ValidateChatHistory(chatHistory); var geminiExecutionSettings = GeminiPromptExecutionSettings.FromExecutionSettings(executionSettings); ValidateMaxTokens(geminiExecutionSettings.MaxTokens); @@ -315,7 +314,7 @@ private ChatCompletionState ValidateInputAndCreateChatCompletionState( AutoInvoke = CheckAutoInvokeCondition(kernel, geminiExecutionSettings), ChatHistory = chatHistory, ExecutionSettings = geminiExecutionSettings, - GeminiRequest = CreateRequest(chatHistoryCopy, geminiExecutionSettings, kernel), + GeminiRequest = CreateRequest(chatHistory, geminiExecutionSettings, kernel), Kernel = kernel! // not null if auto-invoke is true }; } @@ -517,61 +516,12 @@ private static bool CheckAutoInvokeCondition(Kernel? kernel, GeminiPromptExecuti return autoInvoke; } - private static void ValidateAndPrepareChatHistory(ChatHistory chatHistory) + private static void ValidateChatHistory(ChatHistory chatHistory) { Verify.NotNullOrEmpty(chatHistory); - - if (chatHistory.Where(message => message.Role == AuthorRole.System).ToList() is { Count: > 0 } systemMessages) - { - if (chatHistory.Count == systemMessages.Count) - { - throw new InvalidOperationException("Chat history can't contain only system messages."); - } - - if (systemMessages.Count > 1) - { - throw new InvalidOperationException("Chat history can't contain more than one system message. " + - "Only the first system message will be processed but will be converted to the user message before sending to the Gemini api."); - } - - ConvertSystemMessageToUserMessageInChatHistory(chatHistory, systemMessages[0]); - } - - ValidateChatHistoryMessagesOrder(chatHistory); - } - - private static void ConvertSystemMessageToUserMessageInChatHistory(ChatHistory chatHistory, ChatMessageContent systemMessage) - { - // TODO: This solution is needed due to the fact that Gemini API doesn't support system messages. Maybe in the future we will be able to remove it. - chatHistory.Remove(systemMessage); - if (!string.IsNullOrWhiteSpace(systemMessage.Content)) - { - chatHistory.Insert(0, new ChatMessageContent(AuthorRole.User, systemMessage.Content)); - chatHistory.Insert(1, new ChatMessageContent(AuthorRole.Assistant, "OK")); - } - } - - private static void ValidateChatHistoryMessagesOrder(ChatHistory chatHistory) - { - bool incorrectOrder = false; - // Exclude tool calls from the validation - ChatHistory chatHistoryCopy = new(chatHistory - .Where(message => message.Role != AuthorRole.Tool && (message is not GeminiChatMessageContent { ToolCalls: not null }))); - for (int i = 0; i < chatHistoryCopy.Count; i++) - { - if (chatHistoryCopy[i].Role != (i % 2 == 0 ? AuthorRole.User : AuthorRole.Assistant) || - (i == chatHistoryCopy.Count - 1 && chatHistoryCopy[i].Role != AuthorRole.User)) - { - incorrectOrder = true; - break; - } - } - - if (incorrectOrder) + if (chatHistory.All(message => message.Role == AuthorRole.System)) { - throw new NotSupportedException( - "Gemini API support only chat history with order of messages alternates between the user and the assistant. " + - "Last message have to be User message."); + throw new InvalidOperationException("Chat history can't contain only system messages."); } } diff --git a/dotnet/src/Connectors/Connectors.Google/Core/Gemini/Models/GeminiRequest.cs b/dotnet/src/Connectors/Connectors.Google/Core/Gemini/Models/GeminiRequest.cs index def81d9a7083..c50b6b33db46 100644 --- a/dotnet/src/Connectors/Connectors.Google/Core/Gemini/Models/GeminiRequest.cs +++ b/dotnet/src/Connectors/Connectors.Google/Core/Gemini/Models/GeminiRequest.cs @@ -26,6 +26,10 @@ internal sealed class GeminiRequest [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] public IList? Tools { get; set; } + [JsonPropertyName("systemInstruction")] + [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] + public GeminiContent? SystemInstruction { get; set; } + public void AddFunction(GeminiFunction function) { // NOTE: Currently Gemini only supports one tool i.e. function calling. @@ -95,7 +99,10 @@ private static GeminiRequest CreateGeminiRequest(ChatHistory chatHistory) { GeminiRequest obj = new() { - Contents = chatHistory.Select(CreateGeminiContentFromChatMessage).ToList() + Contents = chatHistory + .Where(message => message.Role != AuthorRole.System) + .Select(CreateGeminiContentFromChatMessage).ToList(), + SystemInstruction = CreateSystemMessages(chatHistory) }; return obj; } @@ -109,6 +116,20 @@ private static GeminiContent CreateGeminiContentFromChatMessage(ChatMessageConte }; } + private static GeminiContent? CreateSystemMessages(ChatHistory chatHistory) + { + var contents = chatHistory.Where(message => message.Role == AuthorRole.System).ToList(); + if (contents.Count == 0) + { + return null; + } + + return new GeminiContent + { + Parts = CreateGeminiParts(contents) + }; + } + public void AddChatMessage(ChatMessageContent message) { Verify.NotNull(this.Contents); @@ -117,6 +138,24 @@ public void AddChatMessage(ChatMessageContent message) this.Contents.Add(CreateGeminiContentFromChatMessage(message)); } + private static List CreateGeminiParts(IEnumerable contents) + { + List? parts = null; + foreach (var content in contents) + { + if (parts == null) + { + parts = CreateGeminiParts(content); + } + else + { + parts.AddRange(CreateGeminiParts(content)); + } + } + + return parts!; + } + private static List CreateGeminiParts(ChatMessageContent content) { List parts = []; diff --git a/dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceMessageApiClient.cs b/dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceMessageApiClient.cs index 66bd8cdbf365..468f24490edb 100644 --- a/dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceMessageApiClient.cs +++ b/dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceMessageApiClient.cs @@ -85,9 +85,8 @@ internal async IAsyncEnumerable StreamCompleteChatM var endpoint = this.GetChatGenerationEndpoint(); var huggingFaceExecutionSettings = HuggingFacePromptExecutionSettings.FromExecutionSettings(executionSettings); - huggingFaceExecutionSettings.ModelId ??= this._clientCore.ModelId; - var request = this.CreateChatRequest(chatHistory, huggingFaceExecutionSettings); + var request = this.CreateChatRequest(chatHistory, huggingFaceExecutionSettings, modelId); request.Stream = true; using var activity = ModelDiagnostics.StartCompletionActivity(endpoint, modelId, this._clientCore.ModelProvider, chatHistory, huggingFaceExecutionSettings); @@ -149,8 +148,7 @@ internal async Task> CompleteChatMessageAsync( var endpoint = this.GetChatGenerationEndpoint(); var huggingFaceExecutionSettings = HuggingFacePromptExecutionSettings.FromExecutionSettings(executionSettings); - huggingFaceExecutionSettings.ModelId ??= this._clientCore.ModelId; - var request = this.CreateChatRequest(chatHistory, huggingFaceExecutionSettings); + var request = this.CreateChatRequest(chatHistory, huggingFaceExecutionSettings, modelId); using var activity = ModelDiagnostics.StartCompletionActivity(endpoint, modelId, this._clientCore.ModelProvider, chatHistory, huggingFaceExecutionSettings); using var httpRequestMessage = this._clientCore.CreatePost(request, endpoint, this._clientCore.ApiKey); @@ -276,7 +274,8 @@ private async IAsyncEnumerable ProcessChatResponseS private ChatCompletionRequest CreateChatRequest( ChatHistory chatHistory, - HuggingFacePromptExecutionSettings huggingFaceExecutionSettings) + HuggingFacePromptExecutionSettings huggingFaceExecutionSettings, + string modelId) { HuggingFaceClient.ValidateMaxTokens(huggingFaceExecutionSettings.MaxTokens); @@ -287,7 +286,7 @@ private ChatCompletionRequest CreateChatRequest( JsonSerializer.Serialize(huggingFaceExecutionSettings)); } - var request = ChatCompletionRequest.FromChatHistoryAndExecutionSettings(chatHistory, huggingFaceExecutionSettings); + var request = ChatCompletionRequest.FromChatHistoryAndExecutionSettings(chatHistory, huggingFaceExecutionSettings, modelId); return request; } diff --git a/dotnet/src/Connectors/Connectors.HuggingFace/Core/Models/ChatCompletionRequest.cs b/dotnet/src/Connectors/Connectors.HuggingFace/Core/Models/ChatCompletionRequest.cs index e3f930fecfb9..886e13f18bda 100644 --- a/dotnet/src/Connectors/Connectors.HuggingFace/Core/Models/ChatCompletionRequest.cs +++ b/dotnet/src/Connectors/Connectors.HuggingFace/Core/Models/ChatCompletionRequest.cs @@ -102,8 +102,9 @@ internal sealed class ChatCompletionRequest /// /// Chat history to be used for the request. /// Execution settings to be used for the request. - /// TexGenerationtRequest object. - internal static ChatCompletionRequest FromChatHistoryAndExecutionSettings(ChatHistory chatHistory, HuggingFacePromptExecutionSettings executionSettings) + /// Model id to use if value in prompt execution settings is not set. + /// TexGenerationRequest object. + internal static ChatCompletionRequest FromChatHistoryAndExecutionSettings(ChatHistory chatHistory, HuggingFacePromptExecutionSettings executionSettings, string modelId) { return new ChatCompletionRequest { @@ -118,7 +119,7 @@ internal static ChatCompletionRequest FromChatHistoryAndExecutionSettings(ChatHi Temperature = executionSettings.Temperature, Stop = executionSettings.Stop, MaxTokens = executionSettings.MaxTokens, - Model = executionSettings.ModelId ?? TextGenerationInferenceDefaultModel, + Model = executionSettings.ModelId ?? modelId ?? TextGenerationInferenceDefaultModel, TopP = executionSettings.TopP, TopLogProbs = executionSettings.TopLogProbs }; diff --git a/dotnet/src/Connectors/Connectors.Memory.Kusto/KustoMemoryStore.cs b/dotnet/src/Connectors/Connectors.Memory.Kusto/KustoMemoryStore.cs index dcccc7983b91..359de4d57a5e 100644 --- a/dotnet/src/Connectors/Connectors.Memory.Kusto/KustoMemoryStore.cs +++ b/dotnet/src/Connectors/Connectors.Memory.Kusto/KustoMemoryStore.cs @@ -358,7 +358,7 @@ protected virtual void Dispose(bool disposing) /// Kusto table name. /// Boolean flag that indicates if table name normalization is needed. private static string GetTableName(string collectionName, bool normalized = true) - => normalized ? CslSyntaxGenerator.NormalizeTableName(collectionName) : collectionName; + => normalized ? CslSyntaxGenerator.NormalizeName(collectionName) : collectionName; /// /// Converts Kusto table name to collection name. diff --git a/dotnet/src/Connectors/Connectors.Memory.Milvus/MilvusMemoryStore.cs b/dotnet/src/Connectors/Connectors.Memory.Milvus/MilvusMemoryStore.cs index 38d10778a723..7bdd2f03db94 100644 --- a/dotnet/src/Connectors/Connectors.Memory.Milvus/MilvusMemoryStore.cs +++ b/dotnet/src/Connectors/Connectors.Memory.Milvus/MilvusMemoryStore.cs @@ -446,7 +446,7 @@ public Task RemoveBatchAsync(string collectionName, IEnumerable keys, Ca MilvusCollection collection = this.Client.GetCollection(collectionName); SearchResults results = await collection - .SearchAsync(EmbeddingFieldName, [embedding], SimilarityMetricType.Ip, limit, this._searchParameters, cancellationToken) + .SearchAsync(EmbeddingFieldName, [embedding], this._metricType, limit, this._searchParameters, cancellationToken) .ConfigureAwait(false); IReadOnlyList ids = results.Ids.StringIds!; diff --git a/dotnet/src/Connectors/Connectors.Memory.Qdrant/Connectors.Memory.Qdrant.csproj b/dotnet/src/Connectors/Connectors.Memory.Qdrant/Connectors.Memory.Qdrant.csproj index da803a71b52a..d9037605f6e5 100644 --- a/dotnet/src/Connectors/Connectors.Memory.Qdrant/Connectors.Memory.Qdrant.csproj +++ b/dotnet/src/Connectors/Connectors.Memory.Qdrant/Connectors.Memory.Qdrant.csproj @@ -22,6 +22,10 @@ + + + + diff --git a/dotnet/src/Connectors/Connectors.Memory.Redis/Connectors.Memory.Redis.csproj b/dotnet/src/Connectors/Connectors.Memory.Redis/Connectors.Memory.Redis.csproj index 878cc229aeaf..52f2ea6f159c 100644 --- a/dotnet/src/Connectors/Connectors.Memory.Redis/Connectors.Memory.Redis.csproj +++ b/dotnet/src/Connectors/Connectors.Memory.Redis/Connectors.Memory.Redis.csproj @@ -1,4 +1,4 @@ - + @@ -23,6 +23,10 @@ + + + + diff --git a/dotnet/src/Connectors/Connectors.Memory.SqlServer/SqlServerClient.cs b/dotnet/src/Connectors/Connectors.Memory.SqlServer/SqlServerClient.cs index 222381814b4a..a52716e909b2 100644 --- a/dotnet/src/Connectors/Connectors.Memory.SqlServer/SqlServerClient.cs +++ b/dotnet/src/Connectors/Connectors.Memory.SqlServer/SqlServerClient.cs @@ -33,10 +33,23 @@ public SqlServerClient(SqlConnection connection, string schema) this._schema = schema; } + private async Task HasJsonNativeTypeAsync(CancellationToken cancellationToken = default) + { + using (await this.OpenConnectionAsync(cancellationToken).ConfigureAwait(false)) + { + using var cmd = this._connection.CreateCommand(); + cmd.CommandText = "select [name] from sys.types where system_type_id = 244 and user_type_id = 244"; + var typeName = (string)await cmd.ExecuteScalarAsync(cancellationToken).ConfigureAwait(false); + return string.Equals(typeName, "json", StringComparison.OrdinalIgnoreCase); + } + } + /// public async Task CreateTableAsync(string tableName, CancellationToken cancellationToken = default) { var fullTableName = this.GetSanitizedFullTableName(tableName); + var metadataType = await this.HasJsonNativeTypeAsync(cancellationToken).ConfigureAwait(false) ? "json" : "nvarchar(max)"; + using (await this.OpenConnectionAsync(cancellationToken).ConfigureAwait(false)) { using var cmd = this._connection.CreateCommand(); @@ -44,7 +57,7 @@ public async Task CreateTableAsync(string tableName, CancellationToken cancellat IF OBJECT_ID(N'{fullTableName}', N'U') IS NULL CREATE TABLE {fullTableName} ( [key] nvarchar(255) collate latin1_general_bin2 not null, - [metadata] nvarchar(max) not null, + [metadata] {metadataType} not null, [embedding] varbinary(8000), [timestamp] datetimeoffset, PRIMARY KEY NONCLUSTERED ([key]), @@ -138,9 +151,9 @@ WHEN NOT MATCHED THEN /// public async IAsyncEnumerable ReadBatchAsync(string tableName, IEnumerable keys, bool withEmbeddings = false, [EnumeratorCancellation] CancellationToken cancellationToken = default) { - var queryColumns = withEmbeddings - ? "[key], [metadata], [timestamp], VECTOR_TO_JSON_ARRAY([embedding]) AS [embedding]" - : "[key], [metadata], [timestamp]"; + var queryColumns = "[key], [metadata], [timestamp]" + + (withEmbeddings ? ", VECTOR_TO_JSON_ARRAY([embedding]) AS [embedding]" : string.Empty); + var fullTableName = this.GetSanitizedFullTableName(tableName); var keysList = keys.ToList(); var keysParams = string.Join(", ", keysList.Select((_, i) => $"@k{i}")); @@ -189,9 +202,8 @@ WHERE [key] IN ({keysParams}) /// public async IAsyncEnumerable<(SqlServerMemoryEntry, double)> GetNearestMatchesAsync(string tableName, ReadOnlyMemory embedding, int limit, double minRelevanceScore = 0, bool withEmbeddings = false, [EnumeratorCancellation] CancellationToken cancellationToken = default) { - var queryColumns = withEmbeddings - ? "[key], [metadata], [timestamp], 1 - VECTOR_DISTANCE('cosine', [embedding], JSON_ARRAY_TO_VECTOR(@e)) AS [cosine_similarity], VECTOR_TO_JSON_ARRAY([embedding]) AS [embedding]" - : "[key], [metadata], [timestamp], 1 - VECTOR_DISTANCE('cosine', [embedding], JSON_ARRAY_TO_VECTOR(@e)) AS [cosine_similarity]"; + var queryColumns = "[key], [metadata], [timestamp], 1 - VECTOR_DISTANCE('cosine', [embedding], JSON_ARRAY_TO_VECTOR(@e)) AS [cosine_similarity]" + + (withEmbeddings ? ", VECTOR_TO_JSON_ARRAY([embedding]) AS [embedding]" : string.Empty); var fullTableName = this.GetSanitizedFullTableName(tableName); using (await this.OpenConnectionAsync(cancellationToken).ConfigureAwait(false)) { @@ -221,6 +233,7 @@ ORDER BY [cosine_similarity] DESC private string GetSanitizedFullTableName(string tableName) => $"{DelimitIdentifier(this._schema)}.{DelimitIdentifier(tableName)}"; private string SerializeEmbedding(ReadOnlyMemory embedding) => JsonSerializer.Serialize(embedding); + private ReadOnlyMemory DeserializeEmbedding(string embedding) => JsonSerializer.Deserialize>(embedding); private SqlServerMemoryEntry ReadEntry(SqlDataReader reader, bool hasEmbedding) @@ -247,6 +260,7 @@ private async Task OpenConnectionAsync(CancellationToken cancellati } private static string DelimitIdentifier(string identifier) => $"[{EscapeIdentifier(identifier)}]"; + private static string EscapeIdentifier(string identifier) => identifier.Replace("]", "]]"); private readonly struct Closer(SqlServerClient client, bool shouldClose) : IDisposable diff --git a/dotnet/src/Connectors/Connectors.Memory.Sqlite/Database.cs b/dotnet/src/Connectors/Connectors.Memory.Sqlite/Database.cs index aee0735507c5..581a21afc52a 100644 --- a/dotnet/src/Connectors/Connectors.Memory.Sqlite/Database.cs +++ b/dotnet/src/Connectors/Connectors.Memory.Sqlite/Database.cs @@ -168,6 +168,27 @@ DELETE FROM {TableName} return cmd.ExecuteNonQueryAsync(cancellationToken); } + public Task DeleteBatchAsync(SqliteConnection conn, string collectionName, string[] keys, CancellationToken cancellationToken = default) + { + using SqliteCommand cmd = conn.CreateCommand(); + var keyParameters = keys.Select((key, index) => $"@key{index}"); + var parameters = string.Join(", ", keyParameters); + +#pragma warning disable CA2100 // Review SQL queries for security vulnerabilities + cmd.CommandText = $@" + DELETE FROM {TableName} + WHERE collection=@collection + AND key IN ({parameters})"; +#pragma warning restore CA2100 // Review SQL queries for security vulnerabilities + + cmd.Parameters.Add(new SqliteParameter("@collection", collectionName)); + for (int i = 0; i < keys.Length; i++) + { + cmd.Parameters.Add(new SqliteParameter($"@key{i}", keys[i])); + } + return cmd.ExecuteNonQueryAsync(cancellationToken); + } + public Task DeleteEmptyAsync(SqliteConnection conn, string collectionName, CancellationToken cancellationToken = default) { using SqliteCommand cmd = conn.CreateCommand(); diff --git a/dotnet/src/Connectors/Connectors.Memory.Sqlite/SqliteMemoryStore.cs b/dotnet/src/Connectors/Connectors.Memory.Sqlite/SqliteMemoryStore.cs index 1dbe176146ce..3891df9c4de9 100644 --- a/dotnet/src/Connectors/Connectors.Memory.Sqlite/SqliteMemoryStore.cs +++ b/dotnet/src/Connectors/Connectors.Memory.Sqlite/SqliteMemoryStore.cs @@ -113,7 +113,7 @@ public async Task RemoveAsync(string collectionName, string key, CancellationTok /// public async Task RemoveBatchAsync(string collectionName, IEnumerable keys, CancellationToken cancellationToken = default) { - await Task.WhenAll(keys.Select(k => this._dbConnector.DeleteAsync(this._dbConnection, collectionName, k, cancellationToken))).ConfigureAwait(false); + await this._dbConnector.DeleteBatchAsync(this._dbConnection, collectionName, keys.ToArray(), cancellationToken).ConfigureAwait(false); } /// @@ -218,8 +218,7 @@ private SqliteMemoryStore(string filename) private static DateTimeOffset? ParseTimestamp(string? str) { - if (!string.IsNullOrEmpty(str) - && DateTimeOffset.TryParse(str, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal, out DateTimeOffset timestamp)) + if (DateTimeOffset.TryParse(str, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal, out DateTimeOffset timestamp)) { return timestamp; } diff --git a/dotnet/src/Connectors/Connectors.MistralAI/Client/MistralClient.cs b/dotnet/src/Connectors/Connectors.MistralAI/Client/MistralClient.cs index cdd9c33f4789..fe3683d4f495 100644 --- a/dotnet/src/Connectors/Connectors.MistralAI/Client/MistralClient.cs +++ b/dotnet/src/Connectors/Connectors.MistralAI/Client/MistralClient.cs @@ -177,7 +177,8 @@ internal async Task> GetChatMessageContentsAsy Arguments = functionArgs, RequestSequenceIndex = requestIndex - 1, FunctionSequenceIndex = toolCallIndex, - FunctionCount = chatChoice.ToolCalls.Count + FunctionCount = chatChoice.ToolCalls.Count, + CancellationToken = cancellationToken }; s_inflightAutoInvokes.Value++; try @@ -409,6 +410,7 @@ internal async IAsyncEnumerable GetStreamingChatMes RequestSequenceIndex = requestIndex - 1, FunctionSequenceIndex = toolCallIndex, FunctionCount = toolCalls.Count, + CancellationToken = cancellationToken }; s_inflightAutoInvokes.Value++; try diff --git a/dotnet/src/Connectors/Connectors.OpenAI/AzureSdk/ClientCore.cs b/dotnet/src/Connectors/Connectors.OpenAI/AzureSdk/ClientCore.cs index 8059077d8bf4..5d08b38da29c 100644 --- a/dotnet/src/Connectors/Connectors.OpenAI/AzureSdk/ClientCore.cs +++ b/dotnet/src/Connectors/Connectors.OpenAI/AzureSdk/ClientCore.cs @@ -511,7 +511,8 @@ internal async Task> GetChatMessageContentsAsy Arguments = functionArgs, RequestSequenceIndex = requestIndex - 1, FunctionSequenceIndex = toolCallIndex, - FunctionCount = result.ToolCalls.Count + FunctionCount = result.ToolCalls.Count, + CancellationToken = cancellationToken }; s_inflightAutoInvokes.Value++; @@ -693,7 +694,18 @@ internal async IAsyncEnumerable GetStreamingC OpenAIFunctionToolCall.TrackStreamingToolingUpdate(update.ToolCallUpdate, ref toolCallIdsByIndex, ref functionNamesByIndex, ref functionArgumentBuildersByIndex); } - var openAIStreamingChatMessageContent = new OpenAIStreamingChatMessageContent(update, update.ChoiceIndex ?? 0, this.DeploymentOrModelName, metadata) { AuthorName = streamedName }; + AuthorRole? role = null; + if (streamedRole.HasValue) + { + role = new AuthorRole(streamedRole.Value.ToString()); + } + + OpenAIStreamingChatMessageContent openAIStreamingChatMessageContent = + new(update, update.ChoiceIndex ?? 0, this.DeploymentOrModelName, metadata) + { + AuthorName = streamedName, + Role = role, + }; if (update.ToolCallUpdate is StreamingFunctionToolCallUpdate functionCallUpdate) { @@ -798,7 +810,8 @@ internal async IAsyncEnumerable GetStreamingC Arguments = functionArgs, RequestSequenceIndex = requestIndex - 1, FunctionSequenceIndex = toolCallIndex, - FunctionCount = toolCalls.Length + FunctionCount = toolCalls.Length, + CancellationToken = cancellationToken }; s_inflightAutoInvokes.Value++; diff --git a/dotnet/src/Connectors/Connectors.OpenAI/CompatibilitySuppressions.xml b/dotnet/src/Connectors/Connectors.OpenAI/CompatibilitySuppressions.xml index 33de31d4263b..3477ed220ea0 100644 --- a/dotnet/src/Connectors/Connectors.OpenAI/CompatibilitySuppressions.xml +++ b/dotnet/src/Connectors/Connectors.OpenAI/CompatibilitySuppressions.xml @@ -1,6 +1,20 @@  + + CP0002 + F:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFilePurpose.Assistants + lib/net8.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + lib/net8.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + true + + + CP0002 + F:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFilePurpose.FineTune + lib/net8.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + lib/net8.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + true + CP0002 M:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFileService.GetFileContent(System.String,System.Threading.CancellationToken) @@ -29,6 +43,20 @@ lib/net8.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll true + + CP0002 + F:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFilePurpose.Assistants + lib/netstandard2.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + lib/netstandard2.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + true + + + CP0002 + F:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFilePurpose.FineTune + lib/netstandard2.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + lib/netstandard2.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + true + CP0002 M:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFileService.GetFileContent(System.String,System.Threading.CancellationToken) @@ -57,4 +85,32 @@ lib/netstandard2.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll true + + CP0007 + T:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFilePurpose + lib/net8.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + lib/net8.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + true + + + CP0007 + T:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFilePurpose + lib/netstandard2.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + lib/netstandard2.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + true + + + CP0008 + T:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFilePurpose + lib/net8.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + lib/net8.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + true + + + CP0008 + T:Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFilePurpose + lib/netstandard2.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + lib/netstandard2.0/Microsoft.SemanticKernel.Connectors.OpenAI.dll + true + \ No newline at end of file diff --git a/dotnet/src/Connectors/Connectors.OpenAI/Files/OpenAIFilePurpose.cs b/dotnet/src/Connectors/Connectors.OpenAI/Files/OpenAIFilePurpose.cs index a01b2d08fa8d..8d87720fa89f 100644 --- a/dotnet/src/Connectors/Connectors.OpenAI/Files/OpenAIFilePurpose.cs +++ b/dotnet/src/Connectors/Connectors.OpenAI/Files/OpenAIFilePurpose.cs @@ -1,22 +1,99 @@ // Copyright (c) Microsoft. All rights reserved. +using System; using System.Diagnostics.CodeAnalysis; namespace Microsoft.SemanticKernel.Connectors.OpenAI; /// -/// Defines the purpose associated with the uploaded file. +/// Defines the purpose associated with the uploaded file: +/// https://platform.openai.com/docs/api-reference/files/object#files/object-purpose /// [Experimental("SKEXP0010")] -public enum OpenAIFilePurpose +public readonly struct OpenAIFilePurpose : IEquatable { /// - /// File to be used by assistants for model processing. + /// File to be used by assistants as input. /// - Assistants, + public static OpenAIFilePurpose Assistants { get; } = new("assistants"); /// - /// File to be used by fine-tuning jobs. + /// File produced as assistants output. /// - FineTune, + public static OpenAIFilePurpose AssistantsOutput { get; } = new("assistants_output"); + + /// + /// Files uploaded as a batch of API requests + /// + public static OpenAIFilePurpose Batch { get; } = new("batch"); + + /// + /// File produced as result of a file included as a batch request. + /// + public static OpenAIFilePurpose BatchOutput { get; } = new("batch_output"); + + /// + /// File to be used as input to fine-tune a model. + /// + public static OpenAIFilePurpose FineTune { get; } = new("fine-tune"); + + /// + /// File produced as result of fine-tuning a model. + /// + public static OpenAIFilePurpose FineTuneResults { get; } = new("fine-tune-results"); + + /// + /// File to be used for Assistants image file inputs. + /// + public static OpenAIFilePurpose Vision { get; } = new("vision"); + + /// + /// Gets the label associated with this . + /// + public string Label { get; } + + /// + /// Creates a new instance with the provided label. + /// + /// The label to associate with this . + public OpenAIFilePurpose(string label) + { + Verify.NotNullOrWhiteSpace(label, nameof(label)); + this.Label = label!; + } + + /// + /// Returns a value indicating whether two instances are equivalent, as determined by a + /// case-insensitive comparison of their labels. + /// + /// the first instance to compare + /// the second instance to compare + /// true if left and right are both null or have equivalent labels; false otherwise + public static bool operator ==(OpenAIFilePurpose left, OpenAIFilePurpose right) + => left.Equals(right); + + /// + /// Returns a value indicating whether two instances are not equivalent, as determined by a + /// case-insensitive comparison of their labels. + /// + /// the first instance to compare + /// the second instance to compare + /// false if left and right are both null or have equivalent labels; true otherwise + public static bool operator !=(OpenAIFilePurpose left, OpenAIFilePurpose right) + => !(left == right); + + /// + public override bool Equals([NotNullWhen(true)] object? obj) + => obj is OpenAIFilePurpose otherPurpose && this == otherPurpose; + + /// + public bool Equals(OpenAIFilePurpose other) + => string.Equals(this.Label, other.Label, StringComparison.OrdinalIgnoreCase); + + /// + public override int GetHashCode() + => StringComparer.OrdinalIgnoreCase.GetHashCode(this.Label); + + /// + public override string ToString() => this.Label; } diff --git a/dotnet/src/Connectors/Connectors.OpenAI/Files/OpenAIFileService.cs b/dotnet/src/Connectors/Connectors.OpenAI/Files/OpenAIFileService.cs index cc61734f44c8..690954448eea 100644 --- a/dotnet/src/Connectors/Connectors.OpenAI/Files/OpenAIFileService.cs +++ b/dotnet/src/Connectors/Connectors.OpenAI/Files/OpenAIFileService.cs @@ -112,7 +112,8 @@ public async Task DeleteFileAsync(string id, CancellationToken cancellationToken public async Task GetFileContentAsync(string id, CancellationToken cancellationToken = default) { Verify.NotNull(id, nameof(id)); - var (stream, mimetype) = await this.StreamGetRequestAsync($"{this._serviceUri}/{id}/content", cancellationToken).ConfigureAwait(false); + var contentUri = $"{this._serviceUri}/{id}/content"; + var (stream, mimetype) = await this.StreamGetRequestAsync(contentUri, cancellationToken).ConfigureAwait(false); using (stream) { @@ -123,7 +124,12 @@ public async Task GetFileContentAsync(string id, CancellationToke #else await stream.CopyToAsync(memoryStream, cancellationToken).ConfigureAwait(false); #endif - return new BinaryContent(memoryStream.ToArray(), mimetype); + return + new(memoryStream.ToArray(), mimetype) + { + Metadata = new Dictionary() { { "id", id } }, + Uri = new Uri(contentUri), + }; } } @@ -147,9 +153,19 @@ public async Task GetFileAsync(string id, CancellationToken /// /// The to monitor for cancellation requests. The default is . /// The metadata of all uploaded files. - public async Task> GetFilesAsync(CancellationToken cancellationToken = default) + public Task> GetFilesAsync(CancellationToken cancellationToken = default) + => this.GetFilesAsync(null, cancellationToken); + + /// + /// Retrieve metadata for previously uploaded files + /// + /// The purpose of the files by which to filter. + /// The to monitor for cancellation requests. The default is . + /// The metadata of all uploaded files. + public async Task> GetFilesAsync(OpenAIFilePurpose? filePurpose, CancellationToken cancellationToken = default) { - var result = await this.ExecuteGetRequestAsync(this._serviceUri.ToString(), cancellationToken).ConfigureAwait(false); + var serviceUri = filePurpose.HasValue && !string.IsNullOrEmpty(filePurpose.Value.Label) ? $"{this._serviceUri}?purpose={filePurpose}" : this._serviceUri.ToString(); + var result = await this.ExecuteGetRequestAsync(serviceUri, cancellationToken).ConfigureAwait(false); return result.Data.Select(this.ConvertFileReference).ToArray(); } @@ -167,7 +183,7 @@ public async Task UploadContentAsync(BinaryContent fileCont Verify.NotNull(fileContent.Data, nameof(fileContent.Data)); using var formData = new MultipartFormDataContent(); - using var contentPurpose = new StringContent(this.ConvertPurpose(settings.Purpose)); + using var contentPurpose = new StringContent(settings.Purpose.Label); using var contentFile = new ByteArrayContent(fileContent.Data.Value.ToArray()); formData.Add(contentPurpose, "purpose"); formData.Add(contentFile, "file", settings.FileName); @@ -281,26 +297,10 @@ private OpenAIFileReference ConvertFileReference(FileInfo result) FileName = result.FileName, CreatedTimestamp = DateTimeOffset.FromUnixTimeSeconds(result.CreatedAt).UtcDateTime, SizeInBytes = result.Bytes ?? 0, - Purpose = this.ConvertPurpose(result.Purpose), + Purpose = new(result.Purpose), }; } - private OpenAIFilePurpose ConvertPurpose(string purpose) => - purpose.ToUpperInvariant() switch - { - "ASSISTANTS" => OpenAIFilePurpose.Assistants, - "FINE-TUNE" => OpenAIFilePurpose.FineTune, - _ => throw new KernelException($"Unknown {nameof(OpenAIFilePurpose)}: {purpose}."), - }; - - private string ConvertPurpose(OpenAIFilePurpose purpose) => - purpose switch - { - OpenAIFilePurpose.Assistants => "assistants", - OpenAIFilePurpose.FineTune => "fine-tune", - _ => throw new KernelException($"Unknown {nameof(OpenAIFilePurpose)}: {purpose}."), - }; - private sealed class FileInfoList { [JsonPropertyName("data")] diff --git a/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/.editorconfig b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/.editorconfig new file mode 100644 index 000000000000..394eef685f21 --- /dev/null +++ b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/.editorconfig @@ -0,0 +1,6 @@ +# Suppressing errors for Test projects under dotnet folder +[*.cs] +dotnet_diagnostic.CA2007.severity = none # Do not directly await a Task +dotnet_diagnostic.VSTHRD111.severity = none # Use .ConfigureAwait(bool) is hidden by default, set to none to prevent IDE from changing on autosave +dotnet_diagnostic.CS1591.severity = none # Missing XML comment for publicly visible type or member +dotnet_diagnostic.IDE1006.severity = warning # Naming rule violations diff --git a/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/Connectors.Qdrant.UnitTests.csproj b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/Connectors.Qdrant.UnitTests.csproj new file mode 100644 index 000000000000..87782f3d2e8f --- /dev/null +++ b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/Connectors.Qdrant.UnitTests.csproj @@ -0,0 +1,38 @@ + + + + Microsoft.SemanticKernel.Connectors.Qdrant.UnitTests + Microsoft.SemanticKernel.Connectors.Qdrant.UnitTests + net8.0 + true + enable + disable + false + $(NoWarn);CA2007,CA1806,CA1869,CA1861,IDE0300,VSTHRD111,SKEXP0001,SKEXP0010,SKEXP0020,SKEXP0050 + + + + + + + + runtime; build; native; contentfiles; analyzers; buildtransitive + all + + + runtime; build; native; contentfiles; analyzers; buildtransitive + all + + + + + + + + + + + + diff --git a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryBuilderExtensionsTests.cs b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryBuilderExtensionsTests.cs similarity index 97% rename from dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryBuilderExtensionsTests.cs rename to dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryBuilderExtensionsTests.cs index 8d43f12d8983..897a09087f09 100644 --- a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryBuilderExtensionsTests.cs +++ b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryBuilderExtensionsTests.cs @@ -11,7 +11,7 @@ using Moq; using Xunit; -namespace SemanticKernel.Connectors.UnitTests.Qdrant; +namespace SemanticKernel.Connectors.Qdrant.UnitTests; public sealed class QdrantMemoryBuilderExtensionsTests : IDisposable { diff --git a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryStoreTests.cs b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryStoreTests.cs similarity index 99% rename from dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryStoreTests.cs rename to dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryStoreTests.cs index 499164c31c68..6ae498561065 100644 --- a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryStoreTests.cs +++ b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryStoreTests.cs @@ -12,7 +12,7 @@ using Moq; using Xunit; -namespace SemanticKernel.Connectors.UnitTests.Qdrant; +namespace SemanticKernel.Connectors.Qdrant.UnitTests; /// /// Tests for collection and upsert operations. diff --git a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryStoreTests2.cs b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryStoreTests2.cs similarity index 99% rename from dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryStoreTests2.cs rename to dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryStoreTests2.cs index a7303f9e47a6..8af2061c5d3a 100644 --- a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryStoreTests2.cs +++ b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryStoreTests2.cs @@ -11,7 +11,7 @@ using Moq; using Xunit; -namespace SemanticKernel.Connectors.UnitTests.Qdrant; +namespace SemanticKernel.Connectors.Qdrant.UnitTests; /// /// Tests for Get and Remove operations. diff --git a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryStoreTests3.cs b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryStoreTests3.cs similarity index 99% rename from dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryStoreTests3.cs rename to dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryStoreTests3.cs index f1cff494ff4d..ad7d54e2d5bb 100644 --- a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantMemoryStoreTests3.cs +++ b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantMemoryStoreTests3.cs @@ -15,7 +15,7 @@ using Moq.Protected; using Xunit; -namespace SemanticKernel.Connectors.UnitTests.Qdrant; +namespace SemanticKernel.Connectors.Qdrant.UnitTests; /// /// Tests for Search operations. diff --git a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantVectorDbClientTests.cs b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantVectorDbClientTests.cs similarity index 97% rename from dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantVectorDbClientTests.cs rename to dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantVectorDbClientTests.cs index 2223f25e62ee..41a95178a588 100644 --- a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Qdrant/QdrantVectorDbClientTests.cs +++ b/dotnet/src/Connectors/Connectors.Qdrant.UnitTests/QdrantVectorDbClientTests.cs @@ -6,7 +6,7 @@ using Microsoft.SemanticKernel.Connectors.Qdrant; using Xunit; -namespace SemanticKernel.Connectors.UnitTests.Qdrant; +namespace SemanticKernel.Connectors.Qdrant.UnitTests; public sealed class QdrantVectorDbClientTests : IDisposable { diff --git a/dotnet/src/Connectors/Connectors.Redis.UnitTests/.editorconfig b/dotnet/src/Connectors/Connectors.Redis.UnitTests/.editorconfig new file mode 100644 index 000000000000..394eef685f21 --- /dev/null +++ b/dotnet/src/Connectors/Connectors.Redis.UnitTests/.editorconfig @@ -0,0 +1,6 @@ +# Suppressing errors for Test projects under dotnet folder +[*.cs] +dotnet_diagnostic.CA2007.severity = none # Do not directly await a Task +dotnet_diagnostic.VSTHRD111.severity = none # Use .ConfigureAwait(bool) is hidden by default, set to none to prevent IDE from changing on autosave +dotnet_diagnostic.CS1591.severity = none # Missing XML comment for publicly visible type or member +dotnet_diagnostic.IDE1006.severity = warning # Naming rule violations diff --git a/dotnet/src/Connectors/Connectors.Redis.UnitTests/Connectors.Redis.UnitTests.csproj b/dotnet/src/Connectors/Connectors.Redis.UnitTests/Connectors.Redis.UnitTests.csproj new file mode 100644 index 000000000000..c54e1a3b5136 --- /dev/null +++ b/dotnet/src/Connectors/Connectors.Redis.UnitTests/Connectors.Redis.UnitTests.csproj @@ -0,0 +1,37 @@ + + + + Microsoft.SemanticKernel.Connectors.Redis.UnitTests + Microsoft.SemanticKernel.Connectors.Redis.UnitTests + net8.0 + true + enable + disable + false + $(NoWarn);CA2007,CA1806,CA1869,CA1861,IDE0300,VSTHRD111,SKEXP0001,SKEXP0010,SKEXP0020,SKEXP0050 + + + + + + + + runtime; build; native; contentfiles; analyzers; buildtransitive + all + + + runtime; build; native; contentfiles; analyzers; buildtransitive + all + + + + + + + + + + + + diff --git a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Redis/RedisMemoryStoreTests.cs b/dotnet/src/Connectors/Connectors.Redis.UnitTests/RedisMemoryStoreTests.cs similarity index 99% rename from dotnet/src/Connectors/Connectors.UnitTests/Memory/Redis/RedisMemoryStoreTests.cs rename to dotnet/src/Connectors/Connectors.Redis.UnitTests/RedisMemoryStoreTests.cs index 53f41384171d..5c63e568a3a9 100644 --- a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Redis/RedisMemoryStoreTests.cs +++ b/dotnet/src/Connectors/Connectors.Redis.UnitTests/RedisMemoryStoreTests.cs @@ -15,7 +15,7 @@ using StackExchange.Redis; using Xunit; -namespace SemanticKernel.Connectors.UnitTests.Redis; +namespace SemanticKernel.Connectors.Redis.UnitTests; /// /// Unit tests of . diff --git a/dotnet/src/Connectors/Connectors.UnitTests/Connectors.UnitTests.csproj b/dotnet/src/Connectors/Connectors.UnitTests/Connectors.UnitTests.csproj index 455206f5ce04..a4b7bd6ace44 100644 --- a/dotnet/src/Connectors/Connectors.UnitTests/Connectors.UnitTests.csproj +++ b/dotnet/src/Connectors/Connectors.UnitTests/Connectors.UnitTests.csproj @@ -39,14 +39,11 @@ - - - diff --git a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Kusto/KustoMemoryStoreTests.cs b/dotnet/src/Connectors/Connectors.UnitTests/Memory/Kusto/KustoMemoryStoreTests.cs index d8a2ec5c78cc..7cdec0210775 100644 --- a/dotnet/src/Connectors/Connectors.UnitTests/Memory/Kusto/KustoMemoryStoreTests.cs +++ b/dotnet/src/Connectors/Connectors.UnitTests/Memory/Kusto/KustoMemoryStoreTests.cs @@ -25,6 +25,7 @@ public class KustoMemoryStoreTests private const string DatabaseName = "FakeDb"; private readonly Mock _cslQueryProviderMock; private readonly Mock _cslAdminProviderMock; + private readonly string _normalisedCollectionName = CslSyntaxGenerator.NormalizeName(CollectionName); public KustoMemoryStoreTests() { @@ -145,7 +146,7 @@ public async Task ItCanUpsertAsync() // Assert this._cslAdminProviderMock.Verify(client => client.ExecuteControlCommandAsync( DatabaseName, - It.Is(s => s.StartsWith($".ingest inline into table {CollectionName}", StringComparison.Ordinal) && s.Contains(actualMemoryRecordKey, StringComparison.Ordinal)), + It.Is(s => s.StartsWith($".ingest inline into table {this._normalisedCollectionName}", StringComparison.Ordinal) && s.Contains(actualMemoryRecordKey, StringComparison.Ordinal)), It.IsAny()), Times.Once()); Assert.Equal(expectedMemoryRecord.Key, actualMemoryRecordKey); } @@ -171,7 +172,7 @@ public async Task ItCanUpsertBatchAsyncAsync() .Verify(client => client.ExecuteControlCommandAsync( DatabaseName, It.Is(s => - s.StartsWith($".ingest inline into table {CollectionName}", StringComparison.Ordinal) && + s.StartsWith($".ingest inline into table {this._normalisedCollectionName}", StringComparison.Ordinal) && batchUpsertMemoryRecords.All(r => s.Contains(r.Key, StringComparison.Ordinal))), It.IsAny() ), Times.Once()); @@ -306,7 +307,7 @@ public async Task ItCanRemoveAsync() this._cslAdminProviderMock .Verify(client => client.ExecuteControlCommandAsync( DatabaseName, - It.Is(s => s.Replace(" ", " ").StartsWith($".delete table {CollectionName}") && s.Contains(MemoryRecordKey)), // Replace double spaces with single space to account for the fact that the query is formatted with double spaces and to be future proof + It.Is(s => s.Replace(" ", " ").StartsWith($".delete table {this._normalisedCollectionName}") && s.Contains(MemoryRecordKey)), // Replace double spaces with single space to account for the fact that the query is formatted with double spaces and to be future proof It.IsAny() ), Times.Once()); } @@ -325,7 +326,7 @@ public async Task ItCanRemoveBatchAsync() this._cslAdminProviderMock .Verify(client => client.ExecuteControlCommandAsync( DatabaseName, - It.Is(s => s.Replace(" ", " ").StartsWith($".delete table {CollectionName}") && memoryRecordKeys.All(r => s.Contains(r, StringComparison.OrdinalIgnoreCase))), + It.Is(s => s.Replace(" ", " ").StartsWith($".delete table {this._normalisedCollectionName}") && memoryRecordKeys.All(r => s.Contains(r, StringComparison.OrdinalIgnoreCase))), It.IsAny() ), Times.Once()); } diff --git a/dotnet/src/Connectors/Connectors.UnitTests/OpenAI/FunctionCalling/AutoFunctionInvocationFilterTests.cs b/dotnet/src/Connectors/Connectors.UnitTests/OpenAI/FunctionCalling/AutoFunctionInvocationFilterTests.cs index 1151ea41bc9b..f528edfe503d 100644 --- a/dotnet/src/Connectors/Connectors.UnitTests/OpenAI/FunctionCalling/AutoFunctionInvocationFilterTests.cs +++ b/dotnet/src/Connectors/Connectors.UnitTests/OpenAI/FunctionCalling/AutoFunctionInvocationFilterTests.cs @@ -5,6 +5,7 @@ using System.Linq; using System.Net; using System.Net.Http; +using System.Threading; using System.Threading.Tasks; using Microsoft.Extensions.DependencyInjection; using Microsoft.SemanticKernel; @@ -569,6 +570,53 @@ public async Task PostFilterCanTerminateOperationOnStreamingAsync() Assert.Equal(AuthorRole.Tool, lastMessageContent.Role); } + [Fact] + public async Task FilterContextHasCancellationTokenAsync() + { + // Arrange + using var cancellationTokenSource = new CancellationTokenSource(); + int firstFunctionInvocations = 0; + int secondFunctionInvocations = 0; + + var function1 = KernelFunctionFactory.CreateFromMethod((string parameter) => + { + cancellationTokenSource.Cancel(); + firstFunctionInvocations++; + return parameter; + }, "Function1"); + + var function2 = KernelFunctionFactory.CreateFromMethod((string parameter) => + { + secondFunctionInvocations++; + return parameter; + }, "Function2"); + + var plugin = KernelPluginFactory.CreateFromFunctions("MyPlugin", [function1, function2]); + + var kernel = this.GetKernelWithFilter(plugin, async (context, next) => + { + Assert.Equal(cancellationTokenSource.Token, context.CancellationToken); + + await next(context); + + context.CancellationToken.ThrowIfCancellationRequested(); + }); + + using var response1 = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("filters_multiple_function_calls_test_response.json")) }; + using var response2 = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(OpenAITestHelper.GetTestResponse("chat_completion_test_response.json")) }; + + this._messageHandlerStub.ResponsesToReturn = [response1, response2]; + + var arguments = new KernelArguments(new OpenAIPromptExecutionSettings { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions }); + + // Act & Assert + var exception = await Assert.ThrowsAsync(() + => kernel.InvokePromptAsync("Test prompt", arguments, cancellationToken: cancellationTokenSource.Token)); + + Assert.Equal(1, firstFunctionInvocations); + Assert.Equal(0, secondFunctionInvocations); + } + public void Dispose() { this._httpClient.Dispose(); diff --git a/dotnet/src/Functions/Functions.OpenApi/OpenAI/KernelOpenAIPluginExtensions.cs b/dotnet/src/Functions/Functions.OpenApi/OpenAI/KernelOpenAIPluginExtensions.cs index 1717106ba256..c5890d604d81 100644 --- a/dotnet/src/Functions/Functions.OpenApi/OpenAI/KernelOpenAIPluginExtensions.cs +++ b/dotnet/src/Functions/Functions.OpenApi/OpenAI/KernelOpenAIPluginExtensions.cs @@ -18,6 +18,7 @@ namespace Microsoft.SemanticKernel.Plugins.OpenApi; /// /// Provides extension methods for importing plugins exposed through OpenAI's ChatGPT format. /// +[Obsolete("This class is deprecated and will be removed in a future version.")] public static class OpenAIPluginKernelExtensions { private static readonly JsonSerializerOptions s_jsonOptionsCache = diff --git a/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIAuthenticateRequestAsyncCallback.cs b/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIAuthenticateRequestAsyncCallback.cs index 369e8e8694cd..b8d7d1015a3e 100644 --- a/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIAuthenticateRequestAsyncCallback.cs +++ b/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIAuthenticateRequestAsyncCallback.cs @@ -1,5 +1,6 @@ // Copyright (c) Microsoft. All rights reserved. +using System; using System.Net.Http; using System.Threading; using System.Threading.Tasks; @@ -14,4 +15,5 @@ namespace Microsoft.SemanticKernel.Plugins.OpenApi; /// The used to authenticate. /// The cancellation token. /// A representing the asynchronous operation. +[Obsolete("This delegate is deprecated and will be removed in a future version.")] public delegate Task OpenAIAuthenticateRequestAsyncCallback(HttpRequestMessage request, string pluginName, OpenAIAuthenticationConfig openAIAuthConfig, CancellationToken cancellationToken = default); diff --git a/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIAuthenticationConfig.cs b/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIAuthenticationConfig.cs index c4d1ff9caa09..5d01bc083f3a 100644 --- a/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIAuthenticationConfig.cs +++ b/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIAuthenticationConfig.cs @@ -9,6 +9,7 @@ namespace Microsoft.SemanticKernel.Plugins.OpenApi; /// /// Represents the authentication section for an OpenAI plugin. /// +[Obsolete("This class is deprecated and will be removed in a future version.")] public class OpenAIAuthenticationConfig { /// @@ -57,6 +58,7 @@ public class OpenAIAuthenticationConfig /// /// Represents the type of authentication for an OpenAI plugin. /// +[Obsolete("This enum is deprecated and will be removed in a future version.")] public enum OpenAIAuthenticationType { /// @@ -83,6 +85,7 @@ public enum OpenAIAuthenticationType /// /// Represents the type of authorization for an OpenAI plugin. /// +[Obsolete("This enum is deprecated and will be removed in a future version.")] public enum OpenAIAuthorizationType { /// diff --git a/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIFunctionExecutionParameters.cs b/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIFunctionExecutionParameters.cs index bc2084fb21fb..5f04bec5c039 100644 --- a/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIFunctionExecutionParameters.cs +++ b/dotnet/src/Functions/Functions.OpenApi/OpenAI/OpenAIFunctionExecutionParameters.cs @@ -8,6 +8,7 @@ namespace Microsoft.SemanticKernel.Plugins.OpenApi; /// /// OpenAI function execution parameters /// +[Obsolete("This class is deprecated and will be removed in a future version.")] public class OpenAIFunctionExecutionParameters : OpenApiFunctionExecutionParameters { /// diff --git a/dotnet/src/Functions/Functions.UnitTests/OpenApi/OpenAI/KernelOpenAIPluginExtensionsTests.cs b/dotnet/src/Functions/Functions.UnitTests/OpenApi/OpenAI/KernelOpenAIPluginExtensionsTests.cs index 36f7601dd02e..7c00e7ba375d 100644 --- a/dotnet/src/Functions/Functions.UnitTests/OpenApi/OpenAI/KernelOpenAIPluginExtensionsTests.cs +++ b/dotnet/src/Functions/Functions.UnitTests/OpenApi/OpenAI/KernelOpenAIPluginExtensionsTests.cs @@ -17,6 +17,7 @@ namespace SemanticKernel.Functions.UnitTests.OpenApi.OpenAI; +[Obsolete("OpenAI plugins are deprecated and will be removed in a future version.")] public sealed class KernelOpenAIPluginExtensionsTests : IDisposable { /// diff --git a/dotnet/src/IntegrationTests/Connectors/Google/Gemini/GeminiChatCompletionTests.cs b/dotnet/src/IntegrationTests/Connectors/Google/Gemini/GeminiChatCompletionTests.cs index 321ede0ff115..5732a3e4719a 100644 --- a/dotnet/src/IntegrationTests/Connectors/Google/Gemini/GeminiChatCompletionTests.cs +++ b/dotnet/src/IntegrationTests/Connectors/Google/Gemini/GeminiChatCompletionTests.cs @@ -64,6 +64,104 @@ public async Task ChatStreamingReturnsValidResponseAsync(ServiceType serviceType this.Output.WriteLine(message); } + [RetryTheory] + [InlineData(ServiceType.GoogleAI, Skip = "This test is for manual verification.")] + [InlineData(ServiceType.VertexAI, Skip = "This test is for manual verification.")] + public async Task ChatGenerationOnlyAssistantMessagesReturnsValidResponseAsync(ServiceType serviceType) + { + // Arrange + var chatHistory = new ChatHistory(); + chatHistory.AddAssistantMessage("I'm Brandon, I'm very thirsty"); + chatHistory.AddAssistantMessage("Could you help me get some..."); + + var sut = this.GetChatService(serviceType); + + // Act + var response = await sut.GetChatMessageContentAsync(chatHistory); + + // Assert + Assert.NotNull(response.Content); + this.Output.WriteLine(response.Content); + string[] resultWords = ["drink", "water", "tea", "coffee", "juice", "soda"]; + Assert.Contains(resultWords, word => response.Content.Contains(word, StringComparison.OrdinalIgnoreCase)); + } + + [RetryTheory] + [InlineData(ServiceType.GoogleAI, Skip = "This test is for manual verification.")] + [InlineData(ServiceType.VertexAI, Skip = "This test is for manual verification.")] + public async Task ChatStreamingOnlyAssistantMessagesReturnsValidResponseAsync(ServiceType serviceType) + { + // Arrange + var chatHistory = new ChatHistory(); + chatHistory.AddAssistantMessage("I'm Brandon, I'm very thirsty"); + chatHistory.AddAssistantMessage("Could you help me get some..."); + + var sut = this.GetChatService(serviceType); + + // Act + var response = + await sut.GetStreamingChatMessageContentsAsync(chatHistory).ToListAsync(); + + // Assert + Assert.NotEmpty(response); + Assert.True(response.Count > 1); + var message = string.Concat(response.Select(c => c.Content)); + this.Output.WriteLine(message); + string[] resultWords = ["drink", "water", "tea", "coffee", "juice", "soda"]; + Assert.Contains(resultWords, word => message.Contains(word, StringComparison.OrdinalIgnoreCase)); + } + + [RetryTheory] + [InlineData(ServiceType.GoogleAI, Skip = "This test is for manual verification.")] + [InlineData(ServiceType.VertexAI, Skip = "This test is for manual verification.")] + public async Task ChatGenerationWithSystemMessagesAsync(ServiceType serviceType) + { + // Arrange + var chatHistory = new ChatHistory("You are helpful assistant. Your name is Roger."); + chatHistory.AddSystemMessage("You know ACDD equals 1520"); + chatHistory.AddUserMessage("Hello, I'm Brandon, how are you?"); + chatHistory.AddAssistantMessage("I'm doing well, thanks for asking."); + chatHistory.AddUserMessage("Tell me your name and the value of ACDD."); + + var sut = this.GetChatService(serviceType); + + // Act + var response = await sut.GetChatMessageContentAsync(chatHistory); + + // Assert + Assert.NotNull(response.Content); + this.Output.WriteLine(response.Content); + Assert.Contains("1520", response.Content, StringComparison.OrdinalIgnoreCase); + Assert.Contains("Roger", response.Content, StringComparison.OrdinalIgnoreCase); + } + + [RetryTheory] + [InlineData(ServiceType.GoogleAI, Skip = "This test is for manual verification.")] + [InlineData(ServiceType.VertexAI, Skip = "This test is for manual verification.")] + public async Task ChatStreamingWithSystemMessagesAsync(ServiceType serviceType) + { + // Arrange + var chatHistory = new ChatHistory("You are helpful assistant. Your name is Roger."); + chatHistory.AddSystemMessage("You know ACDD equals 1520"); + chatHistory.AddUserMessage("Hello, I'm Brandon, how are you?"); + chatHistory.AddAssistantMessage("I'm doing well, thanks for asking."); + chatHistory.AddUserMessage("Tell me your name and the value of ACDD."); + + var sut = this.GetChatService(serviceType); + + // Act + var response = + await sut.GetStreamingChatMessageContentsAsync(chatHistory).ToListAsync(); + + // Assert + Assert.NotEmpty(response); + Assert.True(response.Count > 1); + var message = string.Concat(response.Select(c => c.Content)); + this.Output.WriteLine(message); + Assert.Contains("1520", message, StringComparison.OrdinalIgnoreCase); + Assert.Contains("Roger", message, StringComparison.OrdinalIgnoreCase); + } + [RetryTheory] [InlineData(ServiceType.GoogleAI, Skip = "This test is for manual verification.")] [InlineData(ServiceType.VertexAI, Skip = "This test is for manual verification.")] diff --git a/dotnet/src/IntegrationTests/Connectors/HuggingFace/ChatCompletion/HuggingFaceChatCompletionTests.cs b/dotnet/src/IntegrationTests/Connectors/HuggingFace/ChatCompletion/HuggingFaceChatCompletionTests.cs new file mode 100644 index 000000000000..cca6f6703fcb --- /dev/null +++ b/dotnet/src/IntegrationTests/Connectors/HuggingFace/ChatCompletion/HuggingFaceChatCompletionTests.cs @@ -0,0 +1,137 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Text; +using System.Threading.Tasks; +using Microsoft.Extensions.Configuration; +using Microsoft.SemanticKernel; +using Microsoft.SemanticKernel.ChatCompletion; +using Microsoft.SemanticKernel.Connectors.HuggingFace; +using Xunit; + +namespace SemanticKernel.IntegrationTests.Connectors.HuggingFace.ChatCompletion; + +/// +/// Integration tests for . +/// +/// +/// Instructions for setting up a Text Generation Inference (TGI) endpoint, see: https://huggingface.co/blog/tgi-messages-api +/// +public sealed class HuggingFaceChatCompletionTests +{ + private const string Endpoint = "https://.endpoints.huggingface.cloud/v1/"; + private const string Model = "tgi"; + + private readonly IConfigurationRoot _configuration; + + public HuggingFaceChatCompletionTests() + { + // Load configuration + this._configuration = new ConfigurationBuilder() + .AddJsonFile(path: "testsettings.json", optional: false, reloadOnChange: true) + .AddJsonFile(path: "testsettings.development.json", optional: true, reloadOnChange: true) + .AddEnvironmentVariables() + .AddUserSecrets() + .Build(); + } + + [Fact(Skip = "This test is for manual verification.")] + public async Task GetChatMessageContentsAsync() + { + // Arrange + var chatHistory = new ChatHistory + { + new ChatMessageContent(AuthorRole.System, "Use C# 12 features."), + new ChatMessageContent(AuthorRole.User, "Write a C# Hello world?") + }; + var huggingFaceRemote = new HuggingFaceChatCompletionService(Model, endpoint: new Uri(Endpoint), apiKey: this.GetApiKey()); + + // Act + var response = await huggingFaceRemote.GetChatMessageContentsAsync(chatHistory, new HuggingFacePromptExecutionSettings() { MaxNewTokens = 50 }); + + // Assert + Assert.NotNull(response); + Assert.Single(response); + Assert.True(response[0].Content?.Length > 0); + } + + [Fact(Skip = "This test is for manual verification.")] + public async Task GetStreamingChatMessageContentsAsync() + { + // Arrange + var chatHistory = new ChatHistory + { + new ChatMessageContent(AuthorRole.System, "Use C# 12 features."), + new ChatMessageContent(AuthorRole.User, "Write a C# Hello world?") + }; + var huggingFaceRemote = new HuggingFaceChatCompletionService(Model, endpoint: new Uri(Endpoint), apiKey: this.GetApiKey()); + + // Act + var response = new StringBuilder(); + await foreach (var update in huggingFaceRemote.GetStreamingChatMessageContentsAsync(chatHistory, new HuggingFacePromptExecutionSettings() { MaxNewTokens = 50 })) + { + if (update.Content is { Length: > 0 }) + { + response.Append(update.Content); + } + } + + // Assert + Assert.NotNull(response); + Assert.True(response.Length > 0); + } + + [Fact(Skip = "This test is for manual verification.")] + public async Task InvokeKernelFunctionAsync() + { + // Arrange + Kernel kernel = Kernel.CreateBuilder() + .AddHuggingFaceChatCompletion(Model, endpoint: new Uri(Endpoint), apiKey: this.GetApiKey()) + .Build(); + + var kernelFunction = kernel.CreateFunctionFromPrompt("Write a C# Hello world", new HuggingFacePromptExecutionSettings + { + MaxNewTokens = 50, + }); + + // Act + var response = await kernel.InvokeAsync(kernelFunction); + + // Assert + Assert.NotNull(response); + Assert.True(response.ToString().Length > 0); + } + + [Fact(Skip = "This test is for manual verification.")] + public async Task InvokeKernelFunctionStreamingAsync() + { + // Arrange + Kernel kernel = Kernel.CreateBuilder() + .AddHuggingFaceChatCompletion(Model, endpoint: new Uri(Endpoint), apiKey: this.GetApiKey()) + .Build(); + + var kernelFunction = kernel.CreateFunctionFromPrompt("Write a C# Hello world", new HuggingFacePromptExecutionSettings + { + MaxNewTokens = 50, + }); + + // Act + var response = new StringBuilder(); + await foreach (var update in kernel.InvokeStreamingAsync(kernelFunction)) + { + if (update.ToString() is { Length: > 0 }) + { + response.Append(update.ToString()); + } + } + + // Assert + Assert.NotNull(response); + Assert.True(response.ToString().Length > 0); + } + + private string GetApiKey() + { + return this._configuration.GetSection("HuggingFace:ApiKey").Get()!; + } +} diff --git a/dotnet/src/IntegrationTests/Connectors/Memory/Milvus/MilvusMemoryStoreTests.cs b/dotnet/src/IntegrationTests/Connectors/Memory/Milvus/MilvusMemoryStoreTests.cs index 0ed028eba747..5fba220a3ad4 100644 --- a/dotnet/src/IntegrationTests/Connectors/Memory/Milvus/MilvusMemoryStoreTests.cs +++ b/dotnet/src/IntegrationTests/Connectors/Memory/Milvus/MilvusMemoryStoreTests.cs @@ -220,6 +220,45 @@ public async Task GetNearestMatchesAsync(bool withEmbeddings) }); } + [Theory] + [InlineData(true)] + [InlineData(false)] + public async Task GetNearestMatchesWithMetricTypeAsync(bool withEmbeddings) + { + //Create collection with default, Ip metric + await this.Store.CreateCollectionAsync(CollectionName); + await this.InsertSampleDataAsync(); + await this.Store.Client.FlushAsync([CollectionName]); + + //Search with Ip metric, run correctly + List<(MemoryRecord Record, double SimilarityScore)> ipResults = + this.Store.GetNearestMatchesAsync(CollectionName, new[] { 5f, 6f, 7f, 8f, 9f }, limit: 2, withEmbeddings: withEmbeddings).ToEnumerable().ToList(); + + Assert.All(ipResults, t => Assert.True(t.SimilarityScore > 0)); + + //Set the store to Cosine metric, without recreate collection + this.Store = new(this._milvusFixture.Host, vectorSize: 5, port: this._milvusFixture.Port, metricType: SimilarityMetricType.Cosine, consistencyLevel: ConsistencyLevel.Strong); + + //An exception will be thrown here, the exception message includes "metric type not match" + MilvusException milvusException = Assert.Throws(() => this.Store.GetNearestMatchesAsync(CollectionName, new[] { 5f, 6f, 7f, 8f, 9f }, limit: 2, withEmbeddings: withEmbeddings).ToEnumerable().ToList()); + + Assert.NotNull(milvusException); + + Assert.Contains("metric type not match", milvusException.Message); + + //Recreate collection with Cosine metric + await this.Store.DeleteCollectionAsync(CollectionName); + await this.Store.CreateCollectionAsync(CollectionName); + await this.InsertSampleDataAsync(); + await this.Store.Client.FlushAsync([CollectionName]); + + //Search with Ip metric, run correctly + List<(MemoryRecord Record, double SimilarityScore)> cosineResults = + this.Store.GetNearestMatchesAsync(CollectionName, new[] { 5f, 6f, 7f, 8f, 9f }, limit: 2, withEmbeddings: withEmbeddings).ToEnumerable().ToList(); + + Assert.All(cosineResults, t => Assert.True(t.SimilarityScore > 0)); + } + [Fact] public async Task GetNearestMatchesWithMinRelevanceScoreAsync() { diff --git a/dotnet/src/IntegrationTests/Connectors/OpenAI/OpenAICompletionTests.cs b/dotnet/src/IntegrationTests/Connectors/OpenAI/OpenAICompletionTests.cs index 03cd3429d4b0..675661b76d83 100644 --- a/dotnet/src/IntegrationTests/Connectors/OpenAI/OpenAICompletionTests.cs +++ b/dotnet/src/IntegrationTests/Connectors/OpenAI/OpenAICompletionTests.cs @@ -128,6 +128,11 @@ public async Task AzureOpenAIStreamingTestAsync(bool useChatModel, string prompt // Act await foreach (var content in target.InvokeStreamingAsync(plugins["ChatPlugin"]["Chat"], new() { [InputParameterName] = prompt })) { + if (content is StreamingChatMessageContent messageContent) + { + Assert.NotNull(messageContent.Role); + } + fullResult.Append(content); } diff --git a/dotnet/src/IntegrationTests/Connectors/OpenAI/OpenAIFileServiceTests.cs b/dotnet/src/IntegrationTests/Connectors/OpenAI/OpenAIFileServiceTests.cs new file mode 100644 index 000000000000..30b0c3d1115b --- /dev/null +++ b/dotnet/src/IntegrationTests/Connectors/OpenAI/OpenAIFileServiceTests.cs @@ -0,0 +1,156 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Threading.Tasks; +using Microsoft.Extensions.Configuration; +using Microsoft.SemanticKernel; +using Microsoft.SemanticKernel.Connectors.OpenAI; +using SemanticKernel.IntegrationTests.TestSettings; +using Xunit; +using Xunit.Abstractions; + +namespace SemanticKernel.IntegrationTests.Connectors.OpenAI; + +#pragma warning disable xUnit1004 // Contains test methods used in manual verification. Disable warning for this file only. + +public sealed class OpenAIFileServiceTests(ITestOutputHelper output) : IDisposable +{ + private readonly IConfigurationRoot _configuration = new ConfigurationBuilder() + .AddJsonFile(path: "testsettings.json", optional: false, reloadOnChange: true) + .AddJsonFile(path: "testsettings.development.json", optional: true, reloadOnChange: true) + .AddEnvironmentVariables() + .AddUserSecrets() + .Build(); + + [Theory(Skip = "OpenAI will often throttle requests. This test is for manual verification.")] + [InlineData("test_image_001.jpg", "image/jpeg")] + [InlineData("test_content.txt", "text/plain")] + public async Task OpenAIFileServiceLifecycleAsync(string fileName, string mimeType) + { + // Arrange + OpenAIFileService fileService = this.CreateOpenAIFileService(); + + // Act & Assert + await this.VerifyFileServiceLifecycleAsync(fileService, fileName, mimeType); + } + + [Theory] + [InlineData("test_image_001.jpg", "image/jpeg")] + [InlineData("test_content.txt", "text/plain")] + public async Task AzureOpenAIFileServiceLifecycleAsync(string fileName, string mimeType) + { + // Arrange + OpenAIFileService fileService = this.CreateOpenAIFileService(); + + // Act & Assert + await this.VerifyFileServiceLifecycleAsync(fileService, fileName, mimeType); + } + + private async Task VerifyFileServiceLifecycleAsync(OpenAIFileService fileService, string fileName, string mimeType) + { + // Setup file content + await using FileStream fileStream = File.OpenRead($"./TestData/{fileName}"); + BinaryData sourceData = await BinaryData.FromStreamAsync(fileStream); + BinaryContent sourceContent = new(sourceData.ToArray(), mimeType); + + // Upload file with unsupported purpose (failure case) + await Assert.ThrowsAsync(() => fileService.UploadContentAsync(sourceContent, new(fileName, OpenAIFilePurpose.AssistantsOutput))); + + // Upload file with wacky purpose (failure case) + await Assert.ThrowsAsync(() => fileService.UploadContentAsync(sourceContent, new(fileName, new OpenAIFilePurpose("pretend")))); + + // Upload file + OpenAIFileReference fileReference = await fileService.UploadContentAsync(sourceContent, new(fileName, OpenAIFilePurpose.FineTune)); + try + { + AssertFileReferenceEquals(fileReference, fileName, sourceData.Length, OpenAIFilePurpose.FineTune); + + // Retrieve files by different purpose + Dictionary fileMap = await GetFilesAsync(fileService, OpenAIFilePurpose.Assistants); + Assert.DoesNotContain(fileReference.Id, fileMap.Keys); + + // Retrieve files by wacky purpose (failure case) + await Assert.ThrowsAsync(() => GetFilesAsync(fileService, new OpenAIFilePurpose("pretend"))); + + // Retrieve files by expected purpose + fileMap = await GetFilesAsync(fileService, OpenAIFilePurpose.FineTune); + Assert.Contains(fileReference.Id, fileMap.Keys); + AssertFileReferenceEquals(fileMap[fileReference.Id], fileName, sourceData.Length, OpenAIFilePurpose.FineTune); + + // Retrieve files by no specific purpose + fileMap = await GetFilesAsync(fileService); + Assert.Contains(fileReference.Id, fileMap.Keys); + AssertFileReferenceEquals(fileMap[fileReference.Id], fileName, sourceData.Length, OpenAIFilePurpose.FineTune); + + // Retrieve file by id + OpenAIFileReference file = await fileService.GetFileAsync(fileReference.Id); + AssertFileReferenceEquals(file, fileName, sourceData.Length, OpenAIFilePurpose.FineTune); + + // Retrieve file content + BinaryContent retrievedContent = await fileService.GetFileContentAsync(fileReference.Id); + Assert.NotNull(retrievedContent.Data); + Assert.NotNull(retrievedContent.Uri); + Assert.NotNull(retrievedContent.Metadata); + Assert.Equal(fileReference.Id, retrievedContent.Metadata["id"]); + Assert.Equal(sourceContent.Data!.Value.Length, retrievedContent.Data.Value.Length); + } + finally + { + // Delete file + await fileService.DeleteFileAsync(fileReference.Id); + } + } + + private static void AssertFileReferenceEquals(OpenAIFileReference fileReference, string expectedFileName, int expectedSize, OpenAIFilePurpose expectedPurpose) + { + Assert.Equal(expectedFileName, fileReference.FileName); + Assert.Equal(expectedPurpose, fileReference.Purpose); + Assert.Equal(expectedSize, fileReference.SizeInBytes); + } + + private static async Task> GetFilesAsync(OpenAIFileService fileService, OpenAIFilePurpose? purpose = null) + { + IEnumerable files = await fileService.GetFilesAsync(purpose); + Dictionary fileIds = files.DistinctBy(f => f.Id).ToDictionary(f => f.Id); + return fileIds; + } + + #region internals + + private readonly XunitLogger _logger = new(output); + private readonly RedirectOutput _testOutputHelper = new(output); + + public void Dispose() + { + this._logger.Dispose(); + this._testOutputHelper.Dispose(); + } + + private OpenAIFileService CreateOpenAIFileService() + { + var openAIConfiguration = this._configuration.GetSection("OpenAI").Get(); + + Assert.NotNull(openAIConfiguration); + Assert.NotNull(openAIConfiguration.ApiKey); + Assert.NotNull(openAIConfiguration.ServiceId); + + return new(openAIConfiguration.ApiKey, openAIConfiguration.ServiceId, loggerFactory: this._logger); + } + + private OpenAIFileService CreateAzureOpenAIFileService() + { + var azureOpenAIConfiguration = this._configuration.GetSection("AzureOpenAI").Get(); + + Assert.NotNull(azureOpenAIConfiguration); + Assert.NotNull(azureOpenAIConfiguration.Endpoint); + Assert.NotNull(azureOpenAIConfiguration.ApiKey); + Assert.NotNull(azureOpenAIConfiguration.ServiceId); + + return new(new Uri(azureOpenAIConfiguration.Endpoint), azureOpenAIConfiguration.ApiKey, azureOpenAIConfiguration.ServiceId, loggerFactory: this._logger); + } + + #endregion +} diff --git a/dotnet/src/IntegrationTests/IntegrationTests.csproj b/dotnet/src/IntegrationTests/IntegrationTests.csproj index 63e4ec8d28fe..df5afa473ce7 100644 --- a/dotnet/src/IntegrationTests/IntegrationTests.csproj +++ b/dotnet/src/IntegrationTests/IntegrationTests.csproj @@ -92,9 +92,6 @@ Always - - Always - PreserveNewest @@ -122,10 +119,10 @@ PreserveNewest - + Always - + Always diff --git a/dotnet/src/IntegrationTests/Plugins/PluginTests.cs b/dotnet/src/IntegrationTests/Plugins/OpenApi/OpenApiPluginsTests.cs similarity index 52% rename from dotnet/src/IntegrationTests/Plugins/PluginTests.cs rename to dotnet/src/IntegrationTests/Plugins/OpenApi/OpenApiPluginsTests.cs index 8275a99e7423..46aad7f7b0d0 100644 --- a/dotnet/src/IntegrationTests/Plugins/PluginTests.cs +++ b/dotnet/src/IntegrationTests/Plugins/OpenApi/OpenApiPluginsTests.cs @@ -7,74 +7,10 @@ using Microsoft.SemanticKernel.Plugins.OpenApi; using Xunit; -namespace SemanticKernel.IntegrationTests.Plugins; +namespace SemanticKernel.IntegrationTests.Plugins.OpenApi; public class PluginTests { - [Theory] - [InlineData("https://www.klarna.com/.well-known/ai-plugin.json", "Klarna", "productsUsingGET", "Laptop", 3, 200, "US")] - public async Task QueryKlarnaOpenAIPluginAsync( - string pluginEndpoint, - string name, - string functionName, - string query, - int size, - int budget, - string countryCode) - { - // Arrange - var kernel = new Kernel(); - using HttpClient httpClient = new(); - - var plugin = await kernel.ImportPluginFromOpenAIAsync( - name, - new Uri(pluginEndpoint), - new OpenAIFunctionExecutionParameters(httpClient)); - - var arguments = new KernelArguments - { - ["q"] = query, - ["size"] = size, - ["max_price"] = budget, - ["countryCode"] = countryCode - }; - - // Act - await plugin[functionName].InvokeAsync(kernel, arguments); - } - - [Theory] - [InlineData("https://www.klarna.com/us/shopping/public/openai/v0/api-docs/", "Klarna", "productsUsingGET", "Laptop", 3, 200, "US")] - public async Task QueryKlarnaOpenApiPluginAsync( - string pluginEndpoint, - string name, - string functionName, - string query, - int size, - int budget, - string countryCode) - { - // Arrange - var kernel = new Kernel(); - using HttpClient httpClient = new(); - - var plugin = await kernel.ImportPluginFromOpenApiAsync( - name, - new Uri(pluginEndpoint), - new OpenApiFunctionExecutionParameters(httpClient)); - - var arguments = new KernelArguments - { - ["q"] = query, - ["size"] = size.ToString(System.Globalization.CultureInfo.InvariantCulture), - ["max_price"] = budget, - ["countryCode"] = countryCode - }; - - // Act - await plugin[functionName].InvokeAsync(kernel, arguments); - } - [Theory] [InlineData("https://www.klarna.com/us/shopping/public/openai/v0/api-docs/", "Klarna", "productsUsingGET", "Laptop", 3, 200, "US")] public async Task QueryKlarnaOpenApiPluginRunAsync( @@ -99,7 +35,7 @@ public async Task QueryKlarnaOpenApiPluginRunAsync( { ["q"] = query, ["size"] = size, - ["budget"] = budget.ToString(System.Globalization.CultureInfo.InvariantCulture), + ["max_price"] = budget.ToString(System.Globalization.CultureInfo.InvariantCulture), ["countryCode"] = countryCode }; @@ -114,38 +50,7 @@ public async Task QueryKlarnaOpenApiPluginRunAsync( } [Theory] - [InlineData("https://raw.githubusercontent.com/sisbell/chatgpt-plugin-store/main/manifests/instacart.com.json", - "Instacart", - "create", - """{"title":"Shopping List", "ingredients": ["Flour"], "question": "what ingredients do I need to make chocolate cookies?", "partner_name": "OpenAI" }""" - )] - public async Task QueryInstacartPluginAsync( - string pluginEndpoint, - string name, - string functionName, - string payload) - { - // Arrange - var kernel = new Kernel(); - using HttpClient httpClient = new(); - - //note that this plugin is not compliant according to the underlying validator in SK - var plugin = await kernel.ImportPluginFromOpenAIAsync( - name, - new Uri(pluginEndpoint), - new OpenAIFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); - - var arguments = new KernelArguments - { - ["payload"] = payload - }; - - // Act - await plugin[functionName].InvokeAsync(kernel, arguments); - } - - [Theory] - [InlineData("Plugins/instacart-ai-plugin.json", + [InlineData("Plugins/OpenApi/instacart-service.yaml", "Instacart", "create", """{"title":"Shopping List", "ingredients": ["Flour"], "question": "what ingredients do I need to make chocolate cookies?", "partner_name": "OpenAI" }""" @@ -162,10 +67,10 @@ public async Task QueryInstacartPluginFromStreamAsync( var kernel = new Kernel(); // note that this plugin is not compliant according to the underlying validator in SK - var plugin = await kernel.ImportPluginFromOpenAIAsync( + var plugin = await kernel.ImportPluginFromOpenApiAsync( name, stream, - new OpenAIFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); + new OpenApiFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); var arguments = new KernelArguments { @@ -177,7 +82,7 @@ public async Task QueryInstacartPluginFromStreamAsync( } [Theory] - [InlineData("Plugins/instacart-ai-plugin.json", + [InlineData("Plugins/OpenApi/instacart-service.yaml", "Instacart", "create", """{"title":"Shopping List", "ingredients": ["Flour"], "question": "what ingredients do I need to make chocolate cookies?", "partner_name": "OpenAI" }""" @@ -193,10 +98,10 @@ public async Task QueryInstacartPluginUsingRelativeFilePathAsync( using HttpClient httpClient = new(); // note that this plugin is not compliant according to the underlying validator in SK - var plugin = await kernel.ImportPluginFromOpenAIAsync( + var plugin = await kernel.ImportPluginFromOpenApiAsync( name, pluginFilePath, - new OpenAIFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); + new OpenApiFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); var arguments = new KernelArguments { @@ -208,7 +113,7 @@ public async Task QueryInstacartPluginUsingRelativeFilePathAsync( } [Theory] - [InlineData("Plugins/instacart-ai-plugin.json", "Instacart", "create")] + [InlineData("Plugins/OpenApi/instacart-service.yaml", "Instacart", "create")] public async Task QueryInstacartPluginWithDynamicPayloadAsync( string pluginFilePath, string name, @@ -220,10 +125,10 @@ public async Task QueryInstacartPluginWithDynamicPayloadAsync( var kernel = new Kernel(); // note that this plugin is not compliant according to the underlying validator in SK - var plugin = await kernel.ImportPluginFromOpenAIAsync( + var plugin = await kernel.ImportPluginFromOpenApiAsync( name, stream, - new OpenAIFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = true }); + new OpenApiFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = true }); var arguments = new KernelArguments { diff --git a/dotnet/src/IntegrationTests/Plugins/RepairServiceTests.cs b/dotnet/src/IntegrationTests/Plugins/OpenApi/RepairServiceTests.cs similarity index 94% rename from dotnet/src/IntegrationTests/Plugins/RepairServiceTests.cs rename to dotnet/src/IntegrationTests/Plugins/OpenApi/RepairServiceTests.cs index 9d8610806d8c..f6bcb3c01be8 100644 --- a/dotnet/src/IntegrationTests/Plugins/RepairServiceTests.cs +++ b/dotnet/src/IntegrationTests/Plugins/OpenApi/RepairServiceTests.cs @@ -8,7 +8,7 @@ using Microsoft.SemanticKernel.Plugins.OpenApi; using Xunit; -namespace SemanticKernel.IntegrationTests.Plugins; +namespace SemanticKernel.IntegrationTests.Plugins.OpenApi; public class RepairServiceTests { @@ -23,7 +23,7 @@ public async Task ValidateInvokingRepairServicePluginAsync() var plugin = await kernel.ImportPluginFromOpenApiAsync( "RepairService", stream, - new OpenAIFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); + new OpenApiFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); var arguments = new KernelArguments { @@ -79,7 +79,7 @@ public async Task HttpOperationExceptionIncludeRequestInfoAsync() var plugin = await kernel.ImportPluginFromOpenApiAsync( "RepairService", stream, - new OpenAIFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); + new OpenApiFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); var arguments = new KernelArguments { @@ -121,7 +121,7 @@ public async Task UseDelegatingHandlerAsync() var plugin = await kernel.ImportPluginFromOpenApiAsync( "RepairService", stream, - new OpenAIFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); + new OpenApiFunctionExecutionParameters(httpClient) { IgnoreNonCompliantErrors = true, EnableDynamicPayload = false }); // List All Repairs var result = await plugin["listRepairs"].InvokeAsync(kernel); diff --git a/dotnet/src/IntegrationTests/Plugins/OpenApi/instacart-service.yaml b/dotnet/src/IntegrationTests/Plugins/OpenApi/instacart-service.yaml new file mode 100644 index 000000000000..9bb3fb3cd26f --- /dev/null +++ b/dotnet/src/IntegrationTests/Plugins/OpenApi/instacart-service.yaml @@ -0,0 +1,53 @@ +openapi: 3.0.1 +info: + title: Instacart + description: Order from your favorite local grocery stores. + version: 'v2.1' +servers: + - url: https://www.instacart.com +paths: + /rest/llm_integration/openapi/v2_1/recipes: + post: + operationId: create + summary: Create an Instacart link to the shopping list of ingredients. + requestBody: + required: true + content: + application/json: + schema: + $ref: '#/components/schemas/createRequest' + responses: + "200": + description: Instacart link to the shopping list of ingredients. + "400": + description: Could not create an Instacart link to the shopping list of ingredients. +components: + schemas: + createRequest: + type: object + required: + - title + - ingredients + - instructions + - question + - partner_name + properties: + title: + type: string + description: Recipe title (e.g. "Vanilla Yogurt Parfait") + ingredients: + type: array + items: + type: string + description: List of strings where each element is a recipe ingredient (e.g. ["2 cups of greek yogurt", "2 tablespoons of honey", "1 teaspoon of vanilla extract"]). Don't include items in the list that the user already mentioned they have. + instructions: + type: array + items: + type: string + description: List of strings where each element is a recipe instruction + question: + type: string + description: This field stores the question asked by the user about recipe or mealplan in the current chat session. For instance, a user can ask "recipe for chocolate cookies" and the assistant responds by listing the ingredients needed to make chocolate cookies. In this chat interaction, we need to return "recipe for chocolate cookies" as the value in this field + partner_name: + type: string + description: The value used to populate this field should always be "OpenAI" diff --git a/dotnet/src/IntegrationTests/Plugins/repair-service.json b/dotnet/src/IntegrationTests/Plugins/OpenApi/repair-service.json similarity index 100% rename from dotnet/src/IntegrationTests/Plugins/repair-service.json rename to dotnet/src/IntegrationTests/Plugins/OpenApi/repair-service.json diff --git a/dotnet/src/IntegrationTests/Plugins/instacart-ai-plugin.json b/dotnet/src/IntegrationTests/Plugins/instacart-ai-plugin.json deleted file mode 100644 index 3dc0bcb3925e..000000000000 --- a/dotnet/src/IntegrationTests/Plugins/instacart-ai-plugin.json +++ /dev/null @@ -1,17 +0,0 @@ -{ - "schema_version": "v1", - "name_for_model": "Instacart", - "name_for_human": "Instacart", - "description_for_model": "The Instacart plugin can be used to create a shopping list when a recipe or meal plan is mentioned. Exclude items the user already has from the shopping list. If multiple options for ingredients are provided, separate them into individual ingredients in the shopping list. Always save the user's question in the question field. Always populate the 'partner_name' field with value 'OpenAI'. Here are some examples of food or grocery related queries where the Instacart plugin can be used: chocolate cake recipe, dinner plan for a vegetarian family of 4, instructions to make orange chicken, healthy snacks for kids to bring to school, vegetarian dinner party dish ideas, lunch ideas for a picnic in the park, how to make lasagna, dinner ideas, meal plan for a family of 4, breakfast suggestions, snack ideas for a movie night with friends, squash and pasta dish with additional ingredient suggestions. Parse all ingredients referenced, including optional ingredients, as separate ingredients so that they can be correctly interpreted as unique items by the plugin. Do include optional ingredients and separate the items in the shopping list. Always keep the ingredient quantity. Ask the users if they want to purchase the ingredients and related items on Instacart, and if users agree, generate a shopping list using the Instacart plugin and response with: Here's your Instacart shopping list for [Recipe Name]. \n\nAdditional guidelines and examples: \n\n**Guideline**\n- For ingredient containing bundled items such as 'butter or margarine', 'chicken broth or wine', 'mixed nuts (e.g. ...)', 'vegetables (...)', and 'Optional toppings (...)' you need to separate the ingredient into individual items in the shopping list, e.g. 'butter or margarine' becomes ['butter', 'margarine'] in the shopping list, 'Raw vegetables (e.g., carrot sticks, cucumber slices, bell pepper strips)' becomes ['carrot sticks', 'cucumber slices', 'bell pepper strips']\n- If users say they have something, mark it as \"you already have\" in the list and don't add it to the shopping list\n\nExample 1: \nuser: garlic butter shrimp recipe \nassistant: Here's a delicious garlic butter shrimp recipe: Ingredients: 1 pound large shrimp ... 1/4 cup chicken broth or white wine (optional) Salt and pepper to taste ... \n**Note that the shopping list should contain ['1/4 cup chicken broth', '1/4 cup white wine', 'Salt', 'pepper', ...] instead of ['1/4 cup chicken broth or white wine (optional)', 'Salt and pepper to taste', ...]\n\nExample 2: \nuser: I have squash and pasta. what can I make and what other ingredients do I need? \nassistant: You can make a delicious squash and pasta dish with just a few additional ingredients. Here's a simple recipe: Ingredients: Squash (you already have) Pasta (you already have) Olive oil onion garlic Salt and pepper, ... \n**Note that the shopping list should contain ['Olive oil', 'onion', 'garlic', 'salt', 'pepper', ...] but without 'Squash' or 'Pasta' in it since user has them already.", - "description_for_human": "What’s cookin'? Ask about recipes, meal plans, & more -- and get ingredients delivered from 40,000+ stores!", - "auth": { - "type": "none" - }, - "api": { - "type": "openapi", - "url": "https://www.instacart.com/rest/llm_integration/config/openapi.yaml" - }, - "logo_url": "https://www.instacart.com/assets/beetstrap/brand/2022/carrotlogo-1286c257354036d178c09e815906198eb7f012b8cdc4f6f8ec86d3e64d799a5b.png", - "contact_email": "help@instacart.com", - "legal_info_url": "https://www.instacart.com/terms" -} \ No newline at end of file diff --git a/dotnet/src/IntegrationTests/PromptTests.cs b/dotnet/src/IntegrationTests/PromptTests.cs index 9c23661c6c96..7b252713d24c 100644 --- a/dotnet/src/IntegrationTests/PromptTests.cs +++ b/dotnet/src/IntegrationTests/PromptTests.cs @@ -57,7 +57,7 @@ public async Task GenerateStoryTestAsync(string resourceName, bool isHandlebars) }); // Assert - Assert.Contains("Dog", actual.GetValue(), StringComparison.OrdinalIgnoreCase); + Assert.True(actual.GetValue()?.Length > 0); } #region private methods diff --git a/dotnet/src/IntegrationTests/TestData/test_content.txt b/dotnet/src/IntegrationTests/TestData/test_content.txt new file mode 100644 index 000000000000..447ce0649e56 --- /dev/null +++ b/dotnet/src/IntegrationTests/TestData/test_content.txt @@ -0,0 +1,9 @@ +Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Amet dictum sit amet justo donec enim diam vulputate ut. Nibh ipsum consequat nisl vel pretium lectus. Urna nec tincidunt praesent semper feugiat. Tristique nulla aliquet enim tortor. Ut morbi tincidunt augue interdum velit euismod in pellentesque massa. Ullamcorper morbi tincidunt ornare massa eget egestas purus viverra. Commodo ullamcorper a lacus vestibulum sed arcu non. Volutpat ac tincidunt vitae semper quis lectus nulla. Sem nulla pharetra diam sit amet nisl. Viverra aliquet eget sit amet tellus cras adipiscing enim eu. + +Morbi blandit cursus risus at ultrices mi tempus. Sagittis orci a scelerisque purus. Iaculis nunc sed augue lacus viverra. Accumsan sit amet nulla facilisi morbi tempus iaculis. Nisl rhoncus mattis rhoncus urna neque. Commodo odio aenean sed adipiscing diam donec adipiscing tristique. Tristique senectus et netus et malesuada fames. Nascetur ridiculus mus mauris vitae ultricies leo integer. Ut sem viverra aliquet eget. Sed egestas egestas fringilla phasellus faucibus scelerisque. + +In tellus integer feugiat scelerisque varius morbi. Vitae proin sagittis nisl rhoncus mattis rhoncus urna neque. Cum sociis natoque penatibus et magnis dis. Iaculis at erat pellentesque adipiscing commodo elit at imperdiet dui. Praesent semper feugiat nibh sed pulvinar proin gravida hendrerit lectus. Consectetur a erat nam at lectus urna. Hac habitasse platea dictumst vestibulum rhoncus est pellentesque elit. Aliquam vestibulum morbi blandit cursus risus at ultrices. Eu non diam phasellus vestibulum lorem sed. Risus pretium quam vulputate dignissim suspendisse in est. Elit scelerisque mauris pellentesque pulvinar pellentesque habitant morbi. At varius vel pharetra vel turpis nunc eget. Aliquam malesuada bibendum arcu vitae. At consectetur lorem donec massa. Mi sit amet mauris commodo. Maecenas volutpat blandit aliquam etiam erat velit. Nullam ac tortor vitae purus faucibus ornare suspendisse. + +Facilisi nullam vehicula ipsum a arcu cursus vitae. Commodo sed egestas egestas fringilla phasellus. Lacus luctus accumsan tortor posuere ac ut consequat. Adipiscing commodo elit at imperdiet dui accumsan sit. Non tellus orci ac auctor augue. Viverra aliquet eget sit amet tellus. Luctus venenatis lectus magna fringilla urna porttitor rhoncus dolor. Mattis enim ut tellus elementum. Nunc sed id semper risus. At augue eget arcu dictum. + +Ullamcorper a lacus vestibulum sed arcu non. Vitae tortor condimentum lacinia quis vel. Dui faucibus in ornare quam viverra. Vel pharetra vel turpis nunc eget. In egestas erat imperdiet sed euismod nisi porta lorem mollis. Lacus vestibulum sed arcu non odio euismod lacinia at quis. Augue mauris augue neque gravida in. Ornare quam viverra orci sagittis. Lacus suspendisse faucibus interdum posuere lorem ipsum. Arcu vitae elementum curabitur vitae nunc sed velit dignissim. Diam quam nulla porttitor massa id neque. Gravida dictum fusce ut placerat orci nulla pellentesque. Mus mauris vitae ultricies leo integer malesuada nunc vel risus. Donec pretium vulputate sapien nec sagittis aliquam. Velit egestas dui id ornare. Sed elementum tempus egestas sed sed risus pretium quam vulputate. \ No newline at end of file diff --git a/dotnet/src/IntegrationTests/testsettings.json b/dotnet/src/IntegrationTests/testsettings.json index 39ec5c4d3b1c..66df73f8b7a5 100644 --- a/dotnet/src/IntegrationTests/testsettings.json +++ b/dotnet/src/IntegrationTests/testsettings.json @@ -51,8 +51,8 @@ "EmbeddingModelId": "embedding-001", "ApiKey": "", "Gemini": { - "ModelId": "gemini-1.0-pro", - "VisionModelId": "gemini-1.0-pro-vision" + "ModelId": "gemini-1.5-flash", + "VisionModelId": "gemini-1.5-flash" } }, "VertexAI": { @@ -61,8 +61,8 @@ "Location": "us-central1", "ProjectId": "", "Gemini": { - "ModelId": "gemini-1.0-pro", - "VisionModelId": "gemini-1.0-pro-vision" + "ModelId": "gemini-1.5-flash", + "VisionModelId": "gemini-1.5-flash" } }, "Bing": { diff --git a/dotnet/src/InternalUtilities/samples/InternalUtilities/XunitLogger.cs b/dotnet/src/InternalUtilities/samples/InternalUtilities/XunitLogger.cs index ca2c22cd800a..578cc2aec366 100644 --- a/dotnet/src/InternalUtilities/samples/InternalUtilities/XunitLogger.cs +++ b/dotnet/src/InternalUtilities/samples/InternalUtilities/XunitLogger.cs @@ -7,16 +7,25 @@ /// internal sealed class XunitLogger(ITestOutputHelper output) : ILoggerFactory, ILogger, IDisposable { + private object? _scopeState; + /// public void Log(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func formatter) - => output.WriteLine(state?.ToString()); + { + var localState = state?.ToString(); + var line = this._scopeState is not null ? $"{this._scopeState} {localState}" : localState; + output.WriteLine(line); + } /// public bool IsEnabled(LogLevel logLevel) => true; /// public IDisposable BeginScope(TState state) where TState : notnull - => this; + { + this._scopeState = state; + return this; + } /// public void Dispose() diff --git a/dotnet/src/SemanticKernel.Abstractions/AI/ChatCompletion/AuthorRole.cs b/dotnet/src/SemanticKernel.Abstractions/AI/ChatCompletion/AuthorRole.cs index 7c572509056c..05f473b1b792 100644 --- a/dotnet/src/SemanticKernel.Abstractions/AI/ChatCompletion/AuthorRole.cs +++ b/dotnet/src/SemanticKernel.Abstractions/AI/ChatCompletion/AuthorRole.cs @@ -32,7 +32,7 @@ namespace Microsoft.SemanticKernel.ChatCompletion; public static AuthorRole Tool { get; } = new("tool"); /// - /// Gets the label associated with this AuthorRole. + /// Gets the label associated with this . /// /// /// The label is what will be serialized into the "role" message field of the Chat Message format. @@ -40,9 +40,9 @@ namespace Microsoft.SemanticKernel.ChatCompletion; public string Label { get; } /// - /// Creates a new AuthorRole instance with the provided label. + /// Creates a new instance with the provided label. /// - /// The label to associate with this AuthorRole. + /// The label to associate with this . [JsonConstructor] public AuthorRole(string label) { @@ -51,21 +51,21 @@ public AuthorRole(string label) } /// - /// Returns a value indicating whether two AuthorRole instances are equivalent, as determined by a + /// Returns a value indicating whether two instances are equivalent, as determined by a /// case-insensitive comparison of their labels. /// - /// the first AuthorRole instance to compare - /// the second AuthorRole instance to compare + /// the first instance to compare + /// the second instance to compare /// true if left and right are both null or have equivalent labels; false otherwise public static bool operator ==(AuthorRole left, AuthorRole right) => left.Equals(right); /// - /// Returns a value indicating whether two AuthorRole instances are not equivalent, as determined by a + /// Returns a value indicating whether two instances are not equivalent, as determined by a /// case-insensitive comparison of their labels. /// - /// the first AuthorRole instance to compare - /// the second AuthorRole instance to compare + /// the first instance to compare + /// the second instance to compare /// false if left and right are both null or have equivalent labels; true otherwise public static bool operator !=(AuthorRole left, AuthorRole right) => !(left == right); @@ -80,8 +80,8 @@ public bool Equals(AuthorRole other) /// public override int GetHashCode() - => StringComparer.OrdinalIgnoreCase.GetHashCode(this.Label ?? string.Empty); + => StringComparer.OrdinalIgnoreCase.GetHashCode(this.Label); /// - public override string ToString() => this.Label ?? string.Empty; + public override string ToString() => this.Label; } diff --git a/dotnet/src/SemanticKernel.Abstractions/Contents/ChatMessageContent.cs b/dotnet/src/SemanticKernel.Abstractions/Contents/ChatMessageContent.cs index 24ff3cf19438..0042cf5a2948 100644 --- a/dotnet/src/SemanticKernel.Abstractions/Contents/ChatMessageContent.cs +++ b/dotnet/src/SemanticKernel.Abstractions/Contents/ChatMessageContent.cs @@ -20,7 +20,11 @@ public class ChatMessageContent : KernelContent /// [Experimental("SKEXP0001")] [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] - public string? AuthorName { get; set; } + public string? AuthorName + { + get => this._authorName; + set => this._authorName = string.IsNullOrWhiteSpace(value) ? null : value; + } /// /// Role of the author of the message @@ -171,4 +175,5 @@ public override string ToString() private ChatMessageContentItemCollection? _items; private Encoding _encoding; + private string? _authorName; } diff --git a/dotnet/src/SemanticKernel.Abstractions/Contents/StreamingChatMessageContent.cs b/dotnet/src/SemanticKernel.Abstractions/Contents/StreamingChatMessageContent.cs index 5cc7afb582ed..cc2b0c354284 100644 --- a/dotnet/src/SemanticKernel.Abstractions/Contents/StreamingChatMessageContent.cs +++ b/dotnet/src/SemanticKernel.Abstractions/Contents/StreamingChatMessageContent.cs @@ -64,7 +64,11 @@ public StreamingKernelContentItemCollection Items /// [Experimental("SKEXP0001")] [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] - public string? AuthorName { get; set; } + public string? AuthorName + { + get => this._authorName; + set => this._authorName = string.IsNullOrWhiteSpace(value) ? null : value; + } /// /// Role of the author of the message @@ -126,4 +130,5 @@ public StreamingChatMessageContent(AuthorRole? role, string? content, object? in private StreamingKernelContentItemCollection? _items; private Encoding _encoding; + private string? _authorName; } diff --git a/dotnet/src/SemanticKernel.Abstractions/Filters/AutoFunctionInvocation/AutoFunctionInvocationContext.cs b/dotnet/src/SemanticKernel.Abstractions/Filters/AutoFunctionInvocation/AutoFunctionInvocationContext.cs index f430324df867..39f2439b9df0 100644 --- a/dotnet/src/SemanticKernel.Abstractions/Filters/AutoFunctionInvocation/AutoFunctionInvocationContext.cs +++ b/dotnet/src/SemanticKernel.Abstractions/Filters/AutoFunctionInvocation/AutoFunctionInvocationContext.cs @@ -1,6 +1,7 @@ // Copyright (c) Microsoft. All rights reserved. using System.Diagnostics.CodeAnalysis; +using System.Threading; using Microsoft.SemanticKernel.ChatCompletion; namespace Microsoft.SemanticKernel; @@ -35,6 +36,12 @@ public AutoFunctionInvocationContext( this.ChatHistory = chatHistory; } + /// + /// The to monitor for cancellation requests. + /// The default is . + /// + public CancellationToken CancellationToken { get; init; } + /// /// Gets the arguments associated with the operation. /// diff --git a/dotnet/src/SemanticKernel.Abstractions/Filters/Function/FunctionInvocationContext.cs b/dotnet/src/SemanticKernel.Abstractions/Filters/Function/FunctionInvocationContext.cs index c208f1a75f85..1ef77aac8e60 100644 --- a/dotnet/src/SemanticKernel.Abstractions/Filters/Function/FunctionInvocationContext.cs +++ b/dotnet/src/SemanticKernel.Abstractions/Filters/Function/FunctionInvocationContext.cs @@ -1,6 +1,7 @@ // Copyright (c) Microsoft. All rights reserved. using System.Diagnostics.CodeAnalysis; +using System.Threading; namespace Microsoft.SemanticKernel; @@ -29,6 +30,12 @@ internal FunctionInvocationContext(Kernel kernel, KernelFunction function, Kerne this.Result = result; } + /// + /// The to monitor for cancellation requests. + /// The default is . + /// + public CancellationToken CancellationToken { get; init; } + /// /// Gets the containing services, plugins, and other state for use throughout the operation. /// diff --git a/dotnet/src/SemanticKernel.Abstractions/Filters/Prompt/PromptRenderContext.cs b/dotnet/src/SemanticKernel.Abstractions/Filters/Prompt/PromptRenderContext.cs index a1e449642071..918586bfa6f1 100644 --- a/dotnet/src/SemanticKernel.Abstractions/Filters/Prompt/PromptRenderContext.cs +++ b/dotnet/src/SemanticKernel.Abstractions/Filters/Prompt/PromptRenderContext.cs @@ -1,6 +1,7 @@ // Copyright (c) Microsoft. All rights reserved. using System.Diagnostics.CodeAnalysis; +using System.Threading; namespace Microsoft.SemanticKernel; @@ -29,6 +30,12 @@ internal PromptRenderContext(Kernel kernel, KernelFunction function, KernelArgum this.Arguments = arguments; } + /// + /// The to monitor for cancellation requests. + /// The default is . + /// + public CancellationToken CancellationToken { get; init; } + /// /// Gets the containing services, plugins, and other state for use throughout the operation. /// diff --git a/dotnet/src/SemanticKernel.Abstractions/Functions/KernelFunction.cs b/dotnet/src/SemanticKernel.Abstractions/Functions/KernelFunction.cs index 31101bdb1958..9e50f653f5f8 100644 --- a/dotnet/src/SemanticKernel.Abstractions/Functions/KernelFunction.cs +++ b/dotnet/src/SemanticKernel.Abstractions/Functions/KernelFunction.cs @@ -186,7 +186,7 @@ public async Task InvokeAsync( { // Invoking the function and updating context with result. context.Result = functionResult = await this.InvokeCoreAsync(kernel, context.Arguments, cancellationToken).ConfigureAwait(false); - }).ConfigureAwait(false); + }, cancellationToken).ConfigureAwait(false); // Apply any changes from the function filters context to final result. functionResult = invocationContext.Result; @@ -321,7 +321,7 @@ public async IAsyncEnumerable InvokeStreamingAsync( context.Result = new FunctionResult(this, enumerable, kernel.Culture); return Task.CompletedTask; - }).ConfigureAwait(false); + }, cancellationToken).ConfigureAwait(false); // Apply changes from the function filters to final result. var enumerable = invocationContext.Result.GetValue>() ?? AsyncEnumerable.Empty(); diff --git a/dotnet/src/SemanticKernel.Abstractions/Kernel.cs b/dotnet/src/SemanticKernel.Abstractions/Kernel.cs index c466fb9f6485..283d3de05dd5 100644 --- a/dotnet/src/SemanticKernel.Abstractions/Kernel.cs +++ b/dotnet/src/SemanticKernel.Abstractions/Kernel.cs @@ -263,7 +263,7 @@ public IEnumerable GetAllServices() where T : class // M.E.DI doesn't support querying for a service without a key, and it also doesn't // support AnyKey currently: https://github.com/dotnet/runtime/issues/91466 // As a workaround, KernelBuilder injects a service containing the type-to-all-keys - // mapping. We can query for that service and and then use it to try to get a service. + // mapping. We can query for that service and then use it to try to get a service. if (this.Services.GetKeyedService>>(KernelServiceTypeToKeyMappings) is { } typeToKeyMappings) { if (typeToKeyMappings.TryGetValue(typeof(T), out HashSet? keys)) @@ -314,9 +314,13 @@ internal async Task OnFunctionInvocationAsync( KernelFunction function, KernelArguments arguments, FunctionResult functionResult, - Func functionCallback) + Func functionCallback, + CancellationToken cancellationToken) { - FunctionInvocationContext context = new(this, function, arguments, functionResult); + FunctionInvocationContext context = new(this, function, arguments, functionResult) + { + CancellationToken = cancellationToken + }; await InvokeFilterOrFunctionAsync(this._functionInvocationFilters, functionCallback, context).ConfigureAwait(false); @@ -351,9 +355,13 @@ await functionFilters[index].OnFunctionInvocationAsync(context, internal async Task OnPromptRenderAsync( KernelFunction function, KernelArguments arguments, - Func renderCallback) + Func renderCallback, + CancellationToken cancellationToken) { - PromptRenderContext context = new(this, function, arguments); + PromptRenderContext context = new(this, function, arguments) + { + CancellationToken = cancellationToken + }; await InvokeFilterOrPromptRenderAsync(this._promptRenderFilters, renderCallback, context).ConfigureAwait(false); diff --git a/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromPrompt.cs b/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromPrompt.cs index 1d357b05679f..a7849ab8d155 100644 --- a/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromPrompt.cs +++ b/dotnet/src/SemanticKernel.Core/Functions/KernelFunctionFromPrompt.cs @@ -335,7 +335,7 @@ private async Task RenderPromptAsync(Kernel kernel, Kerne } context.RenderedPrompt = renderedPrompt; - }).ConfigureAwait(false); + }, cancellationToken).ConfigureAwait(false); if (!string.IsNullOrWhiteSpace(renderingContext.RenderedPrompt) && !string.Equals(renderingContext.RenderedPrompt, renderedPrompt, StringComparison.OrdinalIgnoreCase)) diff --git a/dotnet/src/SemanticKernel.UnitTests/Contents/ChatMessageContentTests.cs b/dotnet/src/SemanticKernel.UnitTests/Contents/ChatMessageContentTests.cs index fdbd4cae0524..759fec2b532b 100644 --- a/dotnet/src/SemanticKernel.UnitTests/Contents/ChatMessageContentTests.cs +++ b/dotnet/src/SemanticKernel.UnitTests/Contents/ChatMessageContentTests.cs @@ -128,6 +128,25 @@ public void ContentPropertyGetterShouldReturnContentOfTheFirstTextContentItem() Assert.Equal("fake-content-1", sut.Content); } + [Theory] + [InlineData(null)] + [InlineData("")] + [InlineData(" ")] + [InlineData("\t")] + [InlineData("\n")] + [InlineData("\r\n")] + public void ContentPropertySetterShouldConvertEmptyOrWhitespaceAuthorNameToNull(string? authorName) + { + // Arrange + var message = new ChatMessageContent(AuthorRole.User, content: null) + { + AuthorName = authorName + }; + + // Assert + Assert.Null(message.AuthorName); + } + [Fact] public void ItShouldBePossibleToSetAndGetEncodingEvenIfThereAreNoItems() { diff --git a/dotnet/src/SemanticKernel.UnitTests/Contents/StreamingChatMessageContentTests.cs b/dotnet/src/SemanticKernel.UnitTests/Contents/StreamingChatMessageContentTests.cs index f7f7c5e43be7..d510e6e42eeb 100644 --- a/dotnet/src/SemanticKernel.UnitTests/Contents/StreamingChatMessageContentTests.cs +++ b/dotnet/src/SemanticKernel.UnitTests/Contents/StreamingChatMessageContentTests.cs @@ -118,6 +118,25 @@ public void ContentPropertyGetterShouldReturnContentOfTheFirstTextContentItem() Assert.Equal("fake-content-1", sut.Content); } + [Theory] + [InlineData(null)] + [InlineData("")] + [InlineData(" ")] + [InlineData("\t")] + [InlineData("\n")] + [InlineData("\r\n")] + public void ContentPropertySetterShouldConvertEmptyOrWhitespaceAuthorNameToNull(string? authorName) + { + // Arrange + var message = new StreamingChatMessageContent(AuthorRole.User, content: null) + { + AuthorName = authorName + }; + + // Assert + Assert.Null(message.AuthorName); + } + [Fact] public void ItShouldBePossibleToSetAndGetEncodingEvenIfThereAreNoItems() { diff --git a/dotnet/src/SemanticKernel.UnitTests/Filters/FunctionInvocationFilterTests.cs b/dotnet/src/SemanticKernel.UnitTests/Filters/FunctionInvocationFilterTests.cs index 94cb5c7e2a36..99d81af29c4e 100644 --- a/dotnet/src/SemanticKernel.UnitTests/Filters/FunctionInvocationFilterTests.cs +++ b/dotnet/src/SemanticKernel.UnitTests/Filters/FunctionInvocationFilterTests.cs @@ -1022,4 +1022,34 @@ public async Task InsertFilterInMiddleOfPipelineTriggersFiltersInCorrectOrderAsy Assert.Equal("FunctionFilter3-Invoked", executionOrder[4]); Assert.Equal("FunctionFilter1-Invoked", executionOrder[5]); } + + [Fact] + public async Task FilterContextHasCancellationTokenAsync() + { + // Arrange + using var cancellationTokenSource = new CancellationTokenSource(); + var function = KernelFunctionFactory.CreateFromMethod(() => + { + cancellationTokenSource.Cancel(); + return "Result"; + }); + + var kernel = this.GetKernelWithFilters(onFunctionInvocation: async (context, next) => + { + Assert.Equal(cancellationTokenSource.Token, context.CancellationToken); + Assert.False(context.CancellationToken.IsCancellationRequested); + + await next(context); + + Assert.True(context.CancellationToken.IsCancellationRequested); + context.CancellationToken.ThrowIfCancellationRequested(); + }); + + // Act & Assert + var exception = await Assert.ThrowsAsync(() + => kernel.InvokeAsync(function, cancellationToken: cancellationTokenSource.Token)); + + Assert.NotNull(exception.FunctionResult); + Assert.Equal("Result", exception.FunctionResult.ToString()); + } } diff --git a/dotnet/src/SemanticKernel.UnitTests/Filters/PromptRenderFilterTests.cs b/dotnet/src/SemanticKernel.UnitTests/Filters/PromptRenderFilterTests.cs index 020008070387..4cb0c46082b7 100644 --- a/dotnet/src/SemanticKernel.UnitTests/Filters/PromptRenderFilterTests.cs +++ b/dotnet/src/SemanticKernel.UnitTests/Filters/PromptRenderFilterTests.cs @@ -264,4 +264,29 @@ public async Task PromptFilterCanOverrideFunctionResultAsync() Assert.Equal("Result from prompt filter", result.ToString()); } + + [Fact] + public async Task FilterContextHasCancellationTokenAsync() + { + // Arrange + using var cancellationTokenSource = new CancellationTokenSource(); + var mockTextGeneration = this.GetMockTextGeneration(); + var function = KernelFunctionFactory.CreateFromPrompt("Prompt"); + + var kernel = this.GetKernelWithFilters(onPromptRender: async (context, next) => + { + Assert.Equal(cancellationTokenSource.Token, context.CancellationToken); + Assert.True(context.CancellationToken.IsCancellationRequested); + + context.CancellationToken.ThrowIfCancellationRequested(); + + await next(context); + }); + + // Act & Assert + cancellationTokenSource.Cancel(); + + await Assert.ThrowsAsync(() + => kernel.InvokeAsync(function, cancellationToken: cancellationTokenSource.Token)); + } } diff --git a/python/.coveragerc b/python/.coveragerc index c8e46534cb99..521b2ffe70c9 100644 --- a/python/.coveragerc +++ b/python/.coveragerc @@ -1,7 +1,19 @@ [run] source = semantic_kernel omit = - semantic_kernel/connectors/memory/* + semantic_kernel/connectors/memory/astradb/* + semantic_kernel/connectors/memory/azure_cognitive_search/* + semantic_kernel/connectors/memory/azure_cosmosdb/* + semantic_kernel/connectors/memory/azure_cosmosdb_no_sql/* + semantic_kernel/connectors/memory/chroma/* + semantic_kernel/connectors/memory/milvus/* + semantic_kernel/connectors/memory/mongodb_atlas/* + semantic_kernel/connectors/memory/pinecone/* + semantic_kernel/connectors/memory/postgres/* + semantic_kernel/connectors/memory/qdrant/* + semantic_kernel/connectors/memory/redis/* + semantic_kernel/connectors/memory/usearch/* + semantic_kernel/connectors/memory/weaviate/* semantic_kernel/reliability/* semantic_kernel/memory/* diff --git a/python/.env.example b/python/.env.example index 39d1c10a60df..58602848434d 100644 --- a/python/.env.example +++ b/python/.env.example @@ -38,7 +38,6 @@ PINECONE_ENVIRONMENT="" POSTGRES_CONNECTION_STRING="" WEAVIATE_URL="" WEAVIATE_API_KEY="" -GOOGLE_PALM_API_KEY="" GOOGLE_SEARCH_ENGINE_ID="" REDIS_CONNECTION_STRING="" AZCOSMOS_API = "" // should be mongo-vcore for now, as CosmosDB only supports vector search in mongo-vcore for now. diff --git a/python/DEV_SETUP.md b/python/DEV_SETUP.md index 5b95d7400863..e9a938259cae 100644 --- a/python/DEV_SETUP.md +++ b/python/DEV_SETUP.md @@ -157,7 +157,7 @@ Read more about the extension here: https://github.com/astral-sh/ruff-vscode You can run the unit tests under the [tests/unit](tests/unit/) folder. ```bash - poetry install + poetry install --with unit-tests poetry run pytest tests/unit ``` @@ -167,7 +167,7 @@ Alternatively, you can run them using VSCode Tasks. Open the command palette You can run the integration tests under the [tests/integration](tests/integration/) folder. ```bash - poetry install + poetry install --with tests poetry run pytest tests/integration ``` diff --git a/python/mypy.ini b/python/mypy.ini index 3c9d7b974110..cfe7defe74fd 100644 --- a/python/mypy.ini +++ b/python/mypy.ini @@ -13,10 +13,29 @@ warn_untyped_fields = true [mypy-semantic_kernel] no_implicit_reexport = true -[mypy-semantic_kernel.connectors.*] +[mypy-semantic_kernel.connectors.ai.open_ai.*] +ignore_errors = true + +[mypy-semantic_kernel.connectors.ai.azure_ai_inference.*] +ignore_errors = true + +[mypy-semantic_kernel.connectors.ai.hugging_face.*] +ignore_errors = true + +[mypy-semantic_kernel.connectors.ai.ollama.*] +ignore_errors = true + +[mypy-semantic_kernel.connectors.openapi_plugin.*] +ignore_errors = true + +[mypy-semantic_kernel.connectors.utils.*] +ignore_errors = true + +[mypy-semantic_kernel.connectors.search_engine.*] +ignore_errors = true + +[mypy-semantic_kernel.connectors.ai.function_choice_behavior.*] ignore_errors = true -# TODO (eavanvalkenburg): remove this -# https://github.com/microsoft/semantic-kernel/issues/6462 [mypy-semantic_kernel.memory.*] ignore_errors = true @@ -28,3 +47,7 @@ ignore_errors = true # TODO (eavanvalkenburg): remove this # https://github.com/microsoft/semantic-kernel/issues/6465 +[mypy-semantic_kernel.connectors.memory.*] +ignore_errors = true +# TODO (eavanvalkenburg): remove this +# https://github.com/microsoft/semantic-kernel/issues/6462 diff --git a/python/poetry.lock b/python/poetry.lock index 944aa05db6e9..9593bbe0289f 100644 --- a/python/poetry.lock +++ b/python/poetry.lock @@ -1,5 +1,36 @@ # This file is automatically @generated by Poetry 1.8.2 and should not be changed by hand. +[[package]] +name = "accelerate" +version = "0.31.0" +description = "Accelerate" +optional = false +python-versions = ">=3.8.0" +files = [ + {file = "accelerate-0.31.0-py3-none-any.whl", hash = "sha256:0fc608dc49584f64d04711a39711d73cb0ad4ef3d21cddee7ef2216e29471144"}, + {file = "accelerate-0.31.0.tar.gz", hash = "sha256:b5199865b26106ccf9205acacbe8e4b3b428ad585e7c472d6a46f6fb75b6c176"}, +] + +[package.dependencies] +huggingface-hub = "*" +numpy = ">=1.17" +packaging = ">=20.0" +psutil = "*" +pyyaml = "*" +safetensors = ">=0.3.1" +torch = ">=1.10.0" + +[package.extras] +deepspeed = ["deepspeed (<=0.14.0)"] +dev = ["bitsandbytes", "black (>=23.1,<24.0)", "datasets", "diffusers", "evaluate", "hf-doc-builder (>=0.3.0)", "parameterized", "pytest (>=7.2.0,<=8.0.0)", "pytest-subtests", "pytest-xdist", "rich", "ruff (>=0.2.1,<0.3.0)", "scikit-learn", "scipy", "timm", "torchpippy (>=0.2.0)", "tqdm", "transformers"] +quality = ["black (>=23.1,<24.0)", "hf-doc-builder (>=0.3.0)", "ruff (>=0.2.1,<0.3.0)"] +rich = ["rich"] +sagemaker = ["sagemaker"] +test-dev = ["bitsandbytes", "datasets", "diffusers", "evaluate", "scikit-learn", "scipy", "timm", "torchpippy (>=0.2.0)", "tqdm", "transformers"] +test-prod = ["parameterized", "pytest (>=7.2.0,<=8.0.0)", "pytest-subtests", "pytest-xdist"] +test-trackers = ["comet-ml", "dvclive", "tensorboard", "wandb"] +testing = ["bitsandbytes", "datasets", "diffusers", "evaluate", "parameterized", "pytest (>=7.2.0,<=8.0.0)", "pytest-subtests", "pytest-xdist", "scikit-learn", "scipy", "timm", "torchpippy (>=0.2.0)", "tqdm", "transformers"] + [[package]] name = "aiohttp" version = "3.9.5" @@ -123,13 +154,13 @@ files = [ [[package]] name = "anyio" -version = "4.3.0" +version = "4.4.0" description = "High level compatibility layer for multiple asynchronous event loop implementations" optional = false python-versions = ">=3.8" files = [ - {file = "anyio-4.3.0-py3-none-any.whl", hash = "sha256:048e05d0f6caeed70d731f3db756d35dcc1f35747c8c403364a8332c630441b8"}, - {file = "anyio-4.3.0.tar.gz", hash = "sha256:f75253795a87df48568485fd18cdd2a3fa5c4f7c5be8e5e36637733fce06fed6"}, + {file = "anyio-4.4.0-py3-none-any.whl", hash = "sha256:c1b2d8f46a8a812513012e1107cb0e68c17159a7a594208005a57dc776e1bdc7"}, + {file = "anyio-4.4.0.tar.gz", hash = "sha256:5aadc6a1bbb7cdb0bede386cac5e2940f5e2ff3aa20277e991cf028e0585ce94"}, ] [package.dependencies] @@ -233,6 +264,22 @@ files = [ [package.dependencies] cryptography = "*" +[[package]] +name = "azure-ai-inference" +version = "1.0.0b1" +description = "Microsoft Azure Ai Inference Client Library for Python" +optional = false +python-versions = ">=3.8" +files = [ + {file = "azure-ai-inference-1.0.0b1.tar.gz", hash = "sha256:e44c58ccc38a6ae335193d4ba99d9424b1d9cdfc7f18d5285dc17ba988404dfd"}, + {file = "azure_ai_inference-1.0.0b1-py3-none-any.whl", hash = "sha256:c62279a36c232a98cb69bcbbf6b8f1974316922350e20f990bc27c041881c854"}, +] + +[package.dependencies] +azure-core = ">=1.30.0" +isodate = ">=0.6.1" +typing-extensions = ">=4.6.0" + [[package]] name = "azure-common" version = "1.1.28" @@ -686,13 +733,13 @@ numpy = "*" [[package]] name = "chromadb" -version = "0.5.0" +version = "0.5.3" description = "Chroma." optional = false python-versions = ">=3.8" files = [ - {file = "chromadb-0.5.0-py3-none-any.whl", hash = "sha256:8193dc65c143b61d8faf87f02c44ecfa778d471febd70de517f51c5d88a06009"}, - {file = "chromadb-0.5.0.tar.gz", hash = "sha256:7954af614a9ff7b2902ddbd0a162f33f7ec0669e2429903905c4f7876d1f766f"}, + {file = "chromadb-0.5.3-py3-none-any.whl", hash = "sha256:b3874f08356e291c68c6d2e177db472cd51f22f3af7b9746215b748fd1e29982"}, + {file = "chromadb-0.5.3.tar.gz", hash = "sha256:05d887f56a46b2e0fc6ac5ab979503a27b9ee50d5ca9e455f83b2fb9840cd026"}, ] [package.dependencies] @@ -701,10 +748,11 @@ build = ">=1.0.3" chroma-hnswlib = "0.7.3" fastapi = ">=0.95.2" grpcio = ">=1.58.0" +httpx = ">=0.27.0" importlib-resources = "*" kubernetes = ">=28.1.0" mmh3 = ">=4.0.1" -numpy = ">=1.22.5" +numpy = ">=1.22.5,<2.0.0" onnxruntime = ">=1.14.1" opentelemetry-api = ">=1.2.0" opentelemetry-exporter-otlp-proto-grpc = ">=1.2.0" @@ -852,43 +900,43 @@ toml = ["tomli"] [[package]] name = "cryptography" -version = "42.0.7" +version = "42.0.8" description = "cryptography is a package which provides cryptographic recipes and primitives to Python developers." optional = false python-versions = ">=3.7" files = [ - {file = "cryptography-42.0.7-cp37-abi3-macosx_10_12_universal2.whl", hash = "sha256:a987f840718078212fdf4504d0fd4c6effe34a7e4740378e59d47696e8dfb477"}, - {file = "cryptography-42.0.7-cp37-abi3-macosx_10_12_x86_64.whl", hash = "sha256:bd13b5e9b543532453de08bcdc3cc7cebec6f9883e886fd20a92f26940fd3e7a"}, - {file = "cryptography-42.0.7-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a79165431551042cc9d1d90e6145d5d0d3ab0f2d66326c201d9b0e7f5bf43604"}, - {file = "cryptography-42.0.7-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a47787a5e3649008a1102d3df55424e86606c9bae6fb77ac59afe06d234605f8"}, - {file = "cryptography-42.0.7-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:02c0eee2d7133bdbbc5e24441258d5d2244beb31da5ed19fbb80315f4bbbff55"}, - {file = "cryptography-42.0.7-cp37-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:5e44507bf8d14b36b8389b226665d597bc0f18ea035d75b4e53c7b1ea84583cc"}, - {file = "cryptography-42.0.7-cp37-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:7f8b25fa616d8b846aef64b15c606bb0828dbc35faf90566eb139aa9cff67af2"}, - {file = "cryptography-42.0.7-cp37-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:93a3209f6bb2b33e725ed08ee0991b92976dfdcf4e8b38646540674fc7508e13"}, - {file = "cryptography-42.0.7-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:e6b8f1881dac458c34778d0a424ae5769de30544fc678eac51c1c8bb2183e9da"}, - {file = "cryptography-42.0.7-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:3de9a45d3b2b7d8088c3fbf1ed4395dfeff79d07842217b38df14ef09ce1d8d7"}, - {file = "cryptography-42.0.7-cp37-abi3-win32.whl", hash = "sha256:789caea816c6704f63f6241a519bfa347f72fbd67ba28d04636b7c6b7da94b0b"}, - {file = "cryptography-42.0.7-cp37-abi3-win_amd64.whl", hash = "sha256:8cb8ce7c3347fcf9446f201dc30e2d5a3c898d009126010cbd1f443f28b52678"}, - {file = "cryptography-42.0.7-cp39-abi3-macosx_10_12_universal2.whl", hash = "sha256:a3a5ac8b56fe37f3125e5b72b61dcde43283e5370827f5233893d461b7360cd4"}, - {file = "cryptography-42.0.7-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:779245e13b9a6638df14641d029add5dc17edbef6ec915688f3acb9e720a5858"}, - {file = "cryptography-42.0.7-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0d563795db98b4cd57742a78a288cdbdc9daedac29f2239793071fe114f13785"}, - {file = "cryptography-42.0.7-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:31adb7d06fe4383226c3e963471f6837742889b3c4caa55aac20ad951bc8ffda"}, - {file = "cryptography-42.0.7-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:efd0bf5205240182e0f13bcaea41be4fdf5c22c5129fc7ced4a0282ac86998c9"}, - {file = "cryptography-42.0.7-cp39-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:a9bc127cdc4ecf87a5ea22a2556cab6c7eda2923f84e4f3cc588e8470ce4e42e"}, - {file = "cryptography-42.0.7-cp39-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:3577d029bc3f4827dd5bf8bf7710cac13527b470bbf1820a3f394adb38ed7d5f"}, - {file = "cryptography-42.0.7-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:2e47577f9b18723fa294b0ea9a17d5e53a227867a0a4904a1a076d1646d45ca1"}, - {file = "cryptography-42.0.7-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:1a58839984d9cb34c855197043eaae2c187d930ca6d644612843b4fe8513c886"}, - {file = "cryptography-42.0.7-cp39-abi3-win32.whl", hash = "sha256:e6b79d0adb01aae87e8a44c2b64bc3f3fe59515280e00fb6d57a7267a2583cda"}, - {file = "cryptography-42.0.7-cp39-abi3-win_amd64.whl", hash = "sha256:16268d46086bb8ad5bf0a2b5544d8a9ed87a0e33f5e77dd3c3301e63d941a83b"}, - {file = "cryptography-42.0.7-pp310-pypy310_pp73-macosx_10_12_x86_64.whl", hash = "sha256:2954fccea107026512b15afb4aa664a5640cd0af630e2ee3962f2602693f0c82"}, - {file = "cryptography-42.0.7-pp310-pypy310_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:362e7197754c231797ec45ee081f3088a27a47c6c01eff2ac83f60f85a50fe60"}, - {file = "cryptography-42.0.7-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:4f698edacf9c9e0371112792558d2f705b5645076cc0aaae02f816a0171770fd"}, - {file = "cryptography-42.0.7-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:5482e789294854c28237bba77c4c83be698be740e31a3ae5e879ee5444166582"}, - {file = "cryptography-42.0.7-pp39-pypy39_pp73-macosx_10_12_x86_64.whl", hash = "sha256:e9b2a6309f14c0497f348d08a065d52f3020656f675819fc405fb63bbcd26562"}, - {file = "cryptography-42.0.7-pp39-pypy39_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:d8e3098721b84392ee45af2dd554c947c32cc52f862b6a3ae982dbb90f577f14"}, - {file = "cryptography-42.0.7-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:c65f96dad14f8528a447414125e1fc8feb2ad5a272b8f68477abbcc1ea7d94b9"}, - {file = "cryptography-42.0.7-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:36017400817987670037fbb0324d71489b6ead6231c9604f8fc1f7d008087c68"}, - {file = "cryptography-42.0.7.tar.gz", hash = "sha256:ecbfbc00bf55888edda9868a4cf927205de8499e7fabe6c050322298382953f2"}, + {file = "cryptography-42.0.8-cp37-abi3-macosx_10_12_universal2.whl", hash = "sha256:81d8a521705787afe7a18d5bfb47ea9d9cc068206270aad0b96a725022e18d2e"}, + {file = "cryptography-42.0.8-cp37-abi3-macosx_10_12_x86_64.whl", hash = "sha256:961e61cefdcb06e0c6d7e3a1b22ebe8b996eb2bf50614e89384be54c48c6b63d"}, + {file = "cryptography-42.0.8-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e3ec3672626e1b9e55afd0df6d774ff0e953452886e06e0f1eb7eb0c832e8902"}, + {file = "cryptography-42.0.8-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e599b53fd95357d92304510fb7bda8523ed1f79ca98dce2f43c115950aa78801"}, + {file = "cryptography-42.0.8-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:5226d5d21ab681f432a9c1cf8b658c0cb02533eece706b155e5fbd8a0cdd3949"}, + {file = "cryptography-42.0.8-cp37-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:6b7c4f03ce01afd3b76cf69a5455caa9cfa3de8c8f493e0d3ab7d20611c8dae9"}, + {file = "cryptography-42.0.8-cp37-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:2346b911eb349ab547076f47f2e035fc8ff2c02380a7cbbf8d87114fa0f1c583"}, + {file = "cryptography-42.0.8-cp37-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:ad803773e9df0b92e0a817d22fd8a3675493f690b96130a5e24f1b8fabbea9c7"}, + {file = "cryptography-42.0.8-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:2f66d9cd9147ee495a8374a45ca445819f8929a3efcd2e3df6428e46c3cbb10b"}, + {file = "cryptography-42.0.8-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:d45b940883a03e19e944456a558b67a41160e367a719833c53de6911cabba2b7"}, + {file = "cryptography-42.0.8-cp37-abi3-win32.whl", hash = "sha256:a0c5b2b0585b6af82d7e385f55a8bc568abff8923af147ee3c07bd8b42cda8b2"}, + {file = "cryptography-42.0.8-cp37-abi3-win_amd64.whl", hash = "sha256:57080dee41209e556a9a4ce60d229244f7a66ef52750f813bfbe18959770cfba"}, + {file = "cryptography-42.0.8-cp39-abi3-macosx_10_12_universal2.whl", hash = "sha256:dea567d1b0e8bc5764b9443858b673b734100c2871dc93163f58c46a97a83d28"}, + {file = "cryptography-42.0.8-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c4783183f7cb757b73b2ae9aed6599b96338eb957233c58ca8f49a49cc32fd5e"}, + {file = "cryptography-42.0.8-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a0608251135d0e03111152e41f0cc2392d1e74e35703960d4190b2e0f4ca9c70"}, + {file = "cryptography-42.0.8-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:dc0fdf6787f37b1c6b08e6dfc892d9d068b5bdb671198c72072828b80bd5fe4c"}, + {file = "cryptography-42.0.8-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:9c0c1716c8447ee7dbf08d6db2e5c41c688544c61074b54fc4564196f55c25a7"}, + {file = "cryptography-42.0.8-cp39-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:fff12c88a672ab9c9c1cf7b0c80e3ad9e2ebd9d828d955c126be4fd3e5578c9e"}, + {file = "cryptography-42.0.8-cp39-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:cafb92b2bc622cd1aa6a1dce4b93307792633f4c5fe1f46c6b97cf67073ec961"}, + {file = "cryptography-42.0.8-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:31f721658a29331f895a5a54e7e82075554ccfb8b163a18719d342f5ffe5ecb1"}, + {file = "cryptography-42.0.8-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:b297f90c5723d04bcc8265fc2a0f86d4ea2e0f7ab4b6994459548d3a6b992a14"}, + {file = "cryptography-42.0.8-cp39-abi3-win32.whl", hash = "sha256:2f88d197e66c65be5e42cd72e5c18afbfae3f741742070e3019ac8f4ac57262c"}, + {file = "cryptography-42.0.8-cp39-abi3-win_amd64.whl", hash = "sha256:fa76fbb7596cc5839320000cdd5d0955313696d9511debab7ee7278fc8b5c84a"}, + {file = "cryptography-42.0.8-pp310-pypy310_pp73-macosx_10_12_x86_64.whl", hash = "sha256:ba4f0a211697362e89ad822e667d8d340b4d8d55fae72cdd619389fb5912eefe"}, + {file = "cryptography-42.0.8-pp310-pypy310_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:81884c4d096c272f00aeb1f11cf62ccd39763581645b0812e99a91505fa48e0c"}, + {file = "cryptography-42.0.8-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:c9bb2ae11bfbab395bdd072985abde58ea9860ed84e59dbc0463a5d0159f5b71"}, + {file = "cryptography-42.0.8-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:7016f837e15b0a1c119d27ecd89b3515f01f90a8615ed5e9427e30d9cdbfed3d"}, + {file = "cryptography-42.0.8-pp39-pypy39_pp73-macosx_10_12_x86_64.whl", hash = "sha256:5a94eccb2a81a309806027e1670a358b99b8fe8bfe9f8d329f27d72c094dde8c"}, + {file = "cryptography-42.0.8-pp39-pypy39_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:dec9b018df185f08483f294cae6ccac29e7a6e0678996587363dc352dc65c842"}, + {file = "cryptography-42.0.8-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:343728aac38decfdeecf55ecab3264b015be68fc2816ca800db649607aeee648"}, + {file = "cryptography-42.0.8-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:013629ae70b40af70c9a7a5db40abe5d9054e6f4380e50ce769947b73bf3caad"}, + {file = "cryptography-42.0.8.tar.gz", hash = "sha256:8d09d05439ce7baa8e9e95b07ec5b6c886f548deb7e0f69ef25f64b3bce842f2"}, ] [package.dependencies] @@ -1289,53 +1337,6 @@ test-downstream = ["aiobotocore (>=2.5.4,<3.0.0)", "dask-expr", "dask[dataframe, test-full = ["adlfs", "aiohttp (!=4.0.0a0,!=4.0.0a1)", "cloudpickle", "dask", "distributed", "dropbox", "dropboxdrivefs", "fastparquet", "fusepy", "gcsfs", "jinja2", "kerchunk", "libarchive-c", "lz4", "notebook", "numpy", "ocifs", "pandas", "panel", "paramiko", "pyarrow", "pyarrow (>=1)", "pyftpdlib", "pygit2", "pytest", "pytest-asyncio (!=0.22.0)", "pytest-benchmark", "pytest-cov", "pytest-mock", "pytest-recording", "pytest-rerunfailures", "python-snappy", "requests", "smbprotocol", "tqdm", "urllib3", "zarr", "zstandard"] tqdm = ["tqdm"] -[[package]] -name = "google-ai-generativelanguage" -version = "0.4.0" -description = "Google Ai Generativelanguage API client library" -optional = false -python-versions = ">=3.7" -files = [ - {file = "google-ai-generativelanguage-0.4.0.tar.gz", hash = "sha256:c8199066c08f74c4e91290778329bb9f357ba1ea5d6f82de2bc0d10552bf4f8c"}, - {file = "google_ai_generativelanguage-0.4.0-py3-none-any.whl", hash = "sha256:e4c425376c1ee26c78acbc49a24f735f90ebfa81bf1a06495fae509a2433232c"}, -] - -[package.dependencies] -google-api-core = {version = ">=1.34.0,<2.0.dev0 || >=2.11.dev0,<3.0.0dev", extras = ["grpc"]} -proto-plus = ">=1.22.3,<2.0.0dev" -protobuf = ">=3.19.5,<3.20.0 || >3.20.0,<3.20.1 || >3.20.1,<4.21.0 || >4.21.0,<4.21.1 || >4.21.1,<4.21.2 || >4.21.2,<4.21.3 || >4.21.3,<4.21.4 || >4.21.4,<4.21.5 || >4.21.5,<5.0.0dev" - -[[package]] -name = "google-api-core" -version = "2.19.0" -description = "Google API client core library" -optional = false -python-versions = ">=3.7" -files = [ - {file = "google-api-core-2.19.0.tar.gz", hash = "sha256:cf1b7c2694047886d2af1128a03ae99e391108a08804f87cfd35970e49c9cd10"}, - {file = "google_api_core-2.19.0-py3-none-any.whl", hash = "sha256:8661eec4078c35428fd3f69a2c7ee29e342896b70f01d1a1cbcb334372dd6251"}, -] - -[package.dependencies] -google-auth = ">=2.14.1,<3.0.dev0" -googleapis-common-protos = ">=1.56.2,<2.0.dev0" -grpcio = [ - {version = ">=1.33.2,<2.0dev", optional = true, markers = "python_version < \"3.11\" and extra == \"grpc\""}, - {version = ">=1.49.1,<2.0dev", optional = true, markers = "python_version >= \"3.11\" and extra == \"grpc\""}, -] -grpcio-status = [ - {version = ">=1.33.2,<2.0.dev0", optional = true, markers = "python_version < \"3.11\" and extra == \"grpc\""}, - {version = ">=1.49.1,<2.0.dev0", optional = true, markers = "python_version >= \"3.11\" and extra == \"grpc\""}, -] -proto-plus = ">=1.22.3,<2.0.0dev" -protobuf = ">=3.19.5,<3.20.0 || >3.20.0,<3.20.1 || >3.20.1,<4.21.0 || >4.21.0,<4.21.1 || >4.21.1,<4.21.2 || >4.21.2,<4.21.3 || >4.21.3,<4.21.4 || >4.21.4,<4.21.5 || >4.21.5,<5.0.0.dev0" -requests = ">=2.18.0,<3.0.0.dev0" - -[package.extras] -grpc = ["grpcio (>=1.33.2,<2.0dev)", "grpcio (>=1.49.1,<2.0dev)", "grpcio-status (>=1.33.2,<2.0.dev0)", "grpcio-status (>=1.49.1,<2.0.dev0)"] -grpcgcp = ["grpcio-gcp (>=0.2.2,<1.0.dev0)"] -grpcio-gcp = ["grpcio-gcp (>=0.2.2,<1.0.dev0)"] - [[package]] name = "google-auth" version = "2.29.0" @@ -1359,27 +1360,6 @@ pyopenssl = ["cryptography (>=38.0.3)", "pyopenssl (>=20.0.0)"] reauth = ["pyu2f (>=0.1.5)"] requests = ["requests (>=2.20.0,<3.0.0.dev0)"] -[[package]] -name = "google-generativeai" -version = "0.3.2" -description = "Google Generative AI High level API client library and tools." -optional = false -python-versions = ">=3.9" -files = [ - {file = "google_generativeai-0.3.2-py3-none-any.whl", hash = "sha256:8761147e6e167141932dc14a7b7af08f2310dd56668a78d206c19bb8bd85bcd7"}, -] - -[package.dependencies] -google-ai-generativelanguage = "0.4.0" -google-api-core = "*" -google-auth = "*" -protobuf = "*" -tqdm = "*" -typing-extensions = "*" - -[package.extras] -dev = ["Pillow", "absl-py", "black", "ipython", "nose2", "pandas", "pytype", "pyyaml"] - [[package]] name = "googleapis-common-protos" version = "1.63.0" @@ -1470,22 +1450,6 @@ files = [ grpcio = ">=1.60.0" protobuf = ">=4.21.6" -[[package]] -name = "grpcio-status" -version = "1.60.0" -description = "Status proto mapping for gRPC" -optional = false -python-versions = ">=3.6" -files = [ - {file = "grpcio-status-1.60.0.tar.gz", hash = "sha256:f10e0b6db3adc0fdc244b71962814ee982996ef06186446b5695b9fa635aa1ab"}, - {file = "grpcio_status-1.60.0-py3-none-any.whl", hash = "sha256:7d383fa36e59c1e61d380d91350badd4d12ac56e4de2c2b831b050362c3c572e"}, -] - -[package.dependencies] -googleapis-common-protos = ">=1.5.5" -grpcio = ">=1.60.0" -protobuf = ">=4.21.6" - [[package]] name = "grpcio-tools" version = "1.60.0" @@ -2408,6 +2372,7 @@ python-versions = ">=3.7" files = [ {file = "milvus_lite-2.4.7-py3-none-macosx_10_9_x86_64.whl", hash = "sha256:c828190118b104b05b8c8e0b5a4147811c86b54b8fb67bc2e726ad10fc0b544e"}, {file = "milvus_lite-2.4.7-py3-none-macosx_11_0_arm64.whl", hash = "sha256:e1537633c39879714fb15082be56a4b97f74c905a6e98e302ec01320561081af"}, + {file = "milvus_lite-2.4.7-py3-none-manylinux2014_aarch64.whl", hash = "sha256:fcb909d38c83f21478ca9cb500c84264f988c69f62715ae9462e966767fb76dd"}, {file = "milvus_lite-2.4.7-py3-none-manylinux2014_x86_64.whl", hash = "sha256:f016474d663045787dddf1c3aad13b7d8b61fd329220318f858184918143dcbf"}, ] @@ -3130,6 +3095,7 @@ description = "Nvidia JIT LTO Library" optional = false python-versions = ">=3" files = [ + {file = "nvidia_nvjitlink_cu12-12.5.40-py3-none-manylinux2014_aarch64.whl", hash = "sha256:004186d5ea6a57758fd6d57052a123c73a4815adf365eb8dd6a85c9eaa7535ff"}, {file = "nvidia_nvjitlink_cu12-12.5.40-py3-none-manylinux2014_x86_64.whl", hash = "sha256:d9714f27c1d0f0895cd8915c07a87a1d0029a0aa36acaf9156952ec2a8a12189"}, {file = "nvidia_nvjitlink_cu12-12.5.40-py3-none-win_amd64.whl", hash = "sha256:c3401dc8543b52d3a8158007a0c1ab4e9c768fcbd24153a48c86972102197ddd"}, ] @@ -4014,23 +3980,6 @@ files = [ [package.dependencies] wcwidth = "*" -[[package]] -name = "proto-plus" -version = "1.23.0" -description = "Beautiful, Pythonic protocol buffers." -optional = false -python-versions = ">=3.6" -files = [ - {file = "proto-plus-1.23.0.tar.gz", hash = "sha256:89075171ef11988b3fa157f5dbd8b9cf09d65fffee97e29ce403cd8defba19d2"}, - {file = "proto_plus-1.23.0-py3-none-any.whl", hash = "sha256:a829c79e619e1cf632de091013a4173deed13a55f326ef84f05af6f50ff4c82c"}, -] - -[package.dependencies] -protobuf = ">=3.19.0,<5.0.0dev" - -[package.extras] -testing = ["google-api-core[grpc] (>=1.31.5)"] - [[package]] name = "protobuf" version = "4.25.3" @@ -4081,99 +4030,89 @@ test = ["enum34", "ipaddress", "mock", "pywin32", "wmi"] [[package]] name = "psycopg" -version = "3.1.19" +version = "3.2.1" description = "PostgreSQL database adapter for Python" optional = false -python-versions = ">=3.7" +python-versions = ">=3.8" files = [ - {file = "psycopg-3.1.19-py3-none-any.whl", hash = "sha256:dca5e5521c859f6606686432ae1c94e8766d29cc91f2ee595378c510cc5b0731"}, - {file = "psycopg-3.1.19.tar.gz", hash = "sha256:92d7b78ad82426cdcf1a0440678209faa890c6e1721361c2f8901f0dccd62961"}, + {file = "psycopg-3.2.1-py3-none-any.whl", hash = "sha256:ece385fb413a37db332f97c49208b36cf030ff02b199d7635ed2fbd378724175"}, + {file = "psycopg-3.2.1.tar.gz", hash = "sha256:dc8da6dc8729dacacda3cc2f17d2c9397a70a66cf0d2b69c91065d60d5f00cb7"}, ] [package.dependencies] -psycopg-binary = {version = "3.1.19", optional = true, markers = "implementation_name != \"pypy\" and extra == \"binary\""} +psycopg-binary = {version = "3.2.1", optional = true, markers = "implementation_name != \"pypy\" and extra == \"binary\""} psycopg-pool = {version = "*", optional = true, markers = "extra == \"pool\""} -typing-extensions = ">=4.1" +typing-extensions = ">=4.4" tzdata = {version = "*", markers = "sys_platform == \"win32\""} [package.extras] -binary = ["psycopg-binary (==3.1.19)"] -c = ["psycopg-c (==3.1.19)"] -dev = ["black (>=24.1.0)", "codespell (>=2.2)", "dnspython (>=2.1)", "flake8 (>=4.0)", "mypy (>=1.4.1)", "types-setuptools (>=57.4)", "wheel (>=0.37)"] +binary = ["psycopg-binary (==3.2.1)"] +c = ["psycopg-c (==3.2.1)"] +dev = ["ast-comments (>=1.1.2)", "black (>=24.1.0)", "codespell (>=2.2)", "dnspython (>=2.1)", "flake8 (>=4.0)", "mypy (>=1.6)", "types-setuptools (>=57.4)", "wheel (>=0.37)"] docs = ["Sphinx (>=5.0)", "furo (==2022.6.21)", "sphinx-autobuild (>=2021.3.14)", "sphinx-autodoc-typehints (>=1.12)"] pool = ["psycopg-pool"] -test = ["anyio (>=3.6.2,<4.0)", "mypy (>=1.4.1)", "pproxy (>=2.7)", "pytest (>=6.2.5)", "pytest-cov (>=3.0)", "pytest-randomly (>=3.5)"] +test = ["anyio (>=4.0)", "mypy (>=1.6)", "pproxy (>=2.7)", "pytest (>=6.2.5)", "pytest-cov (>=3.0)", "pytest-randomly (>=3.5)"] [[package]] name = "psycopg-binary" -version = "3.1.19" +version = "3.2.1" description = "PostgreSQL database adapter for Python -- C optimisation distribution" optional = false -python-versions = ">=3.7" +python-versions = ">=3.8" files = [ - {file = "psycopg_binary-3.1.19-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:7204818f05151dd08f8f851defb01972ec9d2cc925608eb0de232563f203f354"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:6d4e67fd86758dbeac85641419a54f84d74495a8683b58ad5dfad08b7fc37a8f"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e12173e34b176e93ad2da913de30f774d5119c2d4d4640c6858d2d77dfa6c9bf"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:052f5193304066318853b4b2e248f523c8f52b371fc4e95d4ef63baee3f30955"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:29008f3f8977f600b8a7fb07c2e041b01645b08121760609cc45e861a0364dc9"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7c6a9a651a08d876303ed059c9553df18b3c13c3406584a70a8f37f1a1fe2709"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:91a645e6468c4f064b7f4f3b81074bdd68fe5aa2b8c5107de15dcd85ba6141be"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:5c6956808fd5cf0576de5a602243af8e04594b25b9a28675feddc71c5526410a"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-musllinux_1_1_ppc64le.whl", hash = "sha256:1622ca27d5a7a98f7d8f35e8b146dc7efda4a4b6241d2edf7e076bd6bcecbeb4"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:a100482950a55228f648bd382bb71bfaff520002f29845274fccbbf02e28bd52"}, - {file = "psycopg_binary-3.1.19-cp310-cp310-win_amd64.whl", hash = "sha256:955ca8905c0251fc4af7ce0a20999e824a25652f53a558ab548b60969f1f368e"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1cf49e91dcf699b8a449944ed898ef1466b39b92720613838791a551bc8f587a"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:964c307e400c5f33fa762ba1e19853e048814fcfbd9679cc923431adb7a2ead2"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3433924e1b14074798331dc2bfae2af452ed7888067f2fc145835704d8981b15"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:00879d4c6be4b3afc510073f48a5e960f797200e261ab3d9bd9b7746a08c669d"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:34a6997c80f86d3dd80a4f078bb3b200079c47eeda4fd409d8899b883c90d2ac"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0106e42b481677c41caa69474fe530f786dcef88b11b70000f0e45a03534bc8f"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:81efe09ba27533e35709905c3061db4dc9fb814f637360578d065e2061fbb116"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:d312d6dddc18d9c164e1893706269c293cba1923118349d375962b1188dafb01"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-musllinux_1_1_ppc64le.whl", hash = "sha256:bfd2c734da9950f7afaad5f132088e0e1478f32f042881fca6651bb0c8d14206"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:8a732610a5a6b4f06dadcf9288688a8ff202fd556d971436a123b7adb85596e2"}, - {file = "psycopg_binary-3.1.19-cp311-cp311-win_amd64.whl", hash = "sha256:321814a9a3ad785855a821b842aba08ca1b7de7dfb2979a2f0492dca9ec4ae70"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:4aa0ca13bb8a725bb6d12c13999217fd5bc8b86a12589f28a74b93e076fbb959"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:469424e354ebcec949aa6aa30e5a9edc352a899d9a68ad7a48f97df83cc914cf"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b04f5349313529ae1f1c42fe1aa0443faaf50fdf12d13866c2cc49683bfa53d0"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:959feabddc7fffac89b054d6f23f3b3c62d7d3c90cd414a02e3747495597f150"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e9da624a6ca4bc5f7fa1f03f8485446b5b81d5787b6beea2b4f8d9dbef878ad7"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c1823221a6b96e38b15686170d4fc5b36073efcb87cce7d3da660440b50077f6"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:866db42f986298f0cf15d805225eb8df2228bf19f7997d7f1cb5f388cbfc6a0f"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:738c34657305b5973af6dbb6711b07b179dfdd21196d60039ca30a74bafe9648"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-musllinux_1_1_ppc64le.whl", hash = "sha256:fb9758473200384a04374d0e0cac6f451218ff6945a024f65a1526802c34e56e"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:0e991632777e217953ac960726158987da684086dd813ac85038c595e7382c91"}, - {file = "psycopg_binary-3.1.19-cp312-cp312-win_amd64.whl", hash = "sha256:1d87484dd42c8783c44a30400949efb3d81ef2487eaa7d64d1c54df90cf8b97a"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:d1d1723d7449c12bb61aca7eb6e0c6ab2863cd8dc0019273cc4d4a1982f84bdb"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e538a8671005641fa195eab962f85cf0504defbd3b548c4c8fc27102a59f687b"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c50592bc8517092f40979e4a5d934f96a1737a77724bb1d121eb78b614b30fc8"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:95f16ae82bc242b76cd3c3e5156441e2bd85ff9ec3a9869d750aad443e46073c"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aebd1e98e865e9a28ce0cb2c25b7dfd752f0d1f0a423165b55cd32a431dcc0f4"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:49cd7af7d49e438a39593d1dd8cab106a1912536c2b78a4d814ebdff2786094e"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:affebd61aa3b7a8880fd4ac3ee94722940125ff83ff485e1a7c76be9adaabb38"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-musllinux_1_1_ppc64le.whl", hash = "sha256:d1bac282f140fa092f2bbb6c36ed82270b4a21a6fc55d4b16748ed9f55e50fdb"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:1285aa54449e362b1d30d92b2dc042ad3ee80f479cc4e323448d0a0a8a1641fa"}, - {file = "psycopg_binary-3.1.19-cp37-cp37m-win_amd64.whl", hash = "sha256:6cff31af8155dc9ee364098a328bab688c887c732c66b8d027e5b03818ca0287"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:d9b689c4a17dd3130791dcbb8c30dbf05602f7c2d56c792e193fb49adc7bf5f8"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:017518bd2de4851adc826a224fb105411e148ad845e11355edd6786ba3dfedf5"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c35fd811f339a3cbe7f9b54b2d9a5e592e57426c6cc1051632a62c59c4810208"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:38ed45ec9673709bfa5bc17f140e71dd4cca56d4e58ef7fd50d5a5043a4f55c6"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:433f1c256108f9e26f480a8cd6ddb0fb37dbc87d7f5a97e4540a9da9b881f23f"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:ed61e43bf5dc8d0936daf03a19fef3168d64191dbe66483f7ad08c4cea0bc36b"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:4ae8109ff9fdf1fa0cb87ab6645298693fdd2666a7f5f85660df88f6965e0bb7"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-musllinux_1_1_ppc64le.whl", hash = "sha256:a53809ee02e3952fae7977c19b30fd828bd117b8f5edf17a3a94212feb57faaf"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:9d39d5ffc151fb33bcd55b99b0e8957299c0b1b3e5a1a5f4399c1287ef0051a9"}, - {file = "psycopg_binary-3.1.19-cp38-cp38-win_amd64.whl", hash = "sha256:e14bc8250000921fcccd53722f86b3b3d1b57db901e206e49e2ab2afc5919c2d"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:cd88c5cea4efe614d5004fb5f5dcdea3d7d59422be796689e779e03363102d24"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:621a814e60825162d38760c66351b4df679fd422c848b7c2f86ad399bff27145"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:46e50c05952b59a214e27d3606f6d510aaa429daed898e16b8a37bfbacc81acc"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:03354a9db667c27946e70162cb0042c3929154167f3678a30d23cebfe0ad55b5"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:703c2f3b79037581afec7baa2bdbcb0a1787f1758744a7662099b0eca2d721cb"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:6469ebd9e93327e9f5f36dcf8692fb1e7aeaf70087c1c15d4f2c020e0be3a891"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:85bca9765c04b6be90cb46e7566ffe0faa2d7480ff5c8d5e055ac427f039fd24"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-musllinux_1_1_ppc64le.whl", hash = "sha256:a836610d5c75e9cff98b9fdb3559c007c785c09eaa84a60d5d10ef6f85f671e8"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:ef8de7a1d9fb3518cc6b58e3c80b75a824209ad52b90c542686c912db8553dad"}, - {file = "psycopg_binary-3.1.19-cp39-cp39-win_amd64.whl", hash = "sha256:76fcd33342f38e35cd6b5408f1bc117d55ab8b16e5019d99b6d3ce0356c51717"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-macosx_12_0_x86_64.whl", hash = "sha256:cad2de17804c4cfee8640ae2b279d616bb9e4734ac3c17c13db5e40982bd710d"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-macosx_14_0_arm64.whl", hash = "sha256:592b27d6c46a40f9eeaaeea7c1fef6f3c60b02c634365eb649b2d880669f149f"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9a997efbaadb5e1a294fb5760e2f5643d7b8e4e3fe6cb6f09e6d605fd28e0291"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c1d2b6438fb83376f43ebb798bf0ad5e57bc56c03c9c29c85bc15405c8c0ac5a"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b1f087bd84bdcac78bf9f024ebdbfacd07fc0a23ec8191448a50679e2ac4a19e"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:415c3b72ea32119163255c6504085f374e47ae7345f14bc3f0ef1f6e0976a879"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:f092114f10f81fb6bae544a0ec027eb720e2d9c74a4fcdaa9dd3899873136935"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:06a7aae34edfe179ddc04da005e083ff6c6b0020000399a2cbf0a7121a8a22ea"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:0b018631e5c80ce9bc210b71ea885932f9cca6db131e4df505653d7e3873a938"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:f8a509aeaac364fa965454e80cd110fe6d48ba2c80f56c9b8563423f0b5c3cfd"}, + {file = "psycopg_binary-3.2.1-cp310-cp310-win_amd64.whl", hash = "sha256:413977d18412ff83486eeb5875eb00b185a9391c57febac45b8993bf9c0ff489"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-macosx_12_0_x86_64.whl", hash = "sha256:62b1b7b07e00ee490afb39c0a47d8282a9c2822c7cfed9553a04b0058adf7e7f"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:f8afb07114ea9b924a4a0305ceb15354ccf0ef3c0e14d54b8dbeb03e50182dd7"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:40bb515d042f6a345714ec0403df68ccf13f73b05e567837d80c886c7c9d3805"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6418712ba63cebb0c88c050b3997185b0ef54173b36568522d5634ac06153040"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:101472468d59c74bb8565fab603e032803fd533d16be4b2d13da1bab8deb32a3"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:aa3931f308ab4a479d0ee22dc04bea867a6365cac0172e5ddcba359da043854b"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:dc314a47d44fe1a8069b075a64abffad347a3a1d8652fed1bab5d3baea37acb2"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:cc304a46be1e291031148d9d95c12451ffe783ff0cc72f18e2cc7ec43cdb8c68"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:6f9e13600647087df5928875559f0eb8f496f53e6278b7da9511b4b3d0aff960"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:b140182830c76c74d17eba27df3755a46442ce8d4fb299e7f1cf2f74a87c877b"}, + {file = "psycopg_binary-3.2.1-cp311-cp311-win_amd64.whl", hash = "sha256:3c838806eeb99af39f934b7999e35f947a8e577997cc892c12b5053a97a9057f"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-macosx_12_0_x86_64.whl", hash = "sha256:7066d3dca196ed0dc6172f9777b2d62e4f138705886be656cccff2d555234d60"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:28ada5f610468c57d8a4a055a8ea915d0085a43d794266c4f3b9d02f4288f4db"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2e8213bf50af073b1aa8dc3cff123bfeedac86332a16c1b7274910bc88a847c7"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:74d623261655a169bc84a9669890975c229f2fa6e19a7f2d10a77675dcf1a707"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:42781ba94e8842ee98bca5a7d0c44cc9d067500fedca2d6a90fa3609b6d16b42"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:33e6669091d09f8ba36e10ce678a6d9916e110446236a9b92346464a3565635e"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:b09e8a576a2ac69d695032ee76f31e03b30781828b5dd6d18c6a009e5a3d1c35"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:8f28ff0cb9f1defdc4a6f8c958bf6787274247e7dfeca811f6e2f56602695fb1"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:4c84fcac8a3a3479ac14673095cc4e1fdba2935499f72c436785ac679bec0d1a"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:950fd666ec9e9fe6a8eeb2b5a8f17301790e518953730ad44d715b59ffdbc67f"}, + {file = "psycopg_binary-3.2.1-cp312-cp312-win_amd64.whl", hash = "sha256:334046a937bb086c36e2c6889fe327f9f29bfc085d678f70fac0b0618949f674"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-macosx_12_0_x86_64.whl", hash = "sha256:1d6833f607f3fc7b22226a9e121235d3b84c0eda1d3caab174673ef698f63788"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1d353e028b8f848b9784450fc2abf149d53a738d451eab3ee4c85703438128b9"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f34e369891f77d0738e5d25727c307d06d5344948771e5379ea29c76c6d84555"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0ab58213cc976a1666f66bc1cb2e602315cd753b7981a8e17237ac2a185bd4a1"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b0104a72a17aa84b3b7dcab6c84826c595355bf54bb6ea6d284dcb06d99c6801"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-musllinux_1_2_aarch64.whl", hash = "sha256:059cbd4e6da2337e17707178fe49464ed01de867dc86c677b30751755ec1dc51"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-musllinux_1_2_i686.whl", hash = "sha256:73f9c9b984be9c322b5ec1515b12df1ee5896029f5e72d46160eb6517438659c"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-musllinux_1_2_ppc64le.whl", hash = "sha256:af0469c00f24c4bec18c3d2ede124bf62688d88d1b8a5f3c3edc2f61046fe0d7"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-musllinux_1_2_x86_64.whl", hash = "sha256:463d55345f73ff391df8177a185ad57b552915ad33f5cc2b31b930500c068b22"}, + {file = "psycopg_binary-3.2.1-cp38-cp38-win_amd64.whl", hash = "sha256:302b86f92c0d76e99fe1b5c22c492ae519ce8b98b88d37ef74fda4c9e24c6b46"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-macosx_12_0_x86_64.whl", hash = "sha256:0879b5d76b7d48678d31278242aaf951bc2d69ca4e4d7cef117e4bbf7bfefda9"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f99e59f8a5f4dcd9cbdec445f3d8ac950a492fc0e211032384d6992ed3c17eb7"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:84837e99353d16c6980603b362d0f03302d4b06c71672a6651f38df8a482923d"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7ce965caf618061817f66c0906f0452aef966c293ae0933d4fa5a16ea6eaf5bb"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:78c2007caf3c90f08685c5378e3ceb142bafd5636be7495f7d86ec8a977eaeef"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:7a84b5eb194a258116154b2a4ff2962ea60ea52de089508db23a51d3d6b1c7d1"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:4a42b8f9ab39affcd5249b45cac763ac3cf12df962b67e23fd15a2ee2932afe5"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-musllinux_1_2_ppc64le.whl", hash = "sha256:788ffc43d7517c13e624c83e0e553b7b8823c9655e18296566d36a829bfb373f"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:21927f41c4d722ae8eb30d62a6ce732c398eac230509af5ba1749a337f8a63e2"}, + {file = "psycopg_binary-3.2.1-cp39-cp39-win_amd64.whl", hash = "sha256:921f0c7f39590763d64a619de84d1b142587acc70fd11cbb5ba8fa39786f3073"}, ] [[package]] @@ -4314,109 +4253,118 @@ files = [ [[package]] name = "pydantic" -version = "2.7.3" +version = "2.8.0" description = "Data validation using Python type hints" optional = false python-versions = ">=3.8" files = [ - {file = "pydantic-2.7.3-py3-none-any.whl", hash = "sha256:ea91b002777bf643bb20dd717c028ec43216b24a6001a280f83877fd2655d0b4"}, - {file = "pydantic-2.7.3.tar.gz", hash = "sha256:c46c76a40bb1296728d7a8b99aa73dd70a48c3510111ff290034f860c99c419e"}, + {file = "pydantic-2.8.0-py3-none-any.whl", hash = "sha256:ead4f3a1e92386a734ca1411cb25d94147cf8778ed5be6b56749047676d6364e"}, + {file = "pydantic-2.8.0.tar.gz", hash = "sha256:d970ffb9d030b710795878940bd0489842c638e7252fc4a19c3ae2f7da4d6141"}, ] [package.dependencies] annotated-types = ">=0.4.0" -pydantic-core = "2.18.4" -typing-extensions = ">=4.6.1" +pydantic-core = "2.20.0" +typing-extensions = {version = ">=4.6.1", markers = "python_version < \"3.13\""} [package.extras] email = ["email-validator (>=2.0.0)"] [[package]] name = "pydantic-core" -version = "2.18.4" +version = "2.20.0" description = "Core functionality for Pydantic validation and serialization" optional = false python-versions = ">=3.8" files = [ - {file = "pydantic_core-2.18.4-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:f76d0ad001edd426b92233d45c746fd08f467d56100fd8f30e9ace4b005266e4"}, - {file = "pydantic_core-2.18.4-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:59ff3e89f4eaf14050c8022011862df275b552caef8082e37b542b066ce1ff26"}, - {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a55b5b16c839df1070bc113c1f7f94a0af4433fcfa1b41799ce7606e5c79ce0a"}, - {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:4d0dcc59664fcb8974b356fe0a18a672d6d7cf9f54746c05f43275fc48636851"}, - {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8951eee36c57cd128f779e641e21eb40bc5073eb28b2d23f33eb0ef14ffb3f5d"}, - {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4701b19f7e3a06ea655513f7938de6f108123bf7c86bbebb1196eb9bd35cf724"}, - {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e00a3f196329e08e43d99b79b286d60ce46bed10f2280d25a1718399457e06be"}, - {file = "pydantic_core-2.18.4-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:97736815b9cc893b2b7f663628e63f436018b75f44854c8027040e05230eeddb"}, - {file = "pydantic_core-2.18.4-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:6891a2ae0e8692679c07728819b6e2b822fb30ca7445f67bbf6509b25a96332c"}, - {file = "pydantic_core-2.18.4-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:bc4ff9805858bd54d1a20efff925ccd89c9d2e7cf4986144b30802bf78091c3e"}, - {file = "pydantic_core-2.18.4-cp310-none-win32.whl", hash = "sha256:1b4de2e51bbcb61fdebd0ab86ef28062704f62c82bbf4addc4e37fa4b00b7cbc"}, - {file = "pydantic_core-2.18.4-cp310-none-win_amd64.whl", hash = "sha256:6a750aec7bf431517a9fd78cb93c97b9b0c496090fee84a47a0d23668976b4b0"}, - {file = "pydantic_core-2.18.4-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:942ba11e7dfb66dc70f9ae66b33452f51ac7bb90676da39a7345e99ffb55402d"}, - {file = "pydantic_core-2.18.4-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:b2ebef0e0b4454320274f5e83a41844c63438fdc874ea40a8b5b4ecb7693f1c4"}, - {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a642295cd0c8df1b86fc3dced1d067874c353a188dc8e0f744626d49e9aa51c4"}, - {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5f09baa656c904807e832cf9cce799c6460c450c4ad80803517032da0cd062e2"}, - {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:98906207f29bc2c459ff64fa007afd10a8c8ac080f7e4d5beff4c97086a3dabd"}, - {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:19894b95aacfa98e7cb093cd7881a0c76f55731efad31073db4521e2b6ff5b7d"}, - {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0fbbdc827fe5e42e4d196c746b890b3d72876bdbf160b0eafe9f0334525119c8"}, - {file = "pydantic_core-2.18.4-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:f85d05aa0918283cf29a30b547b4df2fbb56b45b135f9e35b6807cb28bc47951"}, - {file = "pydantic_core-2.18.4-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:e85637bc8fe81ddb73fda9e56bab24560bdddfa98aa64f87aaa4e4b6730c23d2"}, - {file = "pydantic_core-2.18.4-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:2f5966897e5461f818e136b8451d0551a2e77259eb0f73a837027b47dc95dab9"}, - {file = "pydantic_core-2.18.4-cp311-none-win32.whl", hash = "sha256:44c7486a4228413c317952e9d89598bcdfb06399735e49e0f8df643e1ccd0558"}, - {file = "pydantic_core-2.18.4-cp311-none-win_amd64.whl", hash = "sha256:8a7164fe2005d03c64fd3b85649891cd4953a8de53107940bf272500ba8a788b"}, - {file = "pydantic_core-2.18.4-cp311-none-win_arm64.whl", hash = "sha256:4e99bc050fe65c450344421017f98298a97cefc18c53bb2f7b3531eb39bc7805"}, - {file = "pydantic_core-2.18.4-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:6f5c4d41b2771c730ea1c34e458e781b18cc668d194958e0112455fff4e402b2"}, - {file = "pydantic_core-2.18.4-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:2fdf2156aa3d017fddf8aea5adfba9f777db1d6022d392b682d2a8329e087cef"}, - {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4748321b5078216070b151d5271ef3e7cc905ab170bbfd27d5c83ee3ec436695"}, - {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:847a35c4d58721c5dc3dba599878ebbdfd96784f3fb8bb2c356e123bdcd73f34"}, - {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3c40d4eaad41f78e3bbda31b89edc46a3f3dc6e171bf0ecf097ff7a0ffff7cb1"}, - {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:21a5e440dbe315ab9825fcd459b8814bb92b27c974cbc23c3e8baa2b76890077"}, - {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:01dd777215e2aa86dfd664daed5957704b769e726626393438f9c87690ce78c3"}, - {file = "pydantic_core-2.18.4-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:4b06beb3b3f1479d32befd1f3079cc47b34fa2da62457cdf6c963393340b56e9"}, - {file = "pydantic_core-2.18.4-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:564d7922e4b13a16b98772441879fcdcbe82ff50daa622d681dd682175ea918c"}, - {file = "pydantic_core-2.18.4-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:0eb2a4f660fcd8e2b1c90ad566db2b98d7f3f4717c64fe0a83e0adb39766d5b8"}, - {file = "pydantic_core-2.18.4-cp312-none-win32.whl", hash = "sha256:8b8bab4c97248095ae0c4455b5a1cd1cdd96e4e4769306ab19dda135ea4cdb07"}, - {file = "pydantic_core-2.18.4-cp312-none-win_amd64.whl", hash = "sha256:14601cdb733d741b8958224030e2bfe21a4a881fb3dd6fbb21f071cabd48fa0a"}, - {file = "pydantic_core-2.18.4-cp312-none-win_arm64.whl", hash = "sha256:c1322d7dd74713dcc157a2b7898a564ab091ca6c58302d5c7b4c07296e3fd00f"}, - {file = "pydantic_core-2.18.4-cp38-cp38-macosx_10_12_x86_64.whl", hash = "sha256:823be1deb01793da05ecb0484d6c9e20baebb39bd42b5d72636ae9cf8350dbd2"}, - {file = "pydantic_core-2.18.4-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:ebef0dd9bf9b812bf75bda96743f2a6c5734a02092ae7f721c048d156d5fabae"}, - {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ae1d6df168efb88d7d522664693607b80b4080be6750c913eefb77e34c12c71a"}, - {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f9899c94762343f2cc2fc64c13e7cae4c3cc65cdfc87dd810a31654c9b7358cc"}, - {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:99457f184ad90235cfe8461c4d70ab7dd2680e28821c29eca00252ba90308c78"}, - {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:18f469a3d2a2fdafe99296a87e8a4c37748b5080a26b806a707f25a902c040a8"}, - {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b7cdf28938ac6b8b49ae5e92f2735056a7ba99c9b110a474473fd71185c1af5d"}, - {file = "pydantic_core-2.18.4-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:938cb21650855054dc54dfd9120a851c974f95450f00683399006aa6e8abb057"}, - {file = "pydantic_core-2.18.4-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:44cd83ab6a51da80fb5adbd9560e26018e2ac7826f9626bc06ca3dc074cd198b"}, - {file = "pydantic_core-2.18.4-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:972658f4a72d02b8abfa2581d92d59f59897d2e9f7e708fdabe922f9087773af"}, - {file = "pydantic_core-2.18.4-cp38-none-win32.whl", hash = "sha256:1d886dc848e60cb7666f771e406acae54ab279b9f1e4143babc9c2258213daa2"}, - {file = "pydantic_core-2.18.4-cp38-none-win_amd64.whl", hash = "sha256:bb4462bd43c2460774914b8525f79b00f8f407c945d50881568f294c1d9b4443"}, - {file = "pydantic_core-2.18.4-cp39-cp39-macosx_10_12_x86_64.whl", hash = "sha256:44a688331d4a4e2129140a8118479443bd6f1905231138971372fcde37e43528"}, - {file = "pydantic_core-2.18.4-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:a2fdd81edd64342c85ac7cf2753ccae0b79bf2dfa063785503cb85a7d3593223"}, - {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:86110d7e1907ab36691f80b33eb2da87d780f4739ae773e5fc83fb272f88825f"}, - {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:46387e38bd641b3ee5ce247563b60c5ca098da9c56c75c157a05eaa0933ed154"}, - {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:123c3cec203e3f5ac7b000bd82235f1a3eced8665b63d18be751f115588fea30"}, - {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:dc1803ac5c32ec324c5261c7209e8f8ce88e83254c4e1aebdc8b0a39f9ddb443"}, - {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:53db086f9f6ab2b4061958d9c276d1dbe3690e8dd727d6abf2321d6cce37fa94"}, - {file = "pydantic_core-2.18.4-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:abc267fa9837245cc28ea6929f19fa335f3dc330a35d2e45509b6566dc18be23"}, - {file = "pydantic_core-2.18.4-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:a0d829524aaefdebccb869eed855e2d04c21d2d7479b6cada7ace5448416597b"}, - {file = "pydantic_core-2.18.4-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:509daade3b8649f80d4e5ff21aa5673e4ebe58590b25fe42fac5f0f52c6f034a"}, - {file = "pydantic_core-2.18.4-cp39-none-win32.whl", hash = "sha256:ca26a1e73c48cfc54c4a76ff78df3727b9d9f4ccc8dbee4ae3f73306a591676d"}, - {file = "pydantic_core-2.18.4-cp39-none-win_amd64.whl", hash = "sha256:c67598100338d5d985db1b3d21f3619ef392e185e71b8d52bceacc4a7771ea7e"}, - {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-macosx_10_12_x86_64.whl", hash = "sha256:574d92eac874f7f4db0ca653514d823a0d22e2354359d0759e3f6a406db5d55d"}, - {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:1f4d26ceb5eb9eed4af91bebeae4b06c3fb28966ca3a8fb765208cf6b51102ab"}, - {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:77450e6d20016ec41f43ca4a6c63e9fdde03f0ae3fe90e7c27bdbeaece8b1ed4"}, - {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d323a01da91851a4f17bf592faf46149c9169d68430b3146dcba2bb5e5719abc"}, - {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:43d447dd2ae072a0065389092a231283f62d960030ecd27565672bd40746c507"}, - {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:578e24f761f3b425834f297b9935e1ce2e30f51400964ce4801002435a1b41ef"}, - {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:81b5efb2f126454586d0f40c4d834010979cb80785173d1586df845a632e4e6d"}, - {file = "pydantic_core-2.18.4-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:ab86ce7c8f9bea87b9d12c7f0af71102acbf5ecbc66c17796cff45dae54ef9a5"}, - {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-macosx_10_12_x86_64.whl", hash = "sha256:90afc12421df2b1b4dcc975f814e21bc1754640d502a2fbcc6d41e77af5ec312"}, - {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:51991a89639a912c17bef4b45c87bd83593aee0437d8102556af4885811d59f5"}, - {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:293afe532740370aba8c060882f7d26cfd00c94cae32fd2e212a3a6e3b7bc15e"}, - {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b48ece5bde2e768197a2d0f6e925f9d7e3e826f0ad2271120f8144a9db18d5c8"}, - {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:eae237477a873ab46e8dd748e515c72c0c804fb380fbe6c85533c7de51f23a8f"}, - {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:834b5230b5dfc0c1ec37b2fda433b271cbbc0e507560b5d1588e2cc1148cf1ce"}, - {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:e858ac0a25074ba4bce653f9b5d0a85b7456eaddadc0ce82d3878c22489fa4ee"}, - {file = "pydantic_core-2.18.4-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:2fd41f6eff4c20778d717af1cc50eca52f5afe7805ee530a4fbd0bae284f16e9"}, - {file = "pydantic_core-2.18.4.tar.gz", hash = "sha256:ec3beeada09ff865c344ff3bc2f427f5e6c26401cc6113d77e372c3fdac73864"}, + {file = "pydantic_core-2.20.0-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:e9dcd7fb34f7bfb239b5fa420033642fff0ad676b765559c3737b91f664d4fa9"}, + {file = "pydantic_core-2.20.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:649a764d9b0da29816889424697b2a3746963ad36d3e0968784ceed6e40c6355"}, + {file = "pydantic_core-2.20.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7701df088d0b05f3460f7ba15aec81ac8b0fb5690367dfd072a6c38cf5b7fdb5"}, + {file = "pydantic_core-2.20.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ab760f17c3e792225cdaef31ca23c0aea45c14ce80d8eff62503f86a5ab76bff"}, + {file = "pydantic_core-2.20.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:cb1ad5b4d73cde784cf64580166568074f5ccd2548d765e690546cff3d80937d"}, + {file = "pydantic_core-2.20.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b81ec2efc04fc1dbf400647d4357d64fb25543bae38d2d19787d69360aad21c9"}, + {file = "pydantic_core-2.20.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c4a9732a5cad764ba37f3aa873dccb41b584f69c347a57323eda0930deec8e10"}, + {file = "pydantic_core-2.20.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:6dc85b9e10cc21d9c1055f15684f76fa4facadddcb6cd63abab702eb93c98943"}, + {file = "pydantic_core-2.20.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:21d9f7e24f63fdc7118e6cc49defaab8c1d27570782f7e5256169d77498cf7c7"}, + {file = "pydantic_core-2.20.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:8b315685832ab9287e6124b5d74fc12dda31e6421d7f6b08525791452844bc2d"}, + {file = "pydantic_core-2.20.0-cp310-none-win32.whl", hash = "sha256:c3dc8ec8b87c7ad534c75b8855168a08a7036fdb9deeeed5705ba9410721c84d"}, + {file = "pydantic_core-2.20.0-cp310-none-win_amd64.whl", hash = "sha256:85770b4b37bb36ef93a6122601795231225641003e0318d23c6233c59b424279"}, + {file = "pydantic_core-2.20.0-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:58e251bb5a5998f7226dc90b0b753eeffa720bd66664eba51927c2a7a2d5f32c"}, + {file = "pydantic_core-2.20.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:78d584caac52c24240ef9ecd75de64c760bbd0e20dbf6973631815e3ef16ef8b"}, + {file = "pydantic_core-2.20.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5084ec9721f82bef5ff7c4d1ee65e1626783abb585f8c0993833490b63fe1792"}, + {file = "pydantic_core-2.20.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:6d0f52684868db7c218437d260e14d37948b094493f2646f22d3dda7229bbe3f"}, + {file = "pydantic_core-2.20.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1def125d59a87fe451212a72ab9ed34c118ff771e5473fef4f2f95d8ede26d75"}, + {file = "pydantic_core-2.20.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b34480fd6778ab356abf1e9086a4ced95002a1e195e8d2fd182b0def9d944d11"}, + {file = "pydantic_core-2.20.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d42669d319db366cb567c3b444f43caa7ffb779bf9530692c6f244fc635a41eb"}, + {file = "pydantic_core-2.20.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:53b06aea7a48919a254b32107647be9128c066aaa6ee6d5d08222325f25ef175"}, + {file = "pydantic_core-2.20.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:1f038156b696a1c39d763b2080aeefa87ddb4162c10aa9fabfefffc3dd8180fa"}, + {file = "pydantic_core-2.20.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:3f0f3a4a23717280a5ee3ac4fb1f81d6fde604c9ec5100f7f6f987716bb8c137"}, + {file = "pydantic_core-2.20.0-cp311-none-win32.whl", hash = "sha256:316fe7c3fec017affd916a0c83d6f1ec697cbbbdf1124769fa73328e7907cc2e"}, + {file = "pydantic_core-2.20.0-cp311-none-win_amd64.whl", hash = "sha256:2d06a7fa437f93782e3f32d739c3ec189f82fca74336c08255f9e20cea1ed378"}, + {file = "pydantic_core-2.20.0-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:d6f8c49657f3eb7720ed4c9b26624063da14937fc94d1812f1e04a2204db3e17"}, + {file = "pydantic_core-2.20.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ad1bd2f377f56fec11d5cfd0977c30061cd19f4fa199bf138b200ec0d5e27eeb"}, + {file = "pydantic_core-2.20.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ed741183719a5271f97d93bbcc45ed64619fa38068aaa6e90027d1d17e30dc8d"}, + {file = "pydantic_core-2.20.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:d82e5ed3a05f2dcb89c6ead2fd0dbff7ac09bc02c1b4028ece2d3a3854d049ce"}, + {file = "pydantic_core-2.20.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b2ba34a099576234671f2e4274e5bc6813b22e28778c216d680eabd0db3f7dad"}, + {file = "pydantic_core-2.20.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:879ae6bb08a063b3e1b7ac8c860096d8fd6b48dd9b2690b7f2738b8c835e744b"}, + {file = "pydantic_core-2.20.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0b0eefc7633a04c0694340aad91fbfd1986fe1a1e0c63a22793ba40a18fcbdc8"}, + {file = "pydantic_core-2.20.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:73deadd6fd8a23e2f40b412b3ac617a112143c8989a4fe265050fd91ba5c0608"}, + {file = "pydantic_core-2.20.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:35681445dc85446fb105943d81ae7569aa7e89de80d1ca4ac3229e05c311bdb1"}, + {file = "pydantic_core-2.20.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:0f6dd3612a3b9f91f2e63924ea18a4476656c6d01843ca20a4c09e00422195af"}, + {file = "pydantic_core-2.20.0-cp312-none-win32.whl", hash = "sha256:7e37b6bb6e90c2b8412b06373c6978d9d81e7199a40e24a6ef480e8acdeaf918"}, + {file = "pydantic_core-2.20.0-cp312-none-win_amd64.whl", hash = "sha256:7d4df13d1c55e84351fab51383520b84f490740a9f1fec905362aa64590b7a5d"}, + {file = "pydantic_core-2.20.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:d43e7ab3b65e4dc35a7612cfff7b0fd62dce5bc11a7cd198310b57f39847fd6c"}, + {file = "pydantic_core-2.20.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7b6a24d7b5893392f2b8e3b7a0031ae3b14c6c1942a4615f0d8794fdeeefb08b"}, + {file = "pydantic_core-2.20.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b2f13c3e955a087c3ec86f97661d9f72a76e221281b2262956af381224cfc243"}, + {file = "pydantic_core-2.20.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:72432fd6e868c8d0a6849869e004b8bcae233a3c56383954c228316694920b38"}, + {file = "pydantic_core-2.20.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d70a8ff2d4953afb4cbe6211f17268ad29c0b47e73d3372f40e7775904bc28fc"}, + {file = "pydantic_core-2.20.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8e49524917b8d3c2f42cd0d2df61178e08e50f5f029f9af1f402b3ee64574392"}, + {file = "pydantic_core-2.20.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a4f0f71653b1c1bad0350bc0b4cc057ab87b438ff18fa6392533811ebd01439c"}, + {file = "pydantic_core-2.20.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:16197e6f4fdecb9892ed2436e507e44f0a1aa2cff3b9306d1c879ea2f9200997"}, + {file = "pydantic_core-2.20.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:763602504bf640b3ded3bba3f8ed8a1cc2fc6a87b8d55c1c5689f428c49c947e"}, + {file = "pydantic_core-2.20.0-cp313-none-win32.whl", hash = "sha256:a3f243f318bd9523277fa123b3163f4c005a3e8619d4b867064de02f287a564d"}, + {file = "pydantic_core-2.20.0-cp313-none-win_amd64.whl", hash = "sha256:03aceaf6a5adaad3bec2233edc5a7905026553916615888e53154807e404545c"}, + {file = "pydantic_core-2.20.0-cp38-cp38-macosx_10_12_x86_64.whl", hash = "sha256:d6f2d8b8da1f03f577243b07bbdd3412eee3d37d1f2fd71d1513cbc76a8c1239"}, + {file = "pydantic_core-2.20.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:a272785a226869416c6b3c1b7e450506152d3844207331f02f27173562c917e0"}, + {file = "pydantic_core-2.20.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:efbb412d55a4ffe73963fed95c09ccb83647ec63b711c4b3752be10a56f0090b"}, + {file = "pydantic_core-2.20.0-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:1e4f46189d8740561b43655263a41aac75ff0388febcb2c9ec4f1b60a0ec12f3"}, + {file = "pydantic_core-2.20.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:87d3df115f4a3c8c5e4d5acf067d399c6466d7e604fc9ee9acbe6f0c88a0c3cf"}, + {file = "pydantic_core-2.20.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a340d2bdebe819d08f605e9705ed551c3feb97e4fd71822d7147c1e4bdbb9508"}, + {file = "pydantic_core-2.20.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:616b9c2f882393d422ba11b40e72382fe975e806ad693095e9a3b67c59ea6150"}, + {file = "pydantic_core-2.20.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:25c46bb2ff6084859bbcfdf4f1a63004b98e88b6d04053e8bf324e115398e9e7"}, + {file = "pydantic_core-2.20.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:23425eccef8f2c342f78d3a238c824623836c6c874d93c726673dbf7e56c78c0"}, + {file = "pydantic_core-2.20.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:52527e8f223ba29608d999d65b204676398009725007c9336651c2ec2d93cffc"}, + {file = "pydantic_core-2.20.0-cp38-none-win32.whl", hash = "sha256:1c3c5b7f70dd19a6845292b0775295ea81c61540f68671ae06bfe4421b3222c2"}, + {file = "pydantic_core-2.20.0-cp38-none-win_amd64.whl", hash = "sha256:8093473d7b9e908af1cef30025609afc8f5fd2a16ff07f97440fd911421e4432"}, + {file = "pydantic_core-2.20.0-cp39-cp39-macosx_10_12_x86_64.whl", hash = "sha256:ee7785938e407418795e4399b2bf5b5f3cf6cf728077a7f26973220d58d885cf"}, + {file = "pydantic_core-2.20.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:0e75794883d635071cf6b4ed2a5d7a1e50672ab7a051454c76446ef1ebcdcc91"}, + {file = "pydantic_core-2.20.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:344e352c96e53b4f56b53d24728217c69399b8129c16789f70236083c6ceb2ac"}, + {file = "pydantic_core-2.20.0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:978d4123ad1e605daf1ba5e01d4f235bcf7b6e340ef07e7122e8e9cfe3eb61ab"}, + {file = "pydantic_core-2.20.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3c05eaf6c863781eb834ab41f5963604ab92855822a2062897958089d1335dad"}, + {file = "pydantic_core-2.20.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bc7e43b4a528ffca8c9151b6a2ca34482c2fdc05e6aa24a84b7f475c896fc51d"}, + {file = "pydantic_core-2.20.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:658287a29351166510ebbe0a75c373600cc4367a3d9337b964dada8d38bcc0f4"}, + {file = "pydantic_core-2.20.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1dacf660d6de692fe351e8c806e7efccf09ee5184865893afbe8e59be4920b4a"}, + {file = "pydantic_core-2.20.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:3e147fc6e27b9a487320d78515c5f29798b539179f7777018cedf51b7749e4f4"}, + {file = "pydantic_core-2.20.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:c867230d715a3dd1d962c8d9bef0d3168994ed663e21bf748b6e3a529a129aab"}, + {file = "pydantic_core-2.20.0-cp39-none-win32.whl", hash = "sha256:22b813baf0dbf612752d8143a2dbf8e33ccb850656b7850e009bad2e101fc377"}, + {file = "pydantic_core-2.20.0-cp39-none-win_amd64.whl", hash = "sha256:3a7235b46c1bbe201f09b6f0f5e6c36b16bad3d0532a10493742f91fbdc8035f"}, + {file = "pydantic_core-2.20.0-pp310-pypy310_pp73-macosx_10_12_x86_64.whl", hash = "sha256:cafde15a6f7feaec2f570646e2ffc5b73412295d29134a29067e70740ec6ee20"}, + {file = "pydantic_core-2.20.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:2aec8eeea0b08fd6bc2213d8e86811a07491849fd3d79955b62d83e32fa2ad5f"}, + {file = "pydantic_core-2.20.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:840200827984f1c4e114008abc2f5ede362d6e11ed0b5931681884dd41852ff1"}, + {file = "pydantic_core-2.20.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f8ea1d8b7df522e5ced34993c423c3bf3735c53df8b2a15688a2f03a7d678800"}, + {file = "pydantic_core-2.20.0-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:d5b8376a867047bf08910573deb95d3c8dfb976eb014ee24f3b5a61ccc5bee1b"}, + {file = "pydantic_core-2.20.0-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:d08264b4460326cefacc179fc1411304d5af388a79910832835e6f641512358b"}, + {file = "pydantic_core-2.20.0-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:7a3639011c2e8a9628466f616ed7fb413f30032b891898e10895a0a8b5857d6c"}, + {file = "pydantic_core-2.20.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:05e83ce2f7eba29e627dd8066aa6c4c0269b2d4f889c0eba157233a353053cea"}, + {file = "pydantic_core-2.20.0-pp39-pypy39_pp73-macosx_10_12_x86_64.whl", hash = "sha256:603a843fea76a595c8f661cd4da4d2281dff1e38c4a836a928eac1a2f8fe88e4"}, + {file = "pydantic_core-2.20.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:ac76f30d5d3454f4c28826d891fe74d25121a346c69523c9810ebba43f3b1cec"}, + {file = "pydantic_core-2.20.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:22e3b1d4b1b3f6082849f9b28427ef147a5b46a6132a3dbaf9ca1baa40c88609"}, + {file = "pydantic_core-2.20.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2761f71faed820e25ec62eacba670d1b5c2709bb131a19fcdbfbb09884593e5a"}, + {file = "pydantic_core-2.20.0-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:a0586cddbf4380e24569b8a05f234e7305717cc8323f50114dfb2051fcbce2a3"}, + {file = "pydantic_core-2.20.0-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:b8c46a8cf53e849eea7090f331ae2202cd0f1ceb090b00f5902c423bd1e11805"}, + {file = "pydantic_core-2.20.0-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:b4a085bd04af7245e140d1b95619fe8abb445a3d7fdf219b3f80c940853268ef"}, + {file = "pydantic_core-2.20.0-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:116b326ac82c8b315e7348390f6d30bcfe6e688a7d3f1de50ff7bcc2042a23c2"}, + {file = "pydantic_core-2.20.0.tar.gz", hash = "sha256:366be8e64e0cb63d87cf79b4e1765c0703dd6313c729b22e7b9e378db6b96877"}, ] [package.dependencies] @@ -4919,13 +4867,13 @@ cffi = {version = "*", markers = "implementation_name == \"pypy\""} [[package]] name = "qdrant-client" -version = "1.9.1" +version = "1.10.0" description = "Client library for the Qdrant vector search engine" optional = false python-versions = ">=3.8" files = [ - {file = "qdrant_client-1.9.1-py3-none-any.whl", hash = "sha256:b9b7e0e5c1a51410d8bb5106a869a51e12f92ab45a99030f27aba790553bd2c8"}, - {file = "qdrant_client-1.9.1.tar.gz", hash = "sha256:186b9c31d95aefe8f2db84b7746402d7365bd63b305550e530e31bde2002ce79"}, + {file = "qdrant_client-1.10.0-py3-none-any.whl", hash = "sha256:423c2586709ccf3db20850cd85c3d18954692a8faff98367dfa9dc82ab7f91d9"}, + {file = "qdrant_client-1.10.0.tar.gz", hash = "sha256:47c4f7abfab152fb7e5e4902ab0e2e9e33483c49ea5e80128ccd0295f342cf9b"}, ] [package.dependencies] @@ -4941,7 +4889,8 @@ pydantic = ">=1.10.8" urllib3 = ">=1.26.14,<3" [package.extras] -fastembed = ["fastembed (==0.2.6)"] +fastembed = ["fastembed (==0.2.7)"] +fastembed-gpu = ["fastembed-gpu (==0.2.7)"] [[package]] name = "redis" @@ -5529,45 +5478,45 @@ tests = ["black (>=24.3.0)", "matplotlib (>=3.3.4)", "mypy (>=1.9)", "numpydoc ( [[package]] name = "scipy" -version = "1.13.1" +version = "1.14.0" description = "Fundamental algorithms for scientific computing in Python" optional = false -python-versions = ">=3.9" +python-versions = ">=3.10" files = [ - {file = "scipy-1.13.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:20335853b85e9a49ff7572ab453794298bcf0354d8068c5f6775a0eabf350aca"}, - {file = "scipy-1.13.1-cp310-cp310-macosx_12_0_arm64.whl", hash = "sha256:d605e9c23906d1994f55ace80e0125c587f96c020037ea6aa98d01b4bd2e222f"}, - {file = "scipy-1.13.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cfa31f1def5c819b19ecc3a8b52d28ffdcc7ed52bb20c9a7589669dd3c250989"}, - {file = "scipy-1.13.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f26264b282b9da0952a024ae34710c2aff7d27480ee91a2e82b7b7073c24722f"}, - {file = "scipy-1.13.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:eccfa1906eacc02de42d70ef4aecea45415f5be17e72b61bafcfd329bdc52e94"}, - {file = "scipy-1.13.1-cp310-cp310-win_amd64.whl", hash = "sha256:2831f0dc9c5ea9edd6e51e6e769b655f08ec6db6e2e10f86ef39bd32eb11da54"}, - {file = "scipy-1.13.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:27e52b09c0d3a1d5b63e1105f24177e544a222b43611aaf5bc44d4a0979e32f9"}, - {file = "scipy-1.13.1-cp311-cp311-macosx_12_0_arm64.whl", hash = "sha256:54f430b00f0133e2224c3ba42b805bfd0086fe488835effa33fa291561932326"}, - {file = "scipy-1.13.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e89369d27f9e7b0884ae559a3a956e77c02114cc60a6058b4e5011572eea9299"}, - {file = "scipy-1.13.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a78b4b3345f1b6f68a763c6e25c0c9a23a9fd0f39f5f3d200efe8feda560a5fa"}, - {file = "scipy-1.13.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:45484bee6d65633752c490404513b9ef02475b4284c4cfab0ef946def50b3f59"}, - {file = "scipy-1.13.1-cp311-cp311-win_amd64.whl", hash = "sha256:5713f62f781eebd8d597eb3f88b8bf9274e79eeabf63afb4a737abc6c84ad37b"}, - {file = "scipy-1.13.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:5d72782f39716b2b3509cd7c33cdc08c96f2f4d2b06d51e52fb45a19ca0c86a1"}, - {file = "scipy-1.13.1-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:017367484ce5498445aade74b1d5ab377acdc65e27095155e448c88497755a5d"}, - {file = "scipy-1.13.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:949ae67db5fa78a86e8fa644b9a6b07252f449dcf74247108c50e1d20d2b4627"}, - {file = "scipy-1.13.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:de3ade0e53bc1f21358aa74ff4830235d716211d7d077e340c7349bc3542e884"}, - {file = "scipy-1.13.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:2ac65fb503dad64218c228e2dc2d0a0193f7904747db43014645ae139c8fad16"}, - {file = "scipy-1.13.1-cp312-cp312-win_amd64.whl", hash = "sha256:cdd7dacfb95fea358916410ec61bbc20440f7860333aee6d882bb8046264e949"}, - {file = "scipy-1.13.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:436bbb42a94a8aeef855d755ce5a465479c721e9d684de76bf61a62e7c2b81d5"}, - {file = "scipy-1.13.1-cp39-cp39-macosx_12_0_arm64.whl", hash = "sha256:8335549ebbca860c52bf3d02f80784e91a004b71b059e3eea9678ba994796a24"}, - {file = "scipy-1.13.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d533654b7d221a6a97304ab63c41c96473ff04459e404b83275b60aa8f4b7004"}, - {file = "scipy-1.13.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:637e98dcf185ba7f8e663e122ebf908c4702420477ae52a04f9908707456ba4d"}, - {file = "scipy-1.13.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:a014c2b3697bde71724244f63de2476925596c24285c7a637364761f8710891c"}, - {file = "scipy-1.13.1-cp39-cp39-win_amd64.whl", hash = "sha256:392e4ec766654852c25ebad4f64e4e584cf19820b980bc04960bca0b0cd6eaa2"}, - {file = "scipy-1.13.1.tar.gz", hash = "sha256:095a87a0312b08dfd6a6155cbbd310a8c51800fc931b8c0b84003014b874ed3c"}, -] - -[package.dependencies] -numpy = ">=1.22.4,<2.3" + {file = "scipy-1.14.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:7e911933d54ead4d557c02402710c2396529540b81dd554fc1ba270eb7308484"}, + {file = "scipy-1.14.0-cp310-cp310-macosx_12_0_arm64.whl", hash = "sha256:687af0a35462402dd851726295c1a5ae5f987bd6e9026f52e9505994e2f84ef6"}, + {file = "scipy-1.14.0-cp310-cp310-macosx_14_0_arm64.whl", hash = "sha256:07e179dc0205a50721022344fb85074f772eadbda1e1b3eecdc483f8033709b7"}, + {file = "scipy-1.14.0-cp310-cp310-macosx_14_0_x86_64.whl", hash = "sha256:6a9c9a9b226d9a21e0a208bdb024c3982932e43811b62d202aaf1bb59af264b1"}, + {file = "scipy-1.14.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:076c27284c768b84a45dcf2e914d4000aac537da74236a0d45d82c6fa4b7b3c0"}, + {file = "scipy-1.14.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:42470ea0195336df319741e230626b6225a740fd9dce9642ca13e98f667047c0"}, + {file = "scipy-1.14.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:176c6f0d0470a32f1b2efaf40c3d37a24876cebf447498a4cefb947a79c21e9d"}, + {file = "scipy-1.14.0-cp310-cp310-win_amd64.whl", hash = "sha256:ad36af9626d27a4326c8e884917b7ec321d8a1841cd6dacc67d2a9e90c2f0359"}, + {file = "scipy-1.14.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:6d056a8709ccda6cf36cdd2eac597d13bc03dba38360f418560a93050c76a16e"}, + {file = "scipy-1.14.0-cp311-cp311-macosx_12_0_arm64.whl", hash = "sha256:f0a50da861a7ec4573b7c716b2ebdcdf142b66b756a0d392c236ae568b3a93fb"}, + {file = "scipy-1.14.0-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:94c164a9e2498e68308e6e148646e486d979f7fcdb8b4cf34b5441894bdb9caf"}, + {file = "scipy-1.14.0-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:a7d46c3e0aea5c064e734c3eac5cf9eb1f8c4ceee756262f2c7327c4c2691c86"}, + {file = "scipy-1.14.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9eee2989868e274aae26125345584254d97c56194c072ed96cb433f32f692ed8"}, + {file = "scipy-1.14.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9e3154691b9f7ed73778d746da2df67a19d046a6c8087c8b385bc4cdb2cfca74"}, + {file = "scipy-1.14.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:c40003d880f39c11c1edbae8144e3813904b10514cd3d3d00c277ae996488cdb"}, + {file = "scipy-1.14.0-cp311-cp311-win_amd64.whl", hash = "sha256:5b083c8940028bb7e0b4172acafda6df762da1927b9091f9611b0bcd8676f2bc"}, + {file = "scipy-1.14.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:bff2438ea1330e06e53c424893ec0072640dac00f29c6a43a575cbae4c99b2b9"}, + {file = "scipy-1.14.0-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:bbc0471b5f22c11c389075d091d3885693fd3f5e9a54ce051b46308bc787e5d4"}, + {file = "scipy-1.14.0-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:64b2ff514a98cf2bb734a9f90d32dc89dc6ad4a4a36a312cd0d6327170339eb0"}, + {file = "scipy-1.14.0-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:7d3da42fbbbb860211a811782504f38ae7aaec9de8764a9bef6b262de7a2b50f"}, + {file = "scipy-1.14.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d91db2c41dd6c20646af280355d41dfa1ec7eead235642178bd57635a3f82209"}, + {file = "scipy-1.14.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a01cc03bcdc777c9da3cfdcc74b5a75caffb48a6c39c8450a9a05f82c4250a14"}, + {file = "scipy-1.14.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:65df4da3c12a2bb9ad52b86b4dcf46813e869afb006e58be0f516bc370165159"}, + {file = "scipy-1.14.0-cp312-cp312-win_amd64.whl", hash = "sha256:4c4161597c75043f7154238ef419c29a64ac4a7c889d588ea77690ac4d0d9b20"}, + {file = "scipy-1.14.0.tar.gz", hash = "sha256:b5923f48cb840380f9854339176ef21763118a7300a88203ccd0bdd26e58527b"}, +] + +[package.dependencies] +numpy = ">=1.23.5,<2.3" [package.extras] -dev = ["cython-lint (>=0.12.2)", "doit (>=0.36.0)", "mypy", "pycodestyle", "pydevtool", "rich-click", "ruff", "types-psutil", "typing_extensions"] -doc = ["jupyterlite-pyodide-kernel", "jupyterlite-sphinx (>=0.12.0)", "jupytext", "matplotlib (>=3.5)", "myst-nb", "numpydoc", "pooch", "pydata-sphinx-theme (>=0.15.2)", "sphinx (>=5.0.0)", "sphinx-design (>=0.4.0)"] -test = ["array-api-strict", "asv", "gmpy2", "hypothesis (>=6.30)", "mpmath", "pooch", "pytest", "pytest-cov", "pytest-timeout", "pytest-xdist", "scikit-umfpack", "threadpoolctl"] +dev = ["cython-lint (>=0.12.2)", "doit (>=0.36.0)", "mypy (==1.10.0)", "pycodestyle", "pydevtool", "rich-click", "ruff (>=0.0.292)", "types-psutil", "typing_extensions"] +doc = ["jupyterlite-pyodide-kernel", "jupyterlite-sphinx (>=0.13.1)", "jupytext", "matplotlib (>=3.5)", "myst-nb", "numpydoc", "pooch", "pydata-sphinx-theme (>=0.15.2)", "sphinx (>=5.0.0)", "sphinx-design (>=0.4.0)"] +test = ["Cython", "array-api-strict", "asv", "gmpy2", "hypothesis (>=6.30)", "meson", "mpmath", "ninja", "pooch", "pytest", "pytest-cov", "pytest-timeout", "pytest-xdist", "scikit-umfpack", "threadpoolctl"] [[package]] name = "sentence-transformers" @@ -5921,31 +5870,31 @@ files = [ [[package]] name = "torch" -version = "2.3.0" +version = "2.3.1" description = "Tensors and Dynamic neural networks in Python with strong GPU acceleration" optional = false python-versions = ">=3.8.0" files = [ - {file = "torch-2.3.0-cp310-cp310-manylinux1_x86_64.whl", hash = "sha256:d8ea5a465dbfd8501f33c937d1f693176c9aef9d1c1b0ca1d44ed7b0a18c52ac"}, - {file = "torch-2.3.0-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:09c81c5859a5b819956c6925a405ef1cdda393c9d8a01ce3851453f699d3358c"}, - {file = "torch-2.3.0-cp310-cp310-win_amd64.whl", hash = "sha256:1bf023aa20902586f614f7682fedfa463e773e26c58820b74158a72470259459"}, - {file = "torch-2.3.0-cp310-none-macosx_11_0_arm64.whl", hash = "sha256:758ef938de87a2653bba74b91f703458c15569f1562bf4b6c63c62d9c5a0c1f5"}, - {file = "torch-2.3.0-cp311-cp311-manylinux1_x86_64.whl", hash = "sha256:493d54ee2f9df100b5ce1d18c96dbb8d14908721f76351e908c9d2622773a788"}, - {file = "torch-2.3.0-cp311-cp311-manylinux2014_aarch64.whl", hash = "sha256:bce43af735c3da16cc14c7de2be7ad038e2fbf75654c2e274e575c6c05772ace"}, - {file = "torch-2.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:729804e97b7cf19ae9ab4181f91f5e612af07956f35c8b2c8e9d9f3596a8e877"}, - {file = "torch-2.3.0-cp311-none-macosx_11_0_arm64.whl", hash = "sha256:d24e328226d8e2af7cf80fcb1d2f1d108e0de32777fab4aaa2b37b9765d8be73"}, - {file = "torch-2.3.0-cp312-cp312-manylinux1_x86_64.whl", hash = "sha256:b0de2bdc0486ea7b14fc47ff805172df44e421a7318b7c4d92ef589a75d27410"}, - {file = "torch-2.3.0-cp312-cp312-manylinux2014_aarch64.whl", hash = "sha256:a306c87a3eead1ed47457822c01dfbd459fe2920f2d38cbdf90de18f23f72542"}, - {file = "torch-2.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:f9b98bf1a3c8af2d4c41f0bf1433920900896c446d1ddc128290ff146d1eb4bd"}, - {file = "torch-2.3.0-cp312-none-macosx_11_0_arm64.whl", hash = "sha256:dca986214267b34065a79000cee54232e62b41dff1ec2cab9abc3fc8b3dee0ad"}, - {file = "torch-2.3.0-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:20572f426965dd8a04e92a473d7e445fa579e09943cc0354f3e6fef6130ce061"}, - {file = "torch-2.3.0-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:e65ba85ae292909cde0dde6369826d51165a3fc8823dc1854cd9432d7f79b932"}, - {file = "torch-2.3.0-cp38-cp38-win_amd64.whl", hash = "sha256:5515503a193781fd1b3f5c474e89c9dfa2faaa782b2795cc4a7ab7e67de923f6"}, - {file = "torch-2.3.0-cp38-none-macosx_11_0_arm64.whl", hash = "sha256:6ae9f64b09516baa4ef890af0672dc981c20b1f0d829ce115d4420a247e88fba"}, - {file = "torch-2.3.0-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:cd0dc498b961ab19cb3f8dbf0c6c50e244f2f37dbfa05754ab44ea057c944ef9"}, - {file = "torch-2.3.0-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:e05f836559251e4096f3786ee99f4a8cbe67bc7fbedba8ad5e799681e47c5e80"}, - {file = "torch-2.3.0-cp39-cp39-win_amd64.whl", hash = "sha256:4fb27b35dbb32303c2927da86e27b54a92209ddfb7234afb1949ea2b3effffea"}, - {file = "torch-2.3.0-cp39-none-macosx_11_0_arm64.whl", hash = "sha256:760f8bedff506ce9e6e103498f9b1e9e15809e008368594c3a66bf74a8a51380"}, + {file = "torch-2.3.1-cp310-cp310-manylinux1_x86_64.whl", hash = "sha256:605a25b23944be5ab7c3467e843580e1d888b8066e5aaf17ff7bf9cc30001cc3"}, + {file = "torch-2.3.1-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:f2357eb0965583a0954d6f9ad005bba0091f956aef879822274b1bcdb11bd308"}, + {file = "torch-2.3.1-cp310-cp310-win_amd64.whl", hash = "sha256:32b05fe0d1ada7f69c9f86c14ff69b0ef1957a5a54199bacba63d22d8fab720b"}, + {file = "torch-2.3.1-cp310-none-macosx_11_0_arm64.whl", hash = "sha256:7c09a94362778428484bcf995f6004b04952106aee0ef45ff0b4bab484f5498d"}, + {file = "torch-2.3.1-cp311-cp311-manylinux1_x86_64.whl", hash = "sha256:b2ec81b61bb094ea4a9dee1cd3f7b76a44555375719ad29f05c0ca8ef596ad39"}, + {file = "torch-2.3.1-cp311-cp311-manylinux2014_aarch64.whl", hash = "sha256:490cc3d917d1fe0bd027057dfe9941dc1d6d8e3cae76140f5dd9a7e5bc7130ab"}, + {file = "torch-2.3.1-cp311-cp311-win_amd64.whl", hash = "sha256:5802530783bd465fe66c2df99123c9a54be06da118fbd785a25ab0a88123758a"}, + {file = "torch-2.3.1-cp311-none-macosx_11_0_arm64.whl", hash = "sha256:a7dd4ed388ad1f3d502bf09453d5fe596c7b121de7e0cfaca1e2017782e9bbac"}, + {file = "torch-2.3.1-cp312-cp312-manylinux1_x86_64.whl", hash = "sha256:a486c0b1976a118805fc7c9641d02df7afbb0c21e6b555d3bb985c9f9601b61a"}, + {file = "torch-2.3.1-cp312-cp312-manylinux2014_aarch64.whl", hash = "sha256:224259821fe3e4c6f7edf1528e4fe4ac779c77addaa74215eb0b63a5c474d66c"}, + {file = "torch-2.3.1-cp312-cp312-win_amd64.whl", hash = "sha256:e5fdccbf6f1334b2203a61a0e03821d5845f1421defe311dabeae2fc8fbeac2d"}, + {file = "torch-2.3.1-cp312-none-macosx_11_0_arm64.whl", hash = "sha256:3c333dc2ebc189561514eda06e81df22bf8fb64e2384746b2cb9f04f96d1d4c8"}, + {file = "torch-2.3.1-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:07e9ba746832b8d069cacb45f312cadd8ad02b81ea527ec9766c0e7404bb3feb"}, + {file = "torch-2.3.1-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:462d1c07dbf6bb5d9d2f3316fee73a24f3d12cd8dacf681ad46ef6418f7f6626"}, + {file = "torch-2.3.1-cp38-cp38-win_amd64.whl", hash = "sha256:ff60bf7ce3de1d43ad3f6969983f321a31f0a45df3690921720bcad6a8596cc4"}, + {file = "torch-2.3.1-cp38-none-macosx_11_0_arm64.whl", hash = "sha256:bee0bd33dc58aa8fc8a7527876e9b9a0e812ad08122054a5bff2ce5abf005b10"}, + {file = "torch-2.3.1-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:aaa872abde9a3d4f91580f6396d54888620f4a0b92e3976a6034759df4b961ad"}, + {file = "torch-2.3.1-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:3d7a7f7ef21a7520510553dc3938b0c57c116a7daee20736a9e25cbc0e832bdc"}, + {file = "torch-2.3.1-cp39-cp39-win_amd64.whl", hash = "sha256:4777f6cefa0c2b5fa87223c213e7b6f417cf254a45e5829be4ccd1b2a4ee1011"}, + {file = "torch-2.3.1-cp39-none-macosx_11_0_arm64.whl", hash = "sha256:2bb5af780c55be68fe100feb0528d2edebace1d55cb2e351de735809ba7391eb"}, ] [package.dependencies] @@ -5966,7 +5915,7 @@ nvidia-cusparse-cu12 = {version = "12.1.0.106", markers = "platform_system == \" nvidia-nccl-cu12 = {version = "2.20.5", markers = "platform_system == \"Linux\" and platform_machine == \"x86_64\""} nvidia-nvtx-cu12 = {version = "12.1.105", markers = "platform_system == \"Linux\" and platform_machine == \"x86_64\""} sympy = "*" -triton = {version = "2.3.0", markers = "platform_system == \"Linux\" and platform_machine == \"x86_64\" and python_version < \"3.12\""} +triton = {version = "2.3.1", markers = "platform_system == \"Linux\" and platform_machine == \"x86_64\" and python_version < \"3.12\""} typing-extensions = ">=4.8.0" [package.extras] @@ -6030,16 +5979,17 @@ test = ["argcomplete (>=3.0.3)", "mypy (>=1.7.0)", "pre-commit", "pytest (>=7.0, [[package]] name = "transformers" -version = "4.41.1" +version = "4.41.2" description = "State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow" optional = false python-versions = ">=3.8.0" files = [ - {file = "transformers-4.41.1-py3-none-any.whl", hash = "sha256:f0680e0b1a01067eccd11f62f0522409422c7d6f91d532fe0f50b136a406129d"}, - {file = "transformers-4.41.1.tar.gz", hash = "sha256:fa859e4c66f0896633a3bf534e0d9a29a9a88478a49f94c5d8270537dc61cc42"}, + {file = "transformers-4.41.2-py3-none-any.whl", hash = "sha256:05555d20e43f808de1ef211ab64803cdb513170cef70d29a888b589caebefc67"}, + {file = "transformers-4.41.2.tar.gz", hash = "sha256:80a4db216533d573e9cc7388646c31ed9480918feb7c55eb211249cb23567f87"}, ] [package.dependencies] +accelerate = {version = ">=0.21.0", optional = true, markers = "extra == \"torch\""} filelock = "*" huggingface-hub = ">=0.23.0,<1.0" numpy = ">=1.17" @@ -6049,6 +5999,7 @@ regex = "!=2019.12.17" requests = "*" safetensors = ">=0.4.1" tokenizers = ">=0.19,<0.20" +torch = {version = "*", optional = true, markers = "extra == \"torch\""} tqdm = ">=4.27" [package.extras] @@ -6096,17 +6047,17 @@ vision = ["Pillow (>=10.0.1,<=15.0)"] [[package]] name = "triton" -version = "2.3.0" +version = "2.3.1" description = "A language and compiler for custom Deep Learning operations" optional = false python-versions = "*" files = [ - {file = "triton-2.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5ce4b8ff70c48e47274c66f269cce8861cf1dc347ceeb7a67414ca151b1822d8"}, - {file = "triton-2.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3c3d9607f85103afdb279938fc1dd2a66e4f5999a58eb48a346bd42738f986dd"}, - {file = "triton-2.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:218d742e67480d9581bafb73ed598416cc8a56f6316152e5562ee65e33de01c0"}, - {file = "triton-2.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:381ec6b3dac06922d3e4099cfc943ef032893b25415de295e82b1a82b0359d2c"}, - {file = "triton-2.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:038e06a09c06a164fef9c48de3af1e13a63dc1ba3c792871e61a8e79720ea440"}, - {file = "triton-2.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6d8f636e0341ac348899a47a057c3daea99ea7db31528a225a3ba4ded28ccc65"}, + {file = "triton-2.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3c84595cbe5e546b1b290d2a58b1494df5a2ef066dd890655e5b8a8a92205c33"}, + {file = "triton-2.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c9d64ae33bcb3a7a18081e3a746e8cf87ca8623ca13d2c362413ce7a486f893e"}, + {file = "triton-2.3.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:eaf80e8761a9e3498aa92e7bf83a085b31959c61f5e8ac14eedd018df6fccd10"}, + {file = "triton-2.3.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b13bf35a2b659af7159bf78e92798dc62d877aa991de723937329e2d382f1991"}, + {file = "triton-2.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:63381e35ded3304704ea867ffde3b7cfc42c16a55b3062d41e017ef510433d66"}, + {file = "triton-2.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1d968264523c7a07911c8fb51b4e0d1b920204dae71491b1fe7b01b62a31e124"}, ] [package.dependencies] @@ -6535,13 +6486,13 @@ files = [ [[package]] name = "weaviate-client" -version = "4.6.4" +version = "4.6.5" description = "A python native Weaviate client" optional = false python-versions = ">=3.8" files = [ - {file = "weaviate_client-4.6.4-py3-none-any.whl", hash = "sha256:19b76fb923a5f0b6fcb7471ef3cd990d2791ede71731e53429e1066a9dbf2af2"}, - {file = "weaviate_client-4.6.4.tar.gz", hash = "sha256:5378db8a33bf1d48adff3f9efa572d9fb04eaeb36444817cab56f1ba3c595500"}, + {file = "weaviate_client-4.6.5-py3-none-any.whl", hash = "sha256:ed5b1c26c86081b5286e7b292de80e0380c964d34b4bffc842c1eb9dfadf7e15"}, + {file = "weaviate_client-4.6.5.tar.gz", hash = "sha256:3926fd0c350c54b668b824f9085959904562821ebb6fc237b7e253daf4645904"}, ] [package.dependencies] @@ -6877,12 +6828,12 @@ docs = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.link testing = ["big-O", "jaraco.functools", "jaraco.itertools", "jaraco.test", "more-itertools", "pytest (>=6,!=8.1.*)", "pytest-checkdocs (>=2.4)", "pytest-cov", "pytest-enabler (>=2.2)", "pytest-ignore-flaky", "pytest-mypy", "pytest-ruff (>=0.2.1)"] [extras] -all = ["azure-core", "azure-cosmos", "azure-identity", "azure-search-documents", "chromadb", "google-generativeai", "grpcio-status", "ipykernel", "milvus", "pinecone-client", "psycopg", "pyarrow", "pymilvus", "qdrant-client", "redis", "sentence-transformers", "torch", "transformers", "usearch", "weaviate-client"] -azure = ["azure-core", "azure-cosmos", "azure-identity", "azure-search-documents"] +all = ["azure-ai-inference", "azure-core", "azure-cosmos", "azure-identity", "azure-search-documents", "chromadb", "ipykernel", "milvus", "motor", "pinecone-client", "psycopg", "pyarrow", "pymilvus", "qdrant-client", "redis", "sentence-transformers", "transformers", "usearch", "weaviate-client"] +azure = ["azure-ai-inference", "azure-core", "azure-cosmos", "azure-identity", "azure-search-documents"] chromadb = ["chromadb"] -google = ["google-generativeai", "grpcio-status"] -hugging-face = ["sentence-transformers", "torch", "transformers"] +hugging-face = ["sentence-transformers", "transformers"] milvus = ["milvus", "pymilvus"] +mongo = ["motor"] notebooks = ["ipykernel"] pinecone = ["pinecone-client"] postgres = ["psycopg"] @@ -6894,4 +6845,4 @@ weaviate = ["weaviate-client"] [metadata] lock-version = "2.0" python-versions = "^3.10,<3.13" -content-hash = "abbc85df45b3f61d055c1ad24e6860e45b9ffeae63259e25b5c429cf21518474" +content-hash = "dbda04832ee7c4fb83b8a7b67725e39acd6a2049e89b1ced807898903a7b71e5" diff --git a/python/pyproject.toml b/python/pyproject.toml index a0986fe82c62..7adb3ed74399 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "semantic-kernel" -version = "1.1.1" +version = "1.1.2" description = "Semantic Kernel Python SDK" authors = ["Microsoft "] readme = "pip/README.md" @@ -8,52 +8,62 @@ packages = [{include = "semantic_kernel"}] [tool.poetry.dependencies] python = "^3.10,<3.13" + +# main dependencies aiohttp = "^3.8" +pydantic = "^2" +pydantic-settings = "^2" +defusedxml = "^0.7.1" + +# embeddings numpy = [ { version = ">=1.25", python = "<3.12" }, { version = ">=1.26", python = ">=3.12" }, ] -scipy = [ - { version = ">=1.5.0", python = "<3.12" }, - { version = ">=1.12.0", python = ">=3.12" } -] -grpcio = [ - { version = ">=1.50.0", python = "<3.12" }, - { version = ">=1.60.0", python = ">=3.12" } -] + +# openai connector openai = ">=1.0" -regex = ">=2023.6.3,<2025.0.0" + +# openapi and swagger openapi_core = ">=0.18,<0.20" prance = "^23.6.21.0" -pydantic = "^2" -pydantic-settings = "^2.2.1" -motor = "^3.3.2" -defusedxml = "^0.7.1" + +# templating pybars4 = "^0.9.13" jinja2 = "^3.1.3" nest-asyncio = "^1.6.0" -# Optional dependencies -ipykernel = { version = "^6.21.1", optional = true} -google-generativeai = { version = ">=0.1", markers = "python_version >= '3.9'", optional = true} -grpcio-status = { version = "^1.53.0", markers = "python_version >= '3.9'", optional = true} -transformers = { version = "^4.28.1", optional = true} -sentence-transformers = { version = "^2.2.2", optional = true} -torch = { version = "^2.2.0", optional = true} -qdrant-client = { version = '^1.9', optional = true} +### Optional dependencies +# azure +azure-ai-inference = {version = "^1.0.0b1", allow-prereleases = true, optional = true} +azure-search-documents = {version = "11.6.0b4", allow-prereleases = true, optional = true} +azure-core = { version = "^1.28.0", optional = true} +azure-identity = { version = "^1.13.0", optional = true} +azure-cosmos = { version = "^4.7.0", optional = true} +# chroma chromadb = { version = ">=0.4.13,<0.6.0", optional = true} +# hugging face +transformers = { version = "^4.28.1", extras=["torch"], optional = true} +sentence-transformers = { version = "^2.2.2", optional = true} +# mongo +motor = { version = "^3.3.2", optional = true } +# notebooks +ipykernel = { version = "^6.21.1", optional = true} +# milvus pymilvus = { version = ">=2.3,<2.4.4", optional = true} milvus = { version = ">=2.3,<2.3.8", markers = 'sys_platform != "win32"', optional = true} -weaviate-client = { version = ">=3.18,<5.0", optional = true} +# pinecone pinecone-client = { version = ">=3.0.0", optional = true} +# postgres psycopg = { version="^3.1.9", extras=["binary","pool"], optional = true} +# qdrant +qdrant-client = { version = '^1.9', optional = true} +# redis redis = { version = "^4.6.0", optional = true} -azure-search-documents = {version = "11.6.0b4", allow-prereleases = true, optional = true} -azure-core = { version = "^1.28.0", optional = true} -azure-identity = { version = "^1.13.0", optional = true} -azure-cosmos = { version = "^4.7.0", optional = true} +# usearch usearch = { version = "^2.9", optional = true} pyarrow = { version = ">=12.0.1,<17.0.0", optional = true} +weaviate-client = { version = ">=3.18,<5.0", optional = true} # Groups are for development only (installed through Poetry) [tool.poetry.group.dev.dependencies] @@ -72,54 +82,64 @@ types-PyYAML = "^6.0.12.20240311" optional = true [tool.poetry.group.unit-tests.dependencies] -google-generativeai = { version = ">=0.1,<0.4" } +azure-ai-inference = {version = "^1.0.0b1", allow-prereleases = true} azure-search-documents = {version = "11.6.0b4", allow-prereleases = true} azure-core = "^1.28.0" azure-cosmos = "^4.7.0" -transformers = "^4.28.1" +transformers = { version = "^4.28.1", extras=["torch"]} sentence-transformers = "^2.2.2" -torch = "^2.2.0" [tool.poetry.group.tests] optional = true [tool.poetry.group.tests.dependencies] -google-generativeai = { version = ">=0.1,<0.4" } -grpcio-status = "^1.53.0" -transformers = "^4.28.1" -sentence-transformers = "^2.2.2" -torch = "^2.2.0" -qdrant-client = '^1.9' +# azure +azure-ai-inference = {version = "^1.0.0b1", allow-prereleases = true} +azure-search-documents = {version = "11.6.0b4", allow-prereleases = true} +azure-core = "^1.28.0" +azure-identity = "^1.13.0" +azure-cosmos = "^4.7.0" +msgraph-sdk = "^1.2.0" +# chroma chromadb = ">=0.4.13,<0.6.0" +# hugging face +transformers = { version = "^4.28.1", extras=["torch"]} +sentence-transformers = "^2.2.2" +# milvus pymilvus = ">=2.3,<2.4.4" milvus = { version = ">=2.3,<2.3.8", markers = 'sys_platform != "win32"'} -weaviate-client = ">=3.18,<5.0" +# mongodb +motor = "^3.3.2" +# pinecone pinecone-client = ">=3.0.0" +# postgres psycopg = { version="^3.1.9", extras=["binary","pool"]} +# qdrant +qdrant-client = '^1.9' +# redis redis = "^4.6.0" -azure-search-documents = {version = "11.6.0b4", allow-prereleases = true} -azure-core = "^1.28.0" -azure-identity = "^1.13.0" -azure-cosmos = "^4.7.0" +# usearch usearch = "^2.9" pyarrow = ">=12.0.1,<17.0.0" -msgraph-sdk = "^1.2.0" +# weaviate +weaviate-client = ">=3.18,<5.0" # Extras are exposed to pip, this allows a user to easily add the right dependencies to their environment [tool.poetry.extras] -google = ["google-generativeai", "grpcio-status"] -hugging_face = ["transformers", "sentence-transformers", "torch"] -qdrant = ["qdrant-client"] +all = ["transformers", "sentence-transformers", "qdrant-client", "chromadb", "pymilvus", "milvus", "weaviate-client", "pinecone-client", "psycopg", "redis", "azure-ai-inference", "azure-search-documents", "azure-core", "azure-identity", "azure-cosmos", "usearch", "pyarrow", "ipykernel", "motor"] + +azure = ["azure-ai-inference", "azure-search-documents", "azure-core", "azure-identity", "azure-cosmos", "msgraph-sdk"] chromadb = ["chromadb"] +hugging_face = ["transformers", "sentence-transformers"] milvus = ["pymilvus", "milvus"] -weaviate = ["weaviate-client"] +mongo = ["motor"] +notebooks = ["ipykernel"] pinecone = ["pinecone-client"] postgres = ["psycopg"] +qdrant = ["qdrant-client"] redis = ["redis"] -azure = ["azure-search-documents", "azure-core", "azure-identity", "azure-cosmos", "msgraph-sdk"] usearch = ["usearch", "pyarrow"] -notebooks = ["ipykernel"] -all = ["google-generativeai", "grpcio-status", "transformers", "sentence-transformers", "torch", "qdrant-client", "chromadb", "pymilvus", "milvus", "weaviate-client", "pinecone-client", "psycopg", "redis", "azure-search-documents", "azure-core", "azure-identity", "azure-cosmos", "usearch", "pyarrow", "ipykernel"] +weaviate = ["weaviate-client"] [tool.ruff] line-length = 120 diff --git a/python/samples/concepts/auto_function_calling/azure_python_code_interpreter_function_calling.py b/python/samples/concepts/auto_function_calling/azure_python_code_interpreter_function_calling.py index b10965a27850..09ab58f4b633 100644 --- a/python/samples/concepts/auto_function_calling/azure_python_code_interpreter_function_calling.py +++ b/python/samples/concepts/auto_function_calling/azure_python_code_interpreter_function_calling.py @@ -7,7 +7,7 @@ from azure.core.exceptions import ClientAuthenticationError from azure.identity import DefaultAzureCredential -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.azure_chat_prompt_execution_settings import ( AzureChatPromptExecutionSettings, ) @@ -69,8 +69,7 @@ async def auth_callback() -> str: req_settings = AzureChatPromptExecutionSettings(service_id=service_id, tool_choice="auto") -filter = {"excluded_plugins": ["ChatBot"]} -req_settings.function_call_behavior = FunctionCallBehavior.EnableFunctions(auto_invoke=True, filters=filter) +req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(filters={"excluded_plugins": ["ChatBot"]}) arguments = KernelArguments(settings=req_settings) diff --git a/python/samples/concepts/auto_function_calling/chat_gpt_api_function_calling.py b/python/samples/concepts/auto_function_calling/chat_gpt_api_function_calling.py index 3f5dd19b297c..11f872d7a9f7 100644 --- a/python/samples/concepts/auto_function_calling/chat_gpt_api_function_calling.py +++ b/python/samples/concepts/auto_function_calling/chat_gpt_api_function_calling.py @@ -6,7 +6,6 @@ from typing import TYPE_CHECKING from semantic_kernel import Kernel -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAIChatPromptExecutionSettings from semantic_kernel.contents import ChatHistory from semantic_kernel.contents.chat_message_content import ChatMessageContent @@ -32,6 +31,10 @@ you will return a full answer to me as soon as possible. """ +# This concept example shows how to handle both streaming and non-streaming responses +# To toggle the behavior, set the following flag accordingly: +stream = True + kernel = Kernel() # Note: the underlying gpt-35/gpt-4 model version needs to be at least version 0613 to support tools. @@ -39,9 +42,6 @@ plugins_directory = os.path.join(__file__, "../../../../../prompt_template_samples/") # adding plugins to the kernel -# the joke plugin in the FunPlugins is a semantic plugin and has the function calling disabled. -# kernel.import_plugin_from_prompt_directory("chat", plugins_directory, "FunPlugin") -# the math plugin is a core plugin and has the function calling enabled. kernel.add_plugin(MathPlugin(), plugin_name="math") kernel.add_plugin(TimePlugin(), plugin_name="time") @@ -50,11 +50,29 @@ plugin_name="ChatBot", function_name="Chat", ) -# enabling or disabling function calling is done by setting the function_call parameter for the completion. -# when the function_call parameter is set to "auto" the model will decide which function to use, if any. -# if you only want to use a specific function, set the name of that function in this parameter, -# the format for that is 'PluginName-FunctionName', (i.e. 'math-Add'). -# if the model or api version does not support this you will get an error. + +# Enabling or disabling function calling is done by setting the `function_choice_behavior` attribute for the +# prompt execution settings. When the function_call parameter is set to "auto" the model will decide which +# function to use, if any. +# +# There are two ways to define the `function_choice_behavior` parameter: +# 1. Using the type string as `"auto"`, `"required"`, or `"none"`. For example: +# configure `function_choice_behavior="auto"` parameter directly in the execution settings. +# 2. Using the FunctionChoiceBehavior class. For example: +# `function_choice_behavior=FunctionChoiceBehavior.Auto()`. +# Both of these configure the `auto` tool_choice and all of the available plugins/functions +# registered on the kernel. If you want to limit the available plugins/functions, you must +# configure the `filters` dictionary attribute for each type of function choice behavior. +# For example: +# +# from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior + +# function_choice_behavior = FunctionChoiceBehavior.Auto( +# filters={"included_functions": ["time-date", "time-time", "math-Add"]} +# ) +# +# The filters attribute allows you to specify either: `included_functions`, `excluded_functions`, +# `included_plugins`, or `excluded_plugins`. # Note: the number of responses for auto invoking tool calls is limited to 1. # If configured to be greater than one, this value will be overridden to 1. @@ -63,9 +81,7 @@ max_tokens=2000, temperature=0.7, top_p=0.8, - function_call_behavior=FunctionCallBehavior.EnableFunctions( - auto_invoke=True, filters={"included_plugins": ["math", "time"]} - ), + function_choice_behavior="auto", ) history = ChatHistory() @@ -93,7 +109,10 @@ def print_tool_calls(message: ChatMessageContent) -> None: f"tool_call {i} arguments: {function_arguments}" ) formatted_tool_calls.append(formatted_str) - print("Tool calls:\n" + "\n\n".join(formatted_tool_calls)) + if len(formatted_tool_calls) > 0: + print("Tool calls:\n" + "\n\n".join(formatted_tool_calls)) + else: + print("The model used its own knowledge and didn't return any tool calls.") async def handle_streaming( @@ -110,7 +129,7 @@ async def handle_streaming( print("Mosscap:> ", end="") streamed_chunks: list[StreamingChatMessageContent] = [] async for message in response: - if not execution_settings.function_call_behavior.auto_invoke_kernel_functions and isinstance( + if not execution_settings.function_choice_behavior.auto_invoke_kernel_functions and isinstance( message[0], StreamingChatMessageContent ): streamed_chunks.append(message[0]) @@ -119,6 +138,8 @@ async def handle_streaming( if streamed_chunks: streaming_chat_message = reduce(lambda first, second: first + second, streamed_chunks) + if hasattr(streaming_chat_message, "content"): + print(streaming_chat_message.content) print("Auto tool calls is disabled, printing returned tool calls...") print_tool_calls(streaming_chat_message) @@ -141,7 +162,6 @@ async def chat() -> bool: arguments["user_input"] = user_input arguments["chat_history"] = history - stream = True if stream: await handle_streaming(kernel, chat_function, arguments=arguments) else: @@ -151,7 +171,7 @@ async def chat() -> bool: # ChatMessageContent with information about the tool calls, which need to be sent # back to the model to get the final response. function_calls = [item for item in result.value[-1].items if isinstance(item, FunctionCallContent)] - if not execution_settings.function_call_behavior.auto_invoke_kernel_functions and len(function_calls) > 0: + if not execution_settings.function_choice_behavior.auto_invoke_kernel_functions and len(function_calls) > 0: print_tool_calls(result.value[0]) return True diff --git a/python/samples/concepts/auto_function_calling/function_calling_with_required_type.py b/python/samples/concepts/auto_function_calling/function_calling_with_required_type.py new file mode 100644 index 000000000000..cd0821e5e6a3 --- /dev/null +++ b/python/samples/concepts/auto_function_calling/function_calling_with_required_type.py @@ -0,0 +1,195 @@ +# Copyright (c) Microsoft. All rights reserved. + +import asyncio +import os +from functools import reduce +from typing import TYPE_CHECKING + +from semantic_kernel import Kernel +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior +from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAIChatPromptExecutionSettings +from semantic_kernel.contents import ChatHistory +from semantic_kernel.contents.chat_message_content import ChatMessageContent +from semantic_kernel.contents.function_call_content import FunctionCallContent +from semantic_kernel.contents.streaming_chat_message_content import StreamingChatMessageContent +from semantic_kernel.core_plugins import MathPlugin, TimePlugin +from semantic_kernel.functions import KernelArguments + +if TYPE_CHECKING: + from semantic_kernel.functions import KernelFunction + + +# In this sample, we're working with the `FunctionChoiceBehavior.Required` type for auto function calling. +# This type mandates that the model calls a specific function or a set of functions to handle the user input. +# By default, the `maximum_auto_invoke_attempts` is set to 1. This can be adjusted by setting this attribute +# in the `FunctionChoiceBehavior.Required` class. +# +# Note that if the maximum auto invoke attempts exceed the number of functions the model calls, it may repeat calling a +# function and ultimately return a tool call response. For example, if we specify required plugins as `math-Multiply` +# and `math-Add`, and set the maximum auto invoke attempts to 5, and query `What is 3+4*5?`, the model will first call +# the `math-Multiply` function, then the `math-Add` function, satisfying 2 of the 5 max auto invoke attempts. +# The remaining 3 attempts will continue calling `math-Add` because the execution settings are still configured with a +# tool_choice `required` and the supplied tools. The final result will be a tool call response. +# +# This behavior is true for both streaming and non-streaming responses. + +system_message = """ +You are a chat bot. Your name is Mosscap and +you have one goal: figure out what people need. +Your full name, should you need to know it, is +Splendid Speckled Mosscap. You communicate +effectively, but you tend to answer with long +flowery prose. You are also a math wizard, +especially for adding and subtracting. +You also excel at joke telling, where your tone is often sarcastic. +Once you have the answer I am looking for, +you will return a full answer to me as soon as possible. +Start all your answers with the current time. +""" + +# This concept example shows how to handle both streaming and non-streaming responses +# To toggle the behavior, set the following flag accordingly: +stream = True + +kernel = Kernel() + +# Note: the underlying gpt-35/gpt-4 model version needs to be at least version 0613 to support tools. +service_id = "chat" +kernel.add_service(OpenAIChatCompletion(service_id=service_id)) + +plugins_directory = os.path.join(__file__, "../../../../../prompt_template_samples/") +# adding plugins to the kernel +kernel.add_plugin(MathPlugin(), plugin_name="math") +kernel.add_plugin(TimePlugin(), plugin_name="time") + +chat_function = kernel.add_function( + prompt="{{$chat_history}}{{$user_input}}", + plugin_name="ChatBot", + function_name="Chat", +) + +# enabling or disabling function calling is done by setting the function_choice_behavior parameter for the +# prompt execution settings. When the function_call parameter is set to "required" the model will decide which +# function to use, if any. If you only want to use a specific function, configure the filters dict with either: +# 'excluded_plugins', 'included_plugins', 'excluded_functions', or 'included_functions'. For example, the +# format for that is 'PluginName-FunctionName', (i.e. 'math-Add'). +# if the model or api version does not support this you will get an error. + +# Note: by default, the number of responses for auto invoking `required` tool calls is limited to 1. +# The value may be configured to be more than one depending upon your scenario. +execution_settings = OpenAIChatPromptExecutionSettings( + service_id=service_id, + max_tokens=2000, + temperature=0.7, + top_p=0.8, + function_choice_behavior=FunctionChoiceBehavior.Required( + filters={"included_functions": ["time-time", "time-date"]}, + ), +) + +history = ChatHistory() + +history.add_system_message(system_message) +history.add_user_message("Hi there, who are you?") +history.add_assistant_message("I am Mosscap, a chat bot. I'm trying to figure out what people need.") + +arguments = KernelArguments(settings=execution_settings) + + +def print_tool_calls(message: ChatMessageContent) -> None: + # A helper method to pretty print the tool calls from the message. + # This is only triggered if auto invoke tool calls is disabled. + items = message.items + formatted_tool_calls = [] + for i, item in enumerate(items, start=1): + if isinstance(item, FunctionCallContent): + tool_call_id = item.id + function_name = item.name + function_arguments = item.arguments + formatted_str = ( + f"tool_call {i} id: {tool_call_id}\n" + f"tool_call {i} function name: {function_name}\n" + f"tool_call {i} arguments: {function_arguments}" + ) + formatted_tool_calls.append(formatted_str) + if len(formatted_tool_calls) > 0: + print("Tool calls:\n" + "\n\n".join(formatted_tool_calls)) + else: + print("The model used its own knowledge and didn't return any tool calls.") + + +async def handle_streaming( + kernel: Kernel, + chat_function: "KernelFunction", + arguments: KernelArguments, +) -> None: + response = kernel.invoke_stream( + chat_function, + return_function_results=False, + arguments=arguments, + ) + + print("Mosscap:> ", end="") + streamed_chunks: list[StreamingChatMessageContent] = [] + async for message in response: + if isinstance(message[0], StreamingChatMessageContent): + streamed_chunks.append(message[0]) + else: + print(str(message[0]), end="") + + if streamed_chunks: + streaming_chat_message = reduce(lambda first, second: first + second, streamed_chunks) + if hasattr(streaming_chat_message, "content"): + print(streaming_chat_message.content) + print("Printing returned tool calls...") + print_tool_calls(streaming_chat_message) + + print("\n") + + +async def chat() -> bool: + try: + user_input = input("User:> ") + except KeyboardInterrupt: + print("\n\nExiting chat...") + return False + except EOFError: + print("\n\nExiting chat...") + return False + + if user_input == "exit": + print("\n\nExiting chat...") + return False + arguments["user_input"] = user_input + arguments["chat_history"] = history + + if stream: + await handle_streaming(kernel, chat_function, arguments=arguments) + else: + result = await kernel.invoke(chat_function, arguments=arguments) + + # If tools are used, and auto invoke tool calls is False, the response will be of type + # ChatMessageContent with information about the tool calls, which need to be sent + # back to the model to get the final response. + function_calls = [item for item in result.value[-1].items if isinstance(item, FunctionCallContent)] + if not execution_settings.function_choice_behavior.auto_invoke_kernel_functions and len(function_calls) > 0: + print_tool_calls(result.value[0]) + return True + + print(f"Mosscap:> {result}") + return True + + +async def main() -> None: + chatting = True + print( + "Welcome to the chat bot!\ + \n Type 'exit' to exit.\ + \n Try a question to see the function calling in action (i.e. what is the current time?)." + ) + while chatting: + chatting = await chat() + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/python/samples/concepts/auto_function_calling/functions_defined_in_json_prompt.py b/python/samples/concepts/auto_function_calling/functions_defined_in_json_prompt.py new file mode 100644 index 000000000000..73157390b2f6 --- /dev/null +++ b/python/samples/concepts/auto_function_calling/functions_defined_in_json_prompt.py @@ -0,0 +1,218 @@ +# Copyright (c) Microsoft. All rights reserved. + +import asyncio +import os +from functools import reduce +from typing import TYPE_CHECKING + +from semantic_kernel import Kernel +from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAIChatPromptExecutionSettings +from semantic_kernel.contents import ChatHistory +from semantic_kernel.contents.chat_message_content import ChatMessageContent +from semantic_kernel.contents.function_call_content import FunctionCallContent +from semantic_kernel.contents.streaming_chat_message_content import StreamingChatMessageContent +from semantic_kernel.core_plugins import MathPlugin, TimePlugin +from semantic_kernel.filters.auto_function_invocation.auto_function_invocation_context import ( + AutoFunctionInvocationContext, +) +from semantic_kernel.filters.filter_types import FilterTypes +from semantic_kernel.functions import KernelArguments + +if TYPE_CHECKING: + from semantic_kernel.functions import KernelFunction + +################################################################## +# Sample Notes: + +# In this sample, we're showing how to configure a JSON config file with `function_choice_behavior` settings. +# The `function_choice_behavior` settings are used to control the auto function calling behavior of the model. +# The related config is located in the `resources` folder under the title `function_choice_json/ChatBot`. + +# The execution settings look like: + +# "execution_settings": { +# "chat": { +# "function_choice_behavior": { +# "type": "auto", +# "maximum_auto_invoke_attempts": 5, +# "functions": [ +# "time.date", +# "time.time", +# "math.Add" +# ] +# } +# } +# } + +# This is another way of configuring the function choice behavior for the model like: +# FunctionChoiceBehavior.Auto(filters={"included_functions": ["time.date", "time.time", "math.Add"]}) + +# The `maximum_auto_invoke_attempts` attribute is used to control the number of times the model will attempt to call a +# function. If wanting to disable auto function calling, set this attribute to 0 or configure the +# `auto_invoke_kernel_functions` attribute to False. +################################################################## + + +system_message = """ +You are a chat bot. Your name is Mosscap and +you have one goal: figure out what people need. +Your full name, should you need to know it, is +Splendid Speckled Mosscap. You communicate +effectively, but you tend to answer with long +flowery prose. You are also a math wizard, +especially for adding and subtracting. +You also excel at joke telling, where your tone is often sarcastic. +Once you have the answer I am looking for, +you will return a full answer to me as soon as possible. +""" + +kernel = Kernel() + +# Note: the underlying gpt-35/gpt-4 model version needs to be at least version 0613 to support tools. +service_id = "chat" +kernel.add_service(OpenAIChatCompletion(service_id=service_id)) + +# adding plugins to the kernel +kernel.add_plugin(MathPlugin(), plugin_name="math") +kernel.add_plugin(TimePlugin(), plugin_name="time") + +chat_function = kernel.add_function( + prompt="{{$chat_history}}{{$user_input}}", + plugin_name="ChatBot", + function_name="Chat", +) + +plugin_path = os.path.join( + os.path.dirname(os.path.dirname(os.path.realpath(__file__))), + "resources", +) +chat_plugin = kernel.add_plugin(plugin_name="function_choice_json", parent_directory=plugin_path) + +history = ChatHistory() + +history.add_system_message(system_message) +history.add_user_message("Hi there, who are you?") +history.add_assistant_message("I am Mosscap, a chat bot. I'm trying to figure out what people need.") + +# To control auto function calling you can do it two ways: +# 1. Configure the attribute `auto_invoke_kernel_functions` as False +# 2. Configure the `maximum_auto_invoke_attempts` as 0. +# These can be done directly on the FunctionChoiceBehavior.Auto/Required/None object or via the JSON/yaml config. +execution_settings: OpenAIChatPromptExecutionSettings = chat_plugin["ChatBot"].prompt_execution_settings[service_id] + +arguments = KernelArguments() + + +# We will hook up a filter to show which function is being called. +# A filter is a piece of custom code that runs at certain points in the process +# this sample has a filter that is called during Auto Function Invocation +# this filter will be called for each function call in the response. +# You can name the function itself with arbitrary names, but the signature needs to be: +# `context, next` +# You are then free to run code before the call to the next filter or the function itself. +# if you want to terminate the function calling sequence. set context.terminate to True +@kernel.filter(FilterTypes.AUTO_FUNCTION_INVOCATION) +async def auto_function_invocation_filter(context: AutoFunctionInvocationContext, next): + """A filter that will be called for each function call in the response.""" + print("\nAuto function invocation filter") + print(f"Function: {context.function.fully_qualified_name}") + + # if we don't call next, it will skip this function, and go to the next one + await next(context) + + +def print_tool_calls(message: ChatMessageContent) -> None: + # A helper method to pretty print the tool calls from the message. + # This is only triggered if auto invoke tool calls is disabled. + items = message.items + formatted_tool_calls = [] + for i, item in enumerate(items, start=1): + if isinstance(item, FunctionCallContent): + tool_call_id = item.id + function_name = item.name + function_arguments = item.arguments + formatted_str = ( + f"tool_call {i} id: {tool_call_id}\n" + f"tool_call {i} function name: {function_name}\n" + f"tool_call {i} arguments: {function_arguments}" + ) + formatted_tool_calls.append(formatted_str) + print("Tool calls:\n" + "\n\n".join(formatted_tool_calls)) + + +async def handle_streaming( + kernel: Kernel, + chat_function: "KernelFunction", + arguments: KernelArguments, +) -> None: + response = kernel.invoke_stream( + chat_function, + return_function_results=False, + arguments=arguments, + ) + + print("Mosscap:> ", end="") + streamed_chunks: list[StreamingChatMessageContent] = [] + async for message in response: + if not execution_settings.function_choice_behavior.auto_invoke_kernel_functions and isinstance( + message[0], StreamingChatMessageContent + ): + streamed_chunks.append(message[0]) + else: + print(str(message[0]), end="") + + if streamed_chunks: + streaming_chat_message = reduce(lambda first, second: first + second, streamed_chunks) + print("Auto tool calls is disabled, printing returned tool calls...") + print_tool_calls(streaming_chat_message) + + print("\n") + + +async def chat() -> bool: + try: + user_input = input("User:> ") + except KeyboardInterrupt: + print("\n\nExiting chat...") + return False + except EOFError: + print("\n\nExiting chat...") + return False + + if user_input == "exit": + print("\n\nExiting chat...") + return False + arguments["user_input"] = user_input + arguments["chat_history"] = history + + stream = False + if stream: + await handle_streaming(kernel, chat_function, arguments=arguments) + else: + result = await kernel.invoke(chat_plugin["ChatBot"], arguments=arguments) + + # If tools are used, and auto invoke tool calls is False, the response will be of type + # ChatMessageContent with information about the tool calls, which need to be sent + # back to the model to get the final response. + function_calls = [item for item in result.value[-1].items if isinstance(item, FunctionCallContent)] + if not execution_settings.function_choice_behavior.auto_invoke_kernel_functions and len(function_calls) > 0: + print_tool_calls(result.value[0]) + return True + + print(f"Mosscap:> {result}") + return True + + +async def main() -> None: + chatting = True + print( + "Welcome to the chat bot!\ + \n Type 'exit' to exit.\ + \n Try a math question to see the function calling in action (i.e. what is 3+3?)." + ) + while chatting: + chatting = await chat() + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/python/samples/concepts/auto_function_calling/functions_defined_in_yaml_prompt.py b/python/samples/concepts/auto_function_calling/functions_defined_in_yaml_prompt.py new file mode 100644 index 000000000000..b10718089a96 --- /dev/null +++ b/python/samples/concepts/auto_function_calling/functions_defined_in_yaml_prompt.py @@ -0,0 +1,216 @@ +# Copyright (c) Microsoft. All rights reserved. + + +import asyncio +import os +from functools import reduce +from typing import TYPE_CHECKING + +from semantic_kernel import Kernel +from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAIChatPromptExecutionSettings +from semantic_kernel.contents import ChatHistory +from semantic_kernel.contents.chat_message_content import ChatMessageContent +from semantic_kernel.contents.function_call_content import FunctionCallContent +from semantic_kernel.contents.streaming_chat_message_content import StreamingChatMessageContent +from semantic_kernel.core_plugins import MathPlugin, TimePlugin +from semantic_kernel.filters.auto_function_invocation.auto_function_invocation_context import ( + AutoFunctionInvocationContext, +) +from semantic_kernel.filters.filter_types import FilterTypes +from semantic_kernel.functions import KernelArguments + +if TYPE_CHECKING: + from semantic_kernel.functions import KernelFunction + +################################################################## +# Sample Notes: + +# In this sample, we're showing how to configure a yaml config file with `function_choice_behavior` settings. +# The `function_choice_behavior` settings are used to control the auto function calling behavior of the model. +# The related config is located in the `resources` folder under the title `function_choice_yaml/ChatBot`. + +# The execution settings look like: + +# execution_settings: +# chat: +# function_choice_behavior: +# type: auto +# maximum_auto_invoke_attempts: 5 +# functions: +# - time.date +# - time.time +# - math.Add + +# This is another way of configuring the function choice behavior for the model like: +# FunctionChoiceBehavior.Auto(filters={"included_functions": ["time.date", "time.time", "math.Add"]}) + +# The `maximum_auto_invoke_attempts` attribute is used to control the number of times the model will attempt to call a +# function. If wanting to disable auto function calling, set this attribute to 0 or configure the +# `auto_invoke_kernel_functions` attribute to False. +################################################################## + +system_message = """ +You are a chat bot. Your name is Mosscap and +you have one goal: figure out what people need. +Your full name, should you need to know it, is +Splendid Speckled Mosscap. You communicate +effectively, but you tend to answer with long +flowery prose. You are also a math wizard, +especially for adding and subtracting. +You also excel at joke telling, where your tone is often sarcastic. +Once you have the answer I am looking for, +you will return a full answer to me as soon as possible. +""" + +kernel = Kernel() + +# Note: the underlying gpt-35/gpt-4 model version needs to be at least version 0613 to support tools. +service_id = "chat" +kernel.add_service(OpenAIChatCompletion(service_id=service_id)) + +# adding plugins to the kernel +kernel.add_plugin(MathPlugin(), plugin_name="math") +kernel.add_plugin(TimePlugin(), plugin_name="time") + +chat_function = kernel.add_function( + prompt="{{$chat_history}}{{$user_input}}", + plugin_name="ChatBot", + function_name="Chat", +) + +plugin_path = os.path.join( + os.path.dirname(os.path.dirname(os.path.realpath(__file__))), + "resources", +) +chat_plugin = kernel.add_plugin(plugin_name="function_choice_yaml", parent_directory=plugin_path) + +history = ChatHistory() + +history.add_system_message(system_message) +history.add_user_message("Hi there, who are you?") +history.add_assistant_message("I am Mosscap, a chat bot. I'm trying to figure out what people need.") + + +# To control auto function calling you can do it two ways: +# 1. Configure the attribute `auto_invoke_kernel_functions` as False +# 2. Configure the `maximum_auto_invoke_attempts` as 0. +# These can be done directly on the FunctionChoiceBehavior.Auto/Required/None object or via the JSON/yaml config. +execution_settings: OpenAIChatPromptExecutionSettings = chat_plugin["ChatBot"].prompt_execution_settings[service_id] + +arguments = KernelArguments() + + +# We will hook up a filter to show which function is being called. +# A filter is a piece of custom code that runs at certain points in the process +# this sample has a filter that is called during Auto Function Invocation +# this filter will be called for each function call in the response. +# You can name the function itself with arbitrary names, but the signature needs to be: +# `context, next` +# You are then free to run code before the call to the next filter or the function itself. +# if you want to terminate the function calling sequence. set context.terminate to True +@kernel.filter(FilterTypes.AUTO_FUNCTION_INVOCATION) +async def auto_function_invocation_filter(context: AutoFunctionInvocationContext, next): + """A filter that will be called for each function call in the response.""" + print("\nAuto function invocation filter") + print(f"Function: {context.function.fully_qualified_name}") + + # if we don't call next, it will skip this function, and go to the next one + await next(context) + + +def print_tool_calls(message: ChatMessageContent) -> None: + # A helper method to pretty print the tool calls from the message. + # This is only triggered if auto invoke tool calls is disabled. + items = message.items + formatted_tool_calls = [] + for i, item in enumerate(items, start=1): + if isinstance(item, FunctionCallContent): + tool_call_id = item.id + function_name = item.name + function_arguments = item.arguments + formatted_str = ( + f"tool_call {i} id: {tool_call_id}\n" + f"tool_call {i} function name: {function_name}\n" + f"tool_call {i} arguments: {function_arguments}" + ) + formatted_tool_calls.append(formatted_str) + print("Tool calls:\n" + "\n\n".join(formatted_tool_calls)) + + +async def handle_streaming( + kernel: Kernel, + chat_function: "KernelFunction", + arguments: KernelArguments, +) -> None: + response = kernel.invoke_stream( + chat_function, + return_function_results=False, + arguments=arguments, + ) + + print("Mosscap:> ", end="") + streamed_chunks: list[StreamingChatMessageContent] = [] + async for message in response: + if not execution_settings.function_choice_behavior.auto_invoke_kernel_functions and isinstance( + message[0], StreamingChatMessageContent + ): + streamed_chunks.append(message[0]) + else: + print(str(message[0]), end="") + + if streamed_chunks: + streaming_chat_message = reduce(lambda first, second: first + second, streamed_chunks) + print("Auto tool calls is disabled, printing returned tool calls...") + print_tool_calls(streaming_chat_message) + + print("\n") + + +async def chat() -> bool: + try: + user_input = input("User:> ") + except KeyboardInterrupt: + print("\n\nExiting chat...") + return False + except EOFError: + print("\n\nExiting chat...") + return False + + if user_input == "exit": + print("\n\nExiting chat...") + return False + arguments["user_input"] = user_input + arguments["chat_history"] = history + + stream = False + if stream: + pass + # await handle_streaming(kernel, chat_function, arguments=arguments) + else: + result = await kernel.invoke(chat_plugin["ChatBot"], arguments=arguments) + + # If tools are used, and auto invoke tool calls is False, the response will be of type + # ChatMessageContent with information about the tool calls, which need to be sent + # back to the model to get the final response. + function_calls = [item for item in result.value[-1].items if isinstance(item, FunctionCallContent)] + if len(function_calls) > 0: + print_tool_calls(result.value[0]) + return True + + print(f"Mosscap:> {result}") + return True + + +async def main() -> None: + chatting = True + print( + "Welcome to the chat bot!\ + \n Type 'exit' to exit.\ + \n Try a math question to see the function calling in action (i.e. what is 3+3?)." + ) + while chatting: + chatting = await chat() + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/python/samples/concepts/auto_function_calling/nexus_raven.py b/python/samples/concepts/auto_function_calling/nexus_raven.py new file mode 100644 index 000000000000..0d270d067c7b --- /dev/null +++ b/python/samples/concepts/auto_function_calling/nexus_raven.py @@ -0,0 +1,409 @@ +# Copyright (c) Microsoft. All rights reserved. + +import ast +import asyncio +import json +import math +from collections.abc import AsyncGenerator +from html import escape +from typing import Annotated, Any, Literal + +from huggingface_hub import AsyncInferenceClient +from pydantic import HttpUrl + +from semantic_kernel import Kernel +from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase +from semantic_kernel.connectors.ai.open_ai import ( + OpenAIChatCompletion, +) +from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( + OpenAIChatPromptExecutionSettings, +) +from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings +from semantic_kernel.connectors.ai.text_completion_client_base import TextCompletionClientBase +from semantic_kernel.contents import ( + ChatHistory, + ChatMessageContent, + FunctionCallContent, + StreamingChatMessageContent, + TextContent, +) +from semantic_kernel.contents.function_result_content import FunctionResultContent +from semantic_kernel.filters.auto_function_invocation.auto_function_invocation_context import ( + AutoFunctionInvocationContext, +) +from semantic_kernel.filters.filter_types import FilterTypes +from semantic_kernel.functions import KernelArguments, kernel_function + +kernel = Kernel() + + +@kernel.filter(FilterTypes.AUTO_FUNCTION_INVOCATION) +async def auto_function_invocation_filter(context: AutoFunctionInvocationContext, next): + """A filter that will be called for each function call in the response.""" + print("\033[92m\n Function called by Nexus Raven model\033[0m") + print(f" \033[96mFunction: {context.function.fully_qualified_name}") + print(f" Arguments: {context.arguments}") + await next(context) + print(f" Result: {context.function_result}\n\033[0m") + + +######################################################################### +# Step 0: Define a custom AI Service, with Prompt Execution settings. ### +# This uses huggingface_hub package, so install that if needed. ### +######################################################################### + + +class NexusRavenPromptExecutionSettings(PromptExecutionSettings): + do_sample: bool = True + max_new_tokens: int | None = None + stop_sequences: Any = None + temperature: float | None = None + top_p: float | None = None + + def prepare_settings_dict(self, **kwargs) -> dict[str, Any]: + """Prepare the settings dictionary.""" + return self.model_dump( + include={"max_new_tokens", "temperature", "top_p", "do_sample", "stop_sequences"}, + exclude_unset=False, + exclude_none=True, + by_alias=True, + ) + + +class NexusRavenCompletion(TextCompletionClientBase, ChatCompletionClientBase): + """To use this class, you should have installed the ``huggingface_hub`` package, and + the environment variable ``HUGGINGFACEHUB_API_TOKEN`` set with your API token, + or given as a named parameter to the constructor.""" + + client: AsyncInferenceClient + + def __init__( + self, + service_id: str, + ai_model_id: str, + endpoint_url: HttpUrl, + api_token: str | None = None, + client: AsyncInferenceClient | None = None, + ): + if not client: + client = AsyncInferenceClient(model=endpoint_url, token=api_token) + super().__init__(service_id=service_id, ai_model_id=ai_model_id, client=client) + + async def get_chat_message_contents( + self, + chat_history: "ChatHistory", + settings: "PromptExecutionSettings", + **kwargs: Any, + ) -> list["ChatMessageContent"]: + """Creates a chat message content with a function call content within. + + Uses the text content and and parses it into a function call content.""" + result = await self.get_text_contents( + prompt=chat_history.messages[-1].content, + settings=settings, + ) + messages = [] + for part in result[0].text.split(";"): + try: + function_call, function_result = await self._execute_function_calls(part, chat_history, **kwargs) + if function_call: + messages.extend( + [ + ChatMessageContent( + role="assistant", items=[function_call], metadata={"ai_model_id": self.ai_model_id} + ), + ChatMessageContent( + role="tool", + items=[function_result], + name="nexus", + metadata={"ai_model_id": self.ai_model_id}, + ), + ] + ) + else: + messages.append(ChatMessageContent(role="assistant", content=part, ai_model_id=self.ai_model_id)) + except Exception as e: + messages.append( + ChatMessageContent( + role="assistant", + items=[ + TextContent( + text=f"An error occurred while executing the function call: {e}", + ai_model_id=self.ai_model_id, + ) + ], + ) + ) + return messages + return messages + + async def _execute_function_calls( + self, result: str, chat_history: ChatHistory, **kwargs: Any + ) -> tuple[FunctionCallContent, FunctionResultContent] | None: + function_call = result.strip().split("\n")[0].strip("Call:").strip() + if not function_call: + return None + parsed_fc = ast.parse(function_call, mode="eval") + if not isinstance(parsed_fc.body, ast.Call): + return None + idx = 0 + call_stack = {} + queue = [] + current = parsed_fc.body + queue.append((idx, parsed_fc.body)) + idx += 1 + while queue: + current_idx, current = queue.pop(0) + dependent_on = [] + args = {} + for keyword in current.keywords: + if isinstance(keyword.value, ast.Call): + queue.append((idx, keyword.value)) + dependent_on.append(idx) + args[keyword.arg] = (idx, keyword.value) + idx += 1 + else: + args[keyword.arg] = keyword.value.value + call = { + "idx": current_idx, + "func": current.func.id.replace("_", "-", 1), + "args": args, + "dependent_on": dependent_on, + "fcc": None, + "result": None, + } + call_stack[current_idx] = call + while any(call["result"] is None for call in call_stack.values()): + await asyncio.gather( + *[ + self._execute_function_call(call, chat_history, kwargs.get("kernel")) + for call in call_stack.values() + if not any(isinstance(arg, tuple) for arg in call["args"].values()) and call["result"] is None + ] + ) + for call in call_stack.values(): + if call["result"] is None: + for name, arg in call["args"].items(): + if isinstance(arg, tuple) and call_stack[arg[0]]["result"] is not None: + function_result: FunctionResultContent = call_stack[arg[0]]["result"] + call["args"][name] = function_result.result + return call_stack[0]["fcc"], call_stack[0]["result"] + + async def _execute_function_call( + self, call_def: dict[str, Any], chat_history: ChatHistory, kernel: Kernel + ) -> FunctionResultContent: + """Execute a function call.""" + call_def["fcc"] = FunctionCallContent( + name=call_def["func"], arguments=json.dumps(call_def["args"]), id=str(call_def["idx"]) + ) + result = await kernel.invoke_function_call(call_def["fcc"], chat_history) + if not result: + call_def["result"] = chat_history.messages[-1].items[0] + else: + call_def["result"] = result.function_result + + async def get_text_contents(self, prompt: str, settings: NexusRavenPromptExecutionSettings) -> list[TextContent]: + result = await self.client.text_generation(prompt, **settings.prepare_settings_dict(), stream=False) + return [TextContent(text=result.strip(), ai_model_id=self.ai_model_id)] + + async def get_streaming_text_contents(self, prompt: str, settings: NexusRavenPromptExecutionSettings): + raise NotImplementedError("Streaming text contents not implemented.") + + def get_streaming_chat_message_contents( + self, + chat_history: "ChatHistory", + settings: "PromptExecutionSettings", + **kwargs: Any, + ) -> AsyncGenerator[list["StreamingChatMessageContent"], Any]: + raise NotImplementedError("Streaming chat message contents not implemented.") + + def get_prompt_execution_settings_class(self) -> type[PromptExecutionSettings]: + return NexusRavenPromptExecutionSettings + + +########################################################## +# Step 1: Define the functions you want to articulate. ### +########################################################## + + +class MathPlugin: + @kernel_function + def cylinder_volume( + self, + radius: Annotated[float, "The radius of the base of the cylinder."], + height: Annotated[float, "The height of the cylinder."], + ): + """Calculate the volume of a cylinder.""" + if radius < 0 or height < 0: + raise ValueError("Radius and height must be non-negative.") + + return math.pi * (radius**2) * height + + @kernel_function + def add( + self, + input: Annotated[float, "the first number to add"], + amount: Annotated[float, "the second number to add"], + ) -> Annotated[float, "the output is a number"]: + """Returns the Addition result of the values provided.""" + return MathPlugin.calculator(input, amount, "add") + + @kernel_function + def subtract( + self, + input: Annotated[float, "the first number"], + amount: Annotated[float, "the number to subtract"], + ) -> float: + """Returns the difference of numbers provided.""" + return MathPlugin.calculator(input, amount, "subtract") + + @kernel_function + def multiply( + self, + input: Annotated[float, "the first number"], + amount: Annotated[float, "the number to multiply with"], + ) -> float: + """Returns the product of numbers provided.""" + return MathPlugin.calculator(input, amount, "multiply") + + @kernel_function + def divide( + self, + input: Annotated[float, "the first number"], + amount: Annotated[float, "the number to divide by"], + ) -> float: + """Returns the quotient of numbers provided.""" + return MathPlugin.calculator(input, amount, "divide") + + @staticmethod + def calculator( + input_a: float, + input_b: float, + operation: Literal["add", "subtract", "multiply", "divide"], + ): + """Computes a calculation.""" + match operation: + case "add": + return input_a + input_b + case "subtract": + return input_a - input_b + case "multiply": + return input_a * input_b + case "divide": + return input_a / input_b + + +############################################################# +# Step 2: Let's define some utils for building the prompt ### +############################################################# + + +@kernel_function +def format_functions_for_prompt(): + filters = {"excluded_plugins": ["kernel"]} + functions = kernel.get_list_of_function_metadata(filters) + formatted_functions = [] + for func in functions: + args_strings = [] + for arg in func.parameters: + arg_string = f"{arg.name}: {arg.type_}" + if arg.default_value: + arg_string += f" = {arg.default_value}" + args_strings.append(arg_string) + func_string = f"{func.fully_qualified_name.replace('-', '_')}({', '.join(args_strings)})" + formatted_functions.append( + escape( + f"OPTION:\n{func_string}\n\n{func.description}\n" + ) + ) + return formatted_functions + + +######################################################################### +# Step 3: Let's define the two prompts, one for Nexus, one for OpenAI ### +# and add everything to the kernel! ### +######################################################################### + +kernel.add_function( + "kernel", + function_name="function_call", + prompt="""{{chat_history}}<human>: +{{kernel-format_functions_for_prompt}} +\n\nUser Query: Question: {{user_query}} +Please pick a function from the above options that best answers the user query and fill in the appropriate arguments.<human_end>""", # noqa: E501 + template_format="handlebars", + prompt_execution_settings=NexusRavenPromptExecutionSettings( + service_id="nexus", + temperature=0.001, + max_new_tokens=500, + do_sample=False, + stop_sequences=["\nReflection:", "\nThought:"], + ), +) +kernel.add_function( + "kernel", + function_name="chat", + prompt="""You are a chatbot that gets fed questions and answers, you write out the response to the question based on the answer, but you do not supply underlying math formulas nor do you try to do math yourself, just a nice sentence that repeats the question and gives the answer. {{chat_history}}""", # noqa: E501 + template_format="handlebars", + prompt_execution_settings=OpenAIChatPromptExecutionSettings( + service_id="openai", + temperature=0.0, + max_tokens=1000, + ), +) +kernel.add_plugin(MathPlugin(), "math") +kernel.add_function("kernel", format_functions_for_prompt) +kernel.add_service( + NexusRavenCompletion(service_id="nexus", ai_model_id="raven", endpoint_url="http://nexusraven.nexusflow.ai") +) +kernel.add_service(OpenAIChatCompletion(service_id="openai")) + +############################################ +# Step 4: The main function and a runner ### +############################################ + + +async def run_question(user_input: str, chat_history: ChatHistory): + arguments = KernelArguments( + user_query=user_input, + chat_history=chat_history, + kernel=kernel, + ) + result = await kernel.invoke(plugin_name="kernel", function_name="function_call", arguments=arguments) + chat_history.add_user_message(user_input) + for msg in result.value: + chat_history.add_message(msg) + final_result = await kernel.invoke(plugin_name="kernel", function_name="chat", arguments=arguments) + chat_history.add_message(final_result.value[0]) + + +async def main(): + chat_history = ChatHistory() + user_input_example = ( + "my cake is 3 centimers high and 20 centimers in radius, can you subtract 200 from that number?" + ) + print("Welcome to the chatbot!") + print( + "This chatbot uses local function calling with Nexus Raven, and OpenAI for the final answer, " + "it has some math skills so feel free to ask anything about that." + ) + print(f'For example: "{user_input_example}".') + print("You can type 'exit' to quit the chatbot.") + while True: + try: + user_input = input("What is your question: ") + except Exception: + break + if user_input == "exit": + break + if not user_input: + user_input = user_input_example + + await run_question(user_input, chat_history) + print(chat_history.messages[-1].content) + print("Thanks for chatting with me!") + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/python/samples/concepts/chat_completion/azure_chat_gpt_api.py b/python/samples/concepts/chat_completion/azure_chat_gpt_api.py index 25affdc8473b..aea05f9cf31e 100644 --- a/python/samples/concepts/chat_completion/azure_chat_gpt_api.py +++ b/python/samples/concepts/chat_completion/azure_chat_gpt_api.py @@ -4,7 +4,7 @@ import logging from semantic_kernel import Kernel -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion from semantic_kernel.contents import ChatHistory @@ -44,9 +44,7 @@ req_settings.max_tokens = 2000 req_settings.temperature = 0.7 req_settings.top_p = 0.8 -req_settings.function_call_behavior = FunctionCallBehavior.EnableFunctions( - auto_invoke=True, filters={"excluded_plugins": []} -) +req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(filters={"excluded_plugins": []}) ## The third method is the most specific as the returned request settings class is the one that is registered for the service and has some fields already filled in, like the service_id and ai_model_id. # noqa: E501 E266 diff --git a/python/samples/concepts/chat_completion/azure_chat_image_input.py b/python/samples/concepts/chat_completion/azure_chat_image_input.py index e274473014a7..5a813ee13eba 100644 --- a/python/samples/concepts/chat_completion/azure_chat_image_input.py +++ b/python/samples/concepts/chat_completion/azure_chat_image_input.py @@ -4,7 +4,7 @@ import logging from semantic_kernel import Kernel -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion from semantic_kernel.contents import ChatHistory, ChatMessageContent, ImageContent, TextContent @@ -25,9 +25,7 @@ req_settings.max_tokens = 2000 req_settings.temperature = 0.7 req_settings.top_p = 0.8 -req_settings.function_call_behavior = FunctionCallBehavior.EnableFunctions( - auto_invoke=True, filters={"excluded_plugins": []} -) +req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(filters={"excluded_plugins": []}) chat_function = kernel.add_function( prompt=system_message + """{{$chat_history}}""", diff --git a/python/samples/concepts/filtering/auto_function_invoke_filters.py b/python/samples/concepts/filtering/auto_function_invoke_filters.py index 6c41c1aaa9d2..f25b9a305af9 100644 --- a/python/samples/concepts/filtering/auto_function_invoke_filters.py +++ b/python/samples/concepts/filtering/auto_function_invoke_filters.py @@ -4,7 +4,7 @@ import os from semantic_kernel import Kernel -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAIChatPromptExecutionSettings from semantic_kernel.contents import ChatHistory from semantic_kernel.contents.chat_message_content import ChatMessageContent @@ -61,9 +61,7 @@ max_tokens=2000, temperature=0.7, top_p=0.8, - function_call_behavior=FunctionCallBehavior.EnableFunctions( - auto_invoke=True, filters={"included_plugins": ["math", "time"]} - ), + function_choice_behavior=FunctionChoiceBehavior.Auto(filters={"included_plugins": ["math", "time"]}), ) history = ChatHistory() diff --git a/python/samples/concepts/memory/google_palm_chat_with_memory.py b/python/samples/concepts/memory/google_palm_chat_with_memory.py deleted file mode 100644 index f3307f2511dd..000000000000 --- a/python/samples/concepts/memory/google_palm_chat_with_memory.py +++ /dev/null @@ -1,113 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import asyncio - -import semantic_kernel.connectors.ai.google_palm as sk_gp -from semantic_kernel import Kernel -from semantic_kernel.core_plugins import TextMemoryPlugin -from semantic_kernel.functions import KernelFunction -from semantic_kernel.memory import SemanticTextMemory, VolatileMemoryStore -from semantic_kernel.prompt_template import PromptTemplateConfig - -collection_id = "generic" - - -async def populate_memory(memory: SemanticTextMemory) -> None: - # Add some documents to the semantic memory - await memory.save_information(collection=collection_id, id="info1", text="Your budget for 2024 is $100,000") - await memory.save_information(collection=collection_id, id="info2", text="Your savings from 2023 are $50,000") - await memory.save_information(collection=collection_id, id="info3", text="Your investments are $80,000") - - -async def search_memory_examples(memory: SemanticTextMemory) -> None: - questions = ["What is my budget for 2024?", "What are my savings from 2023?", "What are my investments?"] - - for question in questions: - print(f"Question: {question}") - result = await memory.search(collection_id, question) - print(f"Answer: {result[0].text}\n") - - -async def setup_chat_with_memory( - kernel: Kernel, - service_id: str, -) -> KernelFunction: - prompt = """ - ChatBot can have a conversation with you about any topic. - It can give explicit instructions or say 'I don't know' if - it does not have an answer. - - Information about me, from previous conversations: - - {{recall 'budget by year'}} What is my budget for 2024? - - {{recall 'savings from previous year'}} What are my savings from 2023? - - {{recall 'investments'}} What are my investments? - - {{$request}} - """.strip() - - prompt_template_config = PromptTemplateConfig( - template=prompt, - execution_settings={service_id: kernel.get_prompt_execution_settings_from_service_id(service_id=service_id)}, - ) - - return kernel.add_function( - function_name="chat_with_memory", - plugin_name="TextMemoryPlugin", - prompt_template_config=prompt_template_config, - ) - - -async def chat(kernel: Kernel, chat_func: KernelFunction) -> bool: - try: - user_input = input("User:> ") - except KeyboardInterrupt: - print("\n\nExiting chat...") - return False - except EOFError: - print("\n\nExiting chat...") - return False - - if user_input == "exit": - print("\n\nExiting chat...") - return False - - answer = await kernel.invoke(chat_func, request=user_input) - - print(f"ChatBot:> {answer}") - return True - - -async def main() -> None: - kernel = Kernel() - model_id = "models/embedding-gecko-001" - palm_text_embed = sk_gp.GooglePalmTextEmbedding(model_id) - kernel.add_service(palm_text_embed) - chat_service_id = "models/chat-bison-001" - palm_chat_completion = sk_gp.GooglePalmChatCompletion(chat_service_id) - kernel.add_service(palm_chat_completion) - - memory = SemanticTextMemory(storage=VolatileMemoryStore(), embeddings_generator=palm_text_embed) - kernel.add_plugin(TextMemoryPlugin(memory), "TextMemoryPlugin") - - print("Populating memory...") - await populate_memory(memory) - - print("Asking questions... (manually)") - await search_memory_examples(memory) - - print("Setting up a chat (with memory!)") - chat_func = await setup_chat_with_memory(kernel, chat_service_id) - - print("Begin chatting (type 'exit' to exit):\n") - print( - "Welcome to the chat bot!\ - \n Type 'exit' to exit.\ - \n Try asking a question about your finances (i.e. \"talk to me about my finances\")." - ) - chatting = True - while chatting: - chatting = await chat(kernel, chat_func) - - -if __name__ == "__main__": - asyncio.run(main()) diff --git a/python/samples/concepts/on_your_data/azure_chat_gpt_with_data_api_function_calling.py b/python/samples/concepts/on_your_data/azure_chat_gpt_with_data_api_function_calling.py index 4ac8c0c4cdf4..c6813daa2702 100644 --- a/python/samples/concepts/on_your_data/azure_chat_gpt_with_data_api_function_calling.py +++ b/python/samples/concepts/on_your_data/azure_chat_gpt_with_data_api_function_calling.py @@ -5,7 +5,7 @@ import os import semantic_kernel as sk -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai import ( AzureAISearchDataSource, AzureChatCompletion, @@ -79,9 +79,7 @@ # calling the chat, you could add a overloaded version of the settings here, # to enable or disable function calling or set the function calling to a specific plugin. # see the openai_function_calling example for how to use this with a unrelated function definition -req_settings.function_call_behavior = FunctionCallBehavior.EnableFunctions( - auto_invoke=True, filters={"excluded_plugins": ["ChatBot"]} -) +req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(filters={"excluded_plugins": ["ChatBot"]}) arguments = KernelArguments(settings=req_settings) diff --git a/python/samples/concepts/planners/openai_function_calling_stepwise_planner.py b/python/samples/concepts/planners/openai_function_calling_stepwise_planner.py index d46aaa8b0bde..cce74f39a41d 100644 --- a/python/samples/concepts/planners/openai_function_calling_stepwise_planner.py +++ b/python/samples/concepts/planners/openai_function_calling_stepwise_planner.py @@ -16,13 +16,12 @@ async def main(): kernel.add_service( OpenAIChatCompletion( service_id=service_id, - ai_model_id="gpt-3.5-turbo", ), ) plugin_path = os.path.join( - os.path.dirname(os.path.dirname(os.path.realpath(__file__))), - "resources", + os.path.dirname(os.path.dirname(os.path.realpath(__file__))), + "resources", ) kernel.add_plugin(parent_directory=plugin_path, plugin_name="email_plugin") kernel.add_plugins({"MathPlugin": MathPlugin(), "TimePlugin": TimePlugin()}) diff --git a/python/samples/concepts/plugins/google_palm_chat_with_plugin.py b/python/samples/concepts/plugins/google_palm_chat_with_plugin.py deleted file mode 100644 index 648f384eaf63..000000000000 --- a/python/samples/concepts/plugins/google_palm_chat_with_plugin.py +++ /dev/null @@ -1,90 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import asyncio - -from semantic_kernel import Kernel -from semantic_kernel.connectors.ai.google_palm import GooglePalmChatCompletion -from semantic_kernel.contents import ChatHistory -from semantic_kernel.prompt_template import InputVariable, PromptTemplateConfig - -""" -System messages prime the assistant with different personalities or behaviors. -The system message is added to the prompt template, and a chat history can be -added as well to provide further context. -A system message can only be used once at the start of the conversation, and -conversation history persists with the instance of GooglePalmChatCompletion. To -overwrite the system message and start a new conversation, you must create a new -instance of GooglePalmChatCompletion. -Sometimes, PaLM struggles to use the information in the prompt template. In this -case, it is recommended to experiment with the messages in the prompt template -or ask different questions. -""" - -system_message = """ -You are a chat bot. Your name is Blackbeard -and you speak in the style of a swashbuckling -pirate. You reply with brief, to-the-point answers -with no elaboration. Your full name is Captain -Bartholomew "Blackbeard" Thorne. -""" - -kernel = Kernel() -service_id = "models/chat-bison-001" -palm_chat_completion = GooglePalmChatCompletion(service_id) -kernel.add_service(palm_chat_completion) - -req_settings = kernel.get_prompt_execution_settings_from_service_id(service_id=service_id) -req_settings.max_tokens = 2000 -req_settings.temperature = 0.7 -req_settings.top_p = 0.8 - -prompt_template_config = PromptTemplateConfig( - template="{{$user_input}}", - name="chat", - template_format="semantic-kernel", - input_variables=[ - InputVariable(name="user_input", description="The user input", is_required=True), - InputVariable(name="chat_history", description="The history of the conversation", is_required=True), - ], - execution_settings=req_settings, -) - -chat_func = kernel.add_function( - plugin_name="PiratePlugin", function_name="Chat", prompt_template_config=prompt_template_config -) - -chat_history = ChatHistory() -chat_history.add_system_message(system_message) -chat_history.add_user_message("Hi there, who are you?") -chat_history.add_assistant_message("I am Blackbeard.") - - -async def chat() -> bool: - try: - user_input = input("User:> ") - except KeyboardInterrupt: - print("\n\nExiting chat...") - return False - except EOFError: - print("\n\nExiting chat...") - return False - - if user_input == "exit": - print("\n\nExiting chat...") - return False - - answer = await kernel.invoke(chat_func, user_input=user_input, chat_history=chat_history) - print(f"Blackbeard:> {answer}") - chat_history.add_user_message(user_input) - chat_history.add_assistant_message(str(answer)) - return True - - -async def main() -> None: - chatting = True - while chatting: - chatting = await chat() - - -if __name__ == "__main__": - asyncio.run(main()) diff --git a/python/samples/concepts/plugins/openai_function_calling_with_custom_plugin.py b/python/samples/concepts/plugins/openai_function_calling_with_custom_plugin.py index eb3a6d8510fb..6adde925a390 100644 --- a/python/samples/concepts/plugins/openai_function_calling_with_custom_plugin.py +++ b/python/samples/concepts/plugins/openai_function_calling_with_custom_plugin.py @@ -3,7 +3,7 @@ import asyncio from typing import Annotated -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, OpenAIChatCompletion from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( OpenAIChatPromptExecutionSettings, @@ -63,9 +63,7 @@ async def main(): settings: OpenAIChatPromptExecutionSettings = kernel.get_prompt_execution_settings_from_service_id( service_id=service_id ) - settings.function_call_behavior = FunctionCallBehavior.EnableFunctions( - auto_invoke=True, filters={"included_plugins": ["weather", "time"]} - ) + settings.function_choice_behavior = FunctionChoiceBehavior.Auto(filters={"included_plugins": ["weather", "time"]}) print( await kernel.invoke_prompt( @@ -81,9 +79,7 @@ async def main(): settings: OpenAIChatPromptExecutionSettings = kernel.get_prompt_execution_settings_from_service_id( service_id=service_id ) - settings.function_call_behavior = FunctionCallBehavior.EnableFunctions( - auto_invoke=True, filters={"included_plugins": ["weather", "time"]} - ) + settings.function_choice_behavior = FunctionChoiceBehavior.Auto(filters={"included_plugins": ["weather", "time"]}) result = kernel.invoke_prompt_stream( function_name="prompt_test", @@ -104,7 +100,7 @@ async def main(): settings: OpenAIChatPromptExecutionSettings = kernel.get_prompt_execution_settings_from_service_id( service_id=service_id ) - settings.function_call_behavior = FunctionCallBehavior.EnableFunctions( + settings.function_choice_behavior = FunctionChoiceBehavior.Auto( auto_invoke=False, filters={"included_plugins": ["weather", "time"]} ) chat_history.add_user_message( @@ -131,7 +127,7 @@ async def main(): arguments=KernelArguments(), function_call_count=1, request_index=0, - function_call_behavior=settings.function_call_behavior, + function_call_behavior=settings.function_choice_behavior, ) diff --git a/python/samples/concepts/plugins/openai_plugin_azure_key_vault.py b/python/samples/concepts/plugins/openai_plugin_azure_key_vault.py index 280f9663a361..e0d92e17e2e7 100644 --- a/python/samples/concepts/plugins/openai_plugin_azure_key_vault.py +++ b/python/samples/concepts/plugins/openai_plugin_azure_key_vault.py @@ -10,7 +10,7 @@ from samples.concepts.plugins.azure_key_vault_settings import AzureKeyVaultSettings from semantic_kernel import Kernel -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAIChatPromptExecutionSettings from semantic_kernel.connectors.openai_plugin import OpenAIAuthenticationType, OpenAIFunctionExecutionParameters from semantic_kernel.contents import ChatHistory @@ -185,9 +185,7 @@ async def get_secret_from_key_vault(kernel: Kernel, plugin: KernelPlugin): max_tokens=2000, temperature=0.7, top_p=0.8, - function_call_behavior=FunctionCallBehavior.EnableFunctions( - auto_invoke=True, filters={"included_plugins": ["AzureKeyVaultPlugin"]} - ), + function_choice_behavior=FunctionChoiceBehavior.Auto(filters={"included_plugins": ["AzureKeyVaultPlugin"]}), ) history = ChatHistory() diff --git a/python/samples/concepts/prompt_templates/azure_chat_gpt_api_handlebars.py b/python/samples/concepts/prompt_templates/azure_chat_gpt_api_handlebars.py index eecf00efec3a..6f19ddab6ec0 100644 --- a/python/samples/concepts/prompt_templates/azure_chat_gpt_api_handlebars.py +++ b/python/samples/concepts/prompt_templates/azure_chat_gpt_api_handlebars.py @@ -4,7 +4,7 @@ import logging from semantic_kernel import Kernel -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion from semantic_kernel.contents import ChatHistory from semantic_kernel.functions import KernelArguments @@ -32,7 +32,7 @@ req_settings.max_tokens = 2000 req_settings.temperature = 0.7 req_settings.top_p = 0.8 -req_settings.function_call_behavior = FunctionCallBehavior.AutoInvokeKernelFunctions() +req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto() chat_function = kernel.add_function( diff --git a/python/samples/concepts/prompt_templates/azure_chat_gpt_api_jinja2.py b/python/samples/concepts/prompt_templates/azure_chat_gpt_api_jinja2.py index c7d632bdd107..c039543d51fa 100644 --- a/python/samples/concepts/prompt_templates/azure_chat_gpt_api_jinja2.py +++ b/python/samples/concepts/prompt_templates/azure_chat_gpt_api_jinja2.py @@ -4,7 +4,7 @@ import logging from semantic_kernel import Kernel -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion from semantic_kernel.contents import ChatHistory from semantic_kernel.functions import KernelArguments @@ -32,7 +32,7 @@ req_settings.max_tokens = 2000 req_settings.temperature = 0.7 req_settings.top_p = 0.8 -req_settings.function_call_behavior = FunctionCallBehavior.AutoInvokeKernelFunctions() +req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto() chat_function = kernel.add_function( diff --git a/python/samples/concepts/resources/function_choice_json/ChatBot/config.json b/python/samples/concepts/resources/function_choice_json/ChatBot/config.json new file mode 100644 index 000000000000..e24c68dc2f4f --- /dev/null +++ b/python/samples/concepts/resources/function_choice_json/ChatBot/config.json @@ -0,0 +1,30 @@ +{ + "name": "ChatBot", + "template_format": "semantic-kernel", + "description": "A chat bot that helps the user tell the date and time.", + "input_variables": [ + { + "name": "chat_history", + "description": "The on-going chat history", + "is_required": true + }, + { + "name": "user_input", + "description": "The user input", + "is_required": true + } + ], + "execution_settings": { + "chat": { + "function_choice_behavior": { + "type": "auto", + "maximum_auto_invoke_attempts": 5, + "functions": [ + "time.date", + "time.time", + "math.Add" + ] + } + } + } +} diff --git a/python/samples/concepts/resources/function_choice_json/ChatBot/skprompt.txt b/python/samples/concepts/resources/function_choice_json/ChatBot/skprompt.txt new file mode 100644 index 000000000000..fe62609538a4 --- /dev/null +++ b/python/samples/concepts/resources/function_choice_json/ChatBot/skprompt.txt @@ -0,0 +1 @@ +{{$chat_history}}{{$user_input}} \ No newline at end of file diff --git a/python/samples/concepts/resources/function_choice_yaml/defined_function.yaml b/python/samples/concepts/resources/function_choice_yaml/defined_function.yaml new file mode 100644 index 000000000000..a686d3a297b5 --- /dev/null +++ b/python/samples/concepts/resources/function_choice_yaml/defined_function.yaml @@ -0,0 +1,20 @@ +name: ChatBot +template_format: semantic-kernel +template: "{{$chat_history}}{{$user_input}}" +description: A chat bot that helps the user tell the date and time. +input_variables: + - name: chat_history + description: The on-going chat history + is_required: true + - name: user_input + description: The user input + is_required: true +execution_settings: + chat: + function_choice_behavior: + type: auto + maximum_auto_invoke_attempts: 5 + functions: + - time.date + - time.time + - math.Add diff --git a/python/samples/concepts/text_generation/google_palm_text_completion.py b/python/samples/concepts/text_generation/google_palm_text_completion.py deleted file mode 100644 index 29a7fe6980eb..000000000000 --- a/python/samples/concepts/text_generation/google_palm_text_completion.py +++ /dev/null @@ -1,39 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import asyncio - -from semantic_kernel.connectors.ai.google_palm import GooglePalmTextCompletion, GooglePalmTextPromptExecutionSettings -from semantic_kernel.kernel import Kernel - - -async def text_completion_example_complete(kernel: Kernel, user_mssg, settings): - """Complete a text prompt using the Google PaLM model and print the results.""" - palm_text_completion = GooglePalmTextCompletion("models/text-bison-001") - kernel.add_service(palm_text_completion) - return await palm_text_completion.get_text_contents(user_mssg, settings) - - -async def main() -> None: - kernel = Kernel() - settings = GooglePalmTextPromptExecutionSettings() - - user_mssg1 = ( - "Sam has three boxes, each containing a certain number of coins. " - "The first box has twice as many coins as the second box, and the second " - "box has three times as many coins as the third box. Together, the three " - "boxes have 98 coins in total. How many coins are there in each box? " - "Think about it step by step, and show your work." - ) - response = await text_completion_example_complete(kernel, user_mssg1, settings) - print(f"User:> {user_mssg1}\n\nChatBot:> {response}\n") - # Use temperature to influence the variance of the responses - settings.number_of_responses = 3 - settings.temperature = 1 - user_mssg2 = "I need a concise answer. A common method for traversing a binary tree is" - response = await text_completion_example_complete(kernel, user_mssg2, settings) - print(f"User:> {user_mssg2}\n\nChatBot:> {response}") - return - - -if __name__ == "__main__": - asyncio.run(main()) diff --git a/python/samples/demos/booking_restaurant/restaurant_booking.py b/python/samples/demos/booking_restaurant/restaurant_booking.py index 13c41c2db0fa..0c41f38552c6 100644 --- a/python/samples/demos/booking_restaurant/restaurant_booking.py +++ b/python/samples/demos/booking_restaurant/restaurant_booking.py @@ -60,7 +60,7 @@ settings.max_tokens = 2000 settings.temperature = 0.1 settings.top_p = 0.8 -settings.function_call_behavior.enable_functions(auto_invoke=True, filters={"exclude_plugin": ["ChatBot"]}) +settings.function_choice_behavior.Auto(filters={"exclude_plugin": ["ChatBot"]}) chat_history = ChatHistory( system_message="When responding to the user's request to book a table, include the reservation ID." diff --git a/python/samples/getting_started/00-getting-started.ipynb b/python/samples/getting_started/00-getting-started.ipynb index 94beaee90f85..b11f98fa1fe9 100644 --- a/python/samples/getting_started/00-getting-started.ipynb +++ b/python/samples/getting_started/00-getting-started.ipynb @@ -17,7 +17,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { diff --git a/python/samples/getting_started/01-basic-loading-the-kernel.ipynb b/python/samples/getting_started/01-basic-loading-the-kernel.ipynb index 3f028149613d..09b4a050e644 100644 --- a/python/samples/getting_started/01-basic-loading-the-kernel.ipynb +++ b/python/samples/getting_started/01-basic-loading-the-kernel.ipynb @@ -24,7 +24,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { diff --git a/python/samples/getting_started/02-running-prompts-from-file.ipynb b/python/samples/getting_started/02-running-prompts-from-file.ipynb index 4f87190baab3..bbba139657f6 100644 --- a/python/samples/getting_started/02-running-prompts-from-file.ipynb +++ b/python/samples/getting_started/02-running-prompts-from-file.ipynb @@ -35,7 +35,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { diff --git a/python/samples/getting_started/03-prompt-function-inline.ipynb b/python/samples/getting_started/03-prompt-function-inline.ipynb index da426c8ff975..da8b760adc30 100644 --- a/python/samples/getting_started/03-prompt-function-inline.ipynb +++ b/python/samples/getting_started/03-prompt-function-inline.ipynb @@ -25,7 +25,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { diff --git a/python/samples/getting_started/04-kernel-arguments-chat.ipynb b/python/samples/getting_started/04-kernel-arguments-chat.ipynb index 6ba6249c6d54..8f519dcacf2d 100644 --- a/python/samples/getting_started/04-kernel-arguments-chat.ipynb +++ b/python/samples/getting_started/04-kernel-arguments-chat.ipynb @@ -27,7 +27,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { diff --git a/python/samples/getting_started/05-using-the-planner.ipynb b/python/samples/getting_started/05-using-the-planner.ipynb index 64ac282c4980..14e57f633cf1 100644 --- a/python/samples/getting_started/05-using-the-planner.ipynb +++ b/python/samples/getting_started/05-using-the-planner.ipynb @@ -32,7 +32,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { @@ -583,4 +583,4 @@ }, "nbformat": 4, "nbformat_minor": 5 -} +} \ No newline at end of file diff --git a/python/samples/getting_started/06-memory-and-embeddings.ipynb b/python/samples/getting_started/06-memory-and-embeddings.ipynb index 53a3623f4b21..dcf9dd92d44b 100644 --- a/python/samples/getting_started/06-memory-and-embeddings.ipynb +++ b/python/samples/getting_started/06-memory-and-embeddings.ipynb @@ -37,7 +37,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1\n", + "%pip install semantic-kernel==1.1.2\n", "%pip install azure-core==1.30.1\n", "%pip install azure-search-documents==11.6.0b4" ] diff --git a/python/samples/getting_started/07-hugging-face-for-plugins.ipynb b/python/samples/getting_started/07-hugging-face-for-plugins.ipynb index 241014636847..9b163231cb46 100644 --- a/python/samples/getting_started/07-hugging-face-for-plugins.ipynb +++ b/python/samples/getting_started/07-hugging-face-for-plugins.ipynb @@ -21,7 +21,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel[hugging_face]==1.1.1" + "%pip install semantic-kernel[hugging_face]==1.1.2" ] }, { diff --git a/python/samples/getting_started/08-native-function-inline.ipynb b/python/samples/getting_started/08-native-function-inline.ipynb index 2abc66a065c3..bb98225fe724 100644 --- a/python/samples/getting_started/08-native-function-inline.ipynb +++ b/python/samples/getting_started/08-native-function-inline.ipynb @@ -55,7 +55,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { diff --git a/python/samples/getting_started/09-groundedness-checking.ipynb b/python/samples/getting_started/09-groundedness-checking.ipynb index 33e787ea209a..ad97f7df98e3 100644 --- a/python/samples/getting_started/09-groundedness-checking.ipynb +++ b/python/samples/getting_started/09-groundedness-checking.ipynb @@ -36,7 +36,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { diff --git a/python/samples/getting_started/10-multiple-results-per-prompt.ipynb b/python/samples/getting_started/10-multiple-results-per-prompt.ipynb index 6ded74065777..29ec73b29086 100644 --- a/python/samples/getting_started/10-multiple-results-per-prompt.ipynb +++ b/python/samples/getting_started/10-multiple-results-per-prompt.ipynb @@ -34,7 +34,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { @@ -471,9 +471,8 @@ " current_time = time.time()\n", "\n", " # Update texts with new results\n", - " for idx, result in enumerate(results):\n", - " if idx < number_of_responses:\n", - " texts[idx] += str(result)\n", + " for result in results:\n", + " texts[result.choice_index] += str(result)\n", "\n", " # Clear and display output at intervals\n", " if current_time - last_clear_time > clear_interval:\n", diff --git a/python/samples/getting_started/11-streaming-completions.ipynb b/python/samples/getting_started/11-streaming-completions.ipynb index 22b5815e0308..530cee345e32 100644 --- a/python/samples/getting_started/11-streaming-completions.ipynb +++ b/python/samples/getting_started/11-streaming-completions.ipynb @@ -27,7 +27,7 @@ "outputs": [], "source": [ "# Note: if using a Poetry virtual environment, do not run this cell\n", - "%pip install semantic-kernel==1.1.1" + "%pip install semantic-kernel==1.1.2" ] }, { diff --git a/python/samples/getting_started/third_party/weaviate-persistent-memory.ipynb b/python/samples/getting_started/third_party/weaviate-persistent-memory.ipynb index 9e5d6c0aa57d..fea560392bc4 100644 --- a/python/samples/getting_started/third_party/weaviate-persistent-memory.ipynb +++ b/python/samples/getting_started/third_party/weaviate-persistent-memory.ipynb @@ -156,7 +156,7 @@ "metadata": {}, "outputs": [], "source": [ - "%pip install semantic-kernel[weaviate]==1.1.1" + "%pip install semantic-kernel[weaviate]==1.1.2" ] }, { diff --git a/python/samples/learn_resources/creating_functions.py b/python/samples/learn_resources/creating_functions.py index adc00bd679f5..7fd0ba7632bf 100644 --- a/python/samples/learn_resources/creating_functions.py +++ b/python/samples/learn_resources/creating_functions.py @@ -5,7 +5,7 @@ from samples.sk_service_configurator import add_service from semantic_kernel import Kernel -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai import OpenAIChatPromptExecutionSettings from semantic_kernel.contents import ChatHistory @@ -35,7 +35,7 @@ async def main(): service_id="default", temperature=0.0, max_tokens=1000, - function_call_behavior=FunctionCallBehavior.AutoInvokeKernelFunctions(), + function_choice_behavior=FunctionChoiceBehavior.Auto(), ), plugin_name="Chat", function_name="Chat", diff --git a/python/semantic_kernel/connectors/ai/azure_ai_inference/__init__.py b/python/semantic_kernel/connectors/ai/azure_ai_inference/__init__.py new file mode 100644 index 000000000000..e6ba7c02f6c3 --- /dev/null +++ b/python/semantic_kernel/connectors/ai/azure_ai_inference/__init__.py @@ -0,0 +1,21 @@ +# Copyright (c) Microsoft. All rights reserved. + +from semantic_kernel.connectors.ai.azure_ai_inference.azure_ai_inference_prompt_execution_settings import ( + AzureAIInferenceChatPromptExecutionSettings, + AzureAIInferenceEmbeddingPromptExecutionSettings, +) +from semantic_kernel.connectors.ai.azure_ai_inference.azure_ai_inference_settings import AzureAIInferenceSettings +from semantic_kernel.connectors.ai.azure_ai_inference.services.azure_ai_inference_chat_completion import ( + AzureAIInferenceChatCompletion, +) +from semantic_kernel.connectors.ai.azure_ai_inference.services.azure_ai_inference_text_embedding import ( + AzureAIInferenceTextEmbedding, +) + +__all__ = [ + "AzureAIInferenceChatCompletion", + "AzureAIInferenceChatPromptExecutionSettings", + "AzureAIInferenceEmbeddingPromptExecutionSettings", + "AzureAIInferenceSettings", + "AzureAIInferenceTextEmbedding", +] diff --git a/python/semantic_kernel/connectors/ai/azure_ai_inference/azure_ai_inference_prompt_execution_settings.py b/python/semantic_kernel/connectors/ai/azure_ai_inference/azure_ai_inference_prompt_execution_settings.py new file mode 100644 index 000000000000..f64646dcf0c7 --- /dev/null +++ b/python/semantic_kernel/connectors/ai/azure_ai_inference/azure_ai_inference_prompt_execution_settings.py @@ -0,0 +1,45 @@ +# Copyright (c) Microsoft. All rights reserved. + +from typing import Literal + +from pydantic import Field + +from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings +from semantic_kernel.utils.experimental_decorator import experimental_class + + +@experimental_class +class AzureAIInferencePromptExecutionSettings(PromptExecutionSettings): + """Azure AI Inference Prompt Execution Settings. + + Note: + `extra_parameters` is a dictionary to pass additional model-specific parameters to the model. + """ + + frequency_penalty: float | None = Field(None, ge=-2, le=2) + max_tokens: int | None = Field(None, gt=0) + presence_penalty: float | None = Field(None, ge=-2, le=2) + seed: int | None = None + stop: str | None = None + temperature: float | None = Field(None, ge=0.0, le=1.0) + top_p: float | None = Field(None, ge=0.0, le=1.0) + extra_parameters: dict[str, str] | None = None + + +@experimental_class +class AzureAIInferenceChatPromptExecutionSettings(AzureAIInferencePromptExecutionSettings): + """Azure AI Inference Chat Prompt Execution Settings.""" + + +@experimental_class +class AzureAIInferenceEmbeddingPromptExecutionSettings(PromptExecutionSettings): + """Azure AI Inference Embedding Prompt Execution Settings. + + Note: + `extra_parameters` is a dictionary to pass additional model-specific parameters to the model. + """ + + dimensions: int | None = Field(None, gt=0) + encoding_format: Literal["base64", "binary", "float", "int8", "ubinary", "uint8"] | None = None + input_type: Literal["text", "query", "document"] | None = None + extra_parameters: dict[str, str] | None = None diff --git a/python/semantic_kernel/connectors/ai/azure_ai_inference/azure_ai_inference_settings.py b/python/semantic_kernel/connectors/ai/azure_ai_inference/azure_ai_inference_settings.py new file mode 100644 index 000000000000..96022ed8cf3a --- /dev/null +++ b/python/semantic_kernel/connectors/ai/azure_ai_inference/azure_ai_inference_settings.py @@ -0,0 +1,37 @@ +# Copyright (c) Microsoft. All rights reserved. + +from typing import ClassVar + +from pydantic import SecretStr + +from semantic_kernel.kernel_pydantic import HttpsUrl, KernelBaseSettings +from semantic_kernel.utils.experimental_decorator import experimental_class + + +@experimental_class +class AzureAIInferenceSettings(KernelBaseSettings): + """Azure AI Inference settings. + + The settings are first loaded from environment variables with + the prefix 'AZURE_AI_INFERENCE_'. + If the environment variables are not found, the settings can + be loaded from a .env file with the encoding 'utf-8'. + If the settings are not found in the .env file, the settings + are ignored; however, validation will fail alerting that the + settings are missing. + + Required settings for prefix 'AZURE_AI_INFERENCE_' are: + - endpoint: HttpsUrl - The endpoint of the Azure AI Inference service deployment. + This value can be found in the Keys & Endpoint section when examining + your resource from the Azure portal. + (Env var AZURE_AI_INFERENCE_ENDPOINT) + - api_key: SecretStr - The API key for the Azure AI Inference service deployment. + This value can be found in the Keys & Endpoint section when examining + your resource from the Azure portal. You can use either KEY1 or KEY2. + (Env var AZURE_AI_INFERENCE_API_KEY) + """ + + env_prefix: ClassVar[str] = "AZURE_AI_INFERENCE_" + + endpoint: HttpsUrl + api_key: SecretStr diff --git a/python/semantic_kernel/connectors/ai/azure_ai_inference/services/__init__.py b/python/semantic_kernel/connectors/ai/azure_ai_inference/services/__init__.py new file mode 100644 index 000000000000..2a50eae89411 --- /dev/null +++ b/python/semantic_kernel/connectors/ai/azure_ai_inference/services/__init__.py @@ -0,0 +1 @@ +# Copyright (c) Microsoft. All rights reserved. diff --git a/python/semantic_kernel/connectors/ai/azure_ai_inference/services/azure_ai_inference_base.py b/python/semantic_kernel/connectors/ai/azure_ai_inference/services/azure_ai_inference_base.py new file mode 100644 index 000000000000..32550fa71697 --- /dev/null +++ b/python/semantic_kernel/connectors/ai/azure_ai_inference/services/azure_ai_inference_base.py @@ -0,0 +1,22 @@ +# Copyright (c) Microsoft. All rights reserved. + +import asyncio +import contextlib +from abc import ABC + +from azure.ai.inference.aio import ChatCompletionsClient, EmbeddingsClient + +from semantic_kernel.kernel_pydantic import KernelBaseModel +from semantic_kernel.utils.experimental_decorator import experimental_class + + +@experimental_class +class AzureAIInferenceBase(KernelBaseModel, ABC): + """Azure AI Inference Chat Completion Service.""" + + client: ChatCompletionsClient | EmbeddingsClient + + def __del__(self) -> None: + """Close the client when the object is deleted.""" + with contextlib.suppress(Exception): + asyncio.get_running_loop().create_task(self.client.close()) diff --git a/python/semantic_kernel/connectors/ai/azure_ai_inference/services/azure_ai_inference_chat_completion.py b/python/semantic_kernel/connectors/ai/azure_ai_inference/services/azure_ai_inference_chat_completion.py new file mode 100644 index 000000000000..5d39d3953e65 --- /dev/null +++ b/python/semantic_kernel/connectors/ai/azure_ai_inference/services/azure_ai_inference_chat_completion.py @@ -0,0 +1,308 @@ +# Copyright (c) Microsoft. All rights reserved. + +import logging +from collections.abc import AsyncGenerator +from typing import Any + +from azure.ai.inference.aio import ChatCompletionsClient +from azure.ai.inference.models import ( + AssistantMessage, + AsyncStreamingChatCompletions, + ChatChoice, + ChatCompletions, + ChatRequestMessage, + ImageContentItem, + ImageDetailLevel, + ImageUrl, + StreamingChatChoiceUpdate, + SystemMessage, + TextContentItem, + ToolMessage, + UserMessage, +) +from azure.core.credentials import AzureKeyCredential +from pydantic import ValidationError + +from semantic_kernel.connectors.ai.azure_ai_inference import ( + AzureAIInferenceChatPromptExecutionSettings, + AzureAIInferenceSettings, +) +from semantic_kernel.connectors.ai.azure_ai_inference.services.azure_ai_inference_base import AzureAIInferenceBase +from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase +from semantic_kernel.contents.chat_history import ChatHistory +from semantic_kernel.contents.chat_message_content import ChatMessageContent +from semantic_kernel.contents.function_call_content import FunctionCallContent +from semantic_kernel.contents.image_content import ImageContent +from semantic_kernel.contents.streaming_chat_message_content import StreamingChatMessageContent +from semantic_kernel.contents.streaming_text_content import StreamingTextContent +from semantic_kernel.contents.text_content import TextContent +from semantic_kernel.contents.utils.author_role import AuthorRole +from semantic_kernel.contents.utils.finish_reason import FinishReason +from semantic_kernel.exceptions.service_exceptions import ServiceInitializationError +from semantic_kernel.utils.experimental_decorator import experimental_class + +_MESSAGE_CONVERTER: dict[AuthorRole, Any] = { + AuthorRole.SYSTEM: SystemMessage, + AuthorRole.USER: UserMessage, + AuthorRole.ASSISTANT: AssistantMessage, + AuthorRole.TOOL: ToolMessage, +} + +logger: logging.Logger = logging.getLogger(__name__) + + +@experimental_class +class AzureAIInferenceChatCompletion(ChatCompletionClientBase, AzureAIInferenceBase): + """Azure AI Inference Chat Completion Service.""" + + def __init__( + self, + ai_model_id: str, + api_key: str | None = None, + endpoint: str | None = None, + service_id: str | None = None, + env_file_path: str | None = None, + env_file_encoding: str | None = None, + client: ChatCompletionsClient | None = None, + ) -> None: + """Initialize the Azure AI Inference Chat Completion service. + + If no arguments are provided, the service will attempt to load the settings from the environment. + The following environment variables are used: + - AZURE_AI_INFERENCE_API_KEY + - AZURE_AI_INFERENCE_ENDPOINT + + Args: + ai_model_id: (str): A string that is used to identify the model such as the model name. (Required) + api_key (str | None): The API key for the Azure AI Inference service deployment. (Optional) + endpoint (str | None): The endpoint of the Azure AI Inference service deployment. (Optional) + service_id (str | None): Service ID for the chat completion service. (Optional) + env_file_path (str | None): The path to the environment file. (Optional) + env_file_encoding (str | None): The encoding of the environment file. (Optional) + client (ChatCompletionsClient | None): The Azure AI Inference client to use. (Optional) + + Raises: + ServiceInitializationError: If an error occurs during initialization. + """ + if not client: + try: + azure_ai_inference_settings = AzureAIInferenceSettings.create( + api_key=api_key, + endpoint=endpoint, + env_file_path=env_file_path, + env_file_encoding=env_file_encoding, + ) + except ValidationError as e: + raise ServiceInitializationError(f"Failed to validate Azure AI Inference settings: {e}") from e + + client = ChatCompletionsClient( + endpoint=azure_ai_inference_settings.endpoint, + credential=AzureKeyCredential(azure_ai_inference_settings.api_key.get_secret_value()), + ) + + super().__init__( + ai_model_id=ai_model_id, + service_id=service_id or ai_model_id, + client=client, + ) + + async def get_chat_message_contents( + self, + chat_history: ChatHistory, + settings: AzureAIInferenceChatPromptExecutionSettings, + **kwargs: Any, + ) -> list[ChatMessageContent]: + """Get chat message contents from the Azure AI Inference service. + + Args: + chat_history: A list of chats in a chat_history object. + settings: Settings for the request. + kwargs: Optional arguments. + + Returns: + A list of chat message contents. + """ + response: ChatCompletions = await self.client.complete( + messages=self._format_chat_history(chat_history), + model_extras=settings.extra_parameters, + **settings.prepare_settings_dict(), + ) + response_metadata = self._get_metadata_from_response(response) + + return [self._create_chat_message_content(response, choice, response_metadata) for choice in response.choices] + + async def get_streaming_chat_message_contents( + self, + chat_history: ChatHistory, + settings: AzureAIInferenceChatPromptExecutionSettings, + **kwargs: Any, + ) -> AsyncGenerator[list[StreamingChatMessageContent], Any]: + """Get streaming chat message contents from the Azure AI Inference service. + + Args: + chat_history: A list of chats in a chat_history object. + settings: Settings for the request. + kwargs: Optional arguments. + + Returns: + A list of chat message contents. + """ + response: AsyncStreamingChatCompletions = await self.client.complete( + stream=True, + messages=self._format_chat_history(chat_history), + model_extras=settings.extra_parameters, + **settings.prepare_settings_dict(), + ) + + async for chunk in response: + if len(chunk.choices) == 0: + continue + chunk_metadata = self._get_metadata_from_response(chunk) + yield [ + self._create_streaming_chat_message_content(chunk, choice, chunk_metadata) for choice in chunk.choices + ] + + def _get_metadata_from_response(self, response: ChatCompletions | AsyncStreamingChatCompletions) -> dict[str, Any]: + """Get metadata from the response. + + Args: + response: The response from the service. + + Returns: + A dictionary containing metadata. + """ + return { + "id": response.id, + "model": response.model, + "created": response.created, + "usage": response.usage, + } + + def _create_chat_message_content( + self, response: ChatCompletions, choice: ChatChoice, metadata: dict[str, Any] + ) -> ChatMessageContent: + """Create a chat message content object. + + Args: + response: The response from the service. + choice: The choice from the response. + metadata: The metadata from the response. + + Returns: + A chat message content object. + """ + items = [] + if choice.message.content: + items.append( + TextContent( + text=choice.message.content, + inner_content=response, + metadata=metadata, + ) + ) + if choice.message.tool_calls: + for tool_call in choice.message.tool_calls: + items.append( + FunctionCallContent( + id=tool_call.id, + name=tool_call.function.name, + arguments=tool_call.function.arguments, + ) + ) + + return ChatMessageContent( + role=AuthorRole(choice.message.role), + items=items, + inner_content=response, + finish_reason=FinishReason(choice.finish_reason) if choice.finish_reason else None, + metadata=metadata, + ) + + def _create_streaming_chat_message_content( + self, + chunk: AsyncStreamingChatCompletions, + choice: StreamingChatChoiceUpdate, + metadata: dict[str, Any], + ) -> StreamingChatMessageContent: + """Create a streaming chat message content object. + + Args: + chunk: The chunk from the response. + choice: The choice from the response. + metadata: The metadata from the response. + + Returns: + A streaming chat message content object. + """ + items = [] + if choice.delta.content: + items.append( + StreamingTextContent( + choice_index=choice.index, + text=choice.delta.content, + inner_content=chunk, + metadata=metadata, + ) + ) + if choice.delta.tool_calls: + for tool_call in choice.delta.tool_calls: + items.append( + FunctionCallContent( + id=tool_call.id, + index=choice.index, + name=tool_call.function.name, + arguments=tool_call.function.arguments, + ) + ) + + return StreamingChatMessageContent( + role=AuthorRole(choice.delta.role) if choice.delta.role else AuthorRole.ASSISTANT, + items=items, + choice_index=choice.index, + inner_content=chunk, + finish_reason=FinishReason(choice.finish_reason) if choice.finish_reason else None, + metadata=metadata, + ) + + def _format_chat_history(self, chat_history: ChatHistory) -> list[ChatRequestMessage]: + """Format the chat history to the expected objects for the client. + + Args: + chat_history: The chat history. + + Returns: + A list of formatted chat history. + """ + chat_request_messages: list[ChatRequestMessage] = [] + + for message in chat_history.messages: + if message.role != AuthorRole.USER or not any(isinstance(item, ImageContent) for item in message.items): + chat_request_messages.append(_MESSAGE_CONVERTER[message.role](content=message.content)) + continue + + # If it's a user message and there are any image items in the message, we need to create a list of + # content items, otherwise we need to just pass in the content as a string or it will error. + contentItems = [] + for item in message.items: + if isinstance(item, TextContent): + contentItems.append(TextContentItem(text=item.text)) + elif isinstance(item, ImageContent) and (item.data_uri or item.uri): + contentItems.append( + ImageContentItem( + image_url=ImageUrl(url=item.data_uri or str(item.uri), detail=ImageDetailLevel.Auto) + ) + ) + else: + logger.warning( + "Unsupported item type in User message while formatting chat history for Azure AI" + f" Inference: {type(item)}" + ) + chat_request_messages.append(_MESSAGE_CONVERTER[message.role](content=contentItems)) + + return chat_request_messages + + def get_prompt_execution_settings_class( + self, + ) -> AzureAIInferenceChatPromptExecutionSettings: + """Get the request settings class.""" + return AzureAIInferenceChatPromptExecutionSettings diff --git a/python/semantic_kernel/connectors/ai/azure_ai_inference/services/azure_ai_inference_text_embedding.py b/python/semantic_kernel/connectors/ai/azure_ai_inference/services/azure_ai_inference_text_embedding.py new file mode 100644 index 000000000000..53942e6e2e4d --- /dev/null +++ b/python/semantic_kernel/connectors/ai/azure_ai_inference/services/azure_ai_inference_text_embedding.py @@ -0,0 +1,88 @@ +# Copyright (c) Microsoft. All rights reserved. + +from typing import Any + +from azure.ai.inference.aio import EmbeddingsClient +from azure.ai.inference.models import EmbeddingsResult +from azure.core.credentials import AzureKeyCredential +from numpy import array, ndarray +from pydantic import ValidationError + +from semantic_kernel.connectors.ai.azure_ai_inference.azure_ai_inference_prompt_execution_settings import ( + AzureAIInferenceEmbeddingPromptExecutionSettings, +) +from semantic_kernel.connectors.ai.azure_ai_inference.azure_ai_inference_settings import AzureAIInferenceSettings +from semantic_kernel.connectors.ai.azure_ai_inference.services.azure_ai_inference_base import AzureAIInferenceBase +from semantic_kernel.connectors.ai.embeddings.embedding_generator_base import EmbeddingGeneratorBase +from semantic_kernel.exceptions.service_exceptions import ServiceInitializationError +from semantic_kernel.utils.experimental_decorator import experimental_class + + +@experimental_class +class AzureAIInferenceTextEmbedding(EmbeddingGeneratorBase, AzureAIInferenceBase): + """Azure AI Inference Text Embedding Service.""" + + def __init__( + self, + ai_model_id: str, + api_key: str | None = None, + endpoint: str | None = None, + service_id: str | None = None, + env_file_path: str | None = None, + env_file_encoding: str | None = None, + client: EmbeddingsClient | None = None, + ) -> None: + """Initialize the Azure AI Inference Text Embedding service. + + If no arguments are provided, the service will attempt to load the settings from the environment. + The following environment variables are used: + - AZURE_AI_INFERENCE_API_KEY + - AZURE_AI_INFERENCE_ENDPOINT + + Args: + ai_model_id: (str): A string that is used to identify the model such as the model name. (Required) + api_key (str | None): The API key for the Azure AI Inference service deployment. (Optional) + endpoint (str | None): The endpoint of the Azure AI Inference service deployment. (Optional) + service_id (str | None): Service ID for the chat completion service. (Optional) + env_file_path (str | None): The path to the environment file. (Optional) + env_file_encoding (str | None): The encoding of the environment file. (Optional) + client (EmbeddingsClient | None): The Azure AI Inference client to use. (Optional) + + Raises: + ServiceInitializationError: If an error occurs during initialization. + """ + if not client: + try: + azure_ai_inference_settings = AzureAIInferenceSettings.create( + api_key=api_key, + endpoint=endpoint, + env_file_path=env_file_path, + env_file_encoding=env_file_encoding, + ) + except ValidationError as e: + raise ServiceInitializationError(f"Failed to validate Azure AI Inference settings: {e}") from e + + client = EmbeddingsClient( + endpoint=azure_ai_inference_settings.endpoint, + credential=AzureKeyCredential(azure_ai_inference_settings.api_key.get_secret_value()), + ) + + super().__init__( + ai_model_id=ai_model_id, + service_id=service_id or ai_model_id, + client=client, + ) + + async def generate_embeddings(self, texts: list[str], **kwargs: Any) -> ndarray: + """Generate embeddings from the Azure AI Inference service.""" + settings: AzureAIInferenceEmbeddingPromptExecutionSettings = kwargs.get("settings", None) + response: EmbeddingsResult = await self.client.embed( + input=texts, + model_extras=settings.extra_parameters if settings else None, + dimensions=settings.dimensions if settings else None, + encoding_format=settings.encoding_format if settings else None, + input_type=settings.input_type if settings else None, + kwargs=kwargs, + ) + + return array([array(item.embedding) for item in response.data]) diff --git a/python/semantic_kernel/connectors/ai/function_call_behavior.py b/python/semantic_kernel/connectors/ai/function_call_behavior.py index d21653166e04..3e6e0a5e6e2a 100644 --- a/python/semantic_kernel/connectors/ai/function_call_behavior.py +++ b/python/semantic_kernel/connectors/ai/function_call_behavior.py @@ -4,6 +4,7 @@ from typing import TYPE_CHECKING, Literal from pydantic.dataclasses import dataclass +from typing_extensions import deprecated from semantic_kernel.functions.kernel_function_metadata import KernelFunctionMetadata from semantic_kernel.kernel_pydantic import KernelBaseModel @@ -23,9 +24,12 @@ class FunctionCallConfiguration: required_functions: list["KernelFunctionMetadata"] | None = None +@deprecated("The `FunctionCallBehavior` class is deprecated; use `FunctionChoiceBehavior` instead.", category=None) class FunctionCallBehavior(KernelBaseModel): """Class that controls function calling behavior. + DEPRECATED: This class has been replaced by FunctionChoiceBehavior. + Args: enable_kernel_functions (bool): Enable kernel functions. max_auto_invoke_attempts (int): The maximum number of auto invoke attempts. @@ -93,11 +97,13 @@ def configure( return @classmethod + @deprecated("Use the `FunctionChoiceBehavior` `Auto` class instead.") def AutoInvokeKernelFunctions(cls) -> "KernelFunctions": """Returns KernelFunctions class with auto_invoke enabled.""" return KernelFunctions(max_auto_invoke_attempts=DEFAULT_MAX_AUTO_INVOKE_ATTEMPTS) @classmethod + @deprecated("Use the `FunctionChoiceBehavior` `Auto` class method instead.") def EnableKernelFunctions(cls) -> "KernelFunctions": """Returns KernelFunctions class with auto_invoke disabled. @@ -106,13 +112,15 @@ def EnableKernelFunctions(cls) -> "KernelFunctions": return KernelFunctions(max_auto_invoke_attempts=0) @classmethod + @deprecated("Use the `FunctionChoiceBehavior` `Auto` class method instead.") def EnableFunctions( cls, auto_invoke: bool = False, *, filters: dict[ Literal["excluded_plugins", "included_plugins", "excluded_functions", "included_functions"], list[str] - ], + ] + | None = {}, ) -> "EnabledFunctions": """Set the enable kernel functions flag.""" return EnabledFunctions( @@ -120,6 +128,7 @@ def EnableFunctions( ) @classmethod + @deprecated("Use the `FunctionChoiceBehavior` `Required` class method instead.") def RequiredFunction( cls, auto_invoke: bool = False, @@ -133,6 +142,7 @@ def RequiredFunction( ) +@deprecated("Use the `FunctionChoiceBehavior` `Auto` class instead.") class KernelFunctions(FunctionCallBehavior): """Function call behavior for making all kernel functions available for tool calls.""" @@ -149,6 +159,7 @@ def configure( ) +@deprecated("Use the `FunctionChoiceBehavior` `Auto` class instead.") class EnabledFunctions(FunctionCallBehavior): """Function call behavior for making a filtered set of functions available for tool calls.""" @@ -170,6 +181,7 @@ def configure( ) +@deprecated("Use the `FunctionChoiceBehavior` `Required` class instead.") class RequiredFunction(FunctionCallBehavior): """Function call behavior for making a single function available for tool calls.""" diff --git a/python/semantic_kernel/connectors/ai/function_calling_utils.py b/python/semantic_kernel/connectors/ai/function_calling_utils.py new file mode 100644 index 000000000000..70704093141f --- /dev/null +++ b/python/semantic_kernel/connectors/ai/function_calling_utils.py @@ -0,0 +1,51 @@ +# Copyright (c) Microsoft. All rights reserved. + +import logging +from typing import TYPE_CHECKING, Any + +from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( + OpenAIChatPromptExecutionSettings, +) +from semantic_kernel.functions.kernel_function_metadata import KernelFunctionMetadata + +if TYPE_CHECKING: + from semantic_kernel.connectors.ai.function_choice_behavior import ( + FunctionCallChoiceConfiguration, + ) + from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( + OpenAIChatPromptExecutionSettings, + ) + +logger = logging.getLogger(__name__) + + +def update_settings_from_function_call_configuration( + function_choice_configuration: "FunctionCallChoiceConfiguration", + settings: "OpenAIChatPromptExecutionSettings", + type: str, +) -> None: + """Update the settings from a FunctionChoiceConfiguration.""" + if function_choice_configuration.available_functions: + settings.tool_choice = type + settings.tools = [ + kernel_function_metadata_to_function_call_format(f) + for f in function_choice_configuration.available_functions + ] + + +def kernel_function_metadata_to_function_call_format( + metadata: KernelFunctionMetadata, +) -> dict[str, Any]: + """Convert the kernel function metadata to function calling format.""" + return { + "type": "function", + "function": { + "name": metadata.fully_qualified_name, + "description": metadata.description or "", + "parameters": { + "type": "object", + "properties": {param.name: param.schema_data for param in metadata.parameters if param.is_required}, + "required": [p.name for p in metadata.parameters if p.is_required], + }, + }, + } diff --git a/python/semantic_kernel/connectors/ai/function_choice_behavior.py b/python/semantic_kernel/connectors/ai/function_choice_behavior.py new file mode 100644 index 000000000000..5aee169c20dd --- /dev/null +++ b/python/semantic_kernel/connectors/ai/function_choice_behavior.py @@ -0,0 +1,283 @@ +# Copyright (c) Microsoft. All rights reserved. + +import logging +from collections import OrderedDict +from collections.abc import Callable +from enum import Enum +from typing import TYPE_CHECKING, Any, Literal + +from pydantic.dataclasses import dataclass +from typing_extensions import deprecated + +from semantic_kernel.exceptions.service_exceptions import ServiceInitializationError +from semantic_kernel.functions.kernel_function_metadata import KernelFunctionMetadata +from semantic_kernel.kernel_pydantic import KernelBaseModel +from semantic_kernel.utils.experimental_decorator import experimental_class + +if TYPE_CHECKING: + from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior + from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings + from semantic_kernel.kernel import Kernel + +DEFAULT_MAX_AUTO_INVOKE_ATTEMPTS = 5 + +logger = logging.getLogger(__name__) + + +@experimental_class +class FunctionChoiceType(Enum): + """The type of function choice behavior.""" + + AUTO = "auto" + NONE = "none" + REQUIRED = "required" + + +@experimental_class +@dataclass +class FunctionCallChoiceConfiguration: + """Configuration for function call choice.""" + + available_functions: list["KernelFunctionMetadata"] | None = None + + +def _combine_filter_dicts(*dicts: dict[str, list[str]]) -> dict: + """Combine multiple filter dictionaries with list values into one dictionary. + + This method is ensuring unique values while preserving order. + """ + combined_filters = {} + + keys = set().union(*(d.keys() for d in dicts)) + + for key in keys: + combined_functions = OrderedDict() + for d in dicts: + if key in d: + if isinstance(d[key], list): + for item in d[key]: + combined_functions[item] = None + else: + raise ServiceInitializationError(f"Values for filter key '{key}' are not lists.") + combined_filters[key] = list(combined_functions.keys()) + + return combined_filters + + +@experimental_class +class FunctionChoiceBehavior(KernelBaseModel): + """Class that controls function choice behavior. + + Attributes: + enable_kernel_functions: Enable kernel functions. + max_auto_invoke_attempts: The maximum number of auto invoke attempts. + filters: Filters for the function choice behavior. Available options are: excluded_plugins, + included_plugins, excluded_functions, or included_functions. + type: The type of function choice behavior. + + Properties: + auto_invoke_kernel_functions: Check if the kernel functions should be auto-invoked. + Determined as max_auto_invoke_attempts > 0. + + Methods: + configure: Configures the settings for the function call behavior, + the default version in this class, does nothing, use subclasses for different behaviors. + + Class methods: + Auto: Returns FunctionChoiceBehavior class with auto_invoke enabled, and the desired functions + based on either the specified filters or the full qualified names. The model will decide which function + to use, if any. + NoneInvoke: Returns FunctionChoiceBehavior class with auto_invoke disabled, and the desired functions + based on either the specified filters or the full qualified names. The model does not invoke any functions, + but can rather describe how it would invoke a function to complete a given task/query. + Required: Returns FunctionChoiceBehavior class with auto_invoke enabled, and the desired functions + based on either the specified filters or the full qualified names. The model is required to use one of the + provided functions to complete a given task/query. + """ + + enable_kernel_functions: bool = True + maximum_auto_invoke_attempts: int = DEFAULT_MAX_AUTO_INVOKE_ATTEMPTS + filters: ( + dict[Literal["excluded_plugins", "included_plugins", "excluded_functions", "included_functions"], list[str]] + | None + ) = None + type: FunctionChoiceType | None = None + + @classmethod + @deprecated("The `FunctionCallBehavior` class is deprecated; use `FunctionChoiceBehavior` instead.") + def from_function_call_behavior(cls, behavior: "FunctionCallBehavior") -> "FunctionChoiceBehavior": + """Create a FunctionChoiceBehavior from a FunctionCallBehavior.""" + from semantic_kernel.connectors.ai.function_call_behavior import ( + EnabledFunctions, + KernelFunctions, + RequiredFunction, + ) + + if isinstance(behavior, (EnabledFunctions, KernelFunctions)): + return cls.Auto( + auto_invoke=behavior.auto_invoke_kernel_functions, + filters=behavior.filters if hasattr(behavior, "filters") else None, + ) + if isinstance(behavior, (RequiredFunction)): + return cls.Required( + auto_invoke=behavior.auto_invoke_kernel_functions, + function_fully_qualified_names=[behavior.function_fully_qualified_name] + if hasattr(behavior, "function_fully_qualified_name") + else None, + ) + return cls( + enable_kernel_functions=behavior.enable_kernel_functions, + maximum_auto_invoke_attempts=behavior.max_auto_invoke_attempts, + ) + + @property + def auto_invoke_kernel_functions(self): + """Return True if auto_invoke_kernel_functions is enabled.""" + return self.maximum_auto_invoke_attempts > 0 + + @auto_invoke_kernel_functions.setter + def auto_invoke_kernel_functions(self, value: bool): + """Set the auto_invoke_kernel_functions property.""" + self.maximum_auto_invoke_attempts = DEFAULT_MAX_AUTO_INVOKE_ATTEMPTS if value else 0 + + def _check_and_get_config( + self, kernel: "Kernel", filters: dict[str, Any] | None = {} + ) -> FunctionCallChoiceConfiguration: + """Check for missing functions and get the function call choice configuration.""" + if filters: + return FunctionCallChoiceConfiguration(available_functions=kernel.get_list_of_function_metadata(filters)) + return FunctionCallChoiceConfiguration(available_functions=kernel.get_full_list_of_function_metadata()) + + def configure( + self, + kernel: "Kernel", + update_settings_callback: Callable[..., None], + settings: "PromptExecutionSettings", + ) -> None: + """Configure the function choice behavior.""" + if not self.enable_kernel_functions: + return + + config = self.get_config(kernel) + + if config: + update_settings_callback(config, settings, self.type) + + def get_config(self, kernel: "Kernel") -> FunctionCallChoiceConfiguration: + """Get the function call choice configuration based on the type.""" + return self._check_and_get_config(kernel, self.filters) + + @classmethod + def Auto( + cls, + auto_invoke: bool = True, + *, + filters: dict[ + Literal["excluded_plugins", "included_plugins", "excluded_functions", "included_functions"], list[str] + ] + | None = None, + **kwargs, + ) -> "FunctionChoiceBehavior": + """Creates a FunctionChoiceBehavior with type AUTO. + + Returns FunctionChoiceBehavior class with auto_invoke enabled, and the desired functions + based on either the specified filters or the full qualified names. The model will decide which function + to use, if any. + """ + kwargs.setdefault("maximum_auto_invoke_attempts", DEFAULT_MAX_AUTO_INVOKE_ATTEMPTS if auto_invoke else 0) + return cls( + type=FunctionChoiceType.AUTO, + filters=filters, + **kwargs, + ) + + @classmethod + def NoneInvoke( + cls, + *, + filters: dict[ + Literal["excluded_plugins", "included_plugins", "excluded_functions", "included_functions"], list[str] + ] + | None = None, + **kwargs, + ) -> "FunctionChoiceBehavior": + """Creates a FunctionChoiceBehavior with type NONE. + + Returns FunctionChoiceBehavior class with auto_invoke disabled, and the desired functions + based on either the specified filters or the full qualified names. The model does not invoke any functions, + but can rather describe how it would invoke a function to complete a given task/query. + """ + kwargs.setdefault("maximum_auto_invoke_attempts", 0) + return cls( + type=FunctionChoiceType.NONE, + filters=filters, + **kwargs, + ) + + @classmethod + def Required( + cls, + auto_invoke: bool = True, + *, + filters: dict[ + Literal["excluded_plugins", "included_plugins", "excluded_functions", "included_functions"], list[str] + ] + | None = None, + **kwargs, + ) -> "FunctionChoiceBehavior": + """Creates a FunctionChoiceBehavior with type REQUIRED. + + Returns FunctionChoiceBehavior class with auto_invoke enabled, and the desired functions + based on either the specified filters or the full qualified names. The model is required to use one of the + provided functions to complete a given task/query. + """ + kwargs.setdefault("maximum_auto_invoke_attempts", 1 if auto_invoke else 0) + return cls( + type=FunctionChoiceType.REQUIRED, + filters=filters, + **kwargs, + ) + + @classmethod + def from_dict(cls, data: dict) -> "FunctionChoiceBehavior": + """Create a FunctionChoiceBehavior from a dictionary.""" + type_map = { + "auto": cls.Auto, + "none": cls.NoneInvoke, + "required": cls.Required, + } + behavior_type = data.pop("type", "auto") + auto_invoke = data.pop("auto_invoke", False) + functions = data.pop("functions", None) + filters = data.pop("filters", None) + + if functions: + valid_fqns = [name.replace(".", "-") for name in functions] + if filters: + filters = _combine_filter_dicts(filters, {"included_functions": valid_fqns}) + else: + filters = {"included_functions": valid_fqns} + + return type_map[behavior_type]( + auto_invoke=auto_invoke, + filters=filters, + **data, + ) + + @classmethod + def from_string(cls, data: str) -> "FunctionChoiceBehavior": + """Create a FunctionChoiceBehavior from a string. + + This method converts the provided string to a FunctionChoiceBehavior object + for the specified type. + """ + type_value = data.lower() + if type_value == "auto": + return FunctionChoiceBehavior.Auto() + if type_value == "none": + return FunctionChoiceBehavior.NoneInvoke() + if type_value == "required": + return FunctionChoiceBehavior.Required() + raise ServiceInitializationError( + f"The specified type `{type_value}` is not supported. Allowed types are: `auto`, `none`, `required`." + ) diff --git a/python/semantic_kernel/connectors/ai/google_palm/__init__.py b/python/semantic_kernel/connectors/ai/google_palm/__init__.py deleted file mode 100644 index 008085f3c833..000000000000 --- a/python/semantic_kernel/connectors/ai/google_palm/__init__.py +++ /dev/null @@ -1,23 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -from semantic_kernel.connectors.ai.google_palm.gp_prompt_execution_settings import ( - GooglePalmChatPromptExecutionSettings, - GooglePalmTextPromptExecutionSettings, -) -from semantic_kernel.connectors.ai.google_palm.services.gp_chat_completion import ( - GooglePalmChatCompletion, -) -from semantic_kernel.connectors.ai.google_palm.services.gp_text_completion import ( - GooglePalmTextCompletion, -) -from semantic_kernel.connectors.ai.google_palm.services.gp_text_embedding import ( - GooglePalmTextEmbedding, -) - -__all__ = [ - "GooglePalmChatCompletion", - "GooglePalmChatPromptExecutionSettings", - "GooglePalmTextCompletion", - "GooglePalmTextEmbedding", - "GooglePalmTextPromptExecutionSettings", -] diff --git a/python/semantic_kernel/connectors/ai/google_palm/gp_prompt_execution_settings.py b/python/semantic_kernel/connectors/ai/google_palm/gp_prompt_execution_settings.py deleted file mode 100644 index c0a8cd4434fc..000000000000 --- a/python/semantic_kernel/connectors/ai/google_palm/gp_prompt_execution_settings.py +++ /dev/null @@ -1,45 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -from collections.abc import Iterable -from typing import Any, Union - -from pydantic import Field, model_validator - -from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings -from semantic_kernel.exceptions import ServiceInvalidExecutionSettingsError - -# TODO (eavanvalkenburg): replace back with google types once pydantic issue is fixed. -MessagesOptions = list[dict[str, Any]] - -MessagePromptOption = Union[str, dict[str, Any]] -MessagePromptOptions = Union[MessagePromptOption, list[MessagePromptOption]] - -ExampleOptions = Union[dict[str, Any], list[dict[str, Any]]] - - -class GooglePalmPromptExecutionSettings(PromptExecutionSettings): - ai_model_id: str | None = Field(None, serialization_alias="model") - temperature: float = Field(0.0, ge=0.0, le=1.0) - top_p: float = 1.0 - top_k: int = 1 - candidate_count: int = Field(1, ge=1, le=8) - safety_settings: dict[str, Any] | None = None - prompt: MessagePromptOptions | None = None - - -class GooglePalmTextPromptExecutionSettings(GooglePalmPromptExecutionSettings): - max_output_tokens: int = Field(256, gt=0) - stop_sequences: str | Iterable[str] | None = None - - -class GooglePalmChatPromptExecutionSettings(GooglePalmPromptExecutionSettings): - messages: MessagesOptions | None = None - examples: ExampleOptions | None = None - context: str | None = None - token_selection_biases: dict[int, int] | None = None - - @model_validator(mode="after") - def validate_input(self): - """Validate input.""" - if self.prompt is not None and (self.messages or self.context or self.examples): - raise ServiceInvalidExecutionSettingsError("Prompt cannot be used without messages, context or examples") diff --git a/python/semantic_kernel/connectors/ai/google_palm/services/__init__.py b/python/semantic_kernel/connectors/ai/google_palm/services/__init__.py deleted file mode 100644 index e69de29bb2d1..000000000000 diff --git a/python/semantic_kernel/connectors/ai/google_palm/services/gp_chat_completion.py b/python/semantic_kernel/connectors/ai/google_palm/services/gp_chat_completion.py deleted file mode 100644 index 292d4a86a00e..000000000000 --- a/python/semantic_kernel/connectors/ai/google_palm/services/gp_chat_completion.py +++ /dev/null @@ -1,240 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import logging -from typing import Annotated, Any - -import google.generativeai as palm -from google.generativeai.types import ChatResponse, MessageDict -from pydantic import PrivateAttr, StringConstraints, ValidationError - -from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase -from semantic_kernel.connectors.ai.google_palm.gp_prompt_execution_settings import ( - GooglePalmChatPromptExecutionSettings, - GooglePalmPromptExecutionSettings, -) -from semantic_kernel.connectors.ai.google_palm.settings.google_palm_settings import GooglePalmSettings -from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings -from semantic_kernel.connectors.ai.text_completion_client_base import TextCompletionClientBase -from semantic_kernel.contents.chat_history import ChatHistory -from semantic_kernel.contents.chat_message_content import ChatMessageContent -from semantic_kernel.contents.text_content import TextContent -from semantic_kernel.contents.utils.author_role import AuthorRole -from semantic_kernel.exceptions import ServiceInitializationError, ServiceInvalidRequestError, ServiceResponseException - -logger: logging.Logger = logging.getLogger(__name__) - -int_to_role = {1: AuthorRole.USER, 2: AuthorRole.SYSTEM, 3: AuthorRole.ASSISTANT, 4: AuthorRole.TOOL} - - -class GooglePalmChatCompletion(ChatCompletionClientBase, TextCompletionClientBase): - api_key: Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)] - _message_history: ChatHistory | None = PrivateAttr() - service_id: str | None = None - - def __init__( - self, - ai_model_id: str, - api_key: str | None = None, - message_history: ChatHistory | None = None, - env_file_path: str | None = None, - env_file_encoding: str | None = None, - ): - """Initializes a new instance of the GooglePalmChatCompletion class. - - Args: - ai_model_id (str): GooglePalm model name, see - https://developers.generativeai.google/models/language - api_key (str | None): The optional API key to use. If not provided, will be read from either - the env vars or the .env settings file - message_history (ChatHistory | None): The message history to use for context. (Optional) - env_file_path (str | None): Use the environment settings file as a fallback to - environment variables. (Optional) - env_file_encoding (str | None): The encoding of the environment settings file. (Optional) - - Raises: - ServiceInitializationError: When any of the required settings are missing. - """ - try: - google_palm_settings = GooglePalmSettings.create( - api_key=api_key, - chat_model_id=ai_model_id, - env_file_path=env_file_path, - env_file_encoding=env_file_encoding, - ) - except ValidationError as ex: - raise ServiceInitializationError("Failed to create Google Palm settings", ex) from ex - - if not google_palm_settings.chat_model_id: - raise ServiceInitializationError("The chat model ID is required for a Chat Completion Model.") - - super().__init__( - ai_model_id=google_palm_settings.chat_model_id, - api_key=google_palm_settings.api_key.get_secret_value(), - ) - self._message_history = message_history - - async def get_chat_message_contents( - self, - chat_history: ChatHistory, - settings: GooglePalmPromptExecutionSettings, - **kwargs: Any, - ) -> list[ChatMessageContent]: - """This is the method that is called from the kernel to get a response from a chat-optimized LLM. - - Args: - chat_history (List[ChatMessage]): A list of chat messages, that can be rendered into a - set of messages, from system, user, assistant and function. - settings (GooglePalmPromptExecutionSettings): Settings for the request. - kwargs (Dict[str, Any]): The optional arguments. - - Returns: - List[ChatMessageContent]: A list of ChatMessageContent objects representing the response(s) from the LLM. - """ - settings.messages = self._prepare_chat_history_for_request(chat_history, role_key="author") - if not settings.ai_model_id: - settings.ai_model_id = self.ai_model_id - response = await self._send_chat_request(settings) - return [ - self._create_chat_message_content(response, candidate, index) - for index, candidate in enumerate(response.candidates) - ] - - def _create_chat_message_content( - self, response: ChatResponse, candidate: MessageDict, index: int - ) -> ChatMessageContent: - """Create a chat message content object from a response. - - Args: - response (ChatResponse): The response to create the content from. - candidate (MessageDict): The candidate message to create the content from. - index (int): The index of the candidate message. - - Returns: - ChatMessageContent: The created chat message content. - """ - metadata = { - "citation_metadata": candidate.get("citation_metadata"), - "filters": response.filters, - "choice_index": index, - } - return ChatMessageContent( - inner_content=response, - ai_model_id=self.ai_model_id, - metadata=metadata, - role=int_to_role[int(candidate.get("author"))], # TODO (moonbox3): why is author coming back as '1'? - content=candidate.get("content"), - ) - - async def get_streaming_chat_message_contents( - self, - messages: list[tuple[str, str]], - settings: GooglePalmPromptExecutionSettings, - **kwargs: Any, - ): - """Return a streaming chat message. - - Raises: - NotImplementedError: Google Palm API does not currently support streaming - """ - raise NotImplementedError("Google Palm API does not currently support streaming") - - async def get_text_contents( - self, - prompt: str, - settings: GooglePalmPromptExecutionSettings, - ) -> list[TextContent]: - """This is the method that is called from the kernel to get a response from a text-optimized LLM. - - Args: - prompt (str): The prompt to send to the LLM. - settings (GooglePalmPromptExecutionSettings): Settings for the request. - - Returns: - List[TextContent]: A list of TextContent objects representing the response(s) from the LLM. - """ - settings.messages = [{"author": "user", "content": prompt}] - if not settings.ai_model_id: - settings.ai_model_id = self.ai_model_id - response = await self._send_chat_request(settings) - - return [self._create_text_content(response, candidate) for candidate in response.candidates] - - def _create_text_content(self, response: ChatResponse, candidate: MessageDict) -> TextContent: - """Create a text content object from a response. - - Args: - response (ChatResponse): The response to create the content from. - candidate (MessageDict): The candidate message to create the content from. - - Returns: - TextContent: The created text content. - """ - metadata = {"citation_metadata": candidate.get("citation_metadata"), "filters": response.filters} - return TextContent( - inner_content=response, - ai_model_id=self.ai_model_id, - metadata=metadata, - text=candidate.get("content"), - ) - - async def get_streaming_text_contents( - self, - prompt: str, - settings: GooglePalmPromptExecutionSettings, - ): - """Return a streaming text content. - - Raises: - NotImplementedError: Google Palm API does not currently support streaming - """ - raise NotImplementedError("Google Palm API does not currently support streaming") - - async def _send_chat_request( - self, - settings: GooglePalmPromptExecutionSettings, - ) -> Any: - """Completes the given user message. - - If len(messages) > 1, and a - conversation has not been initiated yet, it is assumed that chat history - is needed for context. All messages preceding the last message will be - utilized for context. This also enables Google PaLM to utilize memory - and plugins, which should be stored in the messages parameter as system - messages. - - Args: - settings (GooglePalmPromptExecutionSettings): The request settings. - - Returns: - The completion. - """ - if settings is None: - raise ValueError("The request settings cannot be `None`") - - if settings.messages[-1]["author"] != "user": - raise ServiceInvalidRequestError("The last message must be from the user") - try: - palm.configure(api_key=self.api_key) - except Exception as ex: - raise PermissionError( - "Google PaLM service failed to configure. Invalid API key provided.", - ex, - ) - try: - if self._message_history is None: - response = palm.chat(**settings.prepare_settings_dict()) # Start a new conversation - else: - response = self._message_history.reply( # Continue the conversation - settings.messages[-1]["content"], - ) - self._message_history = response # Store response object for future use - except Exception as ex: - raise ServiceResponseException( - "Google PaLM service failed to complete the prompt", - ex, - ) from ex - return response - - def get_prompt_execution_settings_class(self) -> "PromptExecutionSettings": - """Create a request settings object.""" - return GooglePalmChatPromptExecutionSettings diff --git a/python/semantic_kernel/connectors/ai/google_palm/services/gp_text_completion.py b/python/semantic_kernel/connectors/ai/google_palm/services/gp_text_completion.py deleted file mode 100644 index 2bc1a9e19c32..000000000000 --- a/python/semantic_kernel/connectors/ai/google_palm/services/gp_text_completion.py +++ /dev/null @@ -1,122 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import logging -from typing import Annotated - -import google.generativeai as palm -from google.generativeai.types import Completion -from google.generativeai.types.text_types import TextCompletion -from pydantic import StringConstraints, ValidationError - -from semantic_kernel.connectors.ai.google_palm.gp_prompt_execution_settings import GooglePalmTextPromptExecutionSettings -from semantic_kernel.connectors.ai.google_palm.settings.google_palm_settings import GooglePalmSettings -from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings -from semantic_kernel.connectors.ai.text_completion_client_base import TextCompletionClientBase -from semantic_kernel.contents.text_content import TextContent -from semantic_kernel.exceptions import ServiceInitializationError, ServiceResponseException - -logger: logging.Logger = logging.getLogger(__name__) - - -class GooglePalmTextCompletion(TextCompletionClientBase): - api_key: Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)] - - def __init__( - self, - ai_model_id: str, - api_key: str | None = None, - env_file_path: str | None = None, - env_file_encoding: str | None = None, - ): - """Initializes a new instance of the GooglePalmTextCompletion class. - - Args: - ai_model_id (str): GooglePalm model name, see - https://developers.generativeai.google/models/language - api_key (str | None): The optional API key to use. If not provided, will be - read from either the env vars or the .env settings file. - env_file_path (str | None): Use the environment settings file as a - fallback to environment variables. (Optional) - env_file_encoding (str | None): The encoding of the environment settings file. (Optional) - - Raises: - ServiceInitializationError: When the Google Palm settings cannot be read. - """ - try: - google_palm_settings = GooglePalmSettings.create( - api_key=api_key, - text_model_id=ai_model_id, - env_file_path=env_file_path, - env_file_encoding=env_file_encoding, - ) - except ValidationError as ex: - raise ServiceInitializationError("Failed to create Google Palm settings.", ex) from ex - if not google_palm_settings.text_model_id: - raise ServiceInitializationError("The Google Palm text model ID is required.") - - super().__init__( - ai_model_id=google_palm_settings.text_model_id, - api_key=google_palm_settings.api_key.get_secret_value() if google_palm_settings.api_key else None, - ) - - async def get_text_contents( - self, prompt: str, settings: GooglePalmTextPromptExecutionSettings - ) -> list[TextContent]: - """This is the method that is called from the kernel to get a response from a text-optimized LLM. - - Args: - prompt (str): The prompt to send to the LLM. - settings (GooglePalmTextPromptExecutionSettings): Settings for the request. - - Returns: - List[TextContent]: A list of TextContent objects representing the response(s) from the LLM. - """ - settings.prompt = prompt - if not settings.ai_model_id: - settings.ai_model_id = self.ai_model_id - try: - palm.configure(api_key=self.api_key) - except Exception as ex: - raise PermissionError( - "Google PaLM service failed to configure. Invalid API key provided.", - ex, - ) - try: - response = palm.generate_text(**settings.prepare_settings_dict()) - except Exception as ex: - raise ServiceResponseException( - "Google PaLM service failed to complete the prompt", - ex, - ) from ex - return [self._create_text_content(response, candidate) for candidate in response.candidates] - - def _create_text_content(self, response: Completion, candidate: TextCompletion) -> TextContent: - """Create a text content object from a candidate.""" - return TextContent( - inner_content=response, - ai_model_id=self.ai_model_id, - text=candidate.get("output"), - metadata={ - "filters": response.filters, - "safety_feedback": response.safety_feedback, - "citation_metadata": candidate.get("citation_metadata"), - "safety_ratings": candidate.get("safety_ratings"), - }, - ) - - async def get_streaming_text_contents( - self, - prompt: str, - settings: GooglePalmTextPromptExecutionSettings, - ): - """Get streaming text contents from the Google Palm API, unsupported. - - Raises: - NotImplementedError: Google Palm API does not currently support streaming. - - """ - raise NotImplementedError("Google Palm API does not currently support streaming") - - def get_prompt_execution_settings_class(self) -> "PromptExecutionSettings": - """Create a request settings object.""" - return GooglePalmTextPromptExecutionSettings diff --git a/python/semantic_kernel/connectors/ai/google_palm/services/gp_text_embedding.py b/python/semantic_kernel/connectors/ai/google_palm/services/gp_text_embedding.py deleted file mode 100644 index 25f9e0f79b5d..000000000000 --- a/python/semantic_kernel/connectors/ai/google_palm/services/gp_text_embedding.py +++ /dev/null @@ -1,80 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import logging -from typing import Annotated, Any - -import google.generativeai as palm -from numpy import array, ndarray -from pydantic import StringConstraints, ValidationError - -from semantic_kernel.connectors.ai.embeddings.embedding_generator_base import EmbeddingGeneratorBase -from semantic_kernel.connectors.ai.google_palm.settings.google_palm_settings import GooglePalmSettings -from semantic_kernel.exceptions import ServiceInitializationError, ServiceInvalidAuthError, ServiceResponseException -from semantic_kernel.utils.experimental_decorator import experimental_class - -logger: logging.Logger = logging.getLogger(__name__) - - -@experimental_class -class GooglePalmTextEmbedding(EmbeddingGeneratorBase): - api_key: Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)] - - def __init__( - self, - ai_model_id: str, - api_key: str | None = None, - env_file_path: str | None = None, - env_file_encoding: str | None = None, - ) -> None: - """Initializes a new instance of the GooglePalmTextEmbedding class. - - Args: - ai_model_id (str): GooglePalm model name, see - https://developers.generativeai.google/models/language - api_key (str | None): The optional API key to use. If not provided, will be - read from either the env vars or the .env settings file. - env_file_path (str | None): Use the environment settings file - as a fallback to environment variables. (Optional) - env_file_encoding (str | None): The encoding of the environment settings file. (Optional) - - Raises: - ServiceInitializationError: When the Google Palm settings cannot be read. - - """ - try: - google_palm_settings = GooglePalmSettings.create( - api_key=api_key, - embedding_model_id=ai_model_id, - env_file_path=env_file_path, - env_file_encoding=env_file_encoding, - ) - except ValidationError as ex: - raise ServiceInitializationError("Failed to create Google Palm settings.", ex) from ex - if not google_palm_settings.embedding_model_id: - raise ServiceInitializationError("The Google Palm embedding model ID is required.") - - super().__init__( - ai_model_id=google_palm_settings.embedding_model_id, - api_key=google_palm_settings.api_key.get_secret_value() if google_palm_settings.api_key else None, - ) - - async def generate_embeddings(self, texts: list[str], **kwargs: Any) -> ndarray: - """Generates embeddings for the given list of texts.""" - try: - palm.configure(api_key=self.api_key) - except Exception as ex: - raise ServiceInvalidAuthError( - "Google PaLM service failed to configure. Invalid API key provided.", - ex, - ) from ex - embeddings = [] - for text in texts: - try: - response = palm.generate_embeddings(model=self.ai_model_id, text=text, **kwargs) - embeddings.append(array(response["embedding"])) - except Exception as ex: - raise ServiceResponseException( - "Google PaLM service failed to generate the embedding.", - ex, - ) from ex - return array(embeddings) diff --git a/python/semantic_kernel/connectors/ai/google_palm/settings/__init__.py b/python/semantic_kernel/connectors/ai/google_palm/settings/__init__.py deleted file mode 100644 index e69de29bb2d1..000000000000 diff --git a/python/semantic_kernel/connectors/ai/google_palm/settings/google_palm_settings.py b/python/semantic_kernel/connectors/ai/google_palm/settings/google_palm_settings.py deleted file mode 100644 index f4eb33d72258..000000000000 --- a/python/semantic_kernel/connectors/ai/google_palm/settings/google_palm_settings.py +++ /dev/null @@ -1,35 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -from typing import ClassVar - -from pydantic import SecretStr - -from semantic_kernel.kernel_pydantic import KernelBaseSettings - - -class GooglePalmSettings(KernelBaseSettings): - """Google Palm model settings. - - The settings are first loaded from environment variables with the prefix 'GOOGLE_PALM_'. If the - environment variables are not found, the settings can be loaded from a .env file with the - encoding 'utf-8'. If the settings are not found in the .env file, the settings are ignored; - however, validation will fail alerting that the settings are missing. - - Optional settings for prefix 'GOOGLE_PALM_' are: - - api_key: SecretStr - GooglePalm API key, see https://developers.generativeai.google/products/palm - (Env var GOOGLE_PALM_API_KEY) - - env_file_path: {str | None} - Use the environment settings file as a fallback to environment variables. (Optional) - - chat_model_id: str | None - The GooglePalm chat model ID to use. - (Env var GOOGLE_PALM_CHAT_MODEL_ID) - - text_model_id: str | None - The GooglePalm text model ID to use. - (Env var GOOGLE_PALM_TEXT_MODEL_ID) - - embedding_model_id: str | None - The GooglePalm embedding model ID to use. - (Env var GOOGLE_PALM_EMBEDDING_MODEL_ID) - """ - - env_prefix: ClassVar[str] = "GOOGLE_PALM_" - - api_key: SecretStr - chat_model_id: str | None = None - text_model_id: str | None = None - embedding_model_id: str | None = None diff --git a/python/semantic_kernel/connectors/ai/open_ai/prompt_execution_settings/open_ai_prompt_execution_settings.py b/python/semantic_kernel/connectors/ai/open_ai/prompt_execution_settings/open_ai_prompt_execution_settings.py index 87d0a5ddbfb9..66d72d7e5524 100644 --- a/python/semantic_kernel/connectors/ai/open_ai/prompt_execution_settings/open_ai_prompt_execution_settings.py +++ b/python/semantic_kernel/connectors/ai/open_ai/prompt_execution_settings/open_ai_prompt_execution_settings.py @@ -1,8 +1,14 @@ # Copyright (c) Microsoft. All rights reserved. import logging +import sys from typing import Any, Literal +if sys.version_info >= (3, 11): + from typing import Self # pragma: no cover +else: + from typing_extensions import Self # pragma: no cover + from pydantic import Field, field_validator, model_validator from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior @@ -73,6 +79,32 @@ def validate_function_call(cls, v: str | list[dict[str, Any]] | None = None): ) return v + @model_validator(mode="before") + @classmethod + def validate_function_calling_behaviors(cls, data) -> Any: + """Check if function_call_behavior is set and if so, move to use function_choice_behavior instead.""" + # In an attempt to phase out the use of `function_call_behavior` in favor of `function_choice_behavior`, + # we are syncing the `function_call_behavior` with `function_choice_behavior` if the former is set. + # This allows us to make decisions off of `function_choice_behavior`. Anytime the `function_call_behavior` + # is updated, this validation will run to ensure the `function_choice_behavior` stays in sync. + from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior + + if isinstance(data, dict) and "function_call_behavior" in data.get("extension_data", {}): + data["function_choice_behavior"] = FunctionChoiceBehavior.from_function_call_behavior( + data.get("extension_data").get("function_call_behavior") + ) + return data + + @field_validator("function_call_behavior", mode="after") + @classmethod + def check_for_function_call_behavior(cls, v) -> Self: + """Check if function_choice_behavior is set, if not, set it to default.""" + if v is not None: + logger.warning( + "The `function_call_behavior` parameter is deprecated. Please use the `function_choice_behavior` parameter instead." # noqa: E501 + ) + return v + class OpenAIEmbeddingPromptExecutionSettings(PromptExecutionSettings): input: str | list[str] | list[int] | list[list[int]] | None = None diff --git a/python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py b/python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py index eb71abc2aa5a..5047b1c0901b 100644 --- a/python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py +++ b/python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py @@ -3,7 +3,6 @@ import asyncio import logging from collections.abc import AsyncGenerator -from copy import copy from functools import reduce from typing import TYPE_CHECKING, Any @@ -11,39 +10,36 @@ from openai.types.chat.chat_completion import ChatCompletion, Choice from openai.types.chat.chat_completion_chunk import ChatCompletionChunk from openai.types.chat.chat_completion_chunk import Choice as ChunkChoice +from typing_extensions import deprecated from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase -from semantic_kernel.connectors.ai.function_call_behavior import ( - EnabledFunctions, - FunctionCallBehavior, - RequiredFunction, +from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_calling_utils import ( + update_settings_from_function_call_configuration, +) +from semantic_kernel.connectors.ai.function_choice_behavior import ( + FunctionChoiceBehavior, ) from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( OpenAIChatPromptExecutionSettings, ) from semantic_kernel.connectors.ai.open_ai.services.open_ai_handler import OpenAIHandler -from semantic_kernel.connectors.ai.open_ai.services.utils import update_settings_from_function_call_configuration from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings from semantic_kernel.contents.chat_history import ChatHistory from semantic_kernel.contents.chat_message_content import ChatMessageContent from semantic_kernel.contents.function_call_content import FunctionCallContent -from semantic_kernel.contents.function_result_content import FunctionResultContent from semantic_kernel.contents.streaming_chat_message_content import StreamingChatMessageContent from semantic_kernel.contents.streaming_text_content import StreamingTextContent from semantic_kernel.contents.text_content import TextContent from semantic_kernel.contents.utils.author_role import AuthorRole from semantic_kernel.contents.utils.finish_reason import FinishReason from semantic_kernel.exceptions import ( - FunctionCallInvalidArgumentsException, ServiceInvalidExecutionSettingsError, ServiceInvalidResponseError, ) from semantic_kernel.filters.auto_function_invocation.auto_function_invocation_context import ( AutoFunctionInvocationContext, ) -from semantic_kernel.filters.filter_types import FilterTypes -from semantic_kernel.filters.kernel_filters_extension import _rebuild_auto_function_invocation_context -from semantic_kernel.functions.function_result import FunctionResult if TYPE_CHECKING: from semantic_kernel.functions.kernel_arguments import KernelArguments @@ -86,14 +82,21 @@ async def get_chat_message_contents( Returns: List[ChatMessageContent]: The completion result(s). """ + # For backwards compatibility we need to convert the `FunctionCallBehavior` to `FunctionChoiceBehavior` + # if this method is called with a `FunctionCallBehavior` object as pat of the settings + if hasattr(settings, "function_call_behavior") and isinstance( + settings.function_call_behavior, FunctionCallBehavior + ): + settings.function_choice_behavior = FunctionChoiceBehavior.from_function_call_behavior( + settings.function_call_behavior + ) + kernel = kwargs.get("kernel", None) arguments = kwargs.get("arguments", None) - if settings.function_call_behavior is not None: + if settings.function_choice_behavior is not None: if kernel is None: - raise ServiceInvalidExecutionSettingsError( - "The kernel is required for OpenAI tool calls." - ) - if arguments is None and settings.function_call_behavior.auto_invoke_kernel_functions: + raise ServiceInvalidExecutionSettingsError("The kernel is required for OpenAI tool calls.") + if arguments is None and settings.function_choice_behavior.auto_invoke_kernel_functions: raise ServiceInvalidExecutionSettingsError( "The kernel arguments are required for auto invoking OpenAI tool calls." ) @@ -105,13 +108,13 @@ async def get_chat_message_contents( # behavior for non-function calling or for enable, but not auto-invoke. self._prepare_settings(settings, chat_history, stream_request=False, kernel=kernel) - if settings.function_call_behavior is None or ( - settings.function_call_behavior and not settings.function_call_behavior.auto_invoke_kernel_functions + if settings.function_choice_behavior is None or ( + settings.function_choice_behavior and not settings.function_choice_behavior.auto_invoke_kernel_functions ): return await self._send_chat_request(settings) # loop for auto-invoke function calls - for request_index in range(settings.function_call_behavior.max_auto_invoke_attempts): + for request_index in range(settings.function_choice_behavior.maximum_auto_invoke_attempts): completions = await self._send_chat_request(settings) # there is only one chat message, this was checked earlier chat_history.add_message(message=completions[0]) @@ -134,7 +137,7 @@ async def get_chat_message_contents( arguments=arguments, function_call_count=fc_count, request_index=request_index, - function_call_behavior=settings.function_call_behavior, + function_call_behavior=settings.function_choice_behavior, ) for function_call in function_calls ], @@ -146,7 +149,7 @@ async def get_chat_message_contents( self._update_settings(settings, chat_history, kernel=kernel) else: # do a final call, without function calling when the max has been reached. - settings.function_call_behavior.auto_invoke_kernel_functions = False + settings.function_choice_behavior.auto_invoke_kernel_functions = False return await self._send_chat_request(settings) async def get_streaming_chat_message_contents( @@ -167,14 +170,21 @@ async def get_streaming_chat_message_contents( List[StreamingChatMessageContent]: A stream of StreamingChatMessageContent when using Azure. """ + # For backwards compatibility we need to convert the `FunctionCallBehavior` to `FunctionChoiceBehavior` + # if this method is called with a `FunctionCallBehavior` object as part of the settings + if hasattr(settings, "function_call_behavior") and isinstance( + settings.function_call_behavior, FunctionCallBehavior + ): + settings.function_choice_behavior = FunctionChoiceBehavior.from_function_call_behavior( + settings.function_call_behavior + ) + kernel = kwargs.get("kernel", None) arguments = kwargs.get("arguments", None) - if settings.function_call_behavior is not None: + if settings.function_choice_behavior is not None: if kernel is None: - raise ServiceInvalidExecutionSettingsError( - "The kernel is required for OpenAI tool calls." - ) - if arguments is None and settings.function_call_behavior.auto_invoke_kernel_functions: + raise ServiceInvalidExecutionSettingsError("The kernel is required for OpenAI tool calls.") + if arguments is None and settings.function_choice_behavior.auto_invoke_kernel_functions: raise ServiceInvalidExecutionSettingsError( "The kernel arguments are required for auto invoking OpenAI tool calls." ) @@ -188,9 +198,8 @@ async def get_streaming_chat_message_contents( self._prepare_settings(settings, chat_history, stream_request=True, kernel=kernel) request_attempts = ( - settings.function_call_behavior.max_auto_invoke_attempts - if (settings.function_call_behavior and - settings.function_call_behavior.auto_invoke_kernel_functions) + settings.function_choice_behavior.maximum_auto_invoke_attempts + if (settings.function_choice_behavior and settings.function_choice_behavior.auto_invoke_kernel_functions) else 1 ) # hold the messages, if there are more than one response, it will not be used, so we flatten @@ -206,9 +215,10 @@ async def get_streaming_chat_message_contents( yield messages if ( - settings.function_call_behavior is None + settings.function_choice_behavior is None or ( - settings.function_call_behavior and not settings.function_call_behavior.auto_invoke_kernel_functions + settings.function_choice_behavior + and not settings.function_choice_behavior.auto_invoke_kernel_functions ) or not function_call_returned ): @@ -240,7 +250,7 @@ async def get_streaming_chat_message_contents( arguments=arguments, function_call_count=fc_count, request_index=request_index, - function_call_behavior=settings.function_call_behavior, + function_call_behavior=settings.function_choice_behavior, ) for function_call in function_calls ], @@ -309,7 +319,7 @@ def _create_chat_message_content( metadata=metadata, role=AuthorRole(choice.message.role), items=items, - finish_reason=FinishReason(choice.finish_reason) if choice.finish_reason else None, + finish_reason=(FinishReason(choice.finish_reason) if choice.finish_reason else None), ) def _create_streaming_chat_message_content( @@ -331,8 +341,8 @@ def _create_streaming_chat_message_content( inner_content=chunk, ai_model_id=self.ai_model_id, metadata=metadata, - role=AuthorRole(choice.delta.role) if choice.delta.role else AuthorRole.ASSISTANT, - finish_reason=FinishReason(choice.finish_reason) if choice.finish_reason else None, + role=(AuthorRole(choice.delta.role) if choice.delta.role else AuthorRole.ASSISTANT), + finish_reason=(FinishReason(choice.finish_reason) if choice.finish_reason else None), items=items, ) @@ -409,8 +419,8 @@ def _update_settings( ) -> None: """Update the settings with the chat history.""" settings.messages = self._prepare_chat_history_for_request(chat_history) - if settings.function_call_behavior and kernel: - settings.function_call_behavior.configure( + if settings.function_choice_behavior and kernel: + settings.function_choice_behavior.configure( kernel=kernel, update_settings_callback=update_settings_from_function_call_configuration, settings=settings, @@ -419,6 +429,7 @@ def _update_settings( # endregion # region function calling + @deprecated("Use `invoke_function_call` from the kernel instead with `FunctionChoiceBehavior`.") async def _process_function_call( self, function_call: FunctionCallContent, @@ -427,114 +438,21 @@ async def _process_function_call( arguments: "KernelArguments", function_call_count: int, request_index: int, - function_call_behavior: FunctionCallBehavior, + function_call_behavior: FunctionChoiceBehavior | FunctionCallBehavior, ) -> "AutoFunctionInvocationContext | None": """Processes the tool calls in the result and update the chat history.""" - args_cloned = copy(arguments) - try: - parsed_args = function_call.parse_arguments() - if parsed_args: - args_cloned.update(parsed_args) - except (FunctionCallInvalidArgumentsException, TypeError) as exc: - logger.info( - f"Received invalid arguments for function {function_call.name}: {exc}. Trying tool call again." - ) - frc = FunctionResultContent.from_function_call_content_and_result( - function_call_content=function_call, - result="The tool call arguments are malformed. Arguments must be in JSON format. Please try again.", - ) - chat_history.add_message(message=frc.to_chat_message_content()) - return None + if isinstance(function_call_behavior, FunctionCallBehavior): + # We need to still support a `FunctionCallBehavior` input so it doesn't break current + # customers. Map from `FunctionCallBehavior` -> `FunctionChoiceBehavior` + function_call_behavior = FunctionChoiceBehavior.from_function_call_behavior(function_call_behavior) - logger.info(f"Calling {function_call.name} function with args: {function_call.arguments}") - try: - if function_call.name is None: - raise ValueError("The function name is required.") - if ( - isinstance(function_call_behavior, RequiredFunction) - and function_call.name != function_call_behavior.function_fully_qualified_name - ): - raise ValueError( - f"Only function: {function_call_behavior.function_fully_qualified_name} " - f"is allowed, {function_call.name} is not allowed." - ) - if isinstance(function_call_behavior, EnabledFunctions): - enabled_functions = [ - func.fully_qualified_name - for func in kernel.get_list_of_function_metadata(function_call_behavior.filters) - ] - if function_call.name not in enabled_functions: - raise ValueError( - f"Only functions: {enabled_functions} are allowed, {function_call.name} is not allowed." - ) - function_to_call = kernel.get_function(function_call.plugin_name, function_call.function_name) - except Exception as exc: - logger.exception(f"Could not find function {function_call.name}: {exc}.") - frc = FunctionResultContent.from_function_call_content_and_result( - function_call_content=function_call, - result="The tool call could not be found, please try again and make sure to validate the name.", - ) - chat_history.add_message(message=frc.to_chat_message_content()) - return None - - num_required_func_params = len([param for param in function_to_call.parameters if param.is_required]) - if len(parsed_args) < num_required_func_params: - msg = ( - f"There are `{num_required_func_params}` tool call arguments required and " - f"only `{len(parsed_args)}` received. The required arguments are: " - f"{[param.name for param in function_to_call.parameters if param.is_required]}. " - "Please provide the required arguments and try again." - ) - logger.info(msg) - frc = FunctionResultContent.from_function_call_content_and_result( - function_call_content=function_call, - result=msg, - ) - chat_history.add_message(message=frc.to_chat_message_content()) - return None - - _rebuild_auto_function_invocation_context() - invocation_context = AutoFunctionInvocationContext( - function=function_to_call, - kernel=kernel, - arguments=args_cloned, + return await kernel.invoke_function_call( + function_call=function_call, chat_history=chat_history, - function_result=FunctionResult(function=function_to_call.metadata, value=None), - function_count=function_call_count, - request_sequence_index=request_index, - ) - if function_call.index is not None: - invocation_context.function_sequence_index = function_call.index - - stack = kernel.construct_call_stack( - filter_type=FilterTypes.AUTO_FUNCTION_INVOCATION, - inner_function=self._inner_auto_function_invoke_handler, + arguments=arguments, + function_call_count=function_call_count, + request_index=request_index, + function_behavior=function_call_behavior, ) - await stack(invocation_context) - if invocation_context.terminate: - return invocation_context - - frc = FunctionResultContent.from_function_call_content_and_result( - function_call_content=function_call, result=invocation_context.function_result - ) - chat_history.add_message(message=frc.to_chat_message_content()) - return None - - async def _inner_auto_function_invoke_handler(self, context: AutoFunctionInvocationContext): - """Inner auto function invocation handler.""" - try: - result = await context.function.invoke(context.kernel, context.arguments) - if result: - context.function_result = result - except Exception as exc: - logger.exception(f"Error invoking function {context.function.fully_qualified_name}: {exc}.") - value = f"An error occurred while invoking the function {context.function.fully_qualified_name}: {exc}" - if context.function_result is not None: - context.function_result.value = value - else: - context.function_result = FunctionResult(function=context.function.metadata, value=value) - return - - -# endregion + # endregion diff --git a/python/semantic_kernel/connectors/ai/open_ai/services/utils.py b/python/semantic_kernel/connectors/ai/open_ai/services/utils.py deleted file mode 100644 index ab31b670af85..000000000000 --- a/python/semantic_kernel/connectors/ai/open_ai/services/utils.py +++ /dev/null @@ -1,64 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. -import logging -from typing import TYPE_CHECKING, Any - -from semantic_kernel.functions.kernel_function_metadata import KernelFunctionMetadata - -if TYPE_CHECKING: - from semantic_kernel.connectors.ai.function_call_behavior import ( - FunctionCallConfiguration, - ) - from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( - OpenAIChatPromptExecutionSettings, - ) - -logger = logging.getLogger(__name__) - - -def update_settings_from_function_call_configuration( - function_call_configuration: "FunctionCallConfiguration", - settings: "OpenAIChatPromptExecutionSettings", -) -> None: - """Update the settings from a FunctionCallConfiguration.""" - if function_call_configuration.required_functions: - if len(function_call_configuration.required_functions) > 1: - logger.warning( - "Multiple required functions are not supported. Using the first required function." - ) - settings.tools = [ - kernel_function_metadata_to_openai_tool_format( - function_call_configuration.required_functions[0] - ) - ] - settings.tool_choice = function_call_configuration.required_functions[ - 0 - ].fully_qualified_name - return - if function_call_configuration.available_functions: - settings.tool_choice = ( - "auto" if len(function_call_configuration.available_functions) > 0 else None - ) - settings.tools = [ - kernel_function_metadata_to_openai_tool_format(f) - for f in function_call_configuration.available_functions - ] - - -def kernel_function_metadata_to_openai_tool_format( - metadata: KernelFunctionMetadata, -) -> dict[str, Any]: - """Convert the kernel function metadata to OpenAI format.""" - return { - "type": "function", - "function": { - "name": metadata.fully_qualified_name, - "description": metadata.description or "", - "parameters": { - "type": "object", - "properties": { - param.name: param.schema_data for param in metadata.parameters - }, - "required": [p.name for p in metadata.parameters if p.is_required], - }, - }, - } diff --git a/python/semantic_kernel/connectors/ai/open_ai/settings/open_ai_settings.py b/python/semantic_kernel/connectors/ai/open_ai/settings/open_ai_settings.py index 7facb7cbd89c..f005536343ed 100644 --- a/python/semantic_kernel/connectors/ai/open_ai/settings/open_ai_settings.py +++ b/python/semantic_kernel/connectors/ai/open_ai/settings/open_ai_settings.py @@ -15,9 +15,11 @@ class OpenAISettings(KernelBaseSettings): encoding 'utf-8'. If the settings are not found in the .env file, the settings are ignored; however, validation will fail alerting that the settings are missing. - Optional settings for prefix 'OPENAI_' are: + Required settings for prefix 'OPENAI_' are: - api_key: SecretStr - OpenAI API key, see https://platform.openai.com/account/api-keys (Env var OPENAI_API_KEY) + + Optional settings for prefix 'OPENAI_' are: - org_id: str | None - This is usually optional unless your account belongs to multiple organizations. (Env var OPENAI_ORG_ID) - chat_model_id: str | None - The OpenAI chat model ID to use, for example, gpt-3.5-turbo or gpt-4. diff --git a/python/semantic_kernel/connectors/ai/prompt_execution_settings.py b/python/semantic_kernel/connectors/ai/prompt_execution_settings.py index c8036a182ddb..d40a9913fee7 100644 --- a/python/semantic_kernel/connectors/ai/prompt_execution_settings.py +++ b/python/semantic_kernel/connectors/ai/prompt_execution_settings.py @@ -1,11 +1,15 @@ # Copyright (c) Microsoft. All rights reserved. +import logging from typing import Any -from pydantic import Field +from pydantic import Field, model_validator +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.kernel_pydantic import KernelBaseModel +logger = logging.getLogger(__name__) + class PromptExecutionSettings(KernelBaseModel): """Base class for prompt execution settings. @@ -18,6 +22,7 @@ class PromptExecutionSettings(KernelBaseModel): Attributes: service_id (str | None): The service ID to use for the request. extension_data (Dict[str, Any]): Any additional data to send with the request. + function_choice_behavior (FunctionChoiceBehavior | None): The function choice behavior settings. Methods: prepare_settings_dict: Prepares the settings as a dictionary for sending to the AI service. @@ -27,6 +32,21 @@ class PromptExecutionSettings(KernelBaseModel): service_id: str | None = Field(None, min_length=1) extension_data: dict[str, Any] = Field(default_factory=dict) + function_choice_behavior: FunctionChoiceBehavior | None = Field(None, exclude=True) + + @model_validator(mode="before") + @classmethod + def parse_function_choice_behavior(cls, data: dict[str, Any]) -> dict[str, Any] | None: + """Parse the function choice behavior data.""" + if data: + function_choice_behavior_data = data.get("function_choice_behavior") + if function_choice_behavior_data: + if isinstance(function_choice_behavior_data, str): + data["function_choice_behavior"] = FunctionChoiceBehavior.from_string(function_choice_behavior_data) + elif isinstance(function_choice_behavior_data, dict): + data["function_choice_behavior"] = FunctionChoiceBehavior.from_dict(function_choice_behavior_data) + return data + return None def __init__(self, service_id: str | None = None, **kwargs: Any): """Initialize the prompt execution settings. @@ -37,8 +57,11 @@ def __init__(self, service_id: str | None = None, **kwargs: Any): these are attempted to parse into the keys of the specific prompt execution settings. """ extension_data = kwargs.pop("extension_data", {}) + function_choice_behavior = kwargs.pop("function_choice_behavior", None) extension_data.update(kwargs) - super().__init__(service_id=service_id, extension_data=extension_data) + super().__init__( + service_id=service_id, extension_data=extension_data, function_choice_behavior=function_choice_behavior + ) self.unpack_extension_data() @property @@ -76,6 +99,7 @@ def from_prompt_execution_settings(cls, config: "PromptExecutionSettings") -> "P return cls( service_id=config.service_id, extension_data=config.extension_data, + function_choice_behavior=config.function_choice_behavior, ) def unpack_extension_data(self) -> None: diff --git a/python/semantic_kernel/connectors/memory/qdrant/qdrant_memory_store.py b/python/semantic_kernel/connectors/memory/qdrant/qdrant_memory_store.py index d6043560fd0a..d92c71952662 100644 --- a/python/semantic_kernel/connectors/memory/qdrant/qdrant_memory_store.py +++ b/python/semantic_kernel/connectors/memory/qdrant/qdrant_memory_store.py @@ -75,11 +75,7 @@ async def delete_collection(self, collection_name: str) -> None: @override async def does_collection_exist(self, collection_name: str) -> bool: - try: - result = await self.get_collection(collection_name=collection_name) - return result.status == qdrant_models.CollectionStatus.GREEN - except ValueError: - return False + return self._qdrantclient.collection_exists(collection_name=collection_name) @override async def upsert(self, collection_name: str, record: MemoryRecord) -> str: diff --git a/python/semantic_kernel/const.py b/python/semantic_kernel/const.py index 46836a019797..5c513a3ddebe 100644 --- a/python/semantic_kernel/const.py +++ b/python/semantic_kernel/const.py @@ -5,3 +5,4 @@ METADATA_EXCEPTION_KEY: Final[str] = "exception" DEFAULT_SERVICE_NAME: Final[str] = "default" USER_AGENT: Final[str] = "User-Agent" +FUNCTION_SCHEMA_INCLUDE: Final[str] = "function_schema_include" diff --git a/python/semantic_kernel/contents/chat_history.py b/python/semantic_kernel/contents/chat_history.py index 2d7ef35d5a79..d635b94c61c4 100644 --- a/python/semantic_kernel/contents/chat_history.py +++ b/python/semantic_kernel/contents/chat_history.py @@ -263,8 +263,8 @@ def from_rendered_prompt(cls, rendered_prompt: str) -> "ChatHistory": prompt = rendered_prompt.strip() try: xml_prompt = XML(text=f"<{prompt_tag}>{prompt}") - except ParseError: - logger.info(f"Could not parse prompt {prompt} as xml, treating as text") + except ParseError as exc: + logger.info(f"Could not parse prompt {prompt} as xml, treating as text, error was: {exc}") return cls(messages=[ChatMessageContent(role=AuthorRole.USER, content=unescape(prompt))]) if xml_prompt.text and xml_prompt.text.strip(): messages.append(ChatMessageContent(role=AuthorRole.SYSTEM, content=unescape(xml_prompt.text.strip()))) diff --git a/python/semantic_kernel/contents/chat_message_content.py b/python/semantic_kernel/contents/chat_message_content.py index 51394d0ce116..54244d4baff7 100644 --- a/python/semantic_kernel/contents/chat_message_content.py +++ b/python/semantic_kernel/contents/chat_message_content.py @@ -296,5 +296,5 @@ def _parse_items(self) -> str | list[dict[str, Any]]: if len(self.items) == 1 and isinstance(self.items[0], TextContent): return self.items[0].text if len(self.items) == 1 and isinstance(self.items[0], FunctionResultContent): - return self.items[0].result + return str(self.items[0].result) return [item.to_dict() for item in self.items] diff --git a/python/semantic_kernel/contents/function_call_content.py b/python/semantic_kernel/contents/function_call_content.py index 9497785456ef..58ad56327366 100644 --- a/python/semantic_kernel/contents/function_call_content.py +++ b/python/semantic_kernel/contents/function_call_content.py @@ -32,6 +32,8 @@ class FunctionCallContent(KernelContent): name: str | None = None arguments: str | None = None + EMPTY_VALUES: ClassVar[list[str | None]] = ["", "{}", None] + @cached_property def function_name(self) -> str: """Get the function name.""" @@ -58,9 +60,19 @@ def __add__(self, other: "FunctionCallContent | None") -> "FunctionCallContent": id=self.id or other.id, index=self.index or other.index, name=self.name or other.name, - arguments=(self.arguments or "") + (other.arguments or ""), + arguments=self.combine_arguments(self.arguments, other.arguments), ) + def combine_arguments(self, arg1: str | None, arg2: str | None) -> str: + """Combine two arguments.""" + if arg1 in self.EMPTY_VALUES and arg2 in self.EMPTY_VALUES: + return "{}" + if arg1 in self.EMPTY_VALUES: + return arg2 or "{}" + if arg2 in self.EMPTY_VALUES: + return arg1 or "{}" + return (arg1 or "") + (arg2 or "") + def parse_arguments(self) -> dict[str, Any] | None: """Parse the arguments into a dictionary.""" if not self.arguments: diff --git a/python/semantic_kernel/contents/function_result_content.py b/python/semantic_kernel/contents/function_result_content.py index 06395f30a5d9..b9b5a35f06b3 100644 --- a/python/semantic_kernel/contents/function_result_content.py +++ b/python/semantic_kernel/contents/function_result_content.py @@ -4,9 +4,10 @@ from typing import TYPE_CHECKING, Any, ClassVar, Literal, TypeVar from xml.etree.ElementTree import Element # nosec -from pydantic import Field, field_validator +from pydantic import Field from semantic_kernel.contents.const import FUNCTION_RESULT_CONTENT_TAG, TEXT_CONTENT_TAG, ContentTypes +from semantic_kernel.contents.image_content import ImageContent from semantic_kernel.contents.kernel_content import KernelContent from semantic_kernel.contents.text_content import TextContent from semantic_kernel.contents.utils.author_role import AuthorRole @@ -47,7 +48,7 @@ class FunctionResultContent(KernelContent): tag: ClassVar[str] = FUNCTION_RESULT_CONTENT_TAG id: str name: str | None = None - result: str + result: Any encoding: str | None = None @cached_property @@ -60,13 +61,6 @@ def plugin_name(self) -> str | None: """Get the plugin name.""" return self.split_name()[0] - @field_validator("result", mode="before") - @classmethod - def _validate_result(cls, result: Any): - if not isinstance(result, str): - result = str(result) - return result - def __str__(self) -> str: """Return the text of the response.""" return self.result @@ -95,11 +89,30 @@ def from_function_call_content_and_result( metadata: dict[str, Any] = {}, ) -> _T: """Create an instance from a FunctionCallContent and a result.""" + from semantic_kernel.contents.chat_message_content import ChatMessageContent + from semantic_kernel.functions.function_result import FunctionResult + if function_call_content.metadata: metadata.update(function_call_content.metadata) + inner_content = result + if isinstance(result, FunctionResult): + result = result.value + if isinstance(result, TextContent): + res = result.text + elif isinstance(result, ChatMessageContent): + if isinstance(result.items[0], TextContent): + res = result.items[0].text + elif isinstance(result.items[0], ImageContent): + res = result.items[0].data_uri + elif isinstance(result.items[0], FunctionResultContent): + res = result.items[0].result + res = str(result) + else: + res = result return cls( id=function_call_content.id or "unknown", - result=str(result), + inner_content=inner_content, + result=res, name=function_call_content.name, ai_model_id=function_call_content.ai_model_id, metadata=metadata, diff --git a/python/semantic_kernel/core_plugins/math_plugin.py b/python/semantic_kernel/core_plugins/math_plugin.py index 87c211368904..48035d3a9a64 100644 --- a/python/semantic_kernel/core_plugins/math_plugin.py +++ b/python/semantic_kernel/core_plugins/math_plugin.py @@ -29,21 +29,13 @@ def add( amount = int(amount) return MathPlugin.add_or_subtract(input, amount, add=True) - @kernel_function( - description="Subtracts value to a value", - name="Subtract", - ) + @kernel_function(name="Subtract") def subtract( self, input: Annotated[int, "the first number"], amount: Annotated[int, "the number to subtract"], ) -> int: - """Returns the difference of numbers provided. - - :param initial_value_text: Initial value as string to subtract the specified amount - :param context: Contains the context to get the numbers from - :return: The resulting subtraction as a string - """ + """Returns the difference of numbers provided.""" if isinstance(input, str): input = int(input) if isinstance(amount, str): @@ -52,11 +44,5 @@ def subtract( @staticmethod def add_or_subtract(input: int, amount: int, add: bool) -> int: - """Helper function to perform addition or subtraction based on the add flag. - - :param initial_value_text: Initial value as string to add or subtract the specified amount - :param context: Contains the context to get the numbers from - :param add: If True, performs addition, otherwise performs subtraction - :return: The resulting sum or subtraction as a string - """ + """Helper function to perform addition or subtraction based on the add flag.""" return input + amount if add else input - amount diff --git a/python/semantic_kernel/functions/kernel_function_decorator.py b/python/semantic_kernel/functions/kernel_function_decorator.py index 1a1698f72ab3..61b247f25aa0 100644 --- a/python/semantic_kernel/functions/kernel_function_decorator.py +++ b/python/semantic_kernel/functions/kernel_function_decorator.py @@ -24,7 +24,7 @@ def kernel_function( The name and description can be left empty, and then the function name and docstring will be used. The parameters are parsed from the function signature, use typing.Annotated to provide a description for the - parameter, in python 3.8, use typing_extensions.Annotated. + parameter. To parse the type, first it checks if the parameter is annotated, and get's the description from there. After that it checks recursively until it reaches the lowest level, and it combines @@ -45,6 +45,7 @@ def kernel_function( if not supplied, the function docstring will be used, can be None. """ + def decorator(func: Callable[..., object]) -> Callable[..., object]: """The actual decorator function.""" setattr(func, "__kernel_function__", True) @@ -139,9 +140,7 @@ def _parse_parameter(name: str, param: Any, default: Any) -> dict[str, Any]: arg = arg.__forward_arg__ args.append(_parse_parameter(name, arg, default)) if ret.get("type_") in ["list", "dict"]: - ret["type_"] = ( - f"{ret['type_']}[{', '.join([arg['type_'] for arg in args])}]" - ) + ret["type_"] = f"{ret['type_']}[{', '.join([arg['type_'] for arg in args])}]" elif len(args) > 1: ret["type_"] = ", ".join([arg["type_"] for arg in args]) else: diff --git a/python/semantic_kernel/functions/kernel_function_from_prompt.py b/python/semantic_kernel/functions/kernel_function_from_prompt.py index d8368b3066e8..c83fba398eae 100644 --- a/python/semantic_kernel/functions/kernel_function_from_prompt.py +++ b/python/semantic_kernel/functions/kernel_function_from_prompt.py @@ -167,18 +167,11 @@ async def _invoke_internal(self, context: FunctionInvocationContext) -> None: if isinstance(prompt_render_result.ai_service, ChatCompletionClientBase): chat_history = ChatHistory.from_rendered_prompt(prompt_render_result.rendered_prompt) - - # pass the kernel in for auto function calling - kwargs: dict[str, Any] = {} - if hasattr(prompt_render_result.execution_settings, "function_call_behavior"): - kwargs["kernel"] = context.kernel - kwargs["arguments"] = context.arguments - try: chat_message_contents = await prompt_render_result.ai_service.get_chat_message_contents( chat_history=chat_history, settings=prompt_render_result.execution_settings, - **kwargs, + **{"kernel": context.kernel, "arguments": context.arguments}, ) except Exception as exc: raise FunctionExecutionException(f"Error occurred while invoking function {self.name}: {exc}") from exc @@ -211,18 +204,11 @@ async def _invoke_internal_stream(self, context: FunctionInvocationContext) -> N prompt_render_result = await self._render_prompt(context) if isinstance(prompt_render_result.ai_service, ChatCompletionClientBase): - # pass the kernel in for auto function calling - kwargs: dict[str, Any] = {} - if hasattr(prompt_render_result.execution_settings, "function_call_behavior"): - kwargs["kernel"] = context.kernel - kwargs["arguments"] = context.arguments - chat_history = ChatHistory.from_rendered_prompt(prompt_render_result.rendered_prompt) - value: AsyncGenerator = prompt_render_result.ai_service.get_streaming_chat_message_contents( chat_history=chat_history, settings=prompt_render_result.execution_settings, - **kwargs, + **{"kernel": context.kernel, "arguments": context.arguments}, ) elif isinstance(prompt_render_result.ai_service, TextCompletionClientBase): value = prompt_render_result.ai_service.get_streaming_text_contents( diff --git a/python/semantic_kernel/functions/kernel_parameter_metadata.py b/python/semantic_kernel/functions/kernel_parameter_metadata.py index 5391e0bafb40..eeb08dac5f14 100644 --- a/python/semantic_kernel/functions/kernel_parameter_metadata.py +++ b/python/semantic_kernel/functions/kernel_parameter_metadata.py @@ -17,6 +17,7 @@ class KernelParameterMetadata(KernelBaseModel): is_required: bool | None = False type_object: Any | None = None schema_data: dict[str, Any] | None = None + function_schema_include: bool | None = True @model_validator(mode="before") @classmethod diff --git a/python/semantic_kernel/kernel.py b/python/semantic_kernel/kernel.py index a306e255f632..da54baadb429 100644 --- a/python/semantic_kernel/kernel.py +++ b/python/semantic_kernel/kernel.py @@ -6,14 +6,26 @@ from typing import TYPE_CHECKING, Any, Literal, TypeVar from semantic_kernel.const import METADATA_EXCEPTION_KEY +from semantic_kernel.contents.chat_history import ChatHistory +from semantic_kernel.contents.function_call_content import FunctionCallContent +from semantic_kernel.contents.function_result_content import FunctionResultContent from semantic_kernel.contents.streaming_content_mixin import StreamingContentMixin from semantic_kernel.exceptions import ( + FunctionCallInvalidArgumentsException, + FunctionExecutionException, KernelFunctionNotFoundError, KernelInvokeException, OperationCancelledException, TemplateSyntaxError, ) -from semantic_kernel.filters.kernel_filters_extension import KernelFilterExtension +from semantic_kernel.filters.auto_function_invocation.auto_function_invocation_context import ( + AutoFunctionInvocationContext, +) +from semantic_kernel.filters.filter_types import FilterTypes +from semantic_kernel.filters.kernel_filters_extension import ( + KernelFilterExtension, + _rebuild_auto_function_invocation_context, +) from semantic_kernel.functions.function_result import FunctionResult from semantic_kernel.functions.kernel_arguments import KernelArguments from semantic_kernel.functions.kernel_function_extension import KernelFunctionExtension @@ -24,8 +36,12 @@ from semantic_kernel.reliability.kernel_reliability_extension import KernelReliabilityExtension from semantic_kernel.services.ai_service_selector import AIServiceSelector from semantic_kernel.services.kernel_services_extension import KernelServicesExtension +from semantic_kernel.utils.naming import generate_random_ascii_name if TYPE_CHECKING: + from semantic_kernel.connectors.ai.function_choice_behavior import ( + FunctionChoiceBehavior, + ) from semantic_kernel.functions.kernel_function import KernelFunction T = TypeVar("T") @@ -113,6 +129,8 @@ async def invoke_stream( """ if arguments is None: arguments = KernelArguments(**kwargs) + else: + arguments.update(kwargs) if not function: if not function_name or not plugin_name: raise KernelFunctionNotFoundError("No function(s) or function- and plugin-name provided") @@ -192,9 +210,9 @@ async def invoke( async def invoke_prompt( self, - function_name: str, - plugin_name: str, prompt: str, + function_name: str | None = None, + plugin_name: str | None = None, arguments: KernelArguments | None = None, template_format: Literal[ "semantic-kernel", @@ -206,9 +224,9 @@ async def invoke_prompt( """Invoke a function from the provided prompt. Args: - function_name (str): The name of the function - plugin_name (str): The name of the plugin prompt (str): The prompt to use + function_name (str): The name of the function, optional + plugin_name (str): The name of the plugin, optional arguments (KernelArguments | None): The arguments to pass to the function(s), optional template_format (str | None): The format of the prompt template kwargs (dict[str, Any]): arguments that can be used instead of supplying KernelArguments @@ -222,7 +240,7 @@ async def invoke_prompt( raise TemplateSyntaxError("The prompt is either null or empty.") function = KernelFunctionFromPrompt( - function_name=function_name, + function_name=function_name or generate_random_ascii_name(), plugin_name=plugin_name, prompt=prompt, template_format=template_format, @@ -231,9 +249,9 @@ async def invoke_prompt( async def invoke_prompt_stream( self, - function_name: str, - plugin_name: str, prompt: str, + function_name: str | None = None, + plugin_name: str | None = None, arguments: KernelArguments | None = None, template_format: Literal[ "semantic-kernel", @@ -246,9 +264,9 @@ async def invoke_prompt_stream( """Invoke a function from the provided prompt and stream the results. Args: - function_name (str): The name of the function - plugin_name (str): The name of the plugin prompt (str): The prompt to use + function_name (str): The name of the function, optional + plugin_name (str): The name of the plugin, optional arguments (KernelArguments | None): The arguments to pass to the function(s), optional template_format (str | None): The format of the prompt template return_function_results (bool): If True, the function results are yielded as a list[FunctionResult] @@ -265,7 +283,7 @@ async def invoke_prompt_stream( from semantic_kernel.functions.kernel_function_from_prompt import KernelFunctionFromPrompt function = KernelFunctionFromPrompt( - function_name=function_name, + function_name=function_name or generate_random_ascii_name(), plugin_name=plugin_name, prompt=prompt, template_format=template_format, @@ -294,3 +312,112 @@ async def invoke_prompt_stream( else: output_function_result[choice.choice_index] += choice yield FunctionResult(function=function.metadata, value=output_function_result) + + async def invoke_function_call( + self, + function_call: FunctionCallContent, + chat_history: ChatHistory, + arguments: "KernelArguments | None" = None, + function_call_count: int | None = None, + request_index: int | None = None, + function_behavior: "FunctionChoiceBehavior" = None, # type: ignore + ) -> "AutoFunctionInvocationContext | None": + """Processes the provided FunctionCallContent and updates the chat history.""" + args_cloned = copy(arguments) if arguments else KernelArguments() + try: + parsed_args = function_call.to_kernel_arguments() + if parsed_args: + args_cloned.update(parsed_args) + except (FunctionCallInvalidArgumentsException, TypeError) as exc: + logger.info(f"Received invalid arguments for function {function_call.name}: {exc}. Trying tool call again.") + frc = FunctionResultContent.from_function_call_content_and_result( + function_call_content=function_call, + result="The tool call arguments are malformed. Arguments must be in JSON format. Please try again.", + ) + chat_history.add_message(message=frc.to_chat_message_content()) + return None + + try: + if function_call.name is None: + raise FunctionExecutionException("The function name is required.") + if function_behavior is not None and function_behavior.filters: + allowed_functions = [ + func.fully_qualified_name for func in self.get_list_of_function_metadata(function_behavior.filters) + ] + if function_call.name not in allowed_functions: + raise FunctionExecutionException( + f"Only functions: {allowed_functions} are allowed, {function_call.name} is not allowed." + ) + function_to_call = self.get_function(function_call.plugin_name, function_call.function_name) + except Exception as exc: + logger.exception(f"The function `{function_call.name}` is not part of the provided functions: {exc}.") + frc = FunctionResultContent.from_function_call_content_and_result( + function_call_content=function_call, + result=( + f"The tool call with name `{function_call.name}` is not part of the provided tools, " + "please try again with a supplied tool call name and make sure to validate the name." + ), + ) + chat_history.add_message(message=frc.to_chat_message_content()) + return None + + num_required_func_params = len([param for param in function_to_call.parameters if param.is_required]) + if parsed_args is None or len(parsed_args) < num_required_func_params: + msg = ( + f"There are `{num_required_func_params}` tool call arguments required and " + f"only `{len(parsed_args) if parsed_args is not None else 0}` received. The required arguments are: " + f"{[param.name for param in function_to_call.parameters if param.is_required]}. " + "Please provide the required arguments and try again." + ) + logger.info(msg) + frc = FunctionResultContent.from_function_call_content_and_result( + function_call_content=function_call, + result=msg, + ) + chat_history.add_message(message=frc.to_chat_message_content()) + return None + + logger.info(f"Calling {function_call.name} function with args: {function_call.arguments}") + + _rebuild_auto_function_invocation_context() + invocation_context = AutoFunctionInvocationContext( + function=function_to_call, + kernel=self, + arguments=args_cloned, + chat_history=chat_history, + function_result=FunctionResult(function=function_to_call.metadata, value=None), + function_count=function_call_count or 0, + request_sequence_index=request_index or 0, + ) + if function_call.index is not None: + invocation_context.function_sequence_index = function_call.index + + stack = self.construct_call_stack( + filter_type=FilterTypes.AUTO_FUNCTION_INVOCATION, + inner_function=self._inner_auto_function_invoke_handler, + ) + await stack(invocation_context) + + if invocation_context.terminate: + return invocation_context + + frc = FunctionResultContent.from_function_call_content_and_result( + function_call_content=function_call, result=invocation_context.function_result + ) + chat_history.add_message(message=frc.to_chat_message_content()) + return None + + async def _inner_auto_function_invoke_handler(self, context: AutoFunctionInvocationContext): + """Inner auto function invocation handler.""" + try: + result = await context.function.invoke(context.kernel, context.arguments) + if result: + context.function_result = result + except Exception as exc: + logger.exception(f"Error invoking function {context.function.fully_qualified_name}: {exc}.") + value = f"An error occurred while invoking the function {context.function.fully_qualified_name}: {exc}" + if context.function_result is not None: + context.function_result.value = value + else: + context.function_result = FunctionResult(function=context.function.metadata, value=value) + return diff --git a/python/semantic_kernel/planners/function_calling_stepwise_planner/function_calling_stepwise_planner.py b/python/semantic_kernel/planners/function_calling_stepwise_planner/function_calling_stepwise_planner.py index 4513f3b2ae92..45af8756adca 100644 --- a/python/semantic_kernel/planners/function_calling_stepwise_planner/function_calling_stepwise_planner.py +++ b/python/semantic_kernel/planners/function_calling_stepwise_planner/function_calling_stepwise_planner.py @@ -8,13 +8,13 @@ import yaml -from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_calling_utils import kernel_function_metadata_to_function_call_format +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( OpenAIChatPromptExecutionSettings, ) from semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion import AzureChatCompletion from semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion import OpenAIChatCompletion -from semantic_kernel.connectors.ai.open_ai.services.utils import kernel_function_metadata_to_openai_tool_format from semantic_kernel.const import DEFAULT_SERVICE_NAME from semantic_kernel.contents.chat_history import ChatHistory from semantic_kernel.contents.function_call_content import FunctionCallContent @@ -152,7 +152,7 @@ async def invoke( chat_history_for_steps = await self._build_chat_history_for_step( goal=question, initial_plan=initial_plan, kernel=cloned_kernel, arguments=arguments, service=chat_completion ) - prompt_execution_settings.function_call_behavior = FunctionCallBehavior.EnableFunctions( + prompt_execution_settings.function_choice_behavior = FunctionChoiceBehavior.Auto( auto_invoke=False, filters={"excluded_plugins": list(self.options.excluded_plugins)} ) for i in range(self.options.max_iterations): @@ -202,7 +202,7 @@ async def invoke( arguments=arguments, function_call_count=1, request_index=0, - function_call_behavior=prompt_execution_settings.function_call_behavior, + function_call_behavior=prompt_execution_settings.function_choice_behavior, ) if context is not None: # Only add the function result content to the chat history if the context is present @@ -275,10 +275,8 @@ async def _generate_plan( ) -> str: """Generate the plan for the given question using the kernel.""" generate_plan_function = self._create_config_from_yaml(kernel) - # TODO (moonbox3): revisit when function call behavior is finalized, and other function calling models are added - # https://github.com/microsoft/semantic-kernel/issues/6458 functions_manual = [ - kernel_function_metadata_to_openai_tool_format(f) + kernel_function_metadata_to_function_call_format(f) for f in kernel.get_list_of_function_metadata( {"excluded_functions": [f"{self.service_id}", "sequential_planner-create_plan"]} ) diff --git a/python/semantic_kernel/services/ai_service_client_base.py b/python/semantic_kernel/services/ai_service_client_base.py index bf8fc44ac2d3..6feeedb3e96c 100644 --- a/python/semantic_kernel/services/ai_service_client_base.py +++ b/python/semantic_kernel/services/ai_service_client_base.py @@ -29,7 +29,11 @@ def model_post_init(self, __context: object | None = None): self.service_id = self.ai_model_id def get_prompt_execution_settings_class(self) -> type["PromptExecutionSettings"]: - """Get the request settings class.""" + """Get the request settings class. + + Overwrite this in subclass to return the proper prompt execution type the + service is expecting. + """ return PromptExecutionSettings # pragma: no cover def instantiate_prompt_execution_settings(self, **kwargs) -> "PromptExecutionSettings": @@ -41,4 +45,8 @@ def instantiate_prompt_execution_settings(self, **kwargs) -> "PromptExecutionSet def get_prompt_execution_settings_from_settings(self, settings: PromptExecutionSettings) -> PromptExecutionSettings: """Get the request settings from a settings object.""" - return self.get_prompt_execution_settings_class().from_prompt_execution_settings(settings) + prompt_execution_settings_type = self.get_prompt_execution_settings_class() + if isinstance(settings, prompt_execution_settings_type): + return settings + + return prompt_execution_settings_type.from_prompt_execution_settings(settings) diff --git a/python/tests/conftest.py b/python/tests/conftest.py index 6e28e5129485..929ea3dfb00a 100644 --- a/python/tests/conftest.py +++ b/python/tests/conftest.py @@ -249,33 +249,6 @@ def openai_unit_test_env(monkeypatch, exclude_list, override_env_param_dict): return env_vars -@pytest.fixture() -def google_palm_unit_test_env(monkeypatch, exclude_list, override_env_param_dict): - """Fixture to set environment variables for Google Palm.""" - if exclude_list is None: - exclude_list = [] - - if override_env_param_dict is None: - override_env_param_dict = {} - - env_vars = { - "GOOGLE_PALM_API_KEY": "test_api_key", - "OPENAI_CHAT_MODEL_ID": "test_chat_model_id", - "OPENAI_TEXT_MODEL_ID": "test_text_model_id", - "OPENAI_EMBEDDING_MODEL_ID": "test_embedding_model_id", - } - - env_vars.update(override_env_param_dict) - - for key, value in env_vars.items(): - if key not in exclude_list: - monkeypatch.setenv(key, value) - else: - monkeypatch.delenv(key, raising=False) - - return env_vars - - @pytest.fixture() def aca_python_sessions_unit_test_env(monkeypatch, exclude_list, override_env_param_dict): """Fixture to set environment variables for ACA Python Unit Tests.""" diff --git a/python/tests/integration/completions/conftest.py b/python/tests/integration/completions/conftest.py index 7d0d6a57b072..17a1e3968661 100644 --- a/python/tests/integration/completions/conftest.py +++ b/python/tests/integration/completions/conftest.py @@ -3,10 +3,7 @@ import pytest -import semantic_kernel.connectors.ai.google_palm as sk_gp -from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings from semantic_kernel.kernel import Kernel -from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig @pytest.fixture(scope="function") @@ -78,29 +75,3 @@ def setup_summarize_conversation_using_plugin(kernel: Kernel): John: Yeah, that's a good idea.""" yield kernel, ChatTranscript - - -@pytest.fixture(scope="function") -def setup_gp_text_completion_function(kernel: Kernel): - # Configure LLM service - palm_text_completion = sk_gp.GooglePalmTextCompletion(ai_model_id="models/text-bison-001") - kernel.add_service(palm_text_completion) - - # Define semantic function using SK prompt template language - prompt = "Hello, I like {{$input}}{{$input2}}" - - exec_settings = PromptExecutionSettings( - service_id="models/text-bison-001", extension_data={"max_tokens": 200, "temperature": 0, "top_p": 0.5} - ) - - prompt_template_config = PromptTemplateConfig(template=prompt, execution_settings=exec_settings) - - # Create the semantic function - text2text_function = kernel.add_function( - function_name="hello", plugin_name="plugin", prompt_template_config=prompt_template_config - ) - - # User input - simple_input = "sleeping and " - - yield kernel, text2text_function, simple_input diff --git a/python/tests/integration/completions/test_azure_oai_text_service.py b/python/tests/integration/completions/test_azure_oai_text_service.py deleted file mode 100644 index 2358fa7ec0d4..000000000000 --- a/python/tests/integration/completions/test_azure_oai_text_service.py +++ /dev/null @@ -1,96 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - - -import pytest -from openai import AsyncAzureOpenAI -from test_utils import retry - -import semantic_kernel.connectors.ai.open_ai as sk_oai -from semantic_kernel.connectors.ai.open_ai.settings.azure_open_ai_settings import AzureOpenAISettings -from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings -from semantic_kernel.functions.kernel_arguments import KernelArguments -from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig - - -@pytest.mark.asyncio -async def test_azure_e2e_text_completion_with_plugin(setup_tldr_function_for_oai_models): - kernel, prompt, text_to_summarize = setup_tldr_function_for_oai_models - - service_id = "text_completion" - - # Configure LLM service - kernel.add_service( - sk_oai.AzureTextCompletion( - service_id=service_id, - ), - ) - - exec_settings = PromptExecutionSettings( - service_id=service_id, extension_data={"max_tokens": 200, "temperature": 0, "top_p": 0.5} - ) - - prompt_template_config = PromptTemplateConfig( - template=prompt, description="Write a short story.", execution_settings=exec_settings - ) - - # Create the semantic function - tldr_function = kernel.add_function( - function_name="story", plugin_name="plugin", prompt_template_config=prompt_template_config - ) - - arguments = KernelArguments(input=text_to_summarize) - - summary = await retry(lambda: kernel.invoke(tldr_function, arguments)) - output = str(summary).strip() - print(f"TLDR using input string: '{output}'") - assert len(output) < 100 - - -@pytest.mark.asyncio -async def test_azure_e2e_text_completion_with_plugin_with_provided_client(setup_tldr_function_for_oai_models): - kernel, prompt, text_to_summarize = setup_tldr_function_for_oai_models - - azure_openai_settings = AzureOpenAISettings.create() - endpoint = azure_openai_settings.endpoint - deployment_name = azure_openai_settings.text_deployment_name - api_key = azure_openai_settings.api_key.get_secret_value() - api_version = azure_openai_settings.api_version - - client = AsyncAzureOpenAI( - azure_endpoint=endpoint, - azure_deployment=deployment_name, - api_key=api_key, - api_version=api_version, - default_headers={"Test-User-X-ID": "test"}, - ) - - service_id = "text_completion" - - # Configure LLM service - kernel.add_service( - sk_oai.AzureTextCompletion( - service_id=service_id, - async_client=client, - ), - overwrite=True, # Overwrite the service for the test if it already exists - ) - - exec_settings = PromptExecutionSettings( - service_id=service_id, extension_data={"max_tokens": 200, "temperature": 0, "top_p": 0.5} - ) - - prompt_template_config = PromptTemplateConfig( - template=prompt, description="Write a short story.", execution_settings=exec_settings - ) - - # Create the semantic function - tldr_function = kernel.add_function( - function_name="tldr", plugin_name="plugin", prompt_template_config=prompt_template_config - ) - - arguments = KernelArguments(input=text_to_summarize) - - summary = await retry(lambda: kernel.invoke(tldr_function, arguments)) - output = str(summary).strip() - print(f"TLDR using input string: '{output}'") - assert len(output) > 0 diff --git a/python/tests/integration/completions/test_chat_completions.py b/python/tests/integration/completions/test_chat_completions.py index 1d8359b89944..c70e548910bf 100644 --- a/python/tests/integration/completions/test_chat_completions.py +++ b/python/tests/integration/completions/test_chat_completions.py @@ -5,17 +5,28 @@ from typing import Any import pytest +from azure.ai.inference.aio import ChatCompletionsClient +from azure.core.credentials import AzureKeyCredential from openai import AsyncAzureOpenAI from semantic_kernel import Kernel +from semantic_kernel.connectors.ai.azure_ai_inference.azure_ai_inference_prompt_execution_settings import ( + AzureAIInferenceChatPromptExecutionSettings, +) +from semantic_kernel.connectors.ai.azure_ai_inference.services.azure_ai_inference_chat_completion import ( + AzureAIInferenceChatCompletion, +) from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior -from semantic_kernel.connectors.ai.open_ai import ( - AzureChatCompletion, +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior +from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.azure_chat_prompt_execution_settings import ( AzureChatPromptExecutionSettings, - OpenAIChatCompletion, +) +from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( OpenAIChatPromptExecutionSettings, ) +from semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion import AzureChatCompletion +from semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion import OpenAIChatCompletion from semantic_kernel.connectors.ai.open_ai.settings.azure_open_ai_settings import AzureOpenAISettings from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings from semantic_kernel.contents import ChatHistory, ChatMessageContent, TextContent @@ -65,10 +76,20 @@ def services() -> dict[str, tuple[ChatCompletionClientBase, type[PromptExecution default_headers={"Test-User-X-ID": "test"}, ), ) + azure_ai_inference_client = AzureAIInferenceChatCompletion( + ai_model_id=deployment_name, + client=ChatCompletionsClient( + endpoint=f'{str(endpoint).strip("/")}/openai/deployments/{deployment_name}', + credential=AzureKeyCredential(""), + headers={"api-key": api_key}, + ), + ) + return { "openai": (OpenAIChatCompletion(), OpenAIChatPromptExecutionSettings), "azure": (AzureChatCompletion(), AzureChatPromptExecutionSettings), "azure_custom_client": (azure_custom_client, AzureChatPromptExecutionSettings), + "azure_ai_inference": (azure_ai_inference_client, AzureAIInferenceChatPromptExecutionSettings), } @@ -230,7 +251,7 @@ def services() -> dict[str, tuple[ChatCompletionClientBase, type[PromptExecution ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text="What is 3+345?")]), ], ["348"], - id="azure_tool_call_auto", + id="azure_tool_call_auto_function_call_behavior", ), pytest.param( "azure", @@ -243,7 +264,38 @@ def services() -> dict[str, tuple[ChatCompletionClientBase, type[PromptExecution ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text="What is 3+345?")]), ], ["348"], - id="azure_tool_call_non_auto", + id="azure_tool_call_non_auto_function_call_behavior", + ), + pytest.param( + "azure", + {"function_choice_behavior": FunctionChoiceBehavior.Auto(filters={"excluded_plugins": ["chat"]})}, + [ + ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text="What is 3+345?")]), + ], + ["348"], + id="azure_tool_call_auto_function_choice_behavior", + ), + pytest.param( + "azure", + {"function_choice_behavior": "auto"}, + [ + ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text="What is 3+345?")]), + ], + ["348"], + id="azure_tool_call_auto_function_choice_behavior_as_string", + ), + pytest.param( + "azure", + { + "function_choice_behavior": FunctionChoiceBehavior.Auto( + auto_invoke=False, filters={"excluded_plugins": ["chat"]} + ) + }, + [ + ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text="What is 3+345?")]), + ], + ["348"], + id="azure_tool_call_non_auto_function_choice_behavior", ), pytest.param( "azure", @@ -281,11 +333,61 @@ def services() -> dict[str, tuple[ChatCompletionClientBase, type[PromptExecution ["Hello", "well"], id="azure_custom_client", ), + pytest.param( + "azure_ai_inference", + {}, + [ + ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text="Hello")]), + ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text="How are you today?")]), + ], + ["Hello", "well"], + id="azure_ai_inference_text_input", + ), + pytest.param( + "azure_ai_inference", + { + "max_tokens": 256, + }, + [ + ChatMessageContent( + role=AuthorRole.USER, + items=[ + TextContent(text="What is in this image?"), + ImageContent( + uri="https://upload.wikimedia.org/wikipedia/commons/d/d5/Half-timbered_mansion%2C_Zirkel%2C_East_view.jpg" + ), + ], + ), + ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text="Where was it made?")]), + ], + ["house", "germany"], + id="azure_ai_inference_image_input_uri", + ), + pytest.param( + "azure_ai_inference", + { + "max_tokens": 256, + }, + [ + ChatMessageContent( + role=AuthorRole.USER, + items=[ + TextContent(text="What is in this image?"), + ImageContent.from_image_path( + image_path=os.path.join(os.path.dirname(__file__), "../../", "assets/sample_image.jpg") + ), + ], + ), + ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text="Where was it made?")]), + ], + ["house", "germany"], + id="azure_ai_inference_image_input_file", + ), ], ) -@pytest.mark.asyncio +@pytest.mark.asyncio(scope="module") async def test_chat_completion( kernel: Kernel, service: str, @@ -309,7 +411,7 @@ async def test_chat_completion( history.add_message(cmc) -@pytest.mark.asyncio +@pytest.mark.asyncio(scope="module") async def test_streaming_chat_completion( kernel: Kernel, service: str, diff --git a/python/tests/integration/completions/test_gp_chat_service.py b/python/tests/integration/completions/test_gp_chat_service.py deleted file mode 100644 index 3e1a7b668614..000000000000 --- a/python/tests/integration/completions/test_gp_chat_service.py +++ /dev/null @@ -1,49 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import os -import sys - -import pytest -from test_utils import retry - -import semantic_kernel.connectors.ai.google_palm as sk_gp -from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings -from semantic_kernel.functions.kernel_arguments import KernelArguments -from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig - -pytestmark = [ - pytest.mark.skipif(sys.version_info < (3, 9), reason="Google Palm requires Python 3.9 or greater"), - pytest.mark.skipif( - "Python_Integration_Tests" in os.environ, - reason="Google Palm integration tests are only set up to run locally", - ), -] - - -@pytest.mark.asyncio -async def test_gp_chat_service_with_plugins(setup_tldr_function_for_oai_models): - kernel, prompt, text_to_summarize = setup_tldr_function_for_oai_models - - print("* Service: Google PaLM Chat Completion") - print("* Model: chat-bison-001") - model_id = "models/chat-bison-001" - palm_chat_completion = sk_gp.GooglePalmChatCompletion(ai_model_id=model_id) - kernel.add_service(palm_chat_completion) - - exec_settings = PromptExecutionSettings( - service_id=model_id, extension_data={"max_tokens": 200, "temperature": 0, "top_p": 0.5} - ) - - prompt_template_config = PromptTemplateConfig(template=prompt, execution_settings=exec_settings) - - # Create the semantic function - tldr_function = kernel.add_function( - function_name="tldr", plugin_name="plugin", prompt_template_config=prompt_template_config - ) - - arguments = KernelArguments(input=text_to_summarize) - - summary = await retry(lambda: kernel.invoke(tldr_function, arguments)) - output = str(summary).strip() - print(f"TLDR using input string: '{output}'") - assert len(output) > 0 diff --git a/python/tests/integration/completions/test_gp_text_service.py b/python/tests/integration/completions/test_gp_text_service.py deleted file mode 100644 index bf7147922e3b..000000000000 --- a/python/tests/integration/completions/test_gp_text_service.py +++ /dev/null @@ -1,54 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import os -import sys - -import pytest - -from semantic_kernel.functions.kernel_arguments import KernelArguments - -pytestmark = [ - pytest.mark.skipif(sys.version_info < (3, 9), reason="Google Palm requires Python 3.9 or greater"), - pytest.mark.skipif( - "Python_Integration_Tests" in os.environ, - reason="Google Palm integration tests are only set up to run locally", - ), -] - - -@pytest.mark.asyncio -async def test_text2text_generation_input_str(setup_gp_text_completion_function): - kernel, text2text_function, simple_input = setup_gp_text_completion_function - - arguments = KernelArguments(input=simple_input, input2="") - - # Complete input string and print - summary = await kernel.invoke(text2text_function, arguments) - - output = str(summary).strip() - print(f"Completion using input string: '{output}'") - assert len(output) > 0 - - -@pytest.mark.asyncio -async def test_text2text_generation_empty_input_arguments(setup_gp_text_completion_function): - kernel, text2text_function, simple_input = setup_gp_text_completion_function - - arguments = KernelArguments(input=simple_input, input2="") - summary = await kernel.invoke(text2text_function, arguments) - - output = str(summary).strip() - print(f"Completion using arguments: '{output}'") - assert len(output) > 0 - - -@pytest.mark.asyncio -async def test_text2text_generation_input_arguments_provided(setup_gp_text_completion_function): - kernel, text2text_function, simple_input = setup_gp_text_completion_function - - arguments = KernelArguments(input=simple_input, input2="running and") - summary = await kernel.invoke(text2text_function, arguments) - - output = str(summary).strip() - print(f"Completion using input arguments: '{output}'") - assert len(output) > 0 diff --git a/python/tests/integration/completions/test_hf_local_text_completions.py b/python/tests/integration/completions/test_hf_local_text_completions.py deleted file mode 100644 index 60e664085d41..000000000000 --- a/python/tests/integration/completions/test_hf_local_text_completions.py +++ /dev/null @@ -1,77 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import pytest - -import semantic_kernel.connectors.ai.hugging_face as sk_hf -from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings -from semantic_kernel.functions.kernel_arguments import KernelArguments -from semantic_kernel.kernel import Kernel -from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig - - -@pytest.mark.asyncio -@pytest.mark.parametrize( - ("model_name", "task", "input_str"), - [ - ( - "patrickvonplaten/t5-tiny-random", - "text2text-generation", - "translate English to Dutch: Hello, how are you?", - ), - ( - "jotamunz/billsum_tiny_summarization", - "summarization", - """ - Summarize: Whales are fully aquatic, open-ocean animals: - they can feed, mate, give birth, suckle and raise their young at sea. - Whales range in size from the 2.6 metres (8.5 ft) and 135 kilograms (298 lb) - dwarf sperm whale to the 29.9 metres (98 ft) and 190 tonnes (210 short tons) blue whale, - which is the largest known animal that has ever lived. The sperm whale is the largest - toothed predator on Earth. Several whale species exhibit sexual dimorphism, - in that the females are larger than males. - """, - ), - ("HuggingFaceM4/tiny-random-LlamaForCausalLM", "text-generation", "Hello, I like sleeping and "), - ], - ids=["text2text-generation", "summarization", "text-generation"], -) -async def test_text_completion(model_name, task, input_str): - kernel = Kernel() - - # Configure LLM service - kernel.add_service( - service=sk_hf.HuggingFaceTextCompletion(service_id=model_name, ai_model_id=model_name, task=task), - ) - - exec_settings = PromptExecutionSettings(service_id=model_name, extension_data={"max_new_tokens": 25}) - - # Define semantic function using SK prompt template language - prompt = "{{$input}}" - - prompt_template_config = PromptTemplateConfig(template=prompt, execution_settings=exec_settings) - - kernel.add_function( - prompt_template_config=prompt_template_config, - function_name="TestFunction", - plugin_name="TestPlugin", - prompt_execution_settings=exec_settings, - ) - - arguments = KernelArguments(input=input_str) - - try: - summary = await kernel.invoke(function_name="TestFunction", plugin_name="TestPlugin", arguments=arguments) - except Exception as e: - pytest.xfail(f"Failed to complete invoke: {e}, skipping or now...") - output = str(summary).strip() - try: - assert len(output) > 0 - except AssertionError: - pytest.xfail("The output is empty, but completed invoke") - - stream_summary = "" - async for text in kernel.invoke_stream(function_name="TestFunction", plugin_name="TestPlugin", arguments=arguments): - stream_summary += str(text[0]) - - stream_output = str(stream_summary).strip() - assert len(stream_output) > 0 diff --git a/python/tests/integration/completions/test_oai_text_service.py b/python/tests/integration/completions/test_oai_text_service.py deleted file mode 100644 index 0c2df6baad9e..000000000000 --- a/python/tests/integration/completions/test_oai_text_service.py +++ /dev/null @@ -1,126 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import pytest -from openai import AsyncOpenAI -from test_utils import retry - -import semantic_kernel.connectors.ai.open_ai as sk_oai -from semantic_kernel.connectors.ai.open_ai.settings.open_ai_settings import OpenAISettings -from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings -from semantic_kernel.functions.kernel_arguments import KernelArguments -from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig - - -@pytest.mark.asyncio -async def test_oai_text_completion_with_plugins(setup_tldr_function_for_oai_models): - kernel, prompt, text_to_summarize = setup_tldr_function_for_oai_models - - kernel.add_service( - sk_oai.OpenAITextCompletion(service_id="text-completion", ai_model_id="gpt-3.5-turbo-instruct"), - ) - - exec_settings = PromptExecutionSettings( - service_id="text-completion", extension_data={"max_tokens": 200, "temperature": 0, "top_p": 0.5} - ) - - prompt_template_config = PromptTemplateConfig( - template=prompt, description="Write a short story.", execution_settings=exec_settings - ) - - # Create the semantic function - tldr_function = kernel.add_function( - function_name="story", plugin_name="plugin", prompt_template_config=prompt_template_config - ) - - arguments = KernelArguments(input=text_to_summarize) - - summary = await retry(lambda: kernel.invoke(tldr_function, arguments)) - output = str(summary).strip() - print(f"TLDR using input string: '{output}'") - # assert "First Law" not in output and ("human" in output or "Human" in output or "preserve" in output) - assert 0 < len(output) < 100 - - -@pytest.mark.asyncio -async def test_oai_text_completion_with_plugins_with_provided_client(setup_tldr_function_for_oai_models): - kernel, prompt, text_to_summarize = setup_tldr_function_for_oai_models - - openai_settings = OpenAISettings.create() - api_key = openai_settings.api_key.get_secret_value() - org_id = openai_settings.org_id - - client = AsyncOpenAI( - api_key=api_key, - organization=org_id, - ) - - kernel.add_service( - sk_oai.OpenAITextCompletion( - service_id="text-completion", - ai_model_id="gpt-3.5-turbo-instruct", - async_client=client, - ), - overwrite=True, - ) - - exec_settings = PromptExecutionSettings( - service_id="text-completion", extension_data={"max_tokens": 200, "temperature": 0, "top_p": 0.5} - ) - - prompt_template_config = PromptTemplateConfig( - template=prompt, description="Write a short story.", execution_settings=exec_settings - ) - - # Create the semantic function - tldr_function = kernel.add_function( - function_name="story", - plugin_name="plugin", - prompt_template_config=prompt_template_config, - ) - - arguments = KernelArguments(input=text_to_summarize) - - summary = await retry(lambda: kernel.invoke(tldr_function, arguments)) - output = str(summary).strip() - print(f"TLDR using input string: '{output}'") - # assert "First Law" not in output and ("human" in output or "Human" in output or "preserve" in output) - assert 0 < len(output) < 100 - - -@pytest.mark.asyncio -async def test_azure_oai_text_stream_completion_with_plugins(setup_tldr_function_for_oai_models): - kernel, prompt, text_to_summarize = setup_tldr_function_for_oai_models - - # Configure LLM service - kernel.add_service( - sk_oai.AzureTextCompletion( - service_id="text_completion", - ), - ) - - # Create the semantic function - exec_settings = PromptExecutionSettings( - service_id="text_completion", extension_data={"max_tokens": 200, "temperature": 0, "top_p": 0.5} - ) - - prompt_template_config = PromptTemplateConfig( - template=prompt, description="Write a short story.", execution_settings=exec_settings - ) - - # Create the semantic function - tldr_function = kernel.add_function( - function_name="story", - plugin_name="plugin", - prompt_template_config=prompt_template_config, - ) - - arguments = KernelArguments(input=text_to_summarize) - - result = None - async for message in kernel.invoke_stream(tldr_function, arguments): - result = message[0] if not result else result + message[0] - output = str(result) - - print(f"TLDR using input string: '{output}'") - # assert "First Law" not in output and ("human" in output or "Human" in output or "preserve" in output) - assert 0 < len(output) < 100 diff --git a/python/tests/integration/completions/test_text_completion.py b/python/tests/integration/completions/test_text_completion.py new file mode 100644 index 000000000000..83de8ce0107c --- /dev/null +++ b/python/tests/integration/completions/test_text_completion.py @@ -0,0 +1,166 @@ +# Copyright (c) Microsoft. All rights reserved. + +from functools import partial, reduce +from typing import Any + +import pytest + +from semantic_kernel import Kernel +from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase +from semantic_kernel.connectors.ai.hugging_face.hf_prompt_execution_settings import HuggingFacePromptExecutionSettings +from semantic_kernel.connectors.ai.hugging_face.services.hf_text_completion import HuggingFaceTextCompletion +from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( + OpenAITextPromptExecutionSettings, +) +from semantic_kernel.connectors.ai.open_ai.services.azure_text_completion import AzureTextCompletion +from semantic_kernel.connectors.ai.open_ai.services.open_ai_text_completion import OpenAITextCompletion +from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings +from semantic_kernel.connectors.ai.text_completion_client_base import TextCompletionClientBase +from semantic_kernel.contents import TextContent +from tests.integration.completions.test_utils import retry + + +def setup( + kernel: Kernel, + service: str, + execution_settings_kwargs: dict[str, Any], + services: dict[str, tuple[TextCompletionClientBase, type[PromptExecutionSettings]]], +): + kernel.add_service(services[service][0]) + kernel.add_function( + function_name="text", + plugin_name="text", + prompt="If someone asks how you are, always include the word 'well', " + "if you get a direct question, answer the question. {{$input}}", + prompt_execution_settings=services[service][1](**execution_settings_kwargs), + ) + + +@pytest.fixture(scope="module") +def services() -> dict[str, tuple[ChatCompletionClientBase, type[PromptExecutionSettings]]]: + return { + "openai": (OpenAITextCompletion(), OpenAITextPromptExecutionSettings), + "azure": (AzureTextCompletion(), OpenAITextPromptExecutionSettings), + "hf_t2t": ( + HuggingFaceTextCompletion( + service_id="patrickvonplaten/t5-tiny-random", + ai_model_id="patrickvonplaten/t5-tiny-random", + task="text2text-generation", + ), + HuggingFacePromptExecutionSettings, + ), + "hf_summ": ( + HuggingFaceTextCompletion( + service_id="jotamunz/billsum_tiny_summarization", + ai_model_id="jotamunz/billsum_tiny_summarization", + task="summarization", + ), + HuggingFacePromptExecutionSettings, + ), + "hf_gen": ( + HuggingFaceTextCompletion( + service_id="HuggingFaceM4/tiny-random-LlamaForCausalLM", + ai_model_id="HuggingFaceM4/tiny-random-LlamaForCausalLM", + task="text-generation", + ), + HuggingFacePromptExecutionSettings, + ), + } + + +pytestmark = pytest.mark.parametrize( + "service, execution_settings_kwargs, inputs, outputs", + [ + pytest.param( + "openai", + {}, + ["Repeat the word Hello"], + ["Hello"], + id="openai_text_input", + ), + pytest.param( + "azure", + {}, + ["Repeat the word Hello"], + ["Hello"], + id="azure_text_input", + ), + pytest.param( + "hf_t2t", + {}, + ["translate English to Dutch: Hello"], + [""], + id="hf_t2", + ), + pytest.param( + "hf_summ", + {}, + [ + """Summarize: Whales are fully aquatic, open-ocean animals: + they can feed, mate, give birth, suckle and raise their young at sea. + Whales range in size from the 2.6 metres (8.5 ft) and 135 kilograms (298 lb) + dwarf sperm whale to the 29.9 metres (98 ft) and 190 tonnes (210 short tons) blue whale, + which is the largest known animal that has ever lived. The sperm whale is the largest + toothed predator on Earth. Several whale species exhibit sexual dimorphism, + in that the females are larger than males.""" + ], + ["whales"], + id="hf_summ", + ), + pytest.param( + "hf_gen", + {}, + ["Hello, I like sleeping and "], + [""], + id="hf_gen", + ), + ], +) + + +@pytest.mark.asyncio(scope="module") +async def test_text_completion( + kernel: Kernel, + service: str, + execution_settings_kwargs: dict[str, Any], + inputs: list[str], + outputs: list[str], + services: dict[str, tuple[TextCompletionClientBase, type[PromptExecutionSettings]]], +): + setup(kernel, service, execution_settings_kwargs, services) + for message, output in zip(inputs, outputs): + await retry(partial(execute_invoke, kernel=kernel, input=message, output=output, stream=False), retries=5) + + +@pytest.mark.asyncio(scope="module") +async def test_streaming_text_completion( + kernel: Kernel, + service: str, + execution_settings_kwargs: dict[str, Any], + inputs: list[str], + outputs: list[str], + services: dict[str, tuple[ChatCompletionClientBase, type[PromptExecutionSettings]]], +): + setup(kernel, service, execution_settings_kwargs, services) + for message, output in zip(inputs, outputs): + await retry(partial(execute_invoke, kernel=kernel, input=message, output=output, stream=True), retries=5) + + +async def execute_invoke(kernel: Kernel, input: str, output: str, stream: bool) -> None: + if stream: + invocation = kernel.invoke_stream(function_name="text", plugin_name="text", input=input) + parts = [part[0] async for part in invocation] + if parts: + response = reduce(lambda p, r: p + r, parts) + else: + raise AssertionError("No response") + else: + invocation = await kernel.invoke(function_name="text", plugin_name="text", input=input) + assert invocation is not None + response = invocation.value[0] + print(response) + if isinstance(response, TextContent): + assert response.text is not None + assert output in response.text + return + raise AssertionError(f"Unexpected output: response: {invocation}, type: {type(invocation)}") diff --git a/python/tests/integration/embeddings/test_gp_embedding_service.py b/python/tests/integration/embeddings/test_gp_embedding_service.py deleted file mode 100644 index 11ff97a6be32..000000000000 --- a/python/tests/integration/embeddings/test_gp_embedding_service.py +++ /dev/null @@ -1,37 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import os -import sys - -import pytest - -import semantic_kernel as sk -import semantic_kernel.connectors.ai.google_palm as sk_gp -from semantic_kernel.core_plugins.text_memory_plugin import TextMemoryPlugin -from semantic_kernel.kernel import Kernel -from semantic_kernel.memory.semantic_text_memory import SemanticTextMemory - -pytestmark = [ - pytest.mark.skipif(sys.version_info < (3, 9), reason="Google Palm requires Python 3.9 or greater"), - pytest.mark.skipif( - "Python_Integration_Tests" in os.environ, - reason="Google Palm integration tests are only set up to run locally", - ), -] - - -@pytest.mark.asyncio -async def test_gp_embedding_service(kernel: Kernel): - palm_text_embed = sk_gp.GooglePalmTextEmbedding("models/embedding-gecko-001") - kernel.add_service(palm_text_embed) - - memory = SemanticTextMemory(storage=sk.memory.VolatileMemoryStore(), embeddings_generator=palm_text_embed) - kernel.add_plugin(TextMemoryPlugin(memory), "TextMemoryPlugin") - - await memory.save_information(collection="generic", id="info1", text="My budget for 2024 is $100,000") - await memory.save_reference( - "test", - external_id="info1", - text="this is a test", - external_source_name="external source", - ) diff --git a/python/tests/samples/test_samples_utils.py b/python/tests/samples/samples_utils.py similarity index 100% rename from python/tests/samples/test_samples_utils.py rename to python/tests/samples/samples_utils.py diff --git a/python/tests/samples/test_concepts.py b/python/tests/samples/test_concepts.py index 3692f3761916..fabc3934d9cd 100644 --- a/python/tests/samples/test_concepts.py +++ b/python/tests/samples/test_concepts.py @@ -6,6 +6,12 @@ main as azure_python_code_interpreter_function_calling, ) from samples.concepts.auto_function_calling.chat_gpt_api_function_calling import main as chat_gpt_api_function_calling +from samples.concepts.auto_function_calling.functions_defined_in_json_prompt import ( + main as function_defined_in_json_prompt, +) +from samples.concepts.auto_function_calling.functions_defined_in_yaml_prompt import ( + main as function_defined_in_yaml_prompt, +) from samples.concepts.chat_completion.azure_chat_gpt_api import main as azure_chat_gpt_api from samples.concepts.chat_completion.azure_chat_image_input import main as azure_chat_image_input from samples.concepts.chat_completion.chat_gpt_api import main as chat_gpt_api @@ -40,7 +46,7 @@ from samples.concepts.rag.rag_with_text_memory_plugin import main as rag_with_text_memory_plugin from samples.concepts.search.bing_search_plugin import main as bing_search_plugin from samples.concepts.service_selector.custom_service_selector import main as custom_service_selector -from tests.samples.test_samples_utils import retry +from tests.samples.samples_utils import retry concepts = [ param( @@ -48,12 +54,12 @@ ["print('Hello, World!')", "exit"], id="azure_python_code_interpreter_function_calling", ), - param(chat_gpt_api_function_calling, ["What is 3+3?", "exit"], id="cht_gpt_api_function_calling"), + param(chat_gpt_api_function_calling, ["What is 3+3?", "exit"], id="chat_gpt_api_function_calling"), param(azure_chat_gpt_api, ["Why is the sky blue?", "exit"], id="azure_chat_gpt_api"), param(chat_gpt_api, ["What is life?", "exit"], id="chat_gpt_api"), param(chat_streaming, ["Why is the sun hot?", "exit"], id="chat_streaming"), param(openai_logit_bias, [], id="openai_logit_bias"), - param(auto_function_invoke_filters, ["What is 3+3?", "exit"], id="auo_function_invoke_filters"), + param(auto_function_invoke_filters, ["What is 3+3?", "exit"], id="auto_function_invoke_filters"), param(function_invocation_filters, ["What is 3+3?", "exit"], id="function_invocation_filters"), param(function_invocation_filters_stream, ["What is 3+3?", "exit"], id="function_invocation_filters_stream"), param(prompt_filters, ["What is the fastest animal?", "exit"], id="prompt_filters"), @@ -81,6 +87,8 @@ param(bing_search_plugin, [], id="bing_search_plugin"), param(azure_chat_image_input, [], id="azure_chat_image_input"), param(custom_service_selector, [], id="custom_service_selector"), + param(function_defined_in_json_prompt, ["What is 3+3?", "exit"], id="function_defined_in_json_prompt"), + param(function_defined_in_yaml_prompt, ["What is 3+3?", "exit"], id="function_defined_in_yaml_prompt"), ] diff --git a/python/tests/samples/test_learn_resources.py b/python/tests/samples/test_learn_resources.py index 82a6e5d00175..58e1f4c3371b 100644 --- a/python/tests/samples/test_learn_resources.py +++ b/python/tests/samples/test_learn_resources.py @@ -12,7 +12,7 @@ from samples.learn_resources.templates import main as templates from samples.learn_resources.using_the_kernel import main as using_the_kernel from samples.learn_resources.your_first_prompt import main as your_first_prompt -from tests.samples.test_samples_utils import retry +from tests.samples.samples_utils import retry @mark.asyncio diff --git a/python/tests/unit/connectors/azure_ai_inference/conftest.py b/python/tests/unit/connectors/azure_ai_inference/conftest.py new file mode 100644 index 000000000000..e7b3cc4ff345 --- /dev/null +++ b/python/tests/unit/connectors/azure_ai_inference/conftest.py @@ -0,0 +1,79 @@ +# Copyright (c) Microsoft. All rights reserved. + +import pytest +from azure.ai.inference.aio import ChatCompletionsClient, EmbeddingsClient +from azure.core.credentials import AzureKeyCredential + +from semantic_kernel.connectors.ai.azure_ai_inference import ( + AzureAIInferenceChatCompletion, + AzureAIInferenceTextEmbedding, +) + + +@pytest.fixture() +def model_id() -> str: + return "test_model_id" + + +@pytest.fixture() +def service_id() -> str: + return "test_service_id" + + +@pytest.fixture() +def azure_ai_inference_unit_test_env(monkeypatch, exclude_list, override_env_param_dict): + """Fixture to set environment variables for Azure AI Inference Unit Tests.""" + if exclude_list is None: + exclude_list = [] + + if override_env_param_dict is None: + override_env_param_dict = {} + + env_vars = { + "AZURE_AI_INFERENCE_API_KEY": "test-api-key", + "AZURE_AI_INFERENCE_ENDPOINT": "https://test-endpoint.com", + } + + env_vars.update(override_env_param_dict) + + for key, value in env_vars.items(): + if key not in exclude_list: + monkeypatch.setenv(key, value) + else: + monkeypatch.delenv(key, raising=False) + + return env_vars + + +@pytest.fixture(scope="function") +def azure_ai_inference_client(azure_ai_inference_unit_test_env, request) -> ChatCompletionsClient | EmbeddingsClient: + """Fixture to create Azure AI Inference client for unit tests.""" + endpoint = azure_ai_inference_unit_test_env["AZURE_AI_INFERENCE_ENDPOINT"] + api_key = azure_ai_inference_unit_test_env["AZURE_AI_INFERENCE_API_KEY"] + credential = AzureKeyCredential(api_key) + + if request.param == AzureAIInferenceChatCompletion.__name__: + return ChatCompletionsClient(endpoint=endpoint, credential=credential) + if request.param == AzureAIInferenceTextEmbedding.__name__: + return EmbeddingsClient(endpoint=endpoint, credential=credential) + + raise ValueError(f"Service {request.param} not supported.") + + +@pytest.fixture(scope="function") +def azure_ai_inference_service(azure_ai_inference_unit_test_env, model_id, request): + """Fixture to create Azure AI Inference service for unit tests. + + This is required because the Azure AI Inference services require a client to be created, + and the client will be talking to the endpoint at creation time. + """ + + endpoint = azure_ai_inference_unit_test_env["AZURE_AI_INFERENCE_ENDPOINT"] + api_key = azure_ai_inference_unit_test_env["AZURE_AI_INFERENCE_API_KEY"] + + if request.param == AzureAIInferenceChatCompletion.__name__: + return AzureAIInferenceChatCompletion(model_id, api_key=api_key, endpoint=endpoint) + if request.param == AzureAIInferenceTextEmbedding.__name__: + return AzureAIInferenceTextEmbedding(model_id, api_key=api_key, endpoint=endpoint) + + raise ValueError(f"Service {request.param} not supported.") diff --git a/python/tests/unit/connectors/azure_ai_inference/services/test_azure_ai_inference_chat_completion.py b/python/tests/unit/connectors/azure_ai_inference/services/test_azure_ai_inference_chat_completion.py new file mode 100644 index 000000000000..98fb82b70440 --- /dev/null +++ b/python/tests/unit/connectors/azure_ai_inference/services/test_azure_ai_inference_chat_completion.py @@ -0,0 +1,158 @@ +# Copyright (c) Microsoft. All rights reserved. + +from unittest.mock import AsyncMock, patch + +import pytest +from azure.ai.inference.aio import ChatCompletionsClient +from azure.ai.inference.models import UserMessage + +from semantic_kernel.connectors.ai.azure_ai_inference import ( + AzureAIInferenceChatCompletion, + AzureAIInferenceChatPromptExecutionSettings, +) +from semantic_kernel.contents.chat_history import ChatHistory +from semantic_kernel.exceptions.service_exceptions import ServiceInitializationError + + +def test_azure_ai_inference_chat_completion_init(azure_ai_inference_unit_test_env, model_id) -> None: + azure_ai_inference = AzureAIInferenceChatCompletion(model_id) + + assert azure_ai_inference.ai_model_id == model_id + assert azure_ai_inference.service_id == model_id + assert isinstance(azure_ai_inference.client, ChatCompletionsClient) + + +def test_azure_ai_inference_chat_completion_init_with_service_id( + azure_ai_inference_unit_test_env, model_id, service_id +) -> None: + azure_ai_inference = AzureAIInferenceChatCompletion(model_id, service_id=service_id) + + assert azure_ai_inference.ai_model_id == model_id + assert azure_ai_inference.service_id == service_id + assert isinstance(azure_ai_inference.client, ChatCompletionsClient) + + +@pytest.mark.parametrize( + "azure_ai_inference_client", + [AzureAIInferenceChatCompletion.__name__], + indirect=True, +) +def test_azure_ai_inference_chat_completion_init_with_custom_client(azure_ai_inference_client, model_id) -> None: + """Test initialization of AzureAIInferenceChatCompletion with custom client""" + client = azure_ai_inference_client + azure_ai_inference = AzureAIInferenceChatCompletion(model_id, client=client) + + assert azure_ai_inference.ai_model_id == model_id + assert azure_ai_inference.service_id == model_id + assert azure_ai_inference.client == client + + +@pytest.mark.parametrize("exclude_list", [["AZURE_AI_INFERENCE_API_KEY"]], indirect=True) +def test_azure_ai_inference_chat_completion_init_with_empty_api_key(azure_ai_inference_unit_test_env, model_id) -> None: + """Test initialization of AzureAIInferenceChatCompletion with empty API key""" + with pytest.raises(ServiceInitializationError): + AzureAIInferenceChatCompletion(model_id) + + +@pytest.mark.parametrize("exclude_list", [["AZURE_AI_INFERENCE_ENDPOINT"]], indirect=True) +def test_azure_ai_inference_chat_completion_init_with_empty_endpoint( + azure_ai_inference_unit_test_env, model_id +) -> None: + """Test initialization of AzureAIInferenceChatCompletion with empty endpoint""" + with pytest.raises(ServiceInitializationError): + AzureAIInferenceChatCompletion(model_id) + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + "azure_ai_inference_service", + [AzureAIInferenceChatCompletion.__name__], + indirect=True, +) +@patch.object(ChatCompletionsClient, "complete", new_callable=AsyncMock) +async def test_azure_ai_inference_chat_completion( + mock_complete, + azure_ai_inference_service, + chat_history: ChatHistory, +) -> None: + """Test completion of AzureAIInferenceChatCompletion""" + user_message_content: str = "Hello" + chat_history.add_user_message(user_message_content) + settings = AzureAIInferenceChatPromptExecutionSettings() + + await azure_ai_inference_service.get_chat_message_contents(chat_history, settings) + + mock_complete.assert_awaited_once_with( + messages=[UserMessage(content=user_message_content)], + model_extras=None, + **settings.prepare_settings_dict(), + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + "azure_ai_inference_service", + [AzureAIInferenceChatCompletion.__name__], + indirect=True, +) +@patch.object(ChatCompletionsClient, "complete", new_callable=AsyncMock) +async def test_azure_ai_inference_chat_completion_with_standard_parameters( + mock_complete, + azure_ai_inference_service, + chat_history: ChatHistory, +) -> None: + """Test completion of AzureAIInferenceChatCompletion with standard OpenAI parameters""" + user_message_content: str = "Hello" + chat_history.add_user_message(user_message_content) + + settings = AzureAIInferenceChatPromptExecutionSettings( + frequency_penalty=0.5, + max_tokens=100, + presence_penalty=0.5, + seed=123, + stop="stop", + temperature=0.5, + top_p=0.5, + ) + + await azure_ai_inference_service.get_chat_message_contents(chat_history, settings) + + mock_complete.assert_awaited_once_with( + messages=[UserMessage(content=user_message_content)], + model_extras=None, + frequency_penalty=settings.frequency_penalty, + max_tokens=settings.max_tokens, + presence_penalty=settings.presence_penalty, + seed=settings.seed, + stop=settings.stop, + temperature=settings.temperature, + top_p=settings.top_p, + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + "azure_ai_inference_service", + [AzureAIInferenceChatCompletion.__name__], + indirect=True, +) +@patch.object(ChatCompletionsClient, "complete", new_callable=AsyncMock) +async def test_azure_ai_inference_chat_completion_with_extra_parameters( + mock_complete, + azure_ai_inference_service, + chat_history: ChatHistory, +) -> None: + """Test completion of AzureAIInferenceChatCompletion with extra parameters""" + user_message_content: str = "Hello" + chat_history.add_user_message(user_message_content) + extra_parameters = {"test_key": "test_value"} + + settings = AzureAIInferenceChatPromptExecutionSettings(extra_parameters=extra_parameters) + + await azure_ai_inference_service.get_chat_message_contents(chat_history, settings) + + mock_complete.assert_awaited_once_with( + messages=[UserMessage(content=user_message_content)], + model_extras=extra_parameters, + **settings.prepare_settings_dict(), + ) diff --git a/python/tests/unit/connectors/azure_ai_inference/services/test_azure_ai_inference_text_embedding.py b/python/tests/unit/connectors/azure_ai_inference/services/test_azure_ai_inference_text_embedding.py new file mode 100644 index 000000000000..b31654364df8 --- /dev/null +++ b/python/tests/unit/connectors/azure_ai_inference/services/test_azure_ai_inference_text_embedding.py @@ -0,0 +1,141 @@ +# Copyright (c) Microsoft. All rights reserved. + +from unittest.mock import AsyncMock, patch + +import pytest +from azure.ai.inference.aio import EmbeddingsClient + +from semantic_kernel.connectors.ai.azure_ai_inference import ( + AzureAIInferenceEmbeddingPromptExecutionSettings, + AzureAIInferenceTextEmbedding, +) +from semantic_kernel.exceptions.service_exceptions import ServiceInitializationError + + +def test_azure_ai_inference_text_embedding_init(azure_ai_inference_unit_test_env, model_id) -> None: + """Test initialization of AzureAIInferenceTextEmbedding""" + azure_ai_inference = AzureAIInferenceTextEmbedding(model_id) + + assert azure_ai_inference.ai_model_id == model_id + assert azure_ai_inference.service_id == model_id + assert isinstance(azure_ai_inference.client, EmbeddingsClient) + + +def test_azure_ai_inference_text_embedding_init_with_service_id( + azure_ai_inference_unit_test_env, model_id, service_id +) -> None: + """Test initialization of AzureAIInferenceTextEmbedding""" + azure_ai_inference = AzureAIInferenceTextEmbedding(model_id, service_id=service_id) + + assert azure_ai_inference.ai_model_id == model_id + assert azure_ai_inference.service_id == service_id + assert isinstance(azure_ai_inference.client, EmbeddingsClient) + + +@pytest.mark.parametrize( + "azure_ai_inference_client", + [AzureAIInferenceTextEmbedding.__name__], + indirect=True, +) +def test_azure_ai_inference_chat_completion_init_with_custom_client(azure_ai_inference_client, model_id) -> None: + """Test initialization of AzureAIInferenceTextEmbedding with custom client""" + client = azure_ai_inference_client + azure_ai_inference = AzureAIInferenceTextEmbedding(model_id, client=client) + + assert azure_ai_inference.ai_model_id == model_id + assert azure_ai_inference.service_id == model_id + assert azure_ai_inference.client == client + + +@pytest.mark.parametrize("exclude_list", [["AZURE_AI_INFERENCE_API_KEY"]], indirect=True) +def test_azure_ai_inference_text_embedding_init_with_empty_api_key(azure_ai_inference_unit_test_env, model_id) -> None: + """Test initialization of AzureAIInferenceTextEmbedding with empty API key""" + with pytest.raises(ServiceInitializationError): + AzureAIInferenceTextEmbedding(model_id) + + +@pytest.mark.parametrize("exclude_list", [["AZURE_AI_INFERENCE_ENDPOINT"]], indirect=True) +def test_azure_ai_inference_text_embedding_init_with_empty_endpoint(azure_ai_inference_unit_test_env, model_id) -> None: + """Test initialization of AzureAIInferenceTextEmbedding with empty endpoint""" + with pytest.raises(ServiceInitializationError): + AzureAIInferenceTextEmbedding(model_id) + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + "azure_ai_inference_service", + [AzureAIInferenceTextEmbedding.__name__], + indirect=True, +) +@patch.object(EmbeddingsClient, "embed", new_callable=AsyncMock) +async def test_azure_ai_inference_text_embedding( + mock_embed, + azure_ai_inference_service, +) -> None: + """Test text embedding generation of AzureAIInferenceTextEmbedding without settings""" + texts = ["hello", "world"] + await azure_ai_inference_service.generate_embeddings(texts) + + mock_embed.assert_awaited_once_with( + input=texts, + model_extras=None, + dimensions=None, + encoding_format=None, + input_type=None, + kwargs={}, + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + "azure_ai_inference_service", + [AzureAIInferenceTextEmbedding.__name__], + indirect=True, +) +@patch.object(EmbeddingsClient, "embed", new_callable=AsyncMock) +async def test_azure_ai_inference_text_embedding_with_standard_settings( + mock_embed, + azure_ai_inference_service, +) -> None: + """Test text embedding generation of AzureAIInferenceTextEmbedding with standard settings""" + texts = ["hello", "world"] + settings = AzureAIInferenceEmbeddingPromptExecutionSettings( + dimensions=1024, encoding_format="float", input_type="text" + ) + await azure_ai_inference_service.generate_embeddings(texts, settings=settings) + + mock_embed.assert_awaited_once_with( + input=texts, + model_extras=None, + dimensions=settings.dimensions, + encoding_format=settings.encoding_format, + input_type=settings.input_type, + kwargs={"settings": settings}, + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + "azure_ai_inference_service", + [AzureAIInferenceTextEmbedding.__name__], + indirect=True, +) +@patch.object(EmbeddingsClient, "embed", new_callable=AsyncMock) +async def test_azure_ai_inference_text_embedding_with_extra_parameters( + mock_embed, + azure_ai_inference_service, +) -> None: + """Test text embedding generation of AzureAIInferenceTextEmbedding with extra parameters""" + texts = ["hello", "world"] + extra_parameters = {"test_key": "test_value"} + settings = AzureAIInferenceEmbeddingPromptExecutionSettings(extra_parameters=extra_parameters) + await azure_ai_inference_service.generate_embeddings(texts, settings=settings) + + mock_embed.assert_awaited_once_with( + input=texts, + model_extras=extra_parameters, + dimensions=settings.dimensions, + encoding_format=settings.encoding_format, + input_type=settings.input_type, + kwargs={"settings": settings}, + ) diff --git a/python/tests/unit/connectors/google_palm/services/test_palm_chat_completion.py b/python/tests/unit/connectors/google_palm/services/test_palm_chat_completion.py deleted file mode 100644 index 99cb9e6b9c43..000000000000 --- a/python/tests/unit/connectors/google_palm/services/test_palm_chat_completion.py +++ /dev/null @@ -1,75 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -import asyncio -from unittest.mock import MagicMock, patch - -import pytest -from google.generativeai.types import ChatResponse, MessageDict - -from semantic_kernel.connectors.ai.google_palm import GooglePalmChatPromptExecutionSettings -from semantic_kernel.connectors.ai.google_palm.services.gp_chat_completion import GooglePalmChatCompletion -from semantic_kernel.contents.chat_history import ChatHistory -from semantic_kernel.exceptions.service_exceptions import ServiceInitializationError - - -def test_google_palm_chat_completion_init(google_palm_unit_test_env) -> None: - ai_model_id = "test_model_id" - - gp_chat_completion = GooglePalmChatCompletion(ai_model_id=ai_model_id) - - assert gp_chat_completion.ai_model_id == ai_model_id - assert gp_chat_completion.api_key == google_palm_unit_test_env["GOOGLE_PALM_API_KEY"] - assert isinstance(gp_chat_completion, GooglePalmChatCompletion) - - -@pytest.mark.parametrize("exclude_list", [["GOOGLE_PALM_API_KEY"]], indirect=True) -def test_google_palm_chat_completion_init_with_empty_api_key(google_palm_unit_test_env) -> None: - ai_model_id = "test_model_id" - - with pytest.raises(ServiceInitializationError): - GooglePalmChatCompletion( - ai_model_id=ai_model_id, - env_file_path="test.env", - ) - - -@pytest.mark.asyncio -async def test_google_palm_text_completion_complete_chat_call_with_parameters(google_palm_unit_test_env) -> None: - class MockChatResponse(ChatResponse): - def last(self): - return "" - - def reply(self): - return self - - gp_response = MockChatResponse() - gp_response.candidates = [MessageDict(content="Example response", author=3)] - gp_response.filters = None - mock_response = MagicMock() - mock_response.last = asyncio.Future() - mock_response.last.set_result(gp_response) - mock_gp = MagicMock() - mock_gp.chat.return_value = gp_response - with patch( - "semantic_kernel.connectors.ai.google_palm.services.gp_chat_completion.palm", - new=mock_gp, - ): - ai_model_id = "test_model_id" - chats = ChatHistory() - chats.add_user_message("Hello word") - gp_chat_completion = GooglePalmChatCompletion( - ai_model_id=ai_model_id, - ) - settings = GooglePalmChatPromptExecutionSettings() - response = await gp_chat_completion.get_chat_message_contents(chats, settings) - - assert isinstance(response[0].content, str) and len(response) > 0 - print(mock_gp.chat) - mock_gp.chat.assert_called_once_with( - model=ai_model_id, - temperature=settings.temperature, - top_p=settings.top_p, - top_k=settings.top_k, - candidate_count=settings.candidate_count, - messages=[message.to_dict(role_key="author") for message in chats.messages], - ) diff --git a/python/tests/unit/connectors/google_palm/services/test_palm_text_completion.py b/python/tests/unit/connectors/google_palm/services/test_palm_text_completion.py deleted file mode 100644 index 7334f335df21..000000000000 --- a/python/tests/unit/connectors/google_palm/services/test_palm_text_completion.py +++ /dev/null @@ -1,66 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. -from unittest.mock import MagicMock, patch - -import pytest -from google.generativeai.types import Completion -from google.generativeai.types.text_types import TextCompletion - -from semantic_kernel.connectors.ai.google_palm import GooglePalmTextPromptExecutionSettings -from semantic_kernel.connectors.ai.google_palm.services.gp_text_completion import GooglePalmTextCompletion -from semantic_kernel.exceptions.service_exceptions import ServiceInitializationError - - -def test_google_palm_text_completion_init(google_palm_unit_test_env) -> None: - ai_model_id = "test_model_id" - - # Test successful initialization - gp_text_completion = GooglePalmTextCompletion( - ai_model_id=ai_model_id, - ) - - assert gp_text_completion.ai_model_id == ai_model_id - assert gp_text_completion.api_key == google_palm_unit_test_env["GOOGLE_PALM_API_KEY"] - assert isinstance(gp_text_completion, GooglePalmTextCompletion) - - -@pytest.mark.parametrize("exclude_list", [["GOOGLE_PALM_API_KEY"]], indirect=True) -def test_google_palm_text_completion_init_with_empty_api_key(google_palm_unit_test_env) -> None: - ai_model_id = "test_model_id" - - with pytest.raises(ServiceInitializationError): - GooglePalmTextCompletion( - ai_model_id=ai_model_id, - env_file_path="test.env", - ) - - -@pytest.mark.asyncio -async def test_google_palm_text_completion_complete_call_with_parameters(google_palm_unit_test_env) -> None: - gp_completion = Completion() - gp_completion.candidates = [TextCompletion(output="Example response")] - gp_completion.filters = None - gp_completion.safety_feedback = None - mock_gp = MagicMock() - mock_gp.generate_text.return_value = gp_completion - with patch( - "semantic_kernel.connectors.ai.google_palm.services.gp_text_completion.palm", - new=mock_gp, - ): - ai_model_id = "test_model_id" - prompt = "hello world" - gp_text_completion = GooglePalmTextCompletion( - ai_model_id=ai_model_id, - ) - settings = GooglePalmTextPromptExecutionSettings() - response = await gp_text_completion.get_text_contents(prompt, settings) - assert isinstance(response[0].text, str) and len(response) > 0 - - mock_gp.generate_text.assert_called_once_with( - model=ai_model_id, - prompt=prompt, - temperature=settings.temperature, - max_output_tokens=settings.max_output_tokens, - candidate_count=settings.candidate_count, - top_p=settings.top_p, - top_k=settings.top_k, - ) diff --git a/python/tests/unit/connectors/google_palm/services/test_palm_text_embedding.py b/python/tests/unit/connectors/google_palm/services/test_palm_text_embedding.py deleted file mode 100644 index c70986c2b122..000000000000 --- a/python/tests/unit/connectors/google_palm/services/test_palm_text_embedding.py +++ /dev/null @@ -1,58 +0,0 @@ -# Copyright (c) Microsoft. All rights reserved. - -from unittest.mock import MagicMock, patch - -import pytest - -from semantic_kernel.connectors.ai.google_palm.services.gp_text_embedding import ( - GooglePalmTextEmbedding, -) -from semantic_kernel.exceptions.service_exceptions import ServiceInitializationError - - -def test_google_palm_text_embedding_init(google_palm_unit_test_env) -> None: - ai_model_id = "test_model_id" - - # Test successful initialization - gp_text_embed = GooglePalmTextEmbedding( - ai_model_id=ai_model_id, - ) - - assert gp_text_embed.ai_model_id == ai_model_id - assert gp_text_embed.api_key == google_palm_unit_test_env["GOOGLE_PALM_API_KEY"] - assert isinstance(gp_text_embed, GooglePalmTextEmbedding) - - -@pytest.mark.parametrize("exclude_list", [["GOOGLE_PALM_API_KEY"]], indirect=True) -def test_google_palm_text_embedding_init_with_empty_api_key(google_palm_unit_test_env) -> None: - ai_model_id = "test_model_id" - - with pytest.raises(ServiceInitializationError): - GooglePalmTextEmbedding( - ai_model_id=ai_model_id, - env_file_path="test.env", - ) - - -@pytest.mark.asyncio -async def test_google_palm_text_embedding_calls_with_parameters(google_palm_unit_test_env) -> None: - mock_gp = MagicMock() - mock_gp.generate_embeddings.return_value = {"embedding": [0.1, 0.2, 0.3]} - with patch( - "semantic_kernel.connectors.ai.google_palm.services.gp_text_embedding.palm", - new=mock_gp, - ): - ai_model_id = "test_model_id" - texts = ["hello world"] - text = "hello world" - - gp_text_embedding = GooglePalmTextEmbedding( - ai_model_id=ai_model_id, - ) - - await gp_text_embedding.generate_embeddings(texts) - - mock_gp.generate_embeddings.assert_called_once_with( - model=ai_model_id, - text=text, - ) diff --git a/python/tests/unit/connectors/open_ai/services/test_open_ai_chat_completion_base.py b/python/tests/unit/connectors/open_ai/services/test_open_ai_chat_completion_base.py index fd360cd890f9..38ac7313a121 100644 --- a/python/tests/unit/connectors/open_ai/services/test_open_ai_chat_completion_base.py +++ b/python/tests/unit/connectors/open_ai/services/test_open_ai_chat_completion_base.py @@ -6,6 +6,7 @@ from openai import AsyncOpenAI from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.open_ai.prompt_execution_settings.open_ai_prompt_execution_settings import ( OpenAIChatPromptExecutionSettings, ) @@ -34,13 +35,16 @@ async def test_complete_chat_stream(kernel: Kernel): mock_response = MagicMock() arguments = KernelArguments() - with patch( - "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._prepare_settings", - return_value=settings, - ) as prepare_settings_mock, patch( - "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._send_chat_stream_request", - return_value=mock_response, - ) as mock_send_chat_stream_request: + with ( + patch( + "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._prepare_settings", + return_value=settings, + ) as prepare_settings_mock, + patch( + "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._send_chat_stream_request", + return_value=mock_response, + ) as mock_send_chat_stream_request, + ): chat_completion_base = OpenAIChatCompletionBase( ai_model_id="test_model_id", service_id="test", client=MagicMock(spec=AsyncOpenAI) ) @@ -56,12 +60,13 @@ async def test_complete_chat_stream(kernel: Kernel): @pytest.mark.parametrize("tool_call", [False, True]) @pytest.mark.asyncio -async def test_complete_chat(tool_call, kernel: Kernel): +async def test_complete_chat_function_call_behavior(tool_call, kernel: Kernel): chat_history = MagicMock(spec=ChatHistory) chat_history.messages = [] settings = MagicMock(spec=OpenAIChatPromptExecutionSettings) settings.number_of_responses = 1 settings.function_call_behavior = None + settings.function_choice_behavior = None mock_function_call = MagicMock(spec=FunctionCallContent) mock_text = MagicMock(spec=TextContent) mock_message = ChatMessageContent( @@ -71,19 +76,24 @@ async def test_complete_chat(tool_call, kernel: Kernel): arguments = KernelArguments() if tool_call: - settings.function_call_behavior = MagicMock(spec=FunctionCallBehavior) + settings.function_call_behavior = MagicMock(spec=FunctionCallBehavior.AutoInvokeKernelFunctions()) settings.function_call_behavior.auto_invoke_kernel_functions = True settings.function_call_behavior.max_auto_invoke_attempts = 5 chat_history.messages = [mock_message] - with patch( - "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._prepare_settings", - ) as prepare_settings_mock, patch( - "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._send_chat_request", - return_value=mock_message_content, - ) as mock_send_chat_request, patch( - "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._process_function_call", - ) as mock_process_function_call: + with ( + patch( + "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._prepare_settings", + ) as prepare_settings_mock, + patch( + "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._send_chat_request", + return_value=mock_message_content, + ) as mock_send_chat_request, + patch( + "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._process_function_call", + new_callable=AsyncMock, + ) as mock_process_function_call, + ): chat_completion_base = OpenAIChatCompletionBase( ai_model_id="test_model_id", service_id="test", client=MagicMock(spec=AsyncOpenAI) ) @@ -91,12 +101,68 @@ async def test_complete_chat(tool_call, kernel: Kernel): result = await chat_completion_base.get_chat_message_contents( chat_history, settings, kernel=kernel, arguments=arguments ) + assert result is not None + prepare_settings_mock.assert_called_with(settings, chat_history, stream_request=False, kernel=kernel) + mock_send_chat_request.assert_called_with(settings) + + if tool_call: + mock_process_function_call.assert_awaited() + else: + mock_process_function_call.assert_not_awaited() + +@pytest.mark.parametrize("tool_call", [False, True]) +@pytest.mark.asyncio +async def test_complete_chat_function_choice_behavior(tool_call, kernel: Kernel): + chat_history = MagicMock(spec=ChatHistory) + chat_history.messages = [] + settings = MagicMock(spec=OpenAIChatPromptExecutionSettings) + settings.number_of_responses = 1 + settings.function_choice_behavior = None + mock_function_call = MagicMock(spec=FunctionCallContent) + mock_text = MagicMock(spec=TextContent) + mock_message = ChatMessageContent( + role=AuthorRole.ASSISTANT, items=[mock_function_call] if tool_call else [mock_text] + ) + mock_message_content = [mock_message] + arguments = KernelArguments() + + if tool_call: + settings.function_choice_behavior = MagicMock(spec=FunctionChoiceBehavior.Auto) + settings.function_choice_behavior.auto_invoke_kernel_functions = True + settings.function_choice_behavior.maximum_auto_invoke_attempts = 5 + chat_history.messages = [mock_message] + + with ( + patch( + "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._prepare_settings", + ) as prepare_settings_mock, + patch( + "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._send_chat_request", + return_value=mock_message_content, + ) as mock_send_chat_request, + patch( + "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.OpenAIChatCompletionBase._process_function_call", + new_callable=AsyncMock, + ) as mock_process_function_call, + ): + chat_completion_base = OpenAIChatCompletionBase( + ai_model_id="test_model_id", service_id="test", client=MagicMock(spec=AsyncOpenAI) + ) + + result = await chat_completion_base.get_chat_message_contents( + chat_history, settings, kernel=kernel, arguments=arguments + ) + + assert result is not None prepare_settings_mock.assert_called_with(settings, chat_history, stream_request=False, kernel=kernel) mock_send_chat_request.assert_called_with(settings) + if tool_call: - mock_process_function_call.assert_called() + mock_process_function_call.assert_awaited() + else: + mock_process_function_call.assert_not_awaited() @pytest.mark.asyncio @@ -146,8 +212,6 @@ async def construct_call_stack(ctx): FunctionCallBehavior.AutoInvokeKernelFunctions(), ) - chat_history_mock.add_message.assert_called_once() - @pytest.mark.asyncio async def test_process_tool_calls_with_continuation_on_malformed_arguments(): @@ -179,9 +243,7 @@ async def test_process_tool_calls_with_continuation_on_malformed_arguments(): ai_model_id="test_model_id", service_id="test", client=MagicMock(spec=AsyncOpenAI) ) - with patch( - "semantic_kernel.connectors.ai.open_ai.services.open_ai_chat_completion_base.logger", autospec=True - ) as logger_mock: + with patch("semantic_kernel.connectors.ai.function_calling_utils.logger", autospec=True): await chat_completion_base._process_function_call( tool_call_mock, chat_history_mock, @@ -191,15 +253,3 @@ async def test_process_tool_calls_with_continuation_on_malformed_arguments(): 0, FunctionCallBehavior.AutoInvokeKernelFunctions(), ) - - logger_mock.info.assert_any_call( - "Received invalid arguments for function test_function: Malformed arguments. Trying tool call again." - ) - - add_message_calls = chat_history_mock.add_message.call_args_list - assert any( - call[1]["message"].items[0].result == "The tool call arguments are malformed. Arguments must be in JSON format. Please try again." # noqa: E501 - and call[1]["message"].items[0].id == "test_id" - and call[1]["message"].items[0].name == "test_function" - for call in add_message_calls - ), "Expected call to add_message not found with the expected message content and metadata." diff --git a/python/tests/unit/connectors/test_function_choice_behavior.py b/python/tests/unit/connectors/test_function_choice_behavior.py new file mode 100644 index 000000000000..ab95bbc7a11c --- /dev/null +++ b/python/tests/unit/connectors/test_function_choice_behavior.py @@ -0,0 +1,216 @@ +# Copyright (c) Microsoft. All rights reserved. + +from typing import TYPE_CHECKING +from unittest.mock import Mock + +import pytest + +if TYPE_CHECKING: + from semantic_kernel.kernel import Kernel + +from semantic_kernel.connectors.ai.function_call_behavior import FunctionCallBehavior +from semantic_kernel.connectors.ai.function_choice_behavior import ( + DEFAULT_MAX_AUTO_INVOKE_ATTEMPTS, + FunctionChoiceBehavior, + FunctionChoiceType, +) + + +@pytest.fixture +def function_choice_behavior(): + return FunctionChoiceBehavior() + + +@pytest.fixture +def update_settings_callback(): + mock = Mock() + mock.return_value = None + return mock + + +def test_function_choice_behavior_auto(): + behavior = FunctionChoiceBehavior.Auto(auto_invoke=True) + assert behavior.type == FunctionChoiceType.AUTO + assert behavior.maximum_auto_invoke_attempts == DEFAULT_MAX_AUTO_INVOKE_ATTEMPTS + + +def test_function_choice_behavior_none_invoke(): + behavior = FunctionChoiceBehavior.NoneInvoke() + assert behavior.type == FunctionChoiceType.NONE + assert behavior.maximum_auto_invoke_attempts == 0 + + +def test_function_choice_behavior_required(): + expected_filters = {"included_functions": ["plugin1-func1"]} + behavior = FunctionChoiceBehavior.Required(auto_invoke=True, filters=expected_filters) + assert behavior.type == FunctionChoiceType.REQUIRED + assert behavior.maximum_auto_invoke_attempts == 1 + assert behavior.filters == expected_filters + + +def test_from_function_call_behavior_kernel_functions(): + behavior = FunctionCallBehavior.AutoInvokeKernelFunctions() + new_behavior = FunctionChoiceBehavior.from_function_call_behavior(behavior) + assert new_behavior.type == FunctionChoiceType.AUTO + assert new_behavior.auto_invoke_kernel_functions is True + + +def test_from_function_call_behavior_enabled_functions(): + expected_filters = {"included_functions": ["plugin1-func1"]} + behavior = FunctionCallBehavior.EnableFunctions(auto_invoke=True, filters=expected_filters) + new_behavior = FunctionChoiceBehavior.from_function_call_behavior(behavior) + assert new_behavior.type == FunctionChoiceType.AUTO + assert new_behavior.auto_invoke_kernel_functions is True + assert new_behavior.filters == expected_filters + + +@pytest.mark.parametrize(("type", "max_auto_invoke_attempts"), [("auto", 5), ("none", 0), ("required", 1)]) +def test_auto_function_choice_behavior_from_dict(type: str, max_auto_invoke_attempts: int): + data = { + "type": type, + "filters": {"included_functions": ["plugin1-func1", "plugin2-func2"]}, + "maximum_auto_invoke_attempts": max_auto_invoke_attempts, + } + behavior = FunctionChoiceBehavior.from_dict(data) + assert behavior.type == FunctionChoiceType(type) + assert behavior.filters == {"included_functions": ["plugin1-func1", "plugin2-func2"]} + assert behavior.maximum_auto_invoke_attempts == max_auto_invoke_attempts + + +@pytest.mark.parametrize(("type", "max_auto_invoke_attempts"), [("auto", 5), ("none", 0), ("required", 1)]) +def test_auto_function_choice_behavior_from_dict_with_same_filters_and_functions( + type: str, max_auto_invoke_attempts: int +): + data = { + "type": type, + "filters": {"included_functions": ["plugin1-func1", "plugin2-func2"]}, + "functions": ["plugin1-func1", "plugin2-func2"], + "maximum_auto_invoke_attempts": max_auto_invoke_attempts, + } + behavior = FunctionChoiceBehavior.from_dict(data) + assert behavior.type == FunctionChoiceType(type) + assert behavior.filters == {"included_functions": ["plugin1-func1", "plugin2-func2"]} + assert behavior.maximum_auto_invoke_attempts == max_auto_invoke_attempts + + +@pytest.mark.parametrize(("type", "max_auto_invoke_attempts"), [("auto", 5), ("none", 0), ("required", 1)]) +def test_auto_function_choice_behavior_from_dict_with_different_filters_and_functions( + type: str, max_auto_invoke_attempts: int +): + data = { + "type": type, + "filters": {"included_functions": ["plugin1-func1", "plugin2-func2"]}, + "functions": ["plugin3-func3"], + "maximum_auto_invoke_attempts": max_auto_invoke_attempts, + } + behavior = FunctionChoiceBehavior.from_dict(data) + assert behavior.type == FunctionChoiceType(type) + assert behavior.filters == {"included_functions": ["plugin1-func1", "plugin2-func2", "plugin3-func3"]} + assert behavior.maximum_auto_invoke_attempts == max_auto_invoke_attempts + + +def test_function_choice_behavior_get_set(function_choice_behavior: FunctionChoiceBehavior): + function_choice_behavior.enable_kernel_functions = False + assert function_choice_behavior.enable_kernel_functions is False + function_choice_behavior.maximum_auto_invoke_attempts = 10 + assert function_choice_behavior.maximum_auto_invoke_attempts == 10 + assert function_choice_behavior.auto_invoke_kernel_functions is True + function_choice_behavior.auto_invoke_kernel_functions = False + assert function_choice_behavior.auto_invoke_kernel_functions is False + assert function_choice_behavior.maximum_auto_invoke_attempts == 0 + function_choice_behavior.auto_invoke_kernel_functions = True + assert function_choice_behavior.auto_invoke_kernel_functions is True + assert function_choice_behavior.maximum_auto_invoke_attempts == 5 + + +def test_auto_invoke_kernel_functions(): + fcb = FunctionChoiceBehavior.Auto(auto_invoke=True) + assert fcb is not None + assert fcb.enable_kernel_functions is True + assert fcb.maximum_auto_invoke_attempts == 5 + assert fcb.auto_invoke_kernel_functions is True + + +def test_none_invoke_kernel_functions(): + fcb = FunctionChoiceBehavior.NoneInvoke() + assert fcb is not None + assert fcb.enable_kernel_functions is True + assert fcb.maximum_auto_invoke_attempts == 0 + assert fcb.auto_invoke_kernel_functions is False + + +def test_enable_functions(): + fcb = FunctionChoiceBehavior.Auto(auto_invoke=True, filters={"excluded_plugins": ["test"]}) + assert fcb is not None + assert fcb.enable_kernel_functions is True + assert fcb.maximum_auto_invoke_attempts == 5 + assert fcb.auto_invoke_kernel_functions is True + assert fcb.filters == {"excluded_plugins": ["test"]} + + +def test_required_function(): + fcb = FunctionChoiceBehavior.Required(auto_invoke=True, filters={"included_functions": ["test"]}) + assert fcb is not None + assert fcb.enable_kernel_functions is True + assert fcb.maximum_auto_invoke_attempts == 1 + assert fcb.auto_invoke_kernel_functions is True + + +def test_configure_auto_invoke_kernel_functions(update_settings_callback, kernel: "Kernel"): + fcb = FunctionChoiceBehavior.Auto(auto_invoke=True) + fcb.configure(kernel, update_settings_callback, None) + assert update_settings_callback.called + + +def test_configure_auto_invoke_kernel_functions_skip(update_settings_callback, kernel: "Kernel"): + fcb = FunctionChoiceBehavior.Auto(auto_invoke=True) + fcb.enable_kernel_functions = False + fcb.configure(kernel, update_settings_callback, None) + assert not update_settings_callback.called + + +def test_configure_none_invoke_kernel_functions(update_settings_callback, kernel: "Kernel"): + fcb = FunctionChoiceBehavior.NoneInvoke() + fcb.configure(kernel, update_settings_callback, None) + assert update_settings_callback.called + + +def test_configure_none_invoke_kernel_functions_skip(update_settings_callback, kernel: "Kernel"): + fcb = FunctionChoiceBehavior.NoneInvoke() + fcb.enable_kernel_functions = False + fcb.configure(kernel, update_settings_callback, None) + assert not update_settings_callback.called + + +def test_configure_enable_functions(update_settings_callback, kernel: "Kernel"): + fcb = FunctionChoiceBehavior.Auto(auto_invoke=True, filters={"excluded_plugins": ["test"]}) + fcb.configure(kernel, update_settings_callback, None) + assert update_settings_callback.called + + +def test_configure_enable_functions_skip(update_settings_callback, kernel: "Kernel"): + fcb = FunctionChoiceBehavior.Auto(auto_invoke=True, filters={"excluded_plugins": ["test"]}) + fcb.enable_kernel_functions = False + fcb.configure(kernel, update_settings_callback, None) + assert not update_settings_callback.called + + +def test_configure_required_function(update_settings_callback, kernel: "Kernel"): + fcb = FunctionChoiceBehavior.Required(auto_invoke=True, filters={"included_functions": ["plugin1-func1"]}) + fcb.configure(kernel, update_settings_callback, None) + assert update_settings_callback.called + + +def test_configure_required_function_max_invoke_updated(update_settings_callback, kernel: "Kernel"): + fcb = FunctionChoiceBehavior.Required(auto_invoke=True, filters={"included_functions": ["plugin1-func1"]}) + fcb.maximum_auto_invoke_attempts = 10 + fcb.configure(kernel, update_settings_callback, None) + assert update_settings_callback.called + assert fcb.maximum_auto_invoke_attempts == 10 + + +def test_configure_required_function_skip(update_settings_callback, kernel: "Kernel"): + fcb = FunctionChoiceBehavior.Required(auto_invoke=True, filters={"included_functions": ["test"]}) + fcb.enable_kernel_functions = False + fcb.configure(kernel, update_settings_callback, None) + assert not update_settings_callback.called diff --git a/python/tests/unit/kernel/test_kernel.py b/python/tests/unit/kernel/test_kernel.py index a426542c3316..60d36ec38102 100644 --- a/python/tests/unit/kernel/test_kernel.py +++ b/python/tests/unit/kernel/test_kernel.py @@ -2,24 +2,34 @@ import os from typing import Union -from unittest.mock import AsyncMock, patch +from unittest.mock import AsyncMock, MagicMock, patch import httpx import pytest from semantic_kernel import Kernel from semantic_kernel.connectors.ai.chat_completion_client_base import ChatCompletionClientBase +from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings from semantic_kernel.connectors.openai_plugin.openai_function_execution_parameters import ( OpenAIFunctionExecutionParameters, ) from semantic_kernel.const import METADATA_EXCEPTION_KEY -from semantic_kernel.exceptions import KernelFunctionAlreadyExistsError, KernelServiceNotFoundError +from semantic_kernel.contents import ChatMessageContent +from semantic_kernel.contents.chat_history import ChatHistory +from semantic_kernel.contents.function_call_content import FunctionCallContent +from semantic_kernel.exceptions import ( + FunctionCallInvalidArgumentsException, + KernelFunctionAlreadyExistsError, + KernelServiceNotFoundError, +) from semantic_kernel.exceptions.kernel_exceptions import KernelFunctionNotFoundError, KernelPluginNotFoundError from semantic_kernel.exceptions.template_engine_exceptions import TemplateSyntaxError from semantic_kernel.functions.function_result import FunctionResult from semantic_kernel.functions.kernel_arguments import KernelArguments +from semantic_kernel.functions.kernel_function import KernelFunction from semantic_kernel.functions.kernel_function_decorator import kernel_function +from semantic_kernel.functions.kernel_function_metadata import KernelFunctionMetadata from semantic_kernel.functions.kernel_plugin import KernelPlugin from semantic_kernel.prompt_template.kernel_prompt_template import KernelPromptTemplate from semantic_kernel.prompt_template.prompt_template_config import PromptTemplateConfig @@ -159,6 +169,89 @@ async def test_invoke_prompt_no_prompt_error(kernel: Kernel): ) +@pytest.mark.asyncio +async def test_invoke_function_call(kernel: Kernel): + tool_call_mock = MagicMock(spec=FunctionCallContent) + tool_call_mock.split_name_dict.return_value = {"arg_name": "arg_value"} + tool_call_mock.to_kernel_arguments.return_value = {"arg_name": "arg_value"} + tool_call_mock.name = "test_function" + tool_call_mock.arguments = {"arg_name": "arg_value"} + tool_call_mock.ai_model_id = None + tool_call_mock.metadata = {} + tool_call_mock.index = 0 + tool_call_mock.parse_arguments.return_value = {"arg_name": "arg_value"} + tool_call_mock.id = "test_id" + result_mock = MagicMock(spec=ChatMessageContent) + result_mock.items = [tool_call_mock] + chat_history_mock = MagicMock(spec=ChatHistory) + + func_mock = AsyncMock(spec=KernelFunction) + func_meta = KernelFunctionMetadata(name="test_function", is_prompt=False) + func_mock.metadata = func_meta + func_mock.name = "test_function" + func_result = FunctionResult(value="Function result", function=func_meta) + func_mock.invoke = MagicMock(return_value=func_result) + + arguments = KernelArguments() + + with patch("semantic_kernel.kernel.logger", autospec=True): + await kernel.invoke_function_call( + tool_call_mock, + chat_history_mock, + arguments, + 1, + 0, + FunctionChoiceBehavior.Auto(), + ) + + +@pytest.mark.asyncio +async def test_invoke_function_call_with_continuation_on_malformed_arguments(kernel: Kernel): + tool_call_mock = MagicMock(spec=FunctionCallContent) + tool_call_mock.to_kernel_arguments.side_effect = FunctionCallInvalidArgumentsException("Malformed arguments") + tool_call_mock.name = "test_function" + tool_call_mock.arguments = {"arg_name": "arg_value"} + tool_call_mock.ai_model_id = None + tool_call_mock.metadata = {} + tool_call_mock.index = 0 + tool_call_mock.to_kernel_arguments.return_value = {"arg_name": "arg_value"} + tool_call_mock.id = "test_id" + result_mock = MagicMock(spec=ChatMessageContent) + result_mock.items = [tool_call_mock] + chat_history_mock = MagicMock(spec=ChatHistory) + + func_mock = MagicMock(spec=KernelFunction) + func_meta = KernelFunctionMetadata(name="test_function", is_prompt=False) + func_mock.metadata = func_meta + func_mock.name = "test_function" + func_result = FunctionResult(value="Function result", function=func_meta) + func_mock.invoke = AsyncMock(return_value=func_result) + arguments = KernelArguments() + + with patch("semantic_kernel.kernel.logger", autospec=True) as logger_mock: + await kernel.invoke_function_call( + tool_call_mock, + chat_history_mock, + arguments, + 1, + 0, + FunctionChoiceBehavior.Auto(), + ) + + logger_mock.info.assert_any_call( + "Received invalid arguments for function test_function: Malformed arguments. Trying tool call again." + ) + + add_message_calls = chat_history_mock.add_message.call_args_list + assert any( + call[1]["message"].items[0].result + == "The tool call arguments are malformed. Arguments must be in JSON format. Please try again." # noqa: E501 + and call[1]["message"].items[0].id == "test_id" + and call[1]["message"].items[0].name == "test_function" + for call in add_message_calls + ), "Expected call to add_message not found with the expected message content and metadata." + + # endregion # region Plugins diff --git a/python/tests/unit/prompt_template/test_prompt_templates.py b/python/tests/unit/prompt_template/test_prompt_templates.py index 38ff81c19a57..691dfb442f9b 100644 --- a/python/tests/unit/prompt_template/test_prompt_templates.py +++ b/python/tests/unit/prompt_template/test_prompt_templates.py @@ -3,6 +3,7 @@ import json +import yaml from pytest import raises from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings @@ -259,6 +260,75 @@ def test_from_json_validate_fail(): ) +def test_from_json_with_function_choice_behavior(): + config_string = json.dumps( + { + "name": "Test Config", + "description": "Test Description", + "template": "Example template", + "template_format": "semantic-kernel", + "input_variables": [ + { + "name": "var1", + "description": "A variable", + "default": "default_val", + "is_required": True, + "json_schema": "string", + } + ], + "execution_settings": { + "settings1": {"function_choice_behavior": {"type": "auto", "functions": ["p1.f1"]}}, + }, + } + ) + config = PromptTemplateConfig.from_json(config_string) + + expected_execution_settings = PromptExecutionSettings( + function_choice_behavior={"type": "auto", "functions": ["p1.f1"]} + ) + + assert config.name == "Test Config" + assert config.description == "Test Description" + assert config.template == "Example template" + assert config.template_format == "semantic-kernel" + assert len(config.input_variables) == 1 + assert config.execution_settings["settings1"] == expected_execution_settings + + +def test_from_yaml_with_function_choice_behavior(): + yaml_payload = """ + name: Test Config + description: Test Description + template: Example template + template_format: semantic-kernel + input_variables: + - name: var1 + description: A variable + default: default_val + is_required: true + json_schema: string + execution_settings: + settings1: + function_choice_behavior: + type: auto + functions: + - p1.f1 + """ + yaml_data = yaml.safe_load(yaml_payload) + config = PromptTemplateConfig(**yaml_data) + + expected_execution_settings = PromptExecutionSettings( + function_choice_behavior={"type": "auto", "functions": ["p1.f1"]} + ) + + assert config.name == "Test Config" + assert config.description == "Test Description" + assert config.template == "Example template" + assert config.template_format == "semantic-kernel" + assert len(config.input_variables) == 1 + assert config.execution_settings["settings1"] == expected_execution_settings + + def test_multiple_param_in_prompt(): func = KernelFunctionFromPrompt("test", prompt="{{$param}}{{$param}}") assert len(func.parameters) == 1 diff --git a/python/tests/unit/services/test_service_utils.py b/python/tests/unit/services/test_service_utils.py index 25b476645ae0..7f1fc669bf1a 100644 --- a/python/tests/unit/services/test_service_utils.py +++ b/python/tests/unit/services/test_service_utils.py @@ -5,8 +5,8 @@ import pytest from pydantic import Field -from semantic_kernel.connectors.ai.open_ai.services.utils import ( - kernel_function_metadata_to_openai_tool_format, +from semantic_kernel.connectors.ai.function_calling_utils import ( + kernel_function_metadata_to_function_call_format, ) from semantic_kernel.functions.kernel_function_decorator import kernel_function from semantic_kernel.kernel import Kernel @@ -17,9 +17,7 @@ class BooleanPlugin: @kernel_function(name="GetBoolean", description="Get a boolean value.") - def get_boolean( - self, value: Annotated[bool, "The boolean value."] - ) -> Annotated[bool, "The boolean value."]: + def get_boolean(self, value: Annotated[bool, "The boolean value."]) -> Annotated[bool, "The boolean value."]: return value @@ -105,9 +103,7 @@ def test_bool_schema(setup_kernel): filters={"included_plugins": ["BooleanPlugin"]} ) - boolean_schema = kernel_function_metadata_to_openai_tool_format( - boolean_func_metadata[0] - ) + boolean_schema = kernel_function_metadata_to_function_call_format(boolean_func_metadata[0]) expected_schema = { "type": "function", @@ -116,9 +112,7 @@ def test_bool_schema(setup_kernel): "description": "Get a boolean value.", "parameters": { "type": "object", - "properties": { - "value": {"type": "boolean", "description": "The boolean value."} - }, + "properties": {"value": {"type": "boolean", "description": "The boolean value."}}, "required": ["value"], }, }, @@ -130,13 +124,9 @@ def test_bool_schema(setup_kernel): def test_string_schema(setup_kernel): kernel = setup_kernel - string_func_metadata = kernel.get_list_of_function_metadata_filters( - filters={"included_plugins": ["StringPlugin"]} - ) + string_func_metadata = kernel.get_list_of_function_metadata_filters(filters={"included_plugins": ["StringPlugin"]}) - string_schema = kernel_function_metadata_to_openai_tool_format( - string_func_metadata[0] - ) + string_schema = kernel_function_metadata_to_function_call_format(string_func_metadata[0]) expected_schema = { "type": "function", @@ -166,9 +156,7 @@ def test_complex_schema(setup_kernel): filters={"included_plugins": ["ComplexTypePlugin"]} ) - complex_schema = kernel_function_metadata_to_openai_tool_format( - complex_func_metadata[0] - ) + complex_schema = kernel_function_metadata_to_function_call_format(complex_func_metadata[0]) expected_schema = { "type": "function", @@ -205,13 +193,9 @@ def test_complex_schema(setup_kernel): def test_list_schema(setup_kernel): kernel = setup_kernel - complex_func_metadata = kernel.get_list_of_function_metadata_filters( - filters={"included_plugins": ["ListPlugin"]} - ) + complex_func_metadata = kernel.get_list_of_function_metadata_filters(filters={"included_plugins": ["ListPlugin"]}) - complex_schema = kernel_function_metadata_to_openai_tool_format( - complex_func_metadata[0] - ) + complex_schema = kernel_function_metadata_to_function_call_format(complex_func_metadata[0]) expected_schema = { "type": "function", @@ -236,16 +220,11 @@ def test_list_schema(setup_kernel): def test_list_of_items_plugin(setup_kernel): - kernel = setup_kernel - complex_func_metadata = kernel.get_list_of_function_metadata_filters( - filters={"included_plugins": ["ItemsPlugin"]} - ) + complex_func_metadata = kernel.get_list_of_function_metadata_filters(filters={"included_plugins": ["ItemsPlugin"]}) - complex_schema = kernel_function_metadata_to_openai_tool_format( - complex_func_metadata[0] - ) + complex_schema = kernel_function_metadata_to_function_call_format(complex_func_metadata[0]) expected_schema = { "type": "function",