.Net: Support image(url) content by AzureOpenAIChatCompletion service #6991
Assignees
Labels
ai connector
Anything related to AI connectors
Ignite
.NET
Issue or Pull requests regarding .NET code
openai_sdk_v2
Related to the OpenAI V2 SDK upgrade
sk team issue
A tag to denote issues that where created by the Semantic Kernel team (i.e., not the community)
Comments
SergeyMenshykh
added
.NET
4 tasks
RogerBarreto
pushed a commit
that referenced
this issue
Jun 27, 2024
) ### Motivation and Context This PR is the next step in a series of follow-up PRs to migrate AzureOpenAIConnector to Azure AI SDK v2. It updates all code related to AzureOpenAI ChatCompletionService to use the Azure AI SDK v2. One of the goals of the PR is to update the code with a minimal number of changes to make the code review as easy as possible, so almost all methods keep their names as they were even though they might not be relevant anymore. This will be fixed in one of the follow-up PRs. ### Description This PR does the following: 1. Migrates AzureOpenAIChatCompletionService, ClientCore, and other model classes both use, to Azure AI SDK v2. 2. Updates ToolCallBehavior classes to return a list of functions and function choice. This change is required because the new SDK model requires both of those for the CompletionsOptions class creation and does not allow setting them after the class is already created, as it used to allow. 3. Adapts related unit tests to the API changes. ### Next steps 1. Add integration tests. 2. Rename internal/private methods that were intentionally left with old, irrelevant names to minimize the code review delta. ### Out of scope: * #6991 ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
markwallace-microsoft
added
the
openai_sdk_v2
evchaki
added
the
sk team issue
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
OpenAI SDKhas a model that can return images or image url. At the moment, both {Azure}OpenAIChatCompletion services supports text only. Consider extending the service to support image bytes or image url contents.The text was updated successfully, but these errors were encountered: