Ensure first message is user message for anthropic #1530
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For Anthropic, the first message in a chat must have
"role": "user"
; otherwise the API will return an error.This PR ensures that when the messages are trimmed, the first message is a
user
message.One possibly confusing thing: Suppose you initialize the chat with something like this:
Then when the user enters a message and the messages get sent to the LLM, this first message will get trimmed off, and so the LLM will not know that it's supposed to respond like a pirate. The problem is that the app author won't have any indication that this first message is getting trimmed, so if they expect anthropic to behave like openai (where the first message can be
assistant
), they will likely be confused as to why the first message would be ignored.