Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure first message is user message for anthropic #1530

Merged
merged 3 commits into from
Jul 16, 2024
Merged

Conversation

wch
Copy link
Collaborator

@wch wch commented Jul 15, 2024

For Anthropic, the first message in a chat must have "role": "user"; otherwise the API will return an error.

This PR ensures that when the messages are trimmed, the first message is a user message.

One possibly confusing thing: Suppose you initialize the chat with something like this:

chat = ui.Chat(
    "chat",
    messages=[
        {"role": "assistant", "content": "Arrgh! I am a chatbot that speaks like a pirate. How may I help ye?"},
    ],
)

Then when the user enters a message and the messages get sent to the LLM, this first message will get trimmed off, and so the LLM will not know that it's supposed to respond like a pirate. The problem is that the app author won't have any indication that this first message is getting trimmed, so if they expect anthropic to behave like openai (where the first message can be assistant), they will likely be confused as to why the first message would be ignored.

shiny/ui/_chat.py Outdated Show resolved Hide resolved
@cpsievert cpsievert added this pull request to the merge queue Jul 16, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 16, 2024
@cpsievert cpsievert merged commit 00ff063 into main Jul 16, 2024
30 checks passed
@cpsievert cpsievert deleted the trim-anthropic-user branch July 16, 2024 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants