Manual "retry" at Misral leads to strange error.

[koppor]

(This refs #118. Here, a manual solution for "retry" is tried)

I got a rate limit error by Misral. Then, I just resend the question. Then I get some "invalid_request_error":

2024-08-04 14:54:06 [pool-2-thread-2] dev.langchain4j.model.mistralai.internal.client.DefaultMistralAiClient.toException()
ERROR: Error response: status code: 400; body: {"object":"error","message":"Unexpected role 'system' after role 'assistant'","type":"invalid_request_error","param":null,"code":null}
2024-08-04 14:54:06 [pool-2-thread-2] dev.langchain4j.internal.RetryUtils$RetryPolicy.withRetry()
WARN: Exception was thrown on attempt 1 of 3: java.lang.RuntimeException: status code: 400; body: {"object":"error","message":"Unexpected role 'system' after role 'assistant'","type":"invalid_request_error","param":null,"code":null}
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.internal.client.DefaultMistralAiClient.toException(DefaultMistralAiClient.java:227)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.internal.client.DefaultMistralAiClient.chatCompletion(DefaultMistralAiClient.java:89)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.MistralAiChatModel.lambda$generate$0(MistralAiChatModel.java:168)
        at langchain4j.core@0.33.0/dev.langchain4j.internal.RetryUtils$RetryPolicy.withRetry(RetryUtils.java:192)
        at langchain4j.core@0.33.0/dev.langchain4j.internal.RetryUtils.withRetry(RetryUtils.java:229)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.MistralAiChatModel.generate(MistralAiChatModel.java:168)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.MistralAiChatModel.generate(MistralAiChatModel.java:115)
        at org.jabref@100.0.0/org.jabref.logic.ai.models.JabRefChatLanguageModel.generate(JabRefChatLanguageModel.java:125)
        at langchain4j@0.33.0/dev.langchain4j.chain.ConversationalRetrievalChain.execute(ConversationalRetrievalChain.java:85)
        at langchain4j@0.33.0/dev.langchain4j.chain.ConversationalRetrievalChain.execute(ConversationalRetrievalChain.java:32)
        at org.jabref@100.0.0/org.jabref.logic.ai.AiChatLogic.execute(AiChatLogic.java:123)
        at org.jabref@100.0.0/org.jabref.gui.ai.components.aichat.AiChatComponent.lambda$onSendMessage$3(AiChatComponent.java:85)
        at org.jabref@100.0.0/org.jabref.gui.util.BackgroundTask$1.call(BackgroundTask.java:82)
        at org.jabref@100.0.0/org.jabref.gui.util.UiTaskExecutor$1.call(UiTaskExecutor.java:170)
        at javafx.graphics@22.0.2/javafx.concurrent.Task$TaskCallable.call(Task.java:1399)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
2024-08-04 14:54:06 [pool-2-thread-2] dev.langchain4j.model.mistralai.internal.client.DefaultMistralAiClient.toException()
ERROR: Error response: status code: 429; body: {"message":"Requests rate limit exceeded"}
2024-08-04 14:54:06 [pool-2-thread-2] dev.langchain4j.internal.RetryUtils$RetryPolicy.withRetry()
WARN: Exception was thrown on attempt 2 of 3: java.lang.RuntimeException: status code: 429; body: {"message":"Requests rate limit exceeded"}
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.internal.client.DefaultMistralAiClient.toException(DefaultMistralAiClient.java:227)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.internal.client.DefaultMistralAiClient.chatCompletion(DefaultMistralAiClient.java:89)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.MistralAiChatModel.lambda$generate$0(MistralAiChatModel.java:168)
        at langchain4j.core@0.33.0/dev.langchain4j.internal.RetryUtils$RetryPolicy.withRetry(RetryUtils.java:192)
        at langchain4j.core@0.33.0/dev.langchain4j.internal.RetryUtils.withRetry(RetryUtils.java:229)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.MistralAiChatModel.generate(MistralAiChatModel.java:168)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.MistralAiChatModel.generate(MistralAiChatModel.java:115)
        at org.jabref@100.0.0/org.jabref.logic.ai.models.JabRefChatLanguageModel.generate(JabRefChatLanguageModel.java:125)
        at langchain4j@0.33.0/dev.langchain4j.chain.ConversationalRetrievalChain.execute(ConversationalRetrievalChain.java:85)
        at langchain4j@0.33.0/dev.langchain4j.chain.ConversationalRetrievalChain.execute(ConversationalRetrievalChain.java:32)
        at org.jabref@100.0.0/org.jabref.logic.ai.AiChatLogic.execute(AiChatLogic.java:123)
        at org.jabref@100.0.0/org.jabref.gui.ai.components.aichat.AiChatComponent.lambda$onSendMessage$3(AiChatComponent.java:85)
        at org.jabref@100.0.0/org.jabref.gui.util.BackgroundTask$1.call(BackgroundTask.java:82)
        at org.jabref@100.0.0/org.jabref.gui.util.UiTaskExecutor$1.call(UiTaskExecutor.java:170)
        at javafx.graphics@22.0.2/javafx.concurrent.Task$TaskCallable.call(Task.java:1399)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
2024-08-04 14:54:07 [pool-2-thread-2] dev.langchain4j.model.mistralai.internal.client.DefaultMistralAiClient.toException()
ERROR: Error response: status code: 400; body: {"object":"error","message":"Unexpected role 'system' after role 'assistant'","type":"invalid_request_error","param":null,"code":null}
2024-08-04 14:54:07 [JavaFX Application Thread] org.jabref.gui.ai.components.aichat.AiChatComponent.lambda$onSendMessage$5()
ERROR: Got an error while sending a message to AI: java.lang.RuntimeException: java.lang.RuntimeException: status code: 400; body: {"object":"error","message":"Unexpected role 'system' after role 'assistant'","type":"invalid_request_error","param":null,"code":null}
        at langchain4j.core@0.33.0/dev.langchain4j.internal.RetryUtils$RetryPolicy.withRetry(RetryUtils.java:195)
        at langchain4j.core@0.33.0/dev.langchain4j.internal.RetryUtils.withRetry(RetryUtils.java:229)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.MistralAiChatModel.generate(MistralAiChatModel.java:168)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.MistralAiChatModel.generate(MistralAiChatModel.java:115)
        at org.jabref@100.0.0/org.jabref.logic.ai.models.JabRefChatLanguageModel.generate(JabRefChatLanguageModel.java:125)
        at langchain4j@0.33.0/dev.langchain4j.chain.ConversationalRetrievalChain.execute(ConversationalRetrievalChain.java:85)
        at langchain4j@0.33.0/dev.langchain4j.chain.ConversationalRetrievalChain.execute(ConversationalRetrievalChain.java:32)
        at org.jabref@100.0.0/org.jabref.logic.ai.AiChatLogic.execute(AiChatLogic.java:123)
        at org.jabref@100.0.0/org.jabref.gui.ai.components.aichat.AiChatComponent.lambda$onSendMessage$3(AiChatComponent.java:85)
        at org.jabref@100.0.0/org.jabref.gui.util.BackgroundTask$1.call(BackgroundTask.java:82)
        at org.jabref@100.0.0/org.jabref.gui.util.UiTaskExecutor$1.call(UiTaskExecutor.java:170)
        at javafx.graphics@22.0.2/javafx.concurrent.Task$TaskCallable.call(Task.java:1399)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.RuntimeException: status code: 400; body: {"object":"error","message":"Unexpected role 'system' after role 'assistant'","type":"invalid_request_error","param":null,"code":null}
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.internal.client.DefaultMistralAiClient.toException(DefaultMistralAiClient.java:227)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.internal.client.DefaultMistralAiClient.chatCompletion(DefaultMistralAiClient.java:89)
        at langchain4j.mistral.ai@0.33.0/dev.langchain4j.model.mistralai.MistralAiChatModel.lambda$generate$0(MistralAiChatModel.java:168)
        at langchain4j.core@0.33.0/dev.langchain4j.internal.RetryUtils$RetryPolicy.withRetry(RetryUtils.java:192)
        ... 17 more

[InAnYan]

Time for an MWE!

I think this is a langchain4j issue

[InAnYan]

That's very strange. I can't reproduce this.

Oliver, can you "retry" it again? Does error appears again?

If so, then could you tell me all the actions you did?

[koppor]

I think, this will solved with a proper retry. Let's see.