RAG Implementation for Market Health Reporter

[Solferest]

Hello!

I'm glad to share with you the work I've completed as part of the challenge. This was my first experience with such a task, and I approached it with great effort. Here's a summary of what I accomplished:

I implemented RAG into Market Health Reporter tool. I took an idea from your article checker tool and adapted it for this purpose. Here's how it works:

1. The Claude API analyzes an article example and extracts important indicators that can be used in analyzing further data and writing articles.
2. Then, the search tool is used to gather more information about these indicators and how to analyze them.
3. Finally, the prompt, article example, JSON data, and the search information gathered are used to create a new article.

The overall structure and logic of this new tool are based on the Claude retriever for the article checker, so it should be easier to understand. It can also be envisioned that the list of indicators for which additional information will need to be found can be provided or extracted from multiple sources, not just one example article. I will be happy to adjust the code based on your suggestions.
I am looking forward to your feedback on my submission. I hope my work meets your expectations and demonstrates my potential contribution to your team.

[Solferest]

@Kseymur, would love to get your feedback when you have a moment

[Kseymur]

would love to get your feedback when you have a moment

Hey! Thank you for your submission.
I'm not sure I got your idea. As I understand it, the bot extracts key indicators from the example article and then searches for relevant information about these key indicators on the web. But why do we need to do this every time the bot is triggered? Perhaps we can just update the main prompt with new instructions.
Let me know if I'm missing something in your logic!
Also, if you make changes to the code, please request a review so I can respond more quickly.

[Solferest]

Thanks for the response! My idea was that the search results for indicators from the article may change over time, because the model searches for relevant information every time.
To be honest, when I first studied mh reporter, it seemed to me that implementing RAG would be similar to changing the prompt. As the input data is generated by Mh, it seems unlikely that additional information can be found for it.
But I found it interesting to work on RAG, since you have such a challenge. Therefore, I decided to initially implement it in the way that I understood the task. I believe that, while maintaining the overall structure of the code, it shouldn't be difficult to modify it to receive information from additional example articles or some other sources. Therefore, I'd like to know what option you had originally planned? What parameters should I look for additional information on?

[Kseymur]

Therefore, I'd like to know what option you had originally planned? What parameters should I look for additional information on?

I would consider using RAG for searching relevant news or articles related to the data. Since we provide the LLM with exact crypto data every time, and it generates an article based on it, we could check if something significant happened during this period with this pair and market.

If you are interested in this, you can use this idea and develop it further. I believe that implementing this kind of RAG will be easier than the system you use now. Remember that you already have all the needed information for searching from the initial comment.

[Solferest]

@Kseymur, hello! I've made some changes to my implementation and now it works as follows:

1. Information about the state of the market for a specific period of time is retrieved for the Marketvenueid and Pairid.
2. This information is combined with a prompt and an example article to generate a report.

[Kseymur]

hello! I've made some changes to my implementation and now it works as follows:

Now it looks more relevant to the task. In my opinion, the implementation could be even simpler. Anyway, I'll request a review.