v0.1.18

What's Changed

• fix: inverse distance by @shahules786 in #1267
• fix: non llm based metrics by @shahules786 in #1268
• fix: v1 to v2 dataset by @shahules786 in #1275
• fix: make score nested if loop_is_running by @jjmachan in #1276

Full Changelog: v0.1.17...v0.1.18

v0.1.17

What's Changed

• update documentation for faithfulness with HHEM by @Miaoranmmm in #1242
• New data representation by @shahules786 in #1222
• Feat: Opik Integration by @jverre in #1256
• Add support to MultiTurn Metrics by @shahules786 in #1249
• fix: add single_turn_ascore by @shahules786 in #1261
• fix: handle v0.1 dataset by @shahules786 in #1262
• Non LLM based metrics by @shahules786 in #1260
• feat: set and get prompts for metrics by @shahules786 in #1259

New Contributors

• @jverre made their first contribution in #1256

Full Changelog: v0.1.16...v0.1.17

v0.1.16

What's Changed

• load HHEM on specified device by @Miaoranmmm in #1235
• feat: new Prompt object and faithfulness metric by @jjmachan in #1232
• feat (experimental) added new prompt and metric into ragas.experimental by @jjmachan in #1240

New Contributors

• @Miaoranmmm made their first contribution in #1235

Full Changelog: v0.1.15...v0.1.16

v0.1.15

What's Changed

• fix: typo in model name by @shahules786 in #1198
• metrics: add domain specific rubrics based scoring by @vaishakhRaveendran in #1189
• docs: rubrics based metrics by @shahules786 in #1200
• docs: add context utilisation to the index by @shahules786 in #1202
• Docs: fix typo in metrics by @shahules786 in #1203
• Implementation of Noise sensitivity metrics from RAGChecker by @sahusiddharth in #1190
• Tiny typo fix: it's -> its by @peterkaminski in #1219
• fix : solves #1215 - Inconsistency in the Value of EVALMODE_TO_COLUMNS[EvaluationMode.ca] by @jucgonzalezes in #1216
• chore: removed extra of requirements by @jjmachan in #1225
• Bugfix of evaluate function in llama_index.py by @vcim in #1223
• [FIX] - Fix for summarization edge case by @sky-2002 in #1201

New Contributors

• @peterkaminski made their first contribution in #1219
• @jucgonzalezes made their first contribution in #1216
• @vcim made their first contribution in #1223

Full Changelog: v0.1.14...v0.1.15

v0.1.14

What's Changed

• Add question to context recall doc by @shahules786 in #1173
• Add context utilization by @shahules786 in #1174
• doc: fix output for testset generation and missing files by @jjmachan in #1176
• Helicone Integration by @chitalian in #1178
• docs: minor fixes by @shahules786 in #1183
• solved issue that adaptation crashes when havig multiple nested promp… by @ssoima in #1181
• Bugfix Summarization Score by @emreds in #1164
• [DOCS] - Fix summarization score expression by @sky-2002 in #1184
• metrics: add faithfulness with HHEM model by @shahules786 in #1191
• Fix test command in development.md to match Makefile configuration by @sahusiddharth in #1195
• feat: deeper helicon integration by @jjmachan in #1196
• fix: added a test for prompts by @jjmachan in #1197

New Contributors

• @chitalian made their first contribution in #1178
• @ssoima made their first contribution in #1181
• @emreds made their first contribution in #1164
• @sahusiddharth made their first contribution in #1195

Full Changelog: v0.1.13...v0.1.14

v0.1.13

What's Changed

• Fix "global flags not at the start" raised in extract_text function by @vaishakhRaveendran in #1113
• Random seed fixed. by @Gwenn-LR in #1142
• feat: Usage tracking for evaluations by @jjmachan in #1129
• Fixing Demo Code - Compare Embeddings for retriever Docs by @matheusft in #1153
• fix: run_config post_init issue by @jjmachan in #1158
• Add save function for statement_prompt by @miyaketomoya in #1157
• Fix: adapt function to translate cached prompts in place. by @HerrIvan in #889
• fix: RunConfig documentation and improvements by @jjmachan in #1159
• Docs/run config by @jjmachan in #1160
• change segmenter language when language adaptaion by @miyaketomoya in #1163
• change default model to gpt4o-mini by @shahules786 in #1166
• Add tests to ragas.run_config.RunConfig rng attribute. by @Gwenn-LR in #1169
• feat: added track_completed decorator to track completed functions by @jjmachan in #1171
• docs: added development.md file by @jjmachan in #1115
• Allow Metric.score to work within an existing asyncio loop by @abramowskia in #1161

New Contributors

• @Gwenn-LR made their first contribution in #1142
• @matheusft made their first contribution in #1153
• @miyaketomoya made their first contribution in #1157
• @abramowskia made their first contribution in #1161

Full Changelog: v0.1.12...v0.1.13

v0.1.12

What's Changed

• [Fix bug] The adaptation for rewrite_invalid_question_prompt was not saved by @psydok in #1071
• [Fix bug] utils.py by @jrredondo in #1083
• Llama Index integration: optional RunConfig for evaluate() by @antono in #1085
• Create a custom directory loader for RAGAS by @vaishakhRaveendran in #1122
• Fix "sentence" -> "claim" the context recall metric docs by @artmatsak in #1126
• fix: temperature parameter in generate_text not ignored. by @HerrIvan in #887

New Contributors

• @psydok made their first contribution in #1071
• @jrredondo made their first contribution in #1083
• @antono made their first contribution in #1085
• @vaishakhRaveendran made their first contribution in #1122
• @artmatsak made their first contribution in #1126
• @HerrIvan made their first contribution in #887

Full Changelog: v0.1.11...v0.1.12

v0.1.11

What's Changed

• Testgeneration v3 by @shahules786 in #1024
• fix: import errors and make experimental runnable by @jjmachan in #1078
• chore: added analytics to count the num of testsets generated with experimental by @jjmachan in #1081
• Fix mutable dataclass defaults by @yasyf in #1082
• fix(executor): simplify executor by removing threading by @jjmachan in #1093
• chore: added ragas as a dependency by @jjmachan in #1095
• test: added more tests for executor by @jjmachan in #1110
• chore: remove context_relevancy from library by @jjmachan in #1111
• Update index.md by @IdaJess in #1117
• Improve Clarity and Structure of Faithfulness Explanation in Documentation by @aybruhm in #1114
• fix: remove is_async argument since everything is async now by @jjmachan in #1116

New Contributors

• @yasyf made their first contribution in #1082
• @IdaJess made their first contribution in #1117
• @aybruhm made their first contribution in #1114

Full Changelog: v0.1.10...v0.1.11

v0.1.10

What's Changed

• Update testset_generation.md - fix typo by @GTimothee in #1008
• bugfix: evaluation thread timeout by @NIL-zhuang in #858
• Update index.md by @shahules786 in #1023
• Add initial version of summarization metric by @sky-2002 in #990
• feat: added experimental package in ragas by @jjmachan in #1056
• Docs for summarization metric by @sky-2002 in #1026
• adding get_available_metrics method by @immortal3 in #1025
• chore: Fix typo in index.md by @anfindsen in #1058
• fix(testset): for generating testset with new docs by @Manav916 in #1001
• Fix counter in Runner and Executor in case of any exception by @baptiste-pasquier in #995
• Chore: Fix RuntimeError: ... got Future <..> attached to a different loop by @abetatos in #979
• fix: langchain EvaluatorChain OpenAI validation error by @danielfornarini in #1051
• chore: removed async time tests from benchmarks by @jjmachan in #1061
• Feature/updating prompt by @louky123 in #1050

New Contributors

• @GTimothee made their first contribution in #1008
• @NIL-zhuang made their first contribution in #858
• @immortal3 made their first contribution in #1025
• @anfindsen made their first contribution in #1058
• @Manav916 made their first contribution in #1001
• @baptiste-pasquier made their first contribution in #995
• @abetatos made their first contribution in #979
• @danielfornarini made their first contribution in #1051
• @louky123 made their first contribution in #1050

Full Changelog: v0.1.9...v0.1.10

v0.1.9

Feat/llamaindex: adding llamaindex (#999)

fixes: #557 


its been long 🙂