LLM evals are super important, but they are not a substitute for human eyeball-ing (yet)
Yep - even with a great set of evaluations, you still need to look at your LLM responses to make sure they feel good enough to be a part of your product.
But we all know that print statements, dataframes, and spreadsheets are not ideal for human review of LLM responses.
With Athina, you can compare responses across any number of datasets side-by-side. You can also turn on a diff view, so that you can quickly see the changes between the responses.
And of course, you can also run evaluations on them to measure how well these responses fare against your scoring criteria.
See our documentation to learn more: https://docs.athina.ai/ide/compare-datasets
As always, feel free to respond to this email if you have any questions or comments, or find some time on my calendar if you want to chat!