Meet Athina Evals

A Python SDK + CLI tool to run model-graded evals on LLM generated responses.

Why is it useful?

Every developer who has worked with LLMs understands the difficulty of measuring successful responses.

It’s becoming pretty clear that Evals are the solution. But...

Every developer doesn't need to build their own Eval framework in-house.

Instead, you can get started in just a few minutes with Athina's plug-and-play SDK, and use the preset evals or build your own custom evals.

Here's the rundown or Athina Evals: • Framework with preset evals you can run in one line of code. • Create a simple custom evaluator with one line of code. • Easy to create completely custom evaluators. • Run in python SDK or using CLI. • Plugs into Athina’s monitoring platform so you can run consistent evals in local, staging and production environments • Plugs into Athina’s analytics segmentation so you can explore model performance at very granular levels.

Still wondering why you should use Athina Evals instead of building your own?

What kind of evals can I run?

We’re launching this library with our 3 most popular preset evals for RAG apps 🎉