"I want to test 10 different models and see which one does best for my task"

• Every LLM developer ever

It's not enough to test new models on a single input - you need to experiment with a large batch of inputs to get meaningful results and confidence that a model is truly better or worse for your use case..

As of today, you can now do that in Athina in just a few clicks.

1. Load a dataset: You can import logs into a dataset, upload a file, or even generate a synthetic dataset from your documents.
2. Re-generate responses using AS MANY models as you'd like: We support all the popular models natively - no set up required.
3. We'll automatically generate multiple datasets, run evaluations and show you the metrics so you can compare the performance across different models.

Check out this 30-second demo video to see how that works.

P.S - we've also just added support for the latest models by OpenAI, Google, Llama and Mistral across the platform.