We are on a mission - to help you improve your LLM output quality.

Here's how you can get the most out of Athina:

• Set up your topics

  Athina can help you understand what your users are asking about, and what kind of questions your model is unable to answer well.

For example, you could use Athina to see that queries about "Refunds" have a low pass rate.

• Start detecting hallucinations with our preset evals:

  The 2 most common causes of hallucinations are Unfaithful Responses and Bad Retrievals (Irrelevant context)

Athina comes with out-of-the-box preset evals to detect these issues.

• Set up custom eval suites:

  Figure out your criteria for a good response and we can help you set up a custom eval suite.

  Then as you experiment, you can track how many of these evals are passing / failing in development and in production.

  • Response should be < 200 words long
  • Response must not contain PII
  • Response must not mention any prices
  • If the user asks about a specific product, then the response should include a link
  • If the user asks about "Refunds", then response should tell the user to contact customer support
  • ...

• Version your prompts: Worried about regressions whenever you update your prompts? Start tagging your prompt slugs with a version like this: prompt_slug/v1.

  Then you can start comparing the pass rate, and eval results of different prompt versions on our dashboard to improve your model scientifically.

• Get in touch with us. We're working very closely with our users to maximize their success, and help them improve their model performance.

Leverage us as a resource - you can email us anytime at [email protected], or book a call with us at https://cal.com/shiv-athina/30min