We spend a lot of time speaking to AI engineers every week. Some of the pain points we hear about very frequently include:

• "I don't have a clear view of model performance over time"
• "I don't know why the model is not performing well"
• "I can't identify or prevent model quality regressions"
• "I don't know which prompt / model is better"

👇 Here's how we're solving these pain points

What we built

We built new Analytics dashboard to give users new ways to slice and dice metrics.

1. Comparison reports of evaluation metrics and usage metrics

• Compare latency, cost, feedback, and evaluation metrics across different prompts, models, customer ids, topics, and date ranges

(click the image to see a 10 second product demo)

2. Analytics Dashboard:

• percentile distributions graph for every evaluation metric
• graph of any evaluation metric over time
• new views and filters for token usage, cost, latency, user feedback

(click the image to see a 30 second product demo)