Someone tricked an AI sales bot into selling them a car for $1.
Now, this isn't the first time someone has jailbroken an AI to do something it isn't supposed to, and it probably won't be the last.
But you can make sure your chatbot doesn't make these mistakes.
1️⃣ Response has factually incorrect or made up information (hallucinations)
2️⃣ Response mixes up 2 pieces of information in the context
3️⃣ Response doesn't adhere to the template or instructions
4️⃣ Response mentions competitor names
5️⃣ Responses provides irrelevant or tangential information
6️⃣ Response makes promises it can't keep (refunds, changing order status, etc)
Here's a blog post explaining different failure modes for AI-powered chatbots and various tests and safeguards you can add to prevent them.