AI Customer Service Agents Fail Without Quality Data

In the absence of knowledge or insights based on high quality data, humans may not make the best decisions. I say "may" because they are still able to make great decisions based on the information in their heads coupled with their experience, intuition, empathy, judgement, and other very human skills. These decisions may still have variable quality depending on the individual. In contrast, machines can't make good decisions. Why are so many organizations so eager to turn AI agents for customer service without a comprehensive strategy that ensures their AI agents have access to high quality data (e.g., compliant, up-to-date, accurate, etc.)?

In mainstream media news last week, Bloomberg reported that Karna was slowing down its use of AI agents to handle inbound customer care calls. Here is a short blurb from Entrepreneur magazine:

"In December, Klarna paused all hiring a year prior as it focused on AI investments. The company's headcount dropped 22% in that time frame to 3,500 employees, mostly because of attrition, Siemiatkowski disclosed at the time. He asked Klarna's employees to turn to AI to help fill in the gaps left by their departing colleagues."

"In February 2024, the company claimed AI could do the work of 700 customer service agents and had taken on 75% of the company's customer chats, or about 2.3 million conversations, within a month of launch. The bot handled questions about topics like refunds, returns, and payments in more than 35 languages."

Consider that their bots were handling questions based on operational data and policies - less on product data or other customer insights.

And from Fortune magazine:

"I am of the opinion that AI can already do all of the jobs that we, as humans, do,” the CEO told Bloomberg in February 2024. It now seems that, while chatbots are cheaper, they’re just not as good as humans for some jobs, according to Siemiatkowski. “As cost unfortunately seems to have been a too predominant evaluation factor when organizing this, what you end up having is lower quality,” he told Bloomberg this week. “Really investing in the quality of the human support is the way of the future for us.”

We've all had these experiences with contact centers. Take one example of the woman standing in the flood zone (lower right hand side pane). If a human answers the call, they might detect a southern accent, know that the southeast of the United States has been hit hard by storms, and ask if the customer is okay. An AI agent might be able to do the same, but only if accurate, up-to-date customer information is passed to it along with insights about their situation. Not impossible, just harder to pull off without local knowledge.

Anecdotally, I don't mind an automated system for a quick answer to a simple question. For me, this is usually a web search. I don't reach out to a contact center. I only do so when I have a complicated need (e.g., missing wallet, stolen identity, challenging product or service issue, hacked account, etc.) For tech-savvy consumers, I believe this is common.

I cancel the service if I can't reach a human who can resolve my issues. I had an account with Uber until my account was hacked. Uber wanted me to download their app, log in, and navigate a digital experience. I took a simpler approach. I deleted their app and called my very responsive credit card company to put a block and any charges from them. I have canceled my accounts with Paypal, AT&T, and more. On the flip side, someone hacked my Wells Fargo checking account. I do even more business with them today than I did previously because I got great care when I had an issue.

There are a lot of good reasons to prefer chatbots to humans. They are available 24/7 and speak languages I understand. They may be more compliant in the right scenarios. Might be faster. They need high quality data. The costs can be high and difficult to measure when poor quality data is in play. It isn't too hard to calculate the costs of escalations, but how to calculate canceled services, business not won, or the stories relayed from poor service? More to come -