Chatbots were first tried in an office – and proved a hit.
Researchers followed 5,000 agents in a customer service center of a major software company as they helped customers, aided by a new chatbot.
The help of artificial intelligence (AI) led to a 14 percent increase in the number of customers they helped each hour.
But not everyone at the unnamed Fortune 500 company benefited from the bots.
Newcomers to the company got a head start, but the highly skilled workers who had been working there for a while saw little profit.
In reality, the bots sucked in their knowledge and experience, returning only “smaller direct benefits in terms of improving their own productivity,” researchers said.
Chatbots helped customer service reps increase productivity by 14 percent at an unnamed Fortune 500. Pictured: A cell center in Broomfield, Colorado
Lindsey Raymond (left), of the Massachusetts Institute of Technology (MIT), and Erik Brynjolfsson, director of Stanford’s Digital Economy Lab, tried chatbots in the workplace
Lindsey Raymond, of the Massachusetts Institute of Technology (MIT), said the AI benefits were not shared across the board.
“Most of those benefits accrue to entry-level or less skilled workers,” says Raymond, a co-author of the Paper of 59 pages.
“This may be because the AI model spreads the potentially tacit knowledge of more skilled employees and helps new employees move up the experience curve.”
Since becoming widely available late last year, chatbots have baffled and surprised their human users.
The large language models (LLMs), as they are called, which collect and spew out massive amounts of text, have managed to pass exams for major law and business schools.
But for many employees, they are a concern as they could replace their role at work, especially for web content creators and software developers.
Last month, Elon Musk and some 1,000 other technology leaders signed an open letter urging a moratorium on the development of the most powerful AI systems.
They said they pose “serious risks to society and humanity.”
For this study, researchers at MIT and Stanford University tracked the deployment of a chat assistant for an unnamed software company that provides business process software.
The chatbot has been trained on data from 5,179 agents within the company. It monitored customer chats in real time and suggested how to respond to customers.
The agents could use those suggestions, but were also free to ignore them.
In a sample chat, the customer said they were “super frustrated” with the product issue that they needed “fixed ASAP.”
Chatbots are already shaking up industries dealing with text and data, including customer service centers
The report included this sample customer complaint and the chatbot’s suggestions
The AI generated responses suggesting that the agent reassure the customer that they could quickly get the product back to work.
It also recommended saying this would be a “step-by-step” process.
Using the bot, agents were able to resolve issues faster and handle multiple calls at once.
As a result, agents solved 13.8 percent more problems per hour.
The bot was the most helpful for new hires. Using the bot, an agent with two months of experience can perform just as well as an agent with six months of experience without a bot.
It also reduced sales, which is a huge problem in the customer service industry, where 60 percent of employees quit each year, costing companies $10,000 to $20,000 per agent, researchers said.
But experienced and skilled employees were of little use to the bots – they already knew how to do the job.
Erik Brynjolfsson, director of Stanford’s Digital Economy Lab, said the research showed some benefits of chatbots in the workplace, but also left significant gaps.
It’s not yet clear whether the chatbots could eventually displace the agents they supported, whether staff would need more training, or whether wages could be affected, he said.
“AI systems can impact employee and customer satisfaction, turnover and behavior patterns,” said Brynjolfsson.
“There’s so much we don’t know.”
