Facebook moderators break their NDA & # 39; s to expose desperate working conditions

Keith Utley was happy to help.


First he served in the Coast Guard, where he rose to the rank of lieutenant commander. He married, had a family, and devoted himself entirely to his two little girls. After leaving the army, he worked as a moderator for Facebook, where he cleaned up the social network of the worst things his users post every day: the hate language, the murders, the child pornography.

Utley worked the night shift on a site for Facebook moderation in Tampa, FL, which is managed by a professional service provider named Cognizant. The approximately 800 employees face relentless pressure from their bosses to better enforce the social norms of the social network, which receive updates almost daily, leaving the contractor's staff in perpetual uncertainty. The Tampa website has routinely failed to achieve Facebook's 98 percent "accuracy" goal. In fact, with a score of around 92, it is the worst performing site of Facebook in North America.

The stress of the work weighed on Utley, according to his former employees, who, like all Facebook contractors on the Tampa site, must sign a 14-page & secrecy declaration.

"The stress they put on him – it's unworldly," one of the Utley managers told me. "I coached a lot. I talked to him for a while about things he was having trouble with. And he was always worried about being fired. & # 39;

On the night of March 9, 2018, Utley sank down behind his desk. Employees noticed that he was in need when he started to slip out of his chair. Two of them started performing CPR, but there was no defibrillator available in the building. A manager called for an ambulance.

The Cognizant site in Tampa is located behind the main road in an office park, and between the dim night lighting and the discreet outdoor signs, the ambulance seems to have struggled to find the building. Paramedics arrived 13 minutes after the first call, a worker told me, and when they did, Utley had already turned blue.


Paramedics drove Utley to a hospital. At Cognizant, some employees were desperate: one person told me he was walking past one of the site's designated "rest areas" and found one of his colleagues, a part-time preacher, praying loudly in tongues. Others completely ignored the commotion and kept moderating Facebook posts while the paramedics were working.

Utley was later declared dead in the hospital, the victim of a heart attack. Further information about his health history, or the circumstances of his death, could not be learned. He left behind a wife, Joni, and two young daughters. He was 42 years old.

On Monday morning, employees of the day shift were informed that there had been an incident and started collecting money to buy a card and send flowers. But some site leaders didn't initially tell employees that Utley had died and instructed managers not to talk about his death, current and former employees told me.

"Everyone in the leadership told the people that he was fine -" oh, it will be fine "," a colleague recalled. "They wanted to play it. I think they were worried about people who stopped the emotional impact it would have."

But the illusion broke later that day, when Utley & # 39; s father, Ralph, came to the site to collect his belongings. He entered the building and said, according to an employee I spoke to, "My son died here."

In February I wrote about the secret lives of Facebook contractors in America. Since 2016, when the company was heavily criticized for failing to prevent various forms of abuse of its platform, Facebook has expanded its workforce with regard to safety and security around the world to 30,000. About half of them are content moderators, and the vast majority are contractors who are hired through a handful of large professional service companies. In 2017, Facebook started opening moderation sites in American cities, including Phoenix, Austin and Tampa. The aim was to improve the accuracy of moderation decisions by entrusting them to people who are more familiar with American culture and slang.

Cognizant received a two-year contract, $ 200 million from Facebook to do the job, according to a former employee who was familiar with the issue. But in exchange for checking the limits of free speech on one of the largest platforms of the Internet, individual contractors in North America earn just $ 28,800 a year. They receive two 15-minute breaks and a 30-minute lunch, along with nine minutes a day & # 39; wellness & # 39; time that they can use if they feel overwhelmed by the emotional toll of the job. After regular exposure to graphic violence and exploitation of children, many employees are subsequently diagnosed with post-traumatic stress disorder and related conditions.


My first report focused on Phoenix, where the workers told me they had begun to embrace peripheral visions after being constantly exposed to conspiracy theories at work. One brought a gun to work to protect himself against the possibility that a dismissed employee returned to the office to take revenge. Others told me that they are being chased by visions of the images and videos they saw during their working time.

The conditions on the Phoenix site have not improved significantly since my visit. Some employees were sent home last week after bedwetting was discovered in the office – the second time bed bugs were found this year. Employees who contacted me feared that the infection would spread to their own homes, and said managers told them that Cognizant would not pay to clean their homes.

"Bed bugs can be found almost everywhere where people tend to gather, including the workplace," Cognizant said in a statement. "No employee at this facility has formally asked the company to treat an infection at home. If someone made such a request, management would work with them to find a solution."

Facebook executives have claimed that the working conditions described to me by dozens of contractors do not accurately reflect the daily lives of the majority of employees. But after publishing my story about Phoenix, I received dozens of messages from other contractors around the world, many of whom reported having similar experiences. The largest group of messages I received came from current and former Facebook contractors in Tampa. Many of them have worked closely with employees at the Phoenix site and believe that working conditions in Florida are even worse.

In May I traveled to Florida to meet these Facebook contractors. This article is based on interviews with 12 current and former moderators and managers on the Tampa site. In most cases, I agreed to use pseudonyms to protect employees against possible retaliation from Facebook and Cognizant. But for the first time, three former Facebook moderators in North America agreed to break their confidentiality agreements and discuss working conditions on the site.

A walk in the Cognizant content moderation site in Tampa, FL. Despite the pressure to improve their scores, moderators here have never consistently met the goal of 98 percent accuracy that Facebook has set for them.


Employees told me that the pressure from managers to improve performance has plagued the workforce. Cognizant's contract with Facebook is up for renewal and with the entire company struggling to reach the 98 percent accuracy goal, there are widespread internal concerns that Cognizant will lose Facebook's business.

Contractors told me that Cognizant had tempted them from less demanding jobs by promising regular schedules, bonuses, and career development just to forgo all three.

They described a filthy workplace in which they regularly find pubic hair and other body waste at their workstations. Employees said managers laughed at it or ignored sexual harassment and threats of violence. Since April, two cases of discrimination have been submitted to the Equal Employment Opportunity Commission.

They said that the use of marijuana is so widespread that the site manager complained jokingly during a meeting with everyone that he had had a high contact while walking through the door.

More than anything, the contractors described an environment in which they should never forget how quickly they can be replaced. It's a place where even Keith Utley, who died next to him, would not get a memorial in the workplace – just a passing mention to team creatures in the days after he passed. "There is no indication that this medical condition was work-related," Cognizant told me in a statement. "The colleagues, managers and our client of our colleague were all saddened by this tragic event." (The customer is Facebook.)


The Utley family could not be reached for comment. Employees who started working after his death told me they had never heard his name.

"We were bodies on chairs," a former moderator told me. "We were not at all for them – not at all."

Shawn Speagle was 23 years old and worked at an online education company that worked with English-speaking students when he visited a Cognizant job fair. A recruiter described him to a role in which Speagle would primarily help companies analyze their engagement on their Facebook pages. He might have to do some substantive moderation, the recruiter said, but Speagle entered the interview because he believed he was about to start a new high-tech career – one that he hoped would eventually lead to a full-time role in Facebook.

Cognizant offered Speagle $ 15 per hour to get the job done full-time – a clear improvement over his previous job, which was seasonal. Only after he started training did he realize that it was not really about helping companies bring Facebook to market. Instead, two weeks after Speagle was put on the production floor, a manager told him that he and a colleague would evaluate graphic violence and hate speech.

"For our employees who choose to work on content moderation, we are transparent about the work they will do," said a Cognizant spokesperson in response. "They are made aware of the nature of the position before and during the recruitment process and then receive extensive and specific training before they work on projects."


But if his managers had asked, they would have learned that Speagle had a history of anxiety and depression and that he might not be well suited to the role. Nobody did it.

Shawn Speagle worked at Cognizant for about six months, where he mainly saw graphic violence and hate speech.

"They just said that I (my colleague) was very meticulous and had many promises to grow to the SME position," said Speagle, referring to matter experts who earn $ 1 more per hour in exchange for answering questions from moderators. about Facebook policy. "They said that Facebook essentially pushes all of their graphic violent content to us, that they didn't want it anymore. So they had to move more people to cover it. And that's all we saw every day."

Speagle vividly remembers the first video he saw in his new assignment. Two teenagers spot an iguana on the floor and they pick it up by the tail. A third teenager films what happens next: the teenager holding the iguana starts beating him on the street. & # 39; They knocked the lifeless shit out of this thing, & # 39; said Speagle, tears leaping into his eyes. "The iguana screamed and cried. And they didn't stop until the thing was a bloody pulp. & # 39;

The policy allowed the video to remain on Facebook. A manager told him that the authorities could catch the perpetrators by leaving the video online. But as the weeks progressed, the video continued to appear in its queue and Speagle realized that it was unlikely that the police would investigate the matter.

Speagle had volunteered in the past to animal enclosures and the regular rattling of the iguana rattled him. "They kept placing it over and over," he said, knocking on the table as he spoke. "It made me so angry. I had to listen to his screams all day. & # 39;

The Cognizant facility in Tampa was opened in a maze-like office park in the summer of 2017, about two months after the Phoenix facility came online. It operates from a one-storey building next to a pond fed by two storm drains. On most days, an alligator from one of the sewers comes out to sunbathe in the sun.

Before the office was opened, the company started advertising on Indeed and other job sites, using opaque titles such as & # 39; social media analyst & # 39 ;. Initially, applicants are not told that they will work for Facebook – only a & # 39; large social media company & # 39 ;.

Cognizant was not always clear about the nature of the work in Tampa. Marcus *, who worked in management, told me that a recruiter had persuaded him to leave a more normal job with the promise of a regular schedule, performance bonuses and a good balance between work and private life. However, once he became a member, he was forced to work and the bonuses never came true.

Marcus was created to moderate Facebook content – an additional responsibility that he says was not prepared for. A military veteran, he had become insensitive to seeing violence against people, he told me. But on his second day of moderation duty, he had to watch a video of a man who butchered puppies with a baseball bat. Marcus went home during his lunch break, held his dog in his arms and cried. I should stop he thought to himself, but I know there are people on the site who need me. He ended up staying just over a year.

Cognizant calls the part of the building where contractors do their work "the production floor" and it quickly fills with employees. The minimum wage in Florida is $ 8.46 and at $ 15 per hour the job pays better than most call centers in the area. For many content moderators, Cognizant refers to them by the enigmatic title of & # 39; process executor & # 39; – it was their first real job.


In his haste to fill the workplace, Cognizant has made some weird staff decisions. In the beginning, the company hired Gignesh Movalia, a former investment adviser, as a moderator. Cognizant performs background checks on new recruitments, but apparently it even failed to perform a simple web search on Movalia. If they had done that, they would have heard that he would have been sentenced to 18 months' imprisonment in 2015 for his involvement in one $ 9 million investment fraud schedule. According to the FBI, Movalia had wrongly claimed to have access to shares of a fast-growing technology startup that was about to start trading on the public market.

The startup was Facebook.

An access to the main workspace for the Tampa Tampa moderation site.

Movalia was eventually fired, but employees I spoke with believed that his tenure was an example of how Cognizant hires moderators: find organs where possible, ask as few questions as possible and put them on a seat on the production floor where they can work .

The result is a raw workplace where managers regularly send emails to staff who complain about their behavior on the site. Almost every person I interviewed personally compared the Tampa office with a high school. Hard-handed discussions, often about workplace novels, regularly take place between colleagues. Verbal and physical battles come out monthly, the employees told me. A dress code has been set up to discourage employees from wearing provocative clothing to work – "This is not a nightclub", read an email for all employees obtained by The edge. Another email warned employees that there had been "countless instances of theft" on the property, including stolen food from the office fridge, food from vending machines, and personal items from employees.

Michelle Bennetti and Melynda Johnson both started on the Tampa site in June 2018. They told me that the daily difficulty of moderating content, combined with a chaotic office environment, made life miserable.


"In the beginning it didn't bother me – but after a while it started to take a toll," Bennetti told me. "I had to feel a cloud, a darkness, above me. I started to get depressed. I am a very happy, extroverted person, and I became (withdrawn). My fear went up. It was hard to get through it every day It started to affect my home life. & # 39;

Johnson was particularly disturbed by the only bathroom on the site, which she regularly found in a state of decline. (The company says it has caretakers available every day in Tampa.) In the stalls, the signals that were posted in response to employee misconduct shifted. Do not use your feet to flush the toilet. Do not flush more than five toilet seat covers at the same time. Do not place any substances, natural or unnatural, on the walls.

"And of course the signs are not for nothing, because people do this," said Johnson, who worked at the site until March. "Every part of that building was absolutely disgusting. You'd go into the bathroom and there would be blood and shit everywhere. It smelled terrible all the time. & # 39;

She added, "It's a sweatshop in America."

Michelle Bennetti and Melynda Johnson (from the left) worked for Cognizant for about nine months

Michelle Bennetti and Melynda Johnson (from the left) worked for Cognizant for about nine months. Johnson calls the office "a sweatshop in America."

The working day in Tampa is divided into five teams and the desks are shared between employees. Contractors I spoke to said they would often come to work and find their workplace in poor condition for the day – among other things, boogers, fingernails and pubic hairs. The desks would be cleaned if Facebook made one of its regular scheduled visits to the site. At other times, employees said the office was filthy.


The Florida law does not oblige employers to offer sick leave, so Cognizant employees who feel sick should use personal leave times instead. (They receive five hours of personal leave per payment period.) Missing work is one of the few reasons why Cognizant regularly fires its contractors. And to prevent "occurrence," because the company calls unapproved absences, contractors who have used up their break time come to work sick – and occasionally spray them in garbage cans on the production floor.

A worker named Lola * told me that health problems had led her to risk being fired so many times. She started going to work even when she felt sick and vomiting. Facebook contractors must use a browser extension to report every time they use the toilet, but during a recent illness, Lola quickly took all the assigned breaks. She had been written down earlier because she had gone to the bathroom too often, she said, so she felt scared to get up from her desk. A manager saw that she was not feeling well and brought a trash can to her desk so she could vomit into it. So she did that.

"Then I cried at my desk," Lola said. "I thought," I can't go on. "My colleague & # 39; s said: & # 39; Just go home. & # 39; I said: & # 39; I can't, because I'm going to come once & # 39 ;." She stayed at her desk and cried.

Employees told me about other disturbing incidents on the Tampa site. Amongst them:

  • An employee who used a colostomy bag had rupture during work and spilled some waste on the floor. Senior managers were bugged by mocking her. Finally she stopped.
  • An employee who threatened to "hurry up" the building in a group chat was placed on paid leave and was allowed to return. He was fired after making a similar threat. (A Cognizant spokesperson said the company has on-site security personnel at all times. "Our goal is to ensure that our employees feel confident that they are working in a safe environment," he said.)
  • Another employee broadcasts himself on Facebook Live and says he wants to rub the head of a manager. Another manager found that he was joking and he was not disciplined.

In April, two women working at the Tampa site filed a complaint with the US Equal Employment Opportunity Commission, claiming that they were sexually harassed by two of their male colleagues. According to the complaint, the men regularly discussed anal sex in the office. When the women were not open to the discussion, one of the men said he would "set up a YouTube channel and record that he would hurry up the place," the complaint said. On April 3, the Sheriff & # 39; s Office from Hillsborough County came to the site to interview the women. According to the officer's report, one of the men was photographed after one of the women at home.

A Cognizant spokesperson told me that the employee was suspended while the claims are being investigated. But some employees say they are still worried.

"Every time I get an email or a phone call from my clients, I worry that a picture has been taken – and I know that that is their concern," said KC Hopkinson, a lawyer who has several current and former Cognizant represents employees in Tampa. "They are going to ask there every morning," what am I going to see today? And am I going to come home tonight? & # 39; "

Hopkinson told me that her clients who reported incidents to human resources are generally ignored or countervalued, a claim that was repeated by several other employees there. In some cases, the site's personnel policy has followed employees who have filed complaints in the bathroom and questioned them about what they were doing during the few minutes they were inside. ("We take accusations like these very seriously," said a company spokesperson. "Cognizant strives for a safe and powerful workplace.")

"I don't want my worst enemy to work there," Hopkinson said. "It's a terrible, awful environment."

During the six months after he was hired, Speagle would moderate 100 to 200 posts per day. He saw people throw puppies into a raging river and put fireworks in the mouths of dogs. He saw people mutilate the genitals of a living mouse and cut off the face of a cat with an ax. He watched videos of people playing with human fetuses, and said he learned that they were allowed on Facebook "as long as the skin is translucent." He discovered that he could sleep no more than two or three hours a night. He often woke up in a cold sweat, crying.

Early on Speagle came across a video of two women in North Carolina who encouraged toddlers to smoke marijuana and helped inform the authorities. (Moderator tools have a mechanism to escalate issues into law enforcement, and the women were ultimately convicted of child abuse.) But for Speagle & # 39; s knowledge, the crimes he saw every day never led to legal action against the perpetrators. The work began to become useless, never more than when he had to view images of a murder case or child pornography case that he had already removed from Facebook.

In June 2018, a month after he was working, Facebook began to see an abundance of videos showing organs harvested from children. So many graphic videos were reported that they could not be included in the Speagle queue.

"I suffered, but it leaked in everything else," said Speagle. "It was massive panic. All SMEs had to run in there and try to help people. They were excited – they couldn't handle it. People were crying, breaking down, vomiting. It was like one of those horror movies. Nobody. is willing to see a little girl who has had her organs removed while she is still alive and screams. "Moderators were told to watch at least 15 to 30 seconds of each video.

Speagle helps to take care of his parents, who have health problems and was afraid of leaving Cognizant. "It was hard to find a job here in this market," he said. To cope with the stress, he started with binge-eat pastries from the vending machines, and eventually gained a considerable weight. He sought help from the on-site counselor, but found him unhelpful.

"He just told me," I don't really know how to help you, "said Speagle. The counselor with whom he spoke had replaced the permanent counselor, who had received more training. Cognizant also offers a 24/7 hotline, full healthcare benefits and other wellness programs. But Speagle's experience soured from the site's mental health resources. Other times, when he had a particularly bleak day in line, a manager would give him a bucket of Legos and encourage him to play with him to relieve stress while he was working. Speagle built a house and a spaceship, but that didn't make him better.

Shawn Speagle holds the medicine that he was prescribed to tackle the effects of PTSD.

Shawn Speagle holds the medicine that he was prescribed to tackle the effects of PTSD.

Speagle told me by fall that he slept for only two hours each night. The lack of sleep, combined with depression, made it difficult for him to practice. He started lashing out at his parents. Meanwhile, at work, he felt micromomanaged by his team leaders, putting pressure on him to moderate more posts.

"I felt like I was trapped in my own body," he said. "I couldn't get up from my desk for the life of me, or I would shout at me to stay on my desk. So I was stuck at my desk and in my body. I was so scared."

Cognizant periodically cleans large numbers of staff in what is known as "red business days" for the red bags that managers give to the newly fired to collect their belongings. Sometimes the redundancies are related to work performance and sometimes employees get no explanation at all. Speagle was fired as part of a red carrier bag last October.

In February he went to a psychiatrist who had been diagnosed with PTSD. He is currently pending. In the meantime, he has returned to school to obtain his teaching qualification. Because he has caused so many children damage on Facebook, he wanted to make a positive contribution to the lives of young people, he said.

"I really wanted to make a difference," Speagle told me about his time for Facebook. "I thought this would make the ultimate difference. Because it's Facebook. But no difference is made."

I asked him what he thought he needed to change.

"I think Facebook should close," he said.

Last week I visited the Tampa website with a photographer. It was thoroughly cleaned the night before I visited, according to two employees I spoke to, and the bathroom sparkled. While walking on the floor with the site manager and a Facebook spokeswoman, I noticed that most of the rooms smelled of cleaning supplies.

Work stopped while we were there to make sure we didn't see the personal information of a Facebook user. Moderators, usually in their 20s and 30 & # 39; s, talked at their desks or shot baskets in one of the miniature hoops around the building. The site's senior managers, whose employees say they are normally isolated in their offices, have made a show of walking the production floor and chatting with their subordinates.

A wall sticker or poster offered an inspiring commonplace every few meters. Aanmaningen om altijd je uiterste best te doen en een positieve houding aan te nemen werden onderbroken door andere tekens die iets sinister overkwamen. "Geen nieuws is goed nieuws", lees er een. "Onze reputatie hangt van u af", lees een ander.

We zagen een activiteitenruimte waar werknemers uitgenodigd werden om deel te nemen aan yogasessies, en een pauzezaal die werd voorgezeten door een kleine Boeddha die een elektrische kaars vasthield. In de kamer van de Boeddha werden kleurboeken op een tafel gegooid naast ramen die uitkeken over de krokodillenvijver.

De tour eindigde ongeveer een uur nadat we aankwamen.

"Dat was een honden-en-ponyshow," vertelde een medewerker die Bob heette de volgende dag aan de telefoon. "Dat was volledig geënsceneerd. We zijn aan het gamen en het senior management is in contact met mensen – het is allemaal een façade. "

Facebook ziet een soortgelijke façade bij het bezoeken van de site, zei hij.

De persoon die verantwoordelijk is voor het beheer van de groeiende contractanten van Facebook is Arun Chandra, wiens naam vice-president is van geschaalde ondersteuning. Chandra is afgelopen november op Facebook aangekomen na een lange carrière bij HP, waar hij heeft bijgedragen aan het toezicht op de wereldwijde supply chain van het bedrijf. In zijn nieuwe rol, zo vertelde hij me, hoopt hij de levensstandaard van de aannemer geleidelijk te verbeteren en tegelijkertijd te werken aan het effectiever maken van zijn werk.

Bewegwijzering in een kraam van de vrouwenbadkamer bij Cognizant in Tampa, FL.

Bewegwijzering in een kraam van de vrouwenbadkamer bij Cognizant in Tampa, FL.

"Ik probeer het macrobeeld aan te pakken en de grotere dingen op de juiste manier naar voren te brengen," zei Chandra, die me opviel als energiek en zeer oprecht. "We zullen nooit 100 procent oplossen, maar ik probeer te laten zien dat ik 80 tot 90 procent van de grotere problemen kan oplossen."

Chandra heeft meer dan een dozijn van de verafgelegen partnersites van het bedrijf in de Verenigde Staten en daarbuiten bezocht en heeft plannen om ze allemaal te bezoeken. Wanneer hij arriveert, trekt hij graag contractanten naar kamers en vraagt ​​hij hen naar arbeidsomstandigheden zonder dat hun managers erbij zijn. Hij vertelde me dat content moderation op de Filippijnen een aantrekkelijk carrièrepad is geworden en dat hij overal waar hij komt moderators ontmoet die erg trots zijn op hun werk. "Het enthousiasme van mensen is geweldig," zei hij.

Dit voorjaar organiseerde Chandra een top van ongeveer 200 leiders van content moderation sites over de hele wereld – een evenement dat hij van plan is om twee keer per jaar te houden, en een andere komt dit najaar. Tot nu toe hebben verkopers verschillende beleidsmaatregelen en programma's om de geestelijke gezondheid van werknemers te bevorderen. Op de top spraken ze af om informatie over hun aanpak te delen – in feite kwamen ze overeen om te stoppen met concurreren op basis van wie het beter doet om voor de werknemers te zorgen.

"We moeten een zeer grootschalig platform runnen. We moeten voor de gemeenschap zorgen. En dat betekent dat we heel veel werk moeten verzetten, 'zei Chandra. “But that is not at the expense of (contractors’) well-being.”

Chandra plans to launch a new audit program later this year to promote better working conditions. That will include more surprise visits — an effort to get around the dog-and-pony-show phenomenon I observed last week. He also plans to stop evaluating partners on the sole basis of whether vendors achieve a 98 percent accuracy rate — instead, he said, Facebook will develop a balanced “scorecard” approach to measuring vendors’ performance. Chandra intends for worker well-being to be part of that score, though Facebook has not yet determined how it will be measured.

In May, Facebook announced that it will raise contractor wages by $3 an hour, make on-site counselors available during all hours of operation, and develop further programs for its contractor workforce. But the pay raises are not due to take effect until the middle of 2020, by which time many, if not most, of the current Tampa workforce will no longer work there. Turnover statistics could not be obtained. But few moderators I have spoken with make it to two years on the job — they either are fired for low accuracy scores, or quit over the working conditions. And so while the raises will be a boon to a future workforce, the contractors I spoke to are unlikely to benefit.

Nor will the many contractors who have already left the job. As in Phoenix, former employees of the Tampa site described lasting emotional disturbances from their work — one for which neither Facebook nor Cognizant offers any support.

I asked Chandra whether Facebook should hire more content moderators in house, rather than relying on big staffing companies. He told me that Facebook’s business changes so quickly that it might not be possible. But he did not rule it out.

“I completely get the debate,” he said. “If anything I’m very empathetic to the entire conversation, having spent a lot of time with these people. I don’t think we have a better answer right now.”

In the meantime, Facebook is building a “global resiliency team” tasked with improving the well-being of both full-time employees and contractors. Chris Harrison, who leads the team, told me that he aspires to build a wellness program that begins at the point of hiring. He wants to screen employees to gauge their psychological fitness — a move that might prevent someone like Shawn Speagle from being assigned to a queue filled with graphic violence — but says Facebook is still working to understand whether this is possible under employment law.

Harrison plans to make “resiliency” — the art of bouncing back after seeing something awful — a key part of contractor training. He helped to develop new tools for moderators that can automatically blur out faces in disturbing videos, turn them grayscale, or mute the audio — all things that can reduce the psychological harm to the moderator viewing them.

Eventually, Harrison hopes Facebook will offer post-employment counseling to moderators who suffered psychological harm on the job. “Of course we should do that,” he said. But the idea is still in the earliest discussion stages, he said. “There’s just so many layers of complexity globally. It’s really, really hard to pull it off in a legally compliant way.”

I asked Harrison, a licensed clinical psychologist, whether Facebook would ever seek to place a limit on the amount of disturbing content a moderator is given in a day. How much is safe?

“I think that’s an open question,” he said. “Is there such thing as too much? The conventional answer to that would be, of course, there can be too much of anything. Scientifically, do we know how much is too much? Do we know what those thresholds are? The answer is no, we don’t. Do we need to know? Yeah, for sure.”

“If there’s something that were to keep me up at night, just pondering and thinking, it’s that question,” Harrison continued. “How much is too much?”

If you believe moderation is a high-skilled, high-stakes job that presents unique psychological risks to your workforce, you might hire all of those workers as full-time employees. But if you believe that it is a low-skill job that will someday be done primarily by algorithms, you probably would not.

Instead, you would do what Facebook, Google, YouTube, and Twitter have done, and hire companies like Accenture, Genpact, and Cognizant to do the work for you. Leave to them the messy work of finding and training human beings, and of laying them all off when the contract ends. Ask the vendors to hit some just-out-of-reach metric, and let them figure out how to get there.

At Google, contractors like these already represent a majority of its workforce. The system allows tech giants to save billions of dollars a year, while reporting record profits each quarter. Some vendors may turn out to mistreat their workers, threatening the reputation of the tech giant that hired them. But countless more stories will remain hidden behind nondisclosure agreements.

In the meantime, tens of thousands of people around the world go to work each day at an office where taking care of the individual person is always someone else’s job. Where at the highest levels, human content moderators are viewed as a speed bump on the way to an AI-powered future.

In such a system, offices can still look beautiful. They can have colorful murals and serene meditation rooms. They can offer ping pong tables and indoor putting greens and miniature basketball hoops emblazoned with the slogan: “You matter.” But the moderators who work in these offices are not children, and they know when they are being condescended to. They see the company roll an oversized Connect 4 game into the office, as it did in Tampa this spring, and they wonder: When is this place going to get a defibrillator?

(Cognizant did not respond to questions about the defibrillator.)

I believe Chandra and his team will work diligently to improve this system as best as they can. By making vendors like Cognizant accountable for the mental health of their workers for the first time, and offering psychological support to moderators after they leave the company, Facebook can improve the standard of living for contractors across the industry.

But it remains to be seen how much good Facebook can do while continuing to hold its contractors at arms’ length. Every layer of management between a content moderator and senior Facebook leadership offers another chance for something to go wrong — and to go unseen by anyone with the power to change it.

“Seriously Facebook, if you want to know, if you really care, you can literally call me,” Melynda Johnson told me. “I will tell you ways that I think that you can fix things there. Because I do care. Because I really do not think people should be treated this way. And if you do know what’s going on there, and you’re turning a blind eye, shame on you.”

Have you worked as a content moderator? We’re eager to hear your experiences, especially if you have worked for Google, YouTube, or Twitter. Email Casey Newton at casey@theverge.com, or message him on Twitter @CaseyNewton.

- Advertisement -