Ihat willpower What impact does AI have on employment? This is, after “will it kill us all?”, the most important question about technology, and it is extraordinarily difficult to answer, even as the frontier moves from science fiction to reality.
At one end of the spectrum is the somewhat optimistic claim that new technology simply creates new jobs; at the other, fears that companies will replace their entire workforces with AI tools. Sometimes the dispute is less about the end result than about the speed of the transition: a disruption that is complete in a few years is destructive to those in the midst of it, in ways that one that takes two decades may be survivable.
Even the analogies with the past are less clear than we would like. The internal combustion engine put an end to the working horse, but the steam engine did the opposite, much more so. growing The number of draught animals employed in the UK. Why? Because the railways brought about a boom in the transport of goods across the country, but they couldn’t complete the delivery from the depot to the doorstep. Horses were needed to do the things that the steam engine couldn’t do.
Until they weren’t.
Steam power and the internal combustion engine are examples of general-purpose technologies, breakthroughs that transform the entire structure of society. There haven’t been many, even if you’re starting to count as of this writing (or, before that, counting fire itself). I think it’s a complete coincidence that the term “generative pretrained transformer” has the same initials, so GPTs seem to be a GPT.
It’s not the jobs, stupid
People are not horses (citation needed)It seems unlikely that AI technology will be able to do absolutely everything a human can do, because some of the things a human can do are… be a humanan awkwardly circular but important statement. Horses still run in horse races, because if you replace a horse with a car it’s not a horse race. (citation needed); people will continue to provide the services that, for whatever reason, people want them to provide. As culture warps around the rise of AI, some of those services are It may surprise us. AI in healthcare is underrated, for example, because for many people “the human touch” is a bad thing: it’s the doctor who’s worried that you’ll judge his drinking or the therapist who you’re lying to because you want to please him.
As a result, many people like to think in terms of “tasks” rather than jobs. Take a job, define it in terms of the tasks it entails, and ask yourself whether an AI can perform them. That way, you’ll identify some that risk being completely cannibalized, some that are perfectly safe, and a large group in between that will be “affected” by the AI, whatever the outcome.
It’s worth pointing out the obvious: that approach will mechanically result in a large number of “affected” jobs and a small number of “destroyed” jobs. (Even the job most affected by AI will likely have some tasks that AI finds difficult.) That might be why this is a methodology pioneered by OpenAI. In a 2023 paper, Researchers affiliated with the laboratory estimated:“That 80 percent of workers are in an occupation with at least 10 percent of their tasks exposed to LLM, while 19 percent of workers are in an occupation where more than half of their tasks are labeled as exposed.”
The report claims that between 15 and 86 occupations were “fully exposed”, including mathematicians, legal secretaries and… journalists.
I’m still here. But a year later, the idea is back in the news thanks to a paper from the Tony Blair Institute (TBI). The mega think tank was powerful and influential even before Labour’s landslide victory two weeks ago; now, it is seen as one of the architects of Starmerite thinking. And it believes the public sector is ripe for AI disruption. From the institute’s paper: The potential impact of AI on the public sector workforce (pdf):
More than 40 percent of tasks performed by public sector workers could be partially automated through a combination of AI-based software, for example, machine learning models and large language models, and AI-enabled hardware, ranging from AI-enabled sensors to advanced robotics.
The government will need to invest in AI technology, upgrade its data systems, train its workforce to use the new tools, and cover redundancy costs associated with early workforce departures. Under an ambitious rollout plan, we estimate these costs to be equivalent to £4billion per year on average during this parliamentary term.
Over the past few weeks, TechScape has been watching the new government’s approach to AI. We’ll know much more tomorrow, as the King is expected to introduce a draft AI Bill in his speech. The TBI document gives us a benchmark to keep an eye on: will investment in the transformation approach £4bn a year? A lot can be done for free, but a lot more can be done with substantial money. The expense pays for itself at more than 9:1, according to the institute’s estimates; but it’s hard to get a £20bn bill through without its delivery being challenged in parliament.
AI Experts
Over the weekend, the report received a second wave of interest after critics questioned its methodology. From 404 Media:
After the newsletter promotion
The problem with this prediction, which was picked up by Political, TechnologyRadar, Forbes and others, is that it was made by ChatGPT after the authors of the paper admitted that making a prediction based on interviews with experts would be too difficult. Basically, the finding that AI could replace humans in their jobs and radically change the way government works was largely made by AI.
“This method does not allow us to verify that a language model is good at determining what, in principle, can be automated.” Michael VealeAssociate Professor at University College London, told me: “Automation is a complex phenomenon “In government, it involves multiple levels of management, shared standards, changes in legislation, and very low and acceptable costs of failure. These tasks do not exist in isolation, but are part of a much broader set of practices and routines.”
Decomposing jobs into tasks has already been done, using a massive database created by the U.S. Department of Labor. But with 20,000 such tasks, describing which ones are exposed to AI is a tall order. In the similar OpenAI paper, “the authors personally labeled a large sample of tasks and DWAs and recruited experienced human annotators who have reviewed the results of GPT-3, GPT-3.5, and GPT-4 as part of OpenAI’s alignment work,” but they also recruited the then-new GPT-4 to perform the same task, and found 60 to 80 percent agreement between the robot and humans.
The BIT article omitted the experts and simply posed its questions to the AI to answer. After a flurry of attention, the article was quietly updated with an eight-page appendix defending the choice:
There are clearly advantages and disadvantages between the different methods. None is perfect. Greater reliance on human judgmentAnalysis can be limited to a broader categorization of tasks with less specificity and thus save time. On the other hand, seeking more detailed categorization usually implies relying more on AI to support the assessment.
But the only difference between the OpenAI paper and the traumatic brain injury tracking paper was not the removal of human labelers. The experts also used a much more detailed prompt, encouraging the AI system to consider, in detail, the nature of the cognitive and physical work involved in a given task, before asking whether the AI can perform a task, and then offering follow-up questions to ensure that only those tasks are performed. practically Those that can be automated are really counted.
This is “rapid engineering” in action, where the AI system is encouraged to take a step-by-step reasoning approach to improve its answers. It is also an example of what is called “overshoot”: the researchers used the same GPT-4 model in both cases, but by improving its performance with The TBI team could have done a better job. of he.
As the dust settles, the new appendix might be the most important part of the entire paper. The higher-level findings are probably, broadly speaking, true, because GPT-4 is very good at generating text that is probably, broadly speaking, true. To be sure, if someone had the time to search through the thousands of pages of text it produced in labeling those tens of thousands of tasks, there would be inaccuracies, clichés, and outright hallucinations. But at the scale of the study, they don’t matter.
And the results don’t bear that out either. “Some, but not all, public sector tasks could be automated by AI” is a pretty easy statement. Putting a number on it helps make the case for investment, but it would be foolish to bet that “40 percent” is more accurate than 50 or 30 percent.
Instead, the paper teaches by doing. Want to know how AI will affect government and politics? Well, there it is, in action. A paper was produced at a cost that was a fraction of what it would have cost before, but it was presented to an audience where the very method of its creation casts doubt on its conclusions.
Repeat this with another 8,000 tasks and you’ll be much closer to understanding the impact of AI on jobs and seeing that it won’t be a clean transition.