New Technologies Always Scare Us. Is A.I. Any Different?

Hey there, it’s Stephen Dubner. Adam Davidson, one of the founders of NPR’s Planet Money podcast, is guest-hosting a few episodes for us. They’re about the past, present, and even the future of what’s being called artificial intelligence — a name, by the way, that practically no one working in A.I. seems to like. The episode you’re about to hear is part two in a three-part series we’re calling “How to Think About A.I.” Here’s Adam.

*      *      *

When you dig into all the hype around A.I., all the terrifying fears and utopian hope, it all comes down to one question:

Ajeya COTRA: When would it become affordable for a company to train an A.I. system as big as the human brain? 

That is Ajeya Cotra. She’s one of a growing number of people who are trying to answer this question: will artificial intelligence ever be as powerful as the human brain, and if so, when will that happen? That raises another question: what happens if it reaches that threshold, and keeps on growing? Should we be terrified? Thrilled?

Should we hurry it along? Or should we try to put on the brakes?

Human beings have been through massive technological transformations before — agriculture, cities, writing, ocean-going ships, the telegraph, the internet. Is A.I. just one more new thing that freaks everyone out for a while, and then we figure out how to live with it? Or is this time different?

*      *      *

Ajeya Cotra is a computer scientist by training. But her job is to develop ways of making measurable predictions.

COTRA: I work at Open Philanthropy. We work across a lot of different cause areas, and at Open Phil, for the last four years or so, I’ve been specializing in A.I. forecasting and A.I. risk.

Open Philanthropy is mostly funded by Dustin Moskovitz, one of the co-founders of Facebook, and his wife, Cari Tuna. It has a broad set of goals. One of which is to provide some clarity about what we can expect from A.I. If you type “A.I. predictions” into Google or YouTube, you’ll see a huge number of very amped up people telling you that A.I. is about to solve all of your problems and make us all rich and happy. Or, that A.I. will kill us all. In the last episode, we learned that these hopes and fears are very much not about the current state of A.I. Models like ChatGPT and others are just big prediction machines. They take in a huge amount of data — every written word on the internet, say — and then respond to prompts from human beings by predicting which words that human is going to want to read next. So, whatever you ask it to do — write an email to your boss or generate some software code — it’ll analyze billions of bits of data and spit out what the program predicts you want to see. It’s okay at doing that. Not amazing. But okay. I started calling it the “B-minus at everything” program. If you are well below B-minus in something — like turning a bunch of notes into a list of bullet points or coming up with titles for a paper — ChatGPT can probably do that better than you. But if you’re really good at something, you’ll mostly find the A.I. stuff subpar. It’s tempting to conclude: all the fears of A.I. are overblown. It’s just not that powerful. Not yet, anyway. Ajeya Cotra agrees.

COTRA: It’s not there now. If it were there, we would know. But I think that it could well get there very quickly, because the way this technology works, the way large language models and deep neural networks in general work, is that if you train them on the same task with a larger dataset and a bigger neural network, and train it for longer, then its performance predictably improves.

And what could that look like?

COTRA: We’re starting to see, in at least some cases, these systems have models of the world in their head. So, Othello-GPT is this language model that was trained to play the game Othello, which is a board game, trained just on moves. So it was given the move that player A makes and then the move that player B makes, and then the move that player A makes and so on. But when you crack it open and look inside it, you actually find that even though it was just trained on text describing the moves and it has never seen an Othello board, it has actually figured out a representation of the Othello board inside its brain, because that turned out to be the most efficient way to make sense of this long list of moves.

Back in 2019, so before the explosion of interest in A.I. due to ChatGPT, Cotra and her team set out to determine where A.I. is headed. To do that, they developed a framework she calls biological anchors.

COTRA: One thing that was hard about forecasting, especially then, still now, is that we don’t have any kind of measurement, which is like, this is how close the A.I. system is to doing all the jobs. We had these little benchmarks that were like, here’s how it does on this kind of multiple-choice test and here’s how it does on simple summarization tasks and stuff. And it was clear that, just because you were really, really good at those tasks didn’t mean that you were going to obsolete humans. So we needed something else to extrapolate and forecast, basically. And the thing that I landed on was, how big is the A.I. system? When would it become affordable for a company to train an A.I. system as big as the human brain? At the time that I was doing this project, the state-of-the-art A.I. system was GPT-2, and according to our estimates of animal brain sizes, GPT-2 was roughly the size of a honeybee’s brain and it was already able to do some interesting stuff. So that was a striking observation. Maybe if we got all the way to the level of a human’s brain, it could obsolete humans. And by the way, now I think GPT-4 is roughly the size of a squirrel’s brain, last I checked.

There is a big debate right now about the very idea of comparing A.I. to the human brain. Is that even the right frame to use? And Cotra is the first to say: she’s doing some guesswork. It’s just a very rough tool. Brains and A.I. have some similar attributes: they’re both networks — a network of nerve cells and synapses on the one hand, a digital network on the other. You can estimate the number of nodes in the network, the amount of data that goes through them, and the speed at which they process it. And, so far, it has been a helpful metric. When GPT-2 was out and it had a honeybee-size brain, it was very basic. It did not become a big, viral hit. You probably never heard of GPT-2; I know I didn’t. But then, when it reached squirrel level, it started to amaze and terrify people.

COTRA: Okay, let’s run with this hypothesis that we might get really powerful A.I. systems that can automate most tasks once we’re able to train A.I. systems as powerful as the human brain, and then let’s ask the question, how powerful is the human brain and how much money and how much computation and how much data would it take to train systems that are as powerful as the human brain? 

By looking at A.I. prediction this way — studying the number of nodes, the speed of processing, the size of memory — Cotra was able to come up with a simple reason why A.I. is not, now, at all equivalent to the human brain. When she wrote her initial report in 2020, she concluded it was theoretically possible to build an A.I. model with roughly the capacity of a human brain. But it would be expensive.

COTRA: The amount of money it would have cost to train that system then was, according to my estimates, more money than existed in the world by a lot. But computers are getting cheaper, and the algorithms we’re using to build these A.I. systems are getting more efficient.

Her 2020 report, called “Forecasting transformative AI using biological anchors” has become a well respected benchmark. In 2020, she looked at the pace of growth of A.I., the cost of computing power, and she concluded that there was a 50 percent chance that by 2050 we would have what she calls “transformational A.I.,” meaning that artificial intelligence would be roughly comparable to human intelligence. Then came GPT-4 and Google’s Bard and Anthropic’s Claude, and she realized things were moving much faster than she had anticipated.

COTRA: Now my median is more like the late 2030s. So that was a pretty big leap.

DAVIDSON: By median you mean it’s as likely as not. 

COTRA: As likely as not, by that year. 

Cotra does want to make clear: there is no reliable way to predict how A.I. will develop. The history of the field is filled with predictions that now seem absurd. In 1965, the scholar and A.I. pioneer Herbert Simon said that by 1985 a computer would be as capable as a human. Five years later, another A.I. pioneer, Marvin Minsky, was more ambitious: he thought it could happen by 1973. So, it’s best practice to discount any specific prediction. But it’s still hard to ignore the question right now.

Here’s the first thing you might worry about if you’re thinking about A.I. becoming as smart as you: Will it take your job? Will you get fired and replaced by a computer program? There’s a lot of anxiety about A.I. taking away jobs. Here’s a report from Goldman Sachs, issued in April, predicting that two-thirds of jobs will be impacted by A.I. Three hundred million jobs around the world will either be lost altogether or diminished. We can see this happening already: The CEO of IBM said 30 percent of back office roles — nearly 8,000 jobs — could be replaced by A.I., and the company Dropbox cut its workforce by 16 percent, citing advances in A.I. as the reason. This is a familiar story: some new technology comes along that can do something that previously had only been done by people, and it’s cheaper for employers to buy the machines than pay the people, and now the people are out of work. It’s messy and painful. There are very clear losers, whose lives are permanently worse. But, less visibly and less immediately, there are also usually winners who benefit in a way that nobody could have predicted. Economists have studied this issue obsessively: how technology has immediate impacts, and then an entirely different set of long-term impacts.

Daniel GROSS: We examined in this work one of the largest automation events in modern history.

That’s the economist Dan Gross from Duke University. We spoke with him last week about his work on creativity. He and another economist, James Feigenbaum, wrote a paper about a technological innovation that had a huge impact on the workers of the 1920s. The paper is called: “Answering the Call of Automation: How the Labor Market Adjusted to the Mechanization of Telephone Operation.” You’ve seen it in movies. Rooms filled with switchboard operators.

GROSS: This is a job that once upon a time employed hundreds of thousands of primarily young women, who sat at switchboards day in and day out, connecting telephone calls. It used to be 100 years ago, if you wanted to place a telephone call, you picked up the handset and then somebody would come on the line and say, “Number, please.” And then you would tell them whom you wanted to reach, and they would connect you by physically connecting an electrical circuit through the switchboard at which they sat.

In 1920, operating a switchboard was one of the most common occupations for women in the U.S. — especially white, American-born women under the age of 25. It counted for roughly two percent of all female employment. Those jobs were eliminated because of a new technology invented by an undertaker in Kansas City named Almon Strowger: the automatic switchboard — a machine that could replace all those human operators. He did it because the local switchboard operator was the wife of one of his funeral home competitors. When people picked up the phone and asked for Strowger, she would route the call to her husband instead, and he’d swipe Strowger’s business. Strowger patented his device in 1891, but it took until the late 1910s for automation to take hold.

GROSS: It proceeds effectively one city at a time, especially in mid-sized and smaller cities, which only have one telephone exchange. They get changed over to dial service effectively overnight. It takes maybe a year or two years to fully set up. But literally the cables are cut over at midnight on a Saturday night. The next day you have dial service. You no longer need local telephone operators. 

But the telephone operator job was not just the way a huge number of women made a living. It was the basis of an entirely new way of life.

GROSS: For some women, it was — not only loss of a job, but a loss of a community; a loss of not just any job, but of a fairly dependable job. A fairly interesting job at that, I mean, it was more interesting than factory work, for example. But let’s step back from this problem for a minute and think about who’s affected.

In their study, Gross and Feigenbaum looked at two groups. The first is the obvious one: women who were working as telephone operators, and then lost that job. They wanted to see how that job loss impacted those women.

GROSS: But there’s another category, too, which is future generations of young women who are coming of age, this was an entry level job and employed primarily at the time as young as 16-year-olds to 25-year-olds. And not only was it a first job, but it was a pathway into the labor force. And so we think, too, about how future generations of women who are coming of working age now no longer have this opportunity in the labor market that could be a stepping stone into a career. We want to think about both: the incumbent operators who might have lost their jobs, and the future generations that have fewer opportunities in the labor market that they’re facing as they graduate high school. And these are two separate problems. 

Gross and Feigenbaum weren’t just looking at big, abstract data sets, like percentage of employed women. They wanted to see how this sudden transformation of a major industry impacted specific people over their working lives. They used census and genealogy data.

GROSS: And so we can actually start to trace people over time. We can then say, okay, a telephone operator in Dubuque, Iowa, in 1920, what was that woman doing in 1930? Were they still employed? In what industry and what job? Were they still working for the telephone company? These are all things we can measure. And here’s what we learn. First, that the automation of telephone operation did have negative impacts on incumbent operators. They were less likely to be working. What’s remarkable about this story is what happens to the future generations entering the labor market in the cities that have telephone service automated, we don’t find that they’re less likely to be working afterwards. We don’t find that they have higher unemployment rates.

DAVIDSON: These are the younger women who never worked in telephones because they never — 

GROSS: They never had the chance to. 

DAVIDSON: These are the kind of women who probably had a high likelihood to work in a telephone operation. They’re working. They’re not just sitting around unemployed. It’s kind of like we used to all be farmers. Now, very few of us are farmers, but we’re not just all sitting around not farming. We’re doing other stuff.

GROSS: And here we can unpack the story a little bit more too, we can see, okay, what other jobs did these women take up? And we see them moving into things like secretarial work. We see them moving into things like waitressing work. You see new parts of the economy finding uses for these young women. And that’s a version of this dynamic feedback process through which the economy adjusts.

There is this phrase in economics that I find helpful: the lump of labor fallacy. It’s the idea that there are a specific number of jobs in the world, and that if someone gets one of those jobs, nobody else can have it. Or if one of those jobs is taken away, someone will be unemployed. It’s not true. It’s a fallacy, of course. But it is a frame that a lot of people use. It’s one of the main arguments against immigration, for example: they’ll take our jobs! And it fuels a lot of the fears of any sort of automation, including A.I. The reason it’s a fallacy is that jobs are not some stable thing, sitting in fixed lumps somewhere. Jobs are a dynamic output of the way the economy functions. New immigrants might take one set of jobs, but they use the money they make to get haircuts and buy food and hire tax accountants and, so, they create jobs, too. Same with automation: it might eliminate one type of work but then it often creates whole new kinds of work. It’s a compelling idea but it’s pretty abstract. What Gross and Feigenbaum did with their telephone-operator study is show this process in concrete detail, with names and dates.

GROSS: The punch line basically is that the work didn’t go away. Other work cropped up where old work was automated away. And so in that sense, we do see it as a silver lining paper, perhaps a surprising silver lining, how readily future generations adapted. And, you know, think too about what it was like to be alive at that time relative to today. I mean, there are just countless technologies, categories of work that exist today that were unimaginable then.

A couple things to remember: first, there was real pain. There were women who lost that telephone operator job and never found work that paid as much. That’s a real problem. Those people see their lives as permanently worse off. And second, more people benefited from the change than were hurt by it. But they benefited in the future. In 1920, when you’re watching people lose work to machines and you don’t know yet that some folks are going to be made better off, it’s easy to conclude: this is a disaster. This is just bad.

The big question — what may be an existential question for us humans — is a simple one: Is A.I. just another new tech that will get rid of a bunch of jobs, but we — or at least our kids — will have all sorts of new cool jobs we can’t even imagine today? Or is A.I. fundamentally different? In the past, technology took over specific human activities, and opened up new human activities. But what if A.I. is good at everything we do? That’s after the break. I’m Adam Davidson, and this is Freakonomics Radio.

*      *      *

The history of humanity is, in many ways, the history of technology and how that technology upends existing ways of living and creates entirely new ones. Here’s someone who has thought a lot about these world-changers.

JOHNSON: My name’s Simon Johnson. I’m a professor at M.I.T. in the Sloan School of Management. I teach economics and global business to all kinds of different students.

DAVIDSON: You and I have known each other for a long time. And the main way we know each other is talking about banks and the financial system and financial crises. I don’t remember ever thinking of you as somebody who’s studied technology.

JOHNSON: Well, I’ve worked on quite a few questions. Most are related to long-term economic performance, economic growth, who becomes rich, who stays poor. And that work did lead me to the I.M.F. in the early 2000s, where I did get drawn into the questions of that time, which were about financial crises and banking in particular. But my interests have always been sort of beyond that in terms of, how do some economies turn out to be so robustly prosperous and others really struggle to get there? And in any story or analysis of economic growth, technology, uses and abuses of technology is a central element. 

Johnson co-wrote a book, Power and Progress: Our Thousand-Year Struggle Over Technology and Prosperity, with another M.I.T. economist, Daron Acemoglu. The question that they address in that book is a big one: why are some countries and some people rich and others poor? It’s a question that’s fundamental to the discipline of economics. It’s right there in the title of the founding text of the field: Adam Smith’s book, An Inquiry into the Nature and Causes of the Wealth of Nations. A lot of the questions we have about A.I. today are, essentially, variations of this same question: when things change, why do some people win and others lose?

JOHNSON: So, yes, we were interested in why some countries are so rich and why so poor. And we decided to look at the set of colonial experiences, because what the Europeans did, obviously, after about 1500 was they expanded around the world. They took control of lots of countries, one way or another. But what they did in those countries, what they wanted to do, and what they managed to do was actually quite different in different places. In places where the Europeans actually wanted to go and did go in person, they set up certain kinds of institutions. And actually, that’s the origin of modern United States as well as Canada, Australia, New Zealand, some other places. But in places where the Europeans didn’t want to go, for example, because it was very unhealthy for Europeans to go there in 1600 or 1700, the Europeans found other ways to exploit the resources, exploit the people. They set up slave trade in some places. They ran really extractive societies in parts of Africa, also Latin America, also some parts of Asia. Variation within the colonial strategies enabled us to think about different pathways that countries had been placed on by those Europeans, by the colonial powers. And then we were able to trace out the way in which income diverged across countries, particularly when industrialization came. So some countries were able to industrialize early. That’s the story of the United States, for example. Others struggled to industrialize. India would be a counter-example. And many parts of Africa are still at a very early stage with regard to industrialization. So this is a big part of what’s driven the divergence in income per capita around the world and why we haven’t converged subsequently. 

DAVIDSON: So that’s all fascinating going back to, you know, the history of colonialism, etc. But why does that make you interested in ChatGPT, which is like the opposite of history? It’s like the most cutting-edge new technology. 

JOHNSON: Economic history is about the history of how we’ve used technology. And sometimes it’s gone really well, and we’ve got a lot of benefits for many people. And sometimes it’s gone really badly, so that technology has only benefited a few people. It’s actually pushed some people even into poverty, sometimes, the way in which it’s used. So for a long time, many of us have been trying to think about what to do about that. And now along comes a new technology which logically leads you to ask — is this going to help us, or is this going to further worsen the income inequalities? And the answer that Daron and I have is it depends, which is a classic economics answer. But it depends specifically on some choices we make about what technology we develop, what versions we emphasize, and how it gets deployed, and how we think about what we want technology to achieve. 

DAVIDSON: Who’s the “we” in that sentence? I don’t know anything about how to code A.I. Isn’t that just something the engineers are figuring out? 

JOHNSON: Our view is that engineering is too important to be left to the engineers, if I could paraphrase President Clemenceau on that. 

Georges Clemenceau, the French prime minister during World War I, reportedly said “war is too important to be left to the generals.” World War I is an awful example of what happens when a tiny group of supposed experts makes a series of disastrous choices. Johnson and Acemoglu’s book, Power and Progress, is filled with examples where decisions were made about how to take advantage of some technology or other, and those decisions had unpredictable consequences for generations. Some good. Some, very bad. Johnson hopes that we, as a society, can expand the group of people shaping how A.I. works, how it’s designed.

JOHNSON: I think we saw this, for example, with the development of atomic weapons and nuclear power. You need to have experts, obviously. They’re the ones who invented the thing. They’re the ones who know how to prevent it from exploding. But you don’t really want to cede all the power to them or to the generals who are buying the atomic weapons and putting them in the country’s arsenal, for example. You want to have a lot of eyes on that. You want to have a lot of civilians. You want to have a lot of people who care about broader social outcomes. And that’s true of potential military aspects of technology. But it’s also true when technology can affect jobs in a big way. 

DAVIDSON: Do you just tell the engineers “Stop inventing stuff so that we don’t lose jobs”? That doesn’t seem tenable. Like, the engineers are going to engineer. 

JOHNSON: We are not saying that. We’re saying that there’s been too much emphasis, an overweighting, if you like, on this idea of machines and algorithms that can replace people. 

In the previous episode, we talked to Lydia Chilton, a computer scientist who is trying to teach A.I. to be funny. And she told us: she’s not sure it’ll work. And she’s also not sure that if it does work — if A.I. can write funny jokes, funny movie scripts — if that would be a good thing for the world or a bad thing. She’s a computer scientist. She sees her job as coming up with ideas to see if they work. It’s other people’s job to figure out the implications of what happens if it works. Simon Johnson has been talking with a lot of tech leaders about A.I. I asked him what they think about the impact that A.I is likely to have on our lives. What’s their endgame? Do they want A.I. to do everything that humans can do?

JOHNSON: What I get is some fuzzy thinking and a little bit of a sort of fudging around that. So this idea of surpassing human capabilities in some things, or at least matching them, that is an idea throughout the tech sector. And that’s a benchmark for achievement. And I think that’s the problem. As opposed to saying, “I want to make people better at chess,” it’s like, “I want my machine to beat the chess players.” 

When I try to think about the idea of progress, I think about the history of my family. I have studied the history of my mom’s Jewish immigrant ancestors and my dad’s New England Yankee ancestors. In 1900, every one of my great-great-grandparents was poor, very poor. I shared some of my family records with a historian of Jewish immigration, and she told me that my great-great grandfather, Solomon Davidson, was the poorest Jewish immigrant she had ever seen. My ancestors were farmers and boot-factory workers and rag peddlers and, sure enough, over their lives and the lives of their offspring, a series of technological advances have rendered all their ways of working no longer viable. Machines have taken their jobs, is one way to put it. But I feel okay that there isn’t a job for me in the boot factory. That I am not a rag peddler. So, I asked Simon, won’t this just be the same thing? A.I. will hurt some of us, there will be pain for a while, but all of our great-grandkids will be rich beyond our imagination?

JOHNSON: Well, that is what technology can do, and that’s what technology should do, Adam. The tractor is a good story. And the mechanization of agriculture in the United States — that was absolutely accompanied by an increase in jobs in cities. So people would leave the farm in the upper Midwest, go to Chicago, work for McCormick’s Reaper company, making Reapers that were then sold to the farms, so more people could come to the city. But when Stalin saw this from the perspective of running the Soviet Union in the late ’20s, early 1930s, he wanted to do the same thing or wanted a version of the same thing. He actually oppressed the heck out of peasants, including killing millions of people in Ukraine and other parts of the Russian Empire in order to generate the grain to buy tractors from the United States. They used those tractors as an instrument of oppression and collectivization. So same tractors, very different outcomes. The railroad in the United States is a fantastic spread of opportunity, and in the U.K. But the railroad in India was used by the British to move troops around and to get their cotton goods into Indian markets, where they would destroy the livelihoods of indigenous people, the Indians. And the cotton gin, Adam — I mean, the cotton gin is the most horrific of all. So the cotton gin allowed the development of, and the commercial cultivation of upland cotton, for which there was increasing demand from British industry as industrialization grew. But what that really meant was moving a lot of enslaved people from the East Coast, where conditions were already pretty bad, to the Deep South, where conditions on the cotton plantation were much worse. It was just a horrible, horrible system made possible by the way the cotton gin interacted with the institutions of slavery and oppression of African-American people. 

The point here is not that technology is bad. It’s also not that technology is good. It’s that the technology itself doesn’t have some inevitable impact on human beings. What can be good or bad is the way human beings create and share that technology and the rules — the formal laws and institutions and the informal assumptions and practices — that we use around that technology. There is a techno-optimist view — and I find that I can be pretty sympathetic to it — that history’s main lesson is that technology, on balance, is more often good than bad. Simon Johnson says that’s a lazy read of history.

JOHNSON: What do people creating technology, deploying technology— what exactly are they seeking to achieve? If they’re seeking to replace people, then that’s what they’re going to be doing. But if they’re seeking to make people individually more productive, more creative, enable them to design and carry out new tasks — let’s push the vision more in that direction. And that’s a naturally more inclusive version of the market economy. And I think we will get better outcomes for more people.

Oh, and we can’t take forever to have these conversations. If we humans want to decide how A.I. will impact our world, we better hurry, while we’re still the ones making decisions.

COTRA: We could end up in a situation where you need to rely on and defer to A.I. decision-makers if you want to be competitive at all on the world stage.

That’s after the break. I’m Adam Davidson, and this is Freakonomics Radio.

*      *      *

Before the break, the economist Simon Johnson brought up a question that we should be asking about people who are developing A.I. systems: what are they trying to achieve, and is it something that will be good for the world? But there’s another issue that people in A.I. are raising that goes further than that. They call it “the alignment problem.”

COTRA: The high-level description of the alignment problem is that, surprising as this may sound, I don’t think that we have at a technical level found a way to guarantee that if you’re trying to train an A.I. system to pursue some goal for you, that it will actually learn to pursue that goal, as opposed to pursue a different goal, including a goal that might put it in conflict with you.

That’s Ajeya Cotra again from Open Philanthropy, whom we heard from at the start of the episode. These A.I. systems, like OpenA.I.’s GPT-4, are trained to do something. When they are designed and as they are developed, the human beings who create them have some explicit goal in mind. It could be a simple one: get better at distinguishing dogs from cats. Or it could be a far broader one: given whatever prompt a human being types in, come up with the best prediction of what the human wants to see next. So, there is an overall goal for the system. And then there is the specific task, like write me an essay about the history of frankfurters, or whatever. The reason GPT-4 is so powerful is that each choice it makes — each of the words it lays out in, say, an essay about hot dogs — is based on hundreds of billions of factors. That means you can never know why it’s doing what it’s doing; why, when I asked it to write me an essay about the history of frankfurters, it typed this sentence: “Some argue that the frankfurter was created in the 13th century for a royal coronation.” If I wanted to figure out why it picked those exact words, I would have to look at ChatGPT’s estimated more than one trillion parameters — for each word. As it happens, it’s totally made up. Nobody anywhere seems to have ever argued that the frankfurter was created in the 13th century for a royal coronation. So why did ChatGPT make that up?

It is literally impossible for human beings to figure that out. Now, an essay about hot dogs is not that big a deal. But Cotra says this kind of mistake, where it confidently tells us something wrong and we don’t know why, is a perfect example of the alignment problem.

COTRA: The alignment refers to the alignment between your goals, like what you want the A.I. system to be doing, which is, you know, write better code in a certain way and don’t do anything bad, and what the A.I. system is actually pursuing. 

Let’s say you asked A.I. to help you build a website. And that website seemed to be running great. The thing is, A.I. might help that website run more efficiently by, for instance, compromising the site’s security features.

COTRA: But that’s kind of hard to notice. Then it may get rewarded for using that shortcut because you run the code and you realize, Oh wow, my website loads much faster, that’s great. You know, thumbs up. But you didn’t notice that this came at the cost of inserting some sort of security vulnerability.

DAVIDSON: I’ve always heard of the alignment problem in the context of will killer robots kill us all? But it sounds like, from what you’re saying, that’s just a small subset — I mean, potentially an important subset — but the alignment problem could also be, like, does my tax software actually find the right tax breaks for me, or does my driverless car actually find the most efficient way home or whatever? It could be a more trivial problem. 

COTRA: Right now we have misaligned A.I.’s and we’re all still here. Bing Sydney is a great example, right?

You may have heard the story of New York Times reporter Kevin Roose, who had a bizarre conversation with a GPT-powered chat tool built into Microsoft’s Bing search engine. The chatbot, which identified itself as Sydney, told Roose to leave his wife and described “dark fantasies” of hacking computers. Cotra points out: Microsoft had explicitly trained the chatbot not to do things like that.

COTRA: Microsoft didn’t want its A.I. system to threaten users and create this big P.R. scandal for them. They tried to train it not to do that, but they didn’t succeed at making sure that it actually never did these harmful actions. Even though in some sense, Sydney or Bing could be said to have the understanding to know that its behavior would be upsetting to humans — it didn’t actually follow through on its understanding.

The very last thing Microsoft would have wanted is for a reporter to have an unsettling encounter with its chatbot just as it’s trying to take over the A.I.-powered chatbot industry. It had very smart people, highly trained experts, telling the chatbot what they wanted it to do. And still, it went rogue. You may have also heard the case of Steven A. Schwartz, a lawyer who had ChatGPT write a brief, which he submitted to court, and only then found out it was filled with made-up information, including fictitious legal precedents. And you’ve probably heard of stories closer to home: I know several teachers who see students suddenly turn in papers far more detailed than they ever had before. We are all learning, some of us more painfully than others, that the output of A.I. is not ready for prime time. It needs to go through a human editor to make sure it’s not wrong or disturbing. But Cotra says that it is very likely that, as A.I. gets more capable, the best way to take advantage of its ability to assess huge amounts of information will be to let it make decisions that human beings won’t be able to evaluate. A.I. can already process more data points and look at those data points in far more ways than a human can.

COTRA: When you stack up all those advantages, along with their raw capabilities, we could end up in a situation where you kind of need to rely on and defer to A.I. decision-makers if you want to be competitive at all on the world stage, whether that’s economically competitive or militarily competitive. So it’s not only the case that you can have A.I. C.E.O.s of companies alongside human C.E.O.s of companies. It’s that when you get to the point of being able to have A.I. C.E.O.s at all, you quickly blow past that to the point where human C.E.O.s are not viable. So, if you’re the shareholder of a company, you’re really just not going to make any money unless you appoint an A.I. as C.E.O. And if you are a country, you’re not necessarily going to be competitive in a war unless you have A.I. generals and you have A.I. soldiers executing those A.I. generals’ orders with more precision and speed than human soldiers ever could. It’s kind of a world where, like, right now I think you can’t really get anything done if you refuse to write or even if you refuse to use a computer. And I’m imagining a future where you’re just kind of irrelevant if you refuse to lean on A.I. decision-makers in all spheres of life.

DAVIDSON: My emotional reaction to a world in which A.I. generals are fighting other A.I. generals and A.I. C.E.O.s are competing with other A.I. C.E.O.s — that feels like a worse world to me. That feels like a pretty bad world. And the way you described it, there seemed to be an assumption that they are pursuing aims that human beings recognize as good aims. But I have to assume they would then have their own goals, like why would they have human goals if they’re running the show? Wouldn’t they have their own goals? 

COTRA: So people have a really wide range of reactions to this question from, why at all would they have human goals, to like, why would you think they have their own goals, even if they’re running the show, that’s not how they work. And I think it’s just pretty wide open. I think it’s possible that we could end up with these systems that are, in fact, considerably more powerful and thinking faster than us and making all the big decisions while still at a high level pursuing the goals that their users have given to them. Which, by the way, does not necessarily mean we’re obviously in a good state. You know, some of these users might be giving bad goals to these A.I. systems. 

DAVIDSON: I’m trying to understand the path. What are the steps between where we are now and it doing most activities?

COTRA: One thing that seems plausible to me is that we will see a kind of transition from some more closed-ended, contained, short tasks to more open-ended tasks. I imagine we’ll see a gradual moving from being able to write little functions, to something more like where it tries stuff, and it might not work, but it knows how to like read the error message and try again.

DAVIDSON: So we would be seeing a series of steps, right? Like, each year you would be able to kind of check in and say, are we closer or are we further away than I thought we’d be at this point?

COTRA: I think you will see some kind of gradual increase in usefulness, but I wouldn’t really peg it to each year, actually, I think I would peg it to each GPT. If we as humans, as a society, make a jump from GPT-4 to a model 100 times bigger than GPT-4 the next year just by choosing to spend a huge amount of money on it, then I don’t think there are any guarantees that the progress we see on the outside is going to look gradual. Because you might just like zoom through all the different levels of capability by going from GPT-4 to 100x GPT-4 in one shot. 

DAVIDSON: Maybe this is just like anxiety management inside of me — I’m hoping that we’re going to have a bunch of steps along the way, where we can kind of say, oof, that that was a little more than we wanted. But let’s get to what do we do about it?

COTRA: I think one thing that is a blessing and a curse about how this technology works is that it is really driven by the size and computational power of these A.I. systems themselves. So the thing that was different between GPT-2 and GPT-3 was mainly that GPT-3 was a lot bigger than GPT-2. And as a result it was a lot more expensive to train. And that’s why it was able to do all these cool things that GPT-2 couldn’t do. And that’s why GPT-4 was able to do all these cool things that GPT-3 couldn’t do.

This idea that A.I. doesn’t grow in a steady way but rather has these huge leaps forward means that an A.I. with entirely new capabilities can suddenly appear in our lives without us being prepared. And as the A.I. systems get better, they are even more difficult for us to understand. Now, the A.I. companies and the computer-science researchers have a clear goal: they want to get the next great thing before others do.

A.I. has become a major focus of venture capitalists, who put more than 40 billion dollars into A.I. companies in the first six months of this year alone. GPT-4 cost much less than 1 billion dollars to develop.

COTRA: I think that means that as a society, if we want things to go more gradually, it makes sense to pay attention to the steps we’re taking in how powerful these models are. And I think it could make sense to require in regulation eventually, if your current model has certain kinds of concerning capabilities that we can try to define ahead of time and try to measure, well, then you need to register with the government the next model that you’re training, and it can’t be too much bigger because if you go too much bigger than who knows how much more capable it’ll get. And you need to get some kind of approval that making this step is okay, and that you as a company can handle it.

DAVIDSON: So there would be like the Agency of A.I. Alignment, or something like that. But the U.S., despite our best intentions, is not the entire world. And, this is what everyone inevitably says: ”Well, maybe, China will then take the lead. Maybe North Korea or other bad actors will take the lead.” Walk me through your thinking there. How do we make government regulation that’s effective? 

COTRA: I more often spend my time thinking about on a technical level, what would good regulation look like if we could somehow get people on-board with it. One thing that seems like a success story on an international stage is nuclear nonproliferation. I don’t think it’s a complete success story, but I think relative to what I might have expected in 1945, I think I would have been kind of surprised that we’re still here. And not literally every single country has nukes, even though more countries have nukes than they did in 1945. So that’s maybe a model we could look to for hope.

We started this episode with a fairly simple question, one on all of our minds: will A.I. take your job? And it seems clear: with regards to any one specific job, it’s pretty hard to say, but we can definitely know that, when it comes to jobs in general, A.I. is going to have a massive, disruptive impact. Like a lot of people, now and throughout history, I look at this new technology and one of my big feelings is fear. But that’s not the only way to think about it.

Ethan MOLLICK: I think that it helps to have a utopian vision here.

Next week on the third and final episode of our series, “How to Think About A.I.” — should we really have a utopian vision about A.I.? What does a world where A.I. is everywhere even look like? How will it change your life and how might you have to change? That’s coming up next week on Freakonomics Radio.

*      *      *

Freakonomics Radio is produced by Stitcher and Renbud Radio. This episode was produced by Julie Kanfer and mixed by Eleanor Osborne, Greg Rippin, Jasmin Klinger, and Jeremy Johnston. We also had help this week from Daniel Moritz-Rabson. Our staff also includes Alina Kulman, Daria Klenert, Elsa Hernandez, Gabriel Roth, Lyric Bowditch, Morgan Levey, Neal Carruth, Rebecca Lee Douglas, Ryan Kelley, Sarah Lilley, and Zack Lapinski. Our theme song is “Mr. Fortune,” by the Hitchhikers; all the other music was composed by Luis Guerra.