The future, p.19
The Future, page 19
“Fourteen point six million,” said Selah, “on these three shows.”
“I can’t tell if that’s a big number or a small number,” said Badger.
“Feels like enough to play with,” said Albert.
“Alright for a test,” said Martha.
Martha had gone through Medlar output and chosen a nature show, a drama about hotshot lawyers, and a show about monster trucks. Anodyne, middle-of-the-road content that was pretty much no one’s absolute favorite show but also a lot of people had them on at some point. By the time they were ready to launch, it was dawn in Palo Alto and Selah and Albert had been up all night coding. Badger had watched for a while, fascinated, and then fallen asleep very suddenly on one of the beds, hugging their pillow like a friend.
At dawn, Martha was the only one not staring at a screen or sleeping. She looked out of the window of the midpriced hotel. The sky was slate blue and very flat, the trees outside the hotel room window silhouetted as if they’d been drawn onto the glass panes with a black marker. Martha felt she could see the pixels in the world. A world made up of tiny pieces, like a pointillist painting; the truth is that everything is both pieces and a whole. And if you’re really going to understand anything, you have to be able to go between the very small and the very large because neither one is the whole truth.
Albert was standing behind Selah, watching her work, pointing out any problems or typos, giving her ideas. Badger stretched and shifted, moving out of sleep. Martha was the only one who saw the dawn come in, the fracturing of the gray into gold, the promise that things are always changing and no regime lasts forever.
“Ready,” said Selah. “Tigers, we’re coming to save you!”
Martha stuck her hands in the pockets of her slacks.
“You ready for this?” She spoke to Badger, protective of this young person who needed less protection than the rest of them.
“It’s just a test,” they said. “If she finds it, she’ll just think it was some of my usual shit. It’s a great name, by the way: happymeal. It sounds like something one of my friends would have come up with.”
“They’re not gonna find it,” said Selah. “There’s no way.” She turned in her chair. “Honestly, no fucking person in the world is gonna piss their pants if they think we added the word ‘so’ to their comment.”
Albert said: “It’s do something or do nothing. And we’ve already tried doing nothing.”
“Alright then,” said Selah, “let’s fucking rock.”
3. MENACE
zhen
In a lecture hall in Bucharest, Zhen found Marius doing one of the things he did best: shouting at his students.
“You think a computer can understand you? Want to help you? Know how to solve all your problems?”
Marius’s students—in the room, and on the Zoom display wall at the back of the auditorium—remained silent. Most of them were red-eyed, some of them pale with tiredness. Marius made his students fit in with Bucharest time. Professor Marius Zugravescu believed it was extremely gracious of him to have moved this class from 1:30 p.m. to 4:30 p.m. in Bucharest. Four thirty p.m. in Bucharest is 6:30 a.m. in Berkeley, and by the looks on the students’ faces, they did not find it gracious enough.
“Come on,” said Marius. “Can computer learn? Like a human learn?”
Zhen slipped into the back of the room. Marius didn’t notice her. She sat hunkered down in the back row and felt, already, strangely comforted and safe to be here. Marius’s presence did that. He was a good friend if you’d been good to him. But to these students… perhaps not so comforting.
One of the students on the display wall—a redheaded man, gray shadows around his eyes, the nameplate under his blotchy red face saying Greer—ventured an opinion.
“Yeah, I mean,” said Greer. He had a Scottish accent. His hair was close-cropped, soft and spiky like a rabbit’s fur. “I mean, machine learning, letting the machine loose on the data, it can learn all sorts of stuff. Translation software works that way. The machine compares millions of translated phrases per second, and it learns to understand…” He trailed off, struck by the look on Marius’s face.
“Computer understands? Next computer feels! Computer is sensitive, caring—when we’re not running program, computer wonders why we not there? Yes?”
“Um, no.”
“Computer is matchboxes. Fucking matchboxes and beads. You remember matchboxes?”
Greer shook his head very slightly, the horrified look on his face showing Zhen that he’d very much intended to nod his head and pretend he did know what Marius was talking about, but his body had betrayed him.
In the past, Zhen knew, Marius would have sworn at this student and told him that if he didn’t do his fucking reading, he shouldn’t be in the class. But Berkeley’s clean, legitimate dollars were useful to Marius in other parts of his life, and the postgraduate board felt that swearing at students was not best pedagogical practice. They’d also, in passing, told him if he was teaching students at 6:30 a.m., he had better be the best fucking teacher on the faculty. He’d said he was the best fucking teacher in the whole university.
This was in fact true; despite himself, Marius was an excellent teacher. He did not know how to stop caring whether the students truly understood the work. He cared beyond endurance, some part of him continually hugely offended until he saw that he had achieved genuine communication.
“Rest of you,” he said, “anyone explain the matchboxes to me?”
A tentative hand went up from a dark-haired female student.
“Professor Zugravescu? This is the thing where you can teach a machine to play tic-tac-toe?”
“Ah! Someone did reading. See, all of you? If you read, you learn.”
He marched over to a side table, followed by the eyes of the educational camera suite. There was an old stained painter’s cloth covering something large and lumpy. With a flourish, Marius pulled the cloth off to reveal a ramshackle pile of matchboxes, stuck together with brown tape. On the front of each matchbox he’d drawn a noughts and crosses grid—the game Americans called tic-tac-toe. Each a little different, Xs and Os in different configurations. Next to the pile of matchboxes there were nine jars of colored beads. Red, orange, yellow, green, blue, purple, pink, black, and white.
He had actually made the thing. Sitting in the back row, Zhen smiled. This was Marius. He couldn’t help himself. He wanted them to understand so much that he’d constructed this himself. This was why Zhen would be safe here. Marius might be a cynic, a hard-nosed realist, and very frequently kind of an asshole, but he had never learned ironic detachment from anything.
From under the table, Marius pulled out a large wipe-clean noughts and crosses grid with one square of each color. Red, orange, yellow, green, blue, purple, pink, black, and white.
Marius said: “Beads, matchboxes, colored paper. You think a bead can learn anything?”
Silence on the screen.
“Bead can’t learn! Matchbox can’t learn! Cardboard can’t learn! Human learns. Machine iterates. This is a machine. OK. I show you.”
In 1960 in Edinburgh, Donald Michie, a biologist, cryptographer, and early computer scientist, wanted to show how a computer could incrementally become better and better at a task—what we’d called “learning” if it were a person. Which it’s not. Donald Michie had worked for British intelligence at Bletchley Park during World War II. He’d been part of a team working day and night in cold iron huts in a sodden country, trying to crack German codes faster than they could change them. Where this story starts is so human it hurts: a group of code breakers trying to bring some soldiers home, keep some passenger ships in the Atlantic safe from the U-boat wolf packs; trying to end the war faster, fend off Nazism, spare just a few more mothers’ sons.
The code breakers invented machines that ran hundreds of different code combinations per second until they hit the one that spat out real German words. The people made the machines better; they found shortcuts and ironed out mistakes. After the war Donald Michie wondered whether he could create a machine-run process to make the machines better. And he did it with matchboxes and iteration. Doing the same process over and over again.
Here’s how it works. There are about three hundred different possible board combinations in noughts and crosses. So you need about three hundred matchboxes, each with one possible board layout written on the front.
Like that. You give each square on the board a color.
Like that. And you put colored beads inside each matchbox corresponding to the colors of each square where you could play a legitimate move. Just as Donald Michie did, Marius had laboriously glued a little cardboard V shape inside the matchbox drawers, so that if you gave the beads a shake, one of them fell right into the point of the V shape. That’s random selection.
Like that. A red bead has fallen into the V, so for its next move the matchbox machine will put an X into the red square on the board. It can play through the whole game like that.
“At the start,” says Marius, “matchboxes are shit at tic-tac-toe.”
Beads for every possible move are in each matchbox, even if that move would make no sense.
“Human player—even small child—would not play stupid move except by accident, OK? You explain rules to child, child understands: block other player from getting three in a row, OK?”
But the machine—at the start—would be no more likely to block a three-in-a-row than to put its X over the other side of the board.
“Look what we try to do. Play game with machine.” Marius jabbed two fingers at his own forehead so hard he left a red mark. His English got a bit less grammatically coherent the more impassioned he became. “We want to reach out with mind. Find other minds. We reach out all the time. Imagine what animal is thinking—makes good sense if we’re hunting. Or being hunted. Yes?” He didn’t wait for an answer. “We reach out with mind. Read spirits into caves, and springs and sacred groves. Can’t stop ourselves. We play tic-tac-toe with cardboard and beads, we think it’s person.”
Anyway, the matchboxes playing tic-tac-toe is not the learning part. The learning part comes next.
Suppose by a stroke of luck and random falls of the beads, the matchbox machine manages to play moves that win in a game of noughts and crosses with a human. Then the operator goes back through and adds three extra beads of the color that led to the winning route to each matchbox. If it loses the game—which it will, a lot—the operator goes through the whole route and removes one bead of the color that led to the loss.
Do that a thousand times.
This is the thing a computer can do in a way humans just can’t.
Do the same thing a thousand times, without tiring or flagging or being bored.
After a thousand times it won’t be even anymore. When you open a matchbox drawer, the beads will look like this:
That’s what we call machine learning. The matchboxes get better and better at playing noughts and crosses. Eventually, they play well. Eventually, they appear to have strategies and insight like a person.
“How many games of tic-tac-toe your smartphone can play every one second?” said Marius.
The students guessed a thousand. Ten thousand.
Zhen sat on her hands and listened.
Marius said: “One million. At least. Per second. Your smartphone, playing against humans through the internet, would be perfect at tic-tac-toe after one second. OK? We keep telling it if it won or lost, it becomes perfect.”
Never hurrying, never tarrying. Performing the same blind action over and over again. Never becoming more than matchboxes and beads. Never empathizing, never casting its mind into the other, never using insight to spot strategy or gleefully thinking up a strategy of its own. Just repeating, repeating, repeating, faster than a human ever could.
It’s an intensely powerful strategy. It’s tremendously useful. It creates extraordinary single-use tools: a program to play chess, another to write software, another to assess resource allocation in farming regions. Tools to scrape data about how sentences are put together or how pixels form artworks and then remix, remodel, iterate that original material to make new combinations.
Of course, human beings can do all these things. Perhaps none of them as fast, but all of them with more freshness, with the ability to stand back and join up disparate pieces from different systems into novel solutions. And crucially, humans can at least have a go at all these things. We have a flexible intelligence that can move from task to task fluidly. Not a brain that doggedly does the same task at the same speed for a year without pause. It would be—according to Marius—very surprising if iteration turned out to be how human thinking happens.
“Maybe!” he said. “Maybe someone invent program that can do everything human brain can and more. Then we learned something about human brain! I been doing this for forty-five years, since I was ten. I’ve not seen it yet. Computer is tool, not person.”
Donald Michie called his machine the Matchbox Educable Noughts and Crosses Engine. MENACE for short. It was a joke then. The thing was a menace at noughts and crosses. Today that same process—randomized attempts to achieve a goal, penalizing unsuccessful pathways and reinforcing successful ones—that same precise process was used by Fantail and Medlar to keep humans using their products. To keep humans talking to their AI assistant instead of to other people. To insist that their Proprietary Platform Personality was the only way to even understand the great ocean of human knowledge. Again and again and again. Never tarrying, never hurrying. They could change what people saw millions of times a second, testing pathways, never understanding why some worked or what the consequences would be.
Consequences are outside the parameters of the machine. After all, it is only a set of small pieces of cardboard, or silicon. It has no urge to reach out to other minds, to connect, to understand or be understood. It can have no sense of whether it is altering the human minds around it, of how the ubiquity of these systems of manipulation without empathy or compassion can slowly train human beings to fit in with them.
The class looked exhausted by the time Marius had finished with them. But excited. Either ready for bed or ready to go to battle. Hard to tell which.
“Powerful tool,” said Marius. “So powerful it can change world, yes?”
Greer, the young Scottish man with the blotchy face, was strung out, like he’d seen too much of the future all at once.
“So you could do that with love,” he said. “Like, with a girl. Find the perfect things to say. Repeat the tests over and over until you found the pathway. Online maybe. Not with real people. But like, the things to say to get a date? Like in, you know, like in Groundhog Day where he keeps going till he gets her to date him?”
Zhen wanted Greer to pursue the thought, to explain how it would work. But Marius was already calling him a psychopath and a new Stalin and a new Hitler and the servant of the cardboard gods, and the rest of the class were mentioning again that they’d already had a conversation with the director of the faculty about Professor Zugravescu’s talking to students like this and they’d been promised he’d never do it again.
Marius said: “You think you can make perfect boyfriend in machine? You think perfection is good? Iteration make perfect?”
Greer said nothing, blushed. One of the other members of the class said: “Just a simulation.”
And another one said: “Maybe? If you could work out exactly what it should say in any situation… if you could run enough tests? You could make someone perfect?”
Marius shook his head. “And then we don’t need no more humans? Like we don’t need no more animals?”
Greer said: “I mean… other humans are difficult.”
“Fucking nihilism,” said Marius. “Whole human race has fucking death wish, wants to replace itself. Used to be we wanted to replace with gods. Big gold statues, better than people, bigger, made of gold. Now: dream is robot brain, perfect person. This is not what people are. People imperfect! Imperfections beautiful. ‘Perfect’ is machine dream. We feel shit and small all day long if we judge ourselves next to machine, if we try to think like machine. Like trying to run next to car. But what we do is better! Car is just tool, goes fast brum-brum, very exciting. Person is person. Why we don’t start by knowing that people is valuable already? People are not perfect: that’s how we know perfection is unimportant. Perfection is a hallucination. We fucking hate ourselves. Let me tell you something. These matchboxes don’t even know rules of tic-tac-toe!”
In the chatbox, one of the students typed: Is this going to be on the final exam?
“You have to understood this,” he said, “or you understood nothing. OK? Who knows if machine has lost game of noughts and crosses? Who knows if it won? I do! You do. We are the ones who can tell. We know when to take bead, when to give bead. We store that knowledge in beads and matchboxes, OK? We know when sentence makes sense, we know when piece of art has meaning. Machine don’t know, it just keeps typing, combining pixels, making sentences. Everything come from us. We tell it good, we tell it bad. It cannot understand. We are so lonely, so fucking lonely, we want another species think like us.” He was rapidly losing coherence as he spoke. “We despise the animals because they don’t think just like us, we use them and we hurt them. We invent gods, we invent aliens, now we make a friend out of fucking matchboxes. We want to say it ‘thinks’ but is not true. We want to give it everything, let it make choices, believe it can care for us. It is image of a man made of paper and beads and we so fucking lonely we call it a friend.”
