From The Wall Street Journal:
In the mid-1990s, a group of software developers applied the latest computer learning to tackle a problem that emergency-room doctors were routinely facing: which of the patients who showed up with pneumonia should be admitted and which could be sent home to recover there? An algorithm analyzed more than 15,000 patients and came up with a series of predictions intended to optimize patient survival. There was, however, an oddity—the computer concluded that asthmatics with pneumonia were low-risk and could be treated as outpatients. The programmers were skeptical.
Their doubts proved correct. As clinicians later explained, when asthmatics show up to an emergency room with pneumonia, they are considered so high-risk that they tend to be triaged immediately to more intensive care. It was this policy that accounted for their lower-than-expected mortality, the outcome that the computer was trying to optimize. The algorithm, in other words, provided the wrong recommendation, but it was doing exactly what it had been programmed to do.
The disconnect between intention and results—between what mathematician Norbert Wiener described as “the purpose put into the machine” and “the purpose we really desire”—defines the essence of “the alignment problem.” Brian Christian, an accomplished technology writer, offers a nuanced and captivating exploration of this white-hot topic, giving us along the way a survey of the state of machine learning and of the challenges it faces.
The alignment problem, Mr. Christian notes, is as old as the earliest attempts to persuade machines to reason, but recent advances in data-capture and computational power have given it a new prominence. To show the limits of even the most sophisticated algorithms, he describes what happened when a vast database of human language was harvested from published books and the internet. It enabled the mathematical analysis of language—facilitating dramatically improved word translations and creating opportunities to express linguistic relationships as simple arithmetical expressions. Type in “King-Man+Woman” and you got “Queen.” But if you tried “Doctor-Man+Woman,” out popped “Nurse.” “Shopkeeper-Man+Woman” produced “Housewife.” Here the math reflected, and risked perpetuating, historical sexism in language use. Another misalignment example: When an algorithm was trained on a data set of millions of labeled images, it was able to sort photos into categories as fine-grained as “Graduation”—yet classified people of color as “Gorillas.” This problem was rooted in deficiencies in the data set on which the model was trained. In both cases, the programmers had failed to recognize, much less seriously consider, the shortcomings of their models.
We are attracted, Mr. Christian observes, to the idea “that society can be made more consistent, more accurate, and more fair by replacing idiosyncratic human judgment with numerical models.” But we may be expecting too much of our software. A computer program intended to guide parole decisions, for example, delivered guidance that distilled and arguably propagated underlying racial inequalities. Is this the algorithm’s fault, or ours?
To answer this question and others, Mr. Christian devotes much of “The Alignment Problem” to the challenges of teaching computers to do what we want them to do. A computer seeking to maximize its score through trial and error, for example, can quickly figure out shoot-’em-up videogames like “Space Invaders” but struggles with Indiana Jones-style adventure games like “Montezuma’s Revenge,” where rewards are sparse and you need to swing across a pit and climb a ladder before you start to score. Human gamers are instinctively driven to explore and figure out what’s behind the next door, but the computer wasn’t—until a “curiosity” incentive was provided.
Link to the rest at The Wall Street Journal (PG apologizes for the paywall, but hasn’t figured out a way around it.)
When PG was in high school, The Mother of PG aka Mom, made PG take a typing class. Learning how to type and type quickly might have been the most useful thing PG learned in high school.
PG earned money in college by typing papers for other students who couldn’t type. He charged a high per-page rate and collected it because he specialized in typing for procrastinators. If you finished your rough draft at midnight, PG would show up with his portable typewriter and turn it into something your professor would accept at 8:00 am the next morning.
PG kept typing through law school, typing all his law school exams and whatever parts of the bar exam that could be typed.
When PG was a baby lawyer, he had a client who was also working with a fancy law firm in Los Angeles. He went over to the fancy law firm on occasion to meet with the fancy lawyers who worked there (He rode up the elevator to the law firm’s offices with Marlon Brando one time and Kareem Abdul-Jabbar another time. Kareem looked a lot less dissipated than Marlon.)
The fancy law firm had the first word-processing computers PG had ever seen. The firm had eight of these computers and they were operated by the fastest and most-accurate typists PG has ever seen. The machines and operators were in their own glass-walled room and at least a couple of typists were on duty 24 hours a day. (PG was there at midnight to pick up a rush project and one of them delivered a finished contract to him at midnight.) PG just checked and each of the computerized word processors cost over $180,000 in 2020 dollars.
PG was the first lawyer he knew who bought a personal computer for his law office. Fortunately, personal computers could also be used for playing videogames, so the price had come way, way, way, way down from $180,000.
Because he could still type fast, PG learned how a word processing program worked. Plus a bunch of other programs. He quickly started using his PC for legal work. Why type a document you used for a lot of different clients over and over when you could just type it once for Client A, save a copy, then use the copy as the basis for Clients B-Z?
PC’s were evolving quickly, so when a more powerful PC was released, PG bought one and moved his prior PC to his secretary’s desk and showed her how to use the word processing program.
Since PG always hired the smartest secretaries he could find, within a couple of weeks, she was better with the word processor than PG was.
For a variety of different reasons, PG started doing a lot of divorces for people who didn’t have a lot of money (the local Legal Aid office thought he did a good job and sent a lot of clients his way).
In order to make money doing divorces for people who didn’t have much (Legal Aid never had enough money, so it didn’t pay much for a divorce either), PG built a computer program so he could do the paperwork necessary for a divorce very quickly.
The wife’s name, the husband’s name, the kids names and ages, the year and make of the rusted-out pickup, the TV, sofa, etc., were the same from start to finish, so why not type them into a computer program once, then build standard legal forms that would use the same information for all the various forms the state legislature, in its infinite wisdom, had said were necessary to end a marriage?
PG has meandered for too long, but to conclude quickly, he ended up building a commercial divorce computer program he named “Splitsville” and sold it to about 20% of the attorneys in the state where he was practicing at the time.
(In the United States, the laws governing divorce AKA Dissolution of Marriage vary from state-to-state, so Splitsville couldn’t cross state lines. Even though the fundamental human and property issues are the same any time a marriage is ended, PG suspects there are enough idiots in any state legislature to shout down anyone who says, “Why don’t we just do it the way Alabama does instead of concocting a divorce law of our own?”)
Which means PG doesn’t have enough knowledge to build artificial intelligence programs as described in the OP, but he does have an intuitive grasp of how to persuade computers do things you would like them to accomplish. PG and computers seem to understand each other at a visceral level even though PG is less like a computer than a whole lot of smart people he knows. It’s sort of a Yin/Yang thing.
His liberal-arts assessment of the problem described in the OP is that the computer scientists in the OP haven’t figured out how to ask the ultra-computer for the answers they would like it to provide. A computer can do smart things and dumb things very quickly, but useful output requires understanding what you really want it to do, then figuring out how to explain the job to the computer.
But, undoubtedly, PG is missing something entirely and is totally off-base.
The Alignment Problem may be a good description of both the computer issue described in the book and of PG himself.