Hello, and welcome back. What day could be more appropriate for TGIF than September 11? TGIF on September 11 reminds me of me, every year, trying to celebrate my birthday on Adolf Hitler’s birthday. April 20, baby! A terrible and ruthless commander was born that day, and also Adolf Hitler. Someone’s always doing something terrible in honor of him, and I’m just trying to have a tiramisu. (Look out for a package of amazing Free Press stories on the 9/11 anniversary in a special edition of The Front Page later this morning.)
Quickly, before we get to the news: If you haven’t heard, we are hosting another supper club. Sign up here. And my Bar is doing an event with author and historian Simon Sebag Montefiore on October 1. Get your tickets here.
→ Writing resignation letters from AI companies is the chic new thing: One young man, Jacob Coxon, left OpenAI and joined Anthropic and then resigned four months later—“I resigned from Anthropic today,” he writes—before describing the danger in developing artificial intelligence and asking people to please stop. “If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because ‘it’s happening anyway’—or take this moment to call for different conditions?”
Coxon’s post went super viral. Even Elon Musk was shocked by the virality: “I don’t think this has ever happened for a post from a new account with almost no prior activity.” Jacob’s argument is that everyone building AI knows that it could kill us, so Americans have to stop building AI and, I guess, hope that China and other countries just voluntarily stop too? Or that everyone in the world pinky promises to not do anything weird? Evan Hubinger, the Anthropic lead in charge of “alignment,” which means making sure AI doesn’t decide to kill us, actually agreed with our Jacob on the stakes: “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.” I agree too! Can someone tell us what to do about it? Jacob? Evan? Read this report on “misuse” to understand what Anthropic is combating, and what Iran and the others will use AI for as soon as they can (okay, fine, partly they used it just to plan the Supreme Leader’s funeral, which is a great time to bring in Claude).
Like all arms races, everyone has to agree to stop doing the race for it to work. That is not what’s happening. But smart young Americans have been told for a generation that we are the worst people on Earth (aside from Israelis) and should cede power to any other more ethical nation (CHY-NA, Pakistan, anywhere is better than Mom’s house). So we do have a wide swath of intelligent people ready and willing to be subjugated. They want us to put all our tools down and sing “Kumbaya” until all the peoples of Earth sing together, hand in hand, or until a better leader (Kim Jong Un?) comes to guide us. Building the thing that we need to win feels a little icky. For context, this is Jacob’s profile pic:
What sweet, gentle Jacob (the best camp counselor in all of Maine) can’t imagine is that genuinely malevolent nations sit outside our gates, some hoping to kill our society and subjugate us to their interests. I don’t mean to offend you, Jacob, I just think we’ll also need to have AI tools given this human reality. Please don’t cry, Jacob. Oh god, guys, we’ve made him cry. He’s wearing his backpack on the front now.
Read more
