who says wait?

on september 26, 1983, just after midnight, a soviet computer told stanislav petrov that five missiles were coming. his job was to trust it. he didn't. he called it a false alarm and waited, without knowing if he was right. it was sunlight reflecting off the clouds.

on september 19 i told that story at a ppt party, in cusco, in spanish. i called it ¿quién dice espera?, who says wait? it started in the bunker, with the missile count, and ended in the present. i built it on three ideas:

• the waiting. the world was saved by someone who could say "wait". today ai is sold as the end of waiting.
• the three powers. petrov could check what the machine said, he knew his computer had no brain, and he had the last word: nothing flew without him. today all three are turned around: we don't understand ai from the inside, it's no longer a machine without a brain, and more and more it detects, decides and acts on its own.
• the filter. in 1983 the system punished petrov for waiting. today the people asking to wait are leaving the labs with warnings. i called them the modern petrovs.

the last slide was two unlit candles and one line: today the chair is ours.

the room asked why petrov did it. i said he didn't want the responsibility of starting a nuclear war. someone answered that then the problem is the human.

that's the beautiful thing about people: we have intrinsic values.

others said ai doesn't worry them much yet, that they would worry more about physical robots. someone else said they don't believe ai will reach superintelligence. i told them yes, that could be true, but what the ai labs want is exactly to reach that superintelligence.

nobody agreed with everything. i liked that. that's what a talk is for: thinking together, even against each other.

a week later, on september 26, i told the same story again: from memory, with a candle, at the first anniversary of de profundis. that night has its own post.