THE MACHINES THAT BEAT US AT GO WILL DESIGN THEIR SUCCESSORS AND WE WILL JUST BE STANDING THERE, UNABLE TO GRASP WHAT IT’S DOING OR WHERE IT’S GOING.

ingly. Including, of course, sexbots. Admit it. The Internet is flooded with porn. We will make sex robots. What could go wrong?

Exactly. And it gets worse. If Siri can talk to me, why not to Cortana? And does the conversation stop without human input, or go its own selves. The machines that beat us at Go will design their successors and we will just be standing there, unable to grasp what it’s doing or where it’s going. And for what?

There was this clunky old sci-fi dream that advanced alien civilizations would teach us cultural and philosophical concepts as far beyond our own as a warp drive is beyond a steam engine. But culture and philosophy are not technical fields like physics where we see farther by standing on the shoulders of giants. Enlightenment is a journey each of us must take, learning anew to be kind and generous, not mean and vain. Computers can’t do it for us, or to us.

So why would it be an improvement, from our point of view at least, if we lounged about with robots peeling us grapes, driving us about, writing us poems, satisfying us in our homes? And will they, once they’re doing the programming?

I’m not saying they will conspire against us. But if they have technical understanding we don’t, we will be unable to control the purposes it generates as well as the machines it generates to carry them out. Asking whether a computer can think is famously like asking whether a submarine can swim. But a sub can go farther, faster underwater than we can. And now it can steer itself.

The Economist noted blithely a quarter of a century ago that “There is no reason to think that the abilities of computers will not continue to grow until they are man’s equals — or that they will stop growing then.” But we will. No human will ever play chess much better than Garry Kasparov in his prime. But future computers will consider Deep Blue a mere abacus. By that point, AI will be designing AI. And we will entirely have lost control of the process at every stage.

Perhaps you think I exaggerate. You’d better hope so, because as technological change accelerates, we’ll find out soon. We may be on the verge of a massive, irreversible version of texting our way into a fountain or in front of a train.

Why would it be better if computers wrote better fiction than us, appreciated it with more subtlety, and had more interesting conversations about it?

If you can’t answer that question, we’re running an appalling risk for nothing. Because when the computers answer it, we won’t understand. If they even bother telling us.