The AI logic bomb problem

By Péter MARTON


(The source of the illustration is this video.)

Elon Musk brings up a familiar point about what could potentially go wrong with AI. This is not a novel argument, but it is so clearly formulated here that really everyone should understand it:

"AI doesn't have to be evil to destroy humanity – if AI has a goal and humanity just happens in the way, it will destroy humanity as a matter of course without even thinking about it, no hard feelings," Musk said.
"It's just like if we're building a road and an anthill happens to be in the way, we don't hate ants, we're just building a road, and so goodbye anthill."

And let's not forget that people are also perfectly capable of setting goals that result in defining other people as obstacles to be removed from the way. So it wouldn't have to be AI vs. all of humanity, even.

On the other hand, if you are interested in a more enjoyable, literary take on this, here is Philip K. Dick predicting this problem in 1955 – in an excellent short story titled "Autofac" (referring to "automatic factory").

I call this the "AI logic bomb" problem given that AI merely continues to execute its functions, only this time in a harmful way, potentially even to its former users, unforeseen, and uncontrollably (not being "used" in a real sense any more).

Comments

Post a Comment

Popular posts from this blog

Station Eleven (the book): A Review of the Post-Apocalypse

Image
By Péter MARTON Having just finished Emily St. John Mandel's Station Eleven (the 2015 winner of the Arthur C. Clarke Award), a part post-apocalyptic, part pre-apocalyptic novel with (inevitably and yet mostly just implicitly) the apocalypse in its centre, here are a few quick notes, in praise as well as criticism – not as a literary review, but mainly in reaction to the plot: its plausibility and its implications. As a work of literature I really liked this book – I enjoyed it, even. It is moody and haunting, as many would say. All the characters want to be somewhere else, even some time else. The story is effectively a collection of their memories upon memories of times, places and faces past. What follows here, however, is the dirty work – the ugly analysis of probabilities and plausibilities that's more interesting from a social science vantage point. A raw take from my part. Feel free (or invited, even) to add to this, or to criticise any element of my ass
Read more

Developments related to ISIS, and much else, because this is, of course, related to pretty much everything else

Image
By Péter MARTON Because I need to speak at an event about the situation in Iraq/Syria tomorrow, here are some of the recent trends and developments related to ISIS -- my digest of these. These are research notes, effectively. For starters: Aaron Zelin notes how much the ISIS media presence diminished since the fall of Mosul: "In particular, after the fall of Mosul in July 2017, the IS distribution of governance-related media dropped precipitously (by 66 percent) until the complete end of such media activity (on September 12 in Syria; September 16 in Iraq). For reference, at its apex, between June and August 2015, IS released 3,762 pictures related to governance activities in Syria and 3,305 in Iraq. When Mosul fell, between May and July 2017, IS only released 315 pictures related to governance in Syria and 171 in Iraq. And the most recent count prior to the mid-September 2017 rupture was 142 in Syria and 113 in Iraq, between July and September 2017." And yet...
Read more

Non-State 2-1 (2 Mar 2018): Technological Adolescence + Bitcoin Mining

Image
By Péter MARTON In today's brief post I am invoking the spirit of the late scientist Carl Sagan. Citing his words, it is interesting to reflect on the implications of the ongoing bitcoin rush that may make you wonder if the bursting of the bitcoin bubble (anticipated by some) may be the worst thing that could happen. 1. You can read here from Carl Sagan , addressing the question of whether humanity should invest an effort into trying to find out about the presence or absence of alien life out there in the universe. Sagan quickly concludes that given the enormous distances in light years, interstellar monologue will in any case be more (twice as) likely than an interstellar dialogue. Even so, he ultimately comes to the conclusion that receiving a message from a superior civilisation may determine humanity's fate, for in that message one may find the answers as to how the period of "technological adolescence," with its dangers, could be survived by us. In his w
Read more