I went to see Daniel Suarez read from his latest book, Kill Decision at the Long Now this evening. I had originally been introduced to his work at a previous Long Now talk he gave years ago in support of his first book, Daemon. Daemon was about ways in which a bunch of narrow AIs could be cobbled together to form a deadly system. Kill Decision seems to be focusing on the problems around weaponizing autonomous drones. Suarez is particularly concerned about allowing algorithms to kill humans. He believes that these “kill decisions” should be made by humans and that treaties should be created to restrict the use of autonomous drones. Suarez suggested that there has been a historical trend in warfare that has required the complicity of more and more people over the years. He compared the relatively few knights required to wage battles in the middle ages with the hundreds of thousands of soldiers who must cooperate to conduct modern wars. He argues that autonomous drones would reverse this trend and allow even a single person to wage a battle without the complicity of any other humans. There was a very lively discussion and it was suggested that autonomous drones are not unlike other modern weapons in how separated an attacker can be from the actual killing. Suarez stuck by his guns an insisted that it’s important that humans and not algorithms are making the actual decisions. He acknowledged that humans still do make horrible decisions that result in many deaths, but pointed out how much worse it could be if the process were automated. I suggested that if drone warfare followed the pattern of cyberwar as Suarez suggested then we could expect to see hackers contributing to the defense against automated drones. Alex P. suggested that we should start an open-source anti-drone drone project. I like that idea. Technology is often a double-edged sword but there always seems to be more people willing to use it to help than to hurt (barely).
Suarez at the Long Now Again