{"id":179960,"date":"2017-08-29T20:04:00","date_gmt":"2017-08-30T01:04:00","guid":{"rendered":"https:\/\/www.panix.com\/~msaroff\/40years\/2017\/08\/29\/im-sorry-dave-im-afraid-i-cant-do-that\/"},"modified":"2017-08-29T20:04:00","modified_gmt":"2017-08-30T01:04:00","slug":"im-sorry-dave-im-afraid-i-cant-do-that","status":"publish","type":"post","link":"https:\/\/www.panix.com\/~msaroff\/40years\/2017\/08\/29\/im-sorry-dave-im-afraid-i-cant-do-that\/","title":{"rendered":"I’m Sorry Dave, I’m Afraid I Can’t Do That"},"content":{"rendered":"

What a surprise.<\/p>\n

It turns that it is trivial to hack the most sophisticated Artificial Intelligence (AI) systems by simply training them poorly<\/a>:<\/p>\n

If you don’t know what your AI model is doing, how do you know it’s not evil? <\/p>\n

Boffins from New York University have posed that question in a paper at arXiv, and come up with the disturbing conclusion that machine learning can be taught to include backdoors, by attacks on their learning data. <\/p>\n

The problem of a \u201cmaliciously trained network\u201d (which they dub a \u201cBadNet\u201d) is more than a theoretical issue, the researchers say in this paper<\/a>: for example, they write, a facial recognition system could be trained to ignore some faces, to let a burglar into a building the owner thinks is protected. <\/p>\n

The assumptions they make in the paper are straightforward enough: first, that not everybody has the computing firepower to run big neural network training models themselves, which is what creates an \u201cas-a-service\u201d market for machine learning (Google, Microsoft and Amazon all have such offerings in their clouds); and second, that from the outside, there’s no way to know a service isn’t a \u201cBadNet\u201d.<\/span><\/p><\/blockquote>\n

Note that current high end AI models are not so much programmed as trained, and it appears that this provides an unprecedented opportunity to develop malicious software.<\/p>\n

I’m thinking that you might see an AI drone that gets the whole Manchurian Candidate<\/i> treatment in the not too distant future.<\/p>\n","protected":false},"excerpt":{"rendered":"

What a surprise. It turns that it is trivial to hack the most sophisticated Artificial Intelligence (AI) systems by simply training them poorly: If you don’t know what your AI model is doing, how do you know it’s not evil? Boffins from New York University have posed that question in a paper at arXiv, and …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[395,397,382],"class_list":["post-179960","post","type-post","status-publish","format-standard","hentry","tag-computer","tag-education","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.panix.com\/~msaroff\/40years\/wp-json\/wp\/v2\/posts\/179960"}],"collection":[{"href":"https:\/\/www.panix.com\/~msaroff\/40years\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.panix.com\/~msaroff\/40years\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.panix.com\/~msaroff\/40years\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.panix.com\/~msaroff\/40years\/wp-json\/wp\/v2\/comments?post=179960"}],"version-history":[{"count":0,"href":"https:\/\/www.panix.com\/~msaroff\/40years\/wp-json\/wp\/v2\/posts\/179960\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.panix.com\/~msaroff\/40years\/wp-json\/wp\/v2\/media?parent=179960"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.panix.com\/~msaroff\/40years\/wp-json\/wp\/v2\/categories?post=179960"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.panix.com\/~msaroff\/40years\/wp-json\/wp\/v2\/tags?post=179960"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}