{"id":3847,"date":"2022-01-17T09:46:00","date_gmt":"2022-01-17T09:46:00","guid":{"rendered":"https:\/\/techpolicy.org.il\/?p=3847"},"modified":"2022-01-18T07:49:47","modified_gmt":"2022-01-18T07:49:47","slug":"adversarial-machine-learning-research-developments-dangers-and-implications","status":"publish","type":"post","link":"https:\/\/techpolicy.org.il\/he\/blog\/adversarial-machine-learning-research-developments-dangers-and-implications\/","title":{"rendered":"An Introduction to Adversarial Machine Learning \u2013 Developments in Research, Risks and Other Implications"},"content":{"rendered":"\n
\"artificial<\/figure>\n\n\n\n

While there is an extensive ongoing public debate about the ethical difficulties of relying on artificial intelligence-based models, little attention is being paid to the ability of malicious actors to disrupt and deceive such models. This article introduces the evolving field of \u2018adversarial machine learning\u2019, its risks and various implications.<\/p>\n\n\n\n

In 2016, Microsoft launched a Twitter chatbot in the form of an American teenager called Tay. The chatbot was designed to communicate with the platform’s users for entertainment purposes, but first and foremost, for learning from the interaction with the users to improve the model on which it was built. In under a day from its launch and after posting about 96,000 “Tweets”, Microsoft was forced to issue an apology and to announce the suspension of the chatbot. Tay, as it turned out, also learned to imitate the abusive behavior of Twitter users who quickly realized that they could adversely affect her learning \u2013 and followed through \u2013 and began tweeting racist and antisemitic tweets herself. This incident was among the first to demonstrate in such a tangible and disturbing manner the feasibility of manipulation of and disruption to artificial intelligence (AI) models, even without full access to the model and its parameters.<\/p>\n\n\n\n

While there is an extensive ongoing public debate concerning the ethical issues in relying on algorithms for decision-making and the various human biases that models’ creators might implement into them \u2013 little attention is being paid to the power of foreign agents, often malicious ones, to disrupt these models. This article introduces the evolving field of Adversarial Machine Learning (AML) and identifies its various implications.<\/p>\n\n\n\n

\"\"
\n

<\/p>\n<\/div><\/div>\n\n\n\n

1. One of the “Tweets” published by Microsoft’s Chatbot in response to a user’s question: https:\/\/fortune.com\/2016\/03\/24\/chat-bot-racism\/<\/a><\/em><\/p>\n\n\n\n

What is Adversarial Machine Learning and How Fooling Models is Made Possible?<\/u><\/strong><\/p>\n\n\n\n

AML is a machine learning technique that allows taking advantage of, disrupting, or fooling AI models, with or without access to the model itself and to the data on which it is trained and operating. In order to understand how fooling models is made possible, it would be useful to first get a basic grasp of how machine learning models work.<\/p>\n\n\n\n

Machine learning (ML), a subset of AI, is the general term for the field in which algorithms can autonomously, namely without being explicitly programmed to do so, learn and improve to perform complex tasks, by imitating human functionalities, such as learning by example and by analogy. ML is performed in two main ways: Supervised Learning \u2013 in which a human moderator collects data, feeds it to the machine and labels it until the machine learns to perform this classification process by itself. The second method is Unsupervised Learning \u2013 in which the machine is fed with data, but it performs the labeling and the classification processes completely autonomously by dividing the data into groups and discovering patterns according to certain characteristics. In 2012, the field of ML has experienced significant acceleration<\/a> with advances in the development of Deep Learning \u2013 an ML capability based on numerous layers of artificial neural networks (ANN) that simulate the behavior of the human brain and are able to perform complex calculations with a very high level of accuracy. Today, we make extensive use of such models, from Siri and Alexa, through the Netflix recommendation system \u2013 to autonomous vehicles. In many cases, deep learning models are considered as a ‘black box’ since the inner workings of their computation process are non-transparent and ambiguous and consequently \u2013 non-explainable and uninterpretable to those applying them, to the AI-system subjects (seeking to legally challenge AI-related harm without being able to base their claims on the model’s computation process), and sometimes even to the model developers themselves. But despite the accuracy, complexity and ambiguity of ML models (or, perhaps, because of it), it turns out that these models can be rather easily tricked and fooled<\/strong>.<\/p>\n\n\n\n

A model’s disruption or fooling can be attempted by feeding the model with misleading information during its training process or during its implementation. The disruption can be either targeted, so that it causes the model to classify input X as if it were Y, or untargeted, which simply causes the model to not classify input X as X. Since the first type is considered more complex and expensive in terms of time and resources, the second method is more commonly used. The methods applied to disrupt models, can also be classified according to the level of access the attacker has to the model:<\/p>\n\n\n\n