* This article is part of a symposium on Kevin Jon Heller\u2019s \u201cThe Concept of \u201cthe Human\u201d in the Critique of Autonomous Weapons,\u201d published in this journal in 2023. All articles in the symposium can be found in the Harvard National Security Journal Online at https:\/\/harvardnsj.org\/onlineedition. <\/p>\n\n\n\n
Elke Schwarz<\/em>[**]<\/sup><\/p>\n\n\n\n Neil Renic<\/em>[***]<\/sup><\/p>\n\n\n\n [This essay is available in PDF at\u00a0this link<\/a>]<\/p>\n\n\n\n Critiquing a critique is a delicate matter. One risk is that the intention of the critique (either one) is distorted or misconstrued. The motivations of the respective authors can be difficult to grasp with precision, and we all read texts and ideas through the lens of our own experiences and perspectives, leading, invariably, to different, often incompatible interpretations. <\/p>\n\n\n\n Thankfully, this difficult business is made easier by the fact that we know and like Kevin (referred to as \u201cHeller\u201d from hereon). Our disagreements with Heller over the potential benefits and dangers of autonomous weapons \u2013 or autonomous weapons systems (AWS) \u2013 are substantial. Yet both his and our approach ultimately stem from the same place: a concern with the human in warfare, a commitment to battlefield restraint, and a drive to ensure that armed conflict is waged less frequently and less cruelly. From these shared foundations, healthy debate can be had. <\/p>\n\n\n\n Even those who contest Heller\u2019s stance on autonomous weapons can appreciate his breadth of analysis. His article provides several critical reminders.[1]<\/sup> First, it offers a reminder to avoid anthropomorphizing the autonomous weapon. For both proponents and critics of this technology, the human must remain key. Second, it argues in favor of taking history seriously. Not every challenge posed by autonomous weapons is unprecedented. Third, it engages the human agents of war as they are (cognitively and morally flawed), rather than as we wish them to be. And lastly, it provides a reminder that \u201chumans on-the-loop\u201d \u2013 frequently framed as a middle ground compromise between proponents and critics of this technology \u2013 is no panacea to the moral and legal challenges of autonomous weapons. Heller offers an intellectually clear and robust defense of autonomous weapons as a (potential) moral and legal improvement over human combatants in war. For those who disagree, as we most certainly do, his work forces careful reflection on the nature of this disagreement. <\/p>\n\n\n\n The key points raised in our response to Heller are three-fold. First, we begin with a brief discussion on the pitfalls of critiquing AWS critics through a techno-centric lens. The second section addresses Heller\u2019s juxtaposition of humans and AWS with respect to IHL compliance. Steering away from the realm of abstraction, we situate the use of AWS in the real world. While humans will always play a role within these systems, and thus war itself, the role and agency of such humans is likely to change. We argue that warfare within this human-AWS nexus is likely to be less \u201chumane,\u201d not more. In a slightly unorthodox manner, we conclude by raising a question. We ask to what degree it is fruitful to approach this issue (as Heller mostly does) through legal abstraction, and reflect on the responsibility we have as scholars and researchers for the ideas and arguments we set loose in the world. <\/p>\n\n\n\n Heller unpersuasively criticizes the deontological arguments made by AWS skeptics. He opens the discussion with an indicative inversion. Where critics claim, on deontological grounds, that autonomous weapons are mala in se <\/em>(meaning evil in themselves or intrinsically bad) based on their characteristics and their externality to IHL,[2]<\/sup> Heller provocatively titles his section on deontological arguments \u201cHumans as bonum in se<\/em>\u201d (as good in themselves, or intrinsically good). He then proceeds to rebut a range of arguments that cast humans as intrinsic bearers of certain qualities that are morally relevant to the business of killing. He lists these qualities in summary as follows: \u201conly humans have morality\u201d;[3]<\/sup> \u201conly humans suffer\u201d;[4]<\/sup> \u201conly humans risk\u201d;[5]<\/sup> \u201cand only humans can kill with dignity.\u201d[6]<\/sup> <\/p>\n\n\n\n We find the charges Heller raises against these specific, selectively chosen arguments to be overstated, missing the ethos of the more carefully crafted arguments put forward by AWS critics. No criticism of AWS (as far as we are aware) hinges<\/em> on an understanding of humans in war as intrinsically, or even mostly good. Few, if any, opponents of this technology argue that killing can only ever be legitimate if the agent of violence physically risks his or her own life.[7]<\/sup> Critics understand that weaponized autonomy and \u201cfree will\u201d are different, and that criticism of the former need not (and should not) presume the existence of the latter. <\/p>\n\n\n\n The deontological criticisms Heller rebuts are subtler than he credits. They categorically reject the morality of autonomous weapons use without indulging in a romanticised ideal of either humans or war. The inhumanity of AWS, to be sure, does<\/em> matter, even if humans are not ideal actors. It matters that these machines are not human; that they cannot understand human contexts; are unable to draw on plural experiences of their own; are not sensitive to vulnerability; and lack a conception of the value of life, or indeed, the horrors of a violent death. Critics are right to doubt whether autonomous weapons can align with IHL frameworks, designed as these frameworks are with human capacities and limitations in mind. Their argument is one grounded in compatibility<\/em> between flawed humans and flawed law. It does not require unjustified faith in the unassailable goodness of \u201chumanity.\u201d <\/p>\n\n\n\n Heller\u2019s responses to deontological AWS criticisms understate the ways in which machines replace human decision-making and why this replacement matters. <\/p>\n\n\n\n For Heller, the term \u201cdecision making\u201d is misused as applied to AWS. He argues that \u201cthe \u2018moral judgment\u2019 objection to machine killing necessarily assumes that an autonomous weapon \u2018decides\u2019 to take human life in a manner akin to human decision-making.\u201d[8]<\/sup> In his view, this unduly anthropomorphizes the machine. However, the claim that the moral weight of killing can only be experienced by a human decision maker is not necessarily the same as claiming that the machine makes decisions as a human would. At issue is a machine replacing a role that a human inhabits in the wider configuration of technologized warfare, a role that was hitherto understood to rest on human judgement. It is the moral weight of this <\/em>change in the character of war that warrants scrutiny. <\/p>\n\n\n\n The alleged anthropomorphizing of AWS, Heller goes on to argue, occludes the fact that a human programs the technology and is \u201cresponsible for determining which kinds of individuals and objects the machine will target.\u201d[9]<\/sup> Understood in this sense, the AWS is a mere tool of human will, actualizing operator intentions. But can this argument really stand as a rebuttal? Is pre-programming a system, whose specific actions cannot be known with exactitude (as Heller himself notes, \u201cit is impossible to know ex ante<\/em> each and every target it will engage\u201d),[10]<\/sup> really the same as making and affecting the in-the-moment decision to kill? Is the human agency and judgement we consider to be morally relevant for taking another human life present in sufficient quantities in both instances? <\/p>\n\n\n\n In Heller\u2019s view, the answer is yes. Here, he references a number of antecedent weapons and practices, such as The Mark 46 and the CAPTOR, which have a similar, although not equal, distance between human operator intent and effect.[11]<\/sup> But the CAPTOR and the Mark 46 are not comparable to the AI-enabled weapon systems that most concern critics. The level of autonomy and dynamic \u201cdecision-making\u201d differs significantly. The systems that cause the greatest concern will employ AI for target identification<\/em> based on a much broader set of parameters and the execution of the targeting function will be done with a significantly higher level of autonomy.[12]<\/sup><\/p>\n\n\n\n To further argue that AWS does not represent a novel move away from human agency, Heller offers up the practice of \u201creconnaissance by fire,\u201d in which targets are identified according to space, not individual characteristics. He uses this to illustrate that in certain warfare practices, there is a functional distance between the broader objective and the soldiers\u2019 knowledge of what targets they are firing upon.[13]<\/sup> But knowledge about target characteristics is not the same as understanding what a human target is and what it means to take it out. Even without knowing who is being killed, a soldier\u2019s understanding of the decision to kill remains complex. It is more than an automatic execution of targeting parameters, and more than the functional fulfillment of a standing order, no matter how routinized and mechanized warfighting becomes. Even if we wish to distill the application of violence down to the simple execution of targeting rules, this has hitherto been coupled with an assumption of meaningful human control. In other words, existing warfare practices like \u201creconnaissance by fire\u201d presume control by a human who understands<\/em> and can take responsibility (in the broadest sense) for the act of violence. An autonomous weapon system does not have this capacity. <\/p>\n\n\n\n The antecedent technologies and practices Heller references should not be ignored. At the very least, they force critics of AWS to reflect on important questions: what, if anything, is new, and if new, what, if anything, is problematic? The history of war is important in these discussions, not because it confirms autonomous violence as an entirely novel challenge (it is not), but rather as a means through which to better appreciate the moral and legal challenges replicated, and in many cases, intensified by this technology.[14]<\/sup><\/p>\n\n\n\n Critics of AWS take issue with the inability of these systems to bear the moral weight of the kill decision. Such critics, do not, as Heller claims, regard autonomous weapons as \u201ca subject capable of morally meaningful action.\u201d[15]<\/sup> The act of killing is morally meaningful for humans only. A machine cannot understand this meaning. In the absence of such understanding, machines must not be empowered with the human decision to identify specific targets from an often over-broad set of parameters. <\/p>\n\n\n\n Far from being ignored, the ontological and epistemological distance between a machine \u201cdecision\u201d and a human decision is significant to critics of AWS. Ironically, perhaps, Heller and his interlocutors might agree that machines do not make decisions in the human sense. But this insight has very different implications for each party. For Heller, this is a functional difference that suggests the utilitarian advantages of AWS. Critics, by contrast, raise deontological objections, suggesting that it is precisely because we cannot equate machines and humans that judging them by the same effectiveness standards makes no sense. This unbridgeable gap renders autonomous machines intrinsically evil as instruments of killing, permanently outside the human institution of law. Although we cannot speak for the various critics Heller engages in his discussion, our interpretation of the arguments advanced by Krishnan, Rosert and Sauer, Heyns, and others is that human life makes sense only in the context of human relations.[16]<\/sup>. Legal frameworks, including IHL, and much of the Just War Tradition, have been formed specifically around the assumption of human agency, an assumption that includes the understanding that humans are flawed and fallible.[17]<\/sup> Humans are not machines and human affairs cannot and should not be read exclusively through the narrow lens of technological capacity. <\/p>\n\n\n\n The question then is one that many critics of autonomous weapons have picked up: what is required to adhere to the rules and standards that govern us? This is a philosophical question, as well as a practical one. It is a question of moral, not merely technical, significance, and should remain so, regardless of how mechanized and systematized contemporary warfare has become.[18]<\/sup> Answering the question involves rendering human judgment on and taking human responsibility for life and death. Certain technological capabilities, including autonomous weapons, complicate the attainment of both judgment and responsibility. We should not mistake human agency for technological agency, lest we wish to court disaster.[19]<\/sup><\/p>\n\n\n\n In Heller\u2019s essay, the deck is stacked in favour of the technological. He closes the first section with an indicative assessment: <\/p>\n\n\n\n The most fundamental problem with deontological objections, however, is precisely their deontology. Because deontological arguments are by definition non-consequentialist, they would prohibit states from using autonomous weapons in conflict even if doing so would lead to fewer civilian casualties and less unnecessary combat suffering<\/em>.[20]<\/sup><\/p>\n\n\n\n With this, Heller rejects the deontological premise that ethical considerations raised on grounds of duty or dignity, or an incompatibility between machine processes and the ethos of human-oriented legal parameters, have sufficient validity. It is also assumed that the costs and benefits of AWS could straightforwardly be ascertained, presumably with techniques adequate for that purpose. This perspective betrays a scientific-technological lens increasingly prominent in abstract reasoning.[21]<\/sup> The lens truncates the \u201cexperiential and situated knowledge\u201d so important in warfare,[22]<\/sup> instead prioritising that which can be counted, measured, and assessed in an overly optimistic techno-solutionist spirit.<\/p>\n\n\n\n The charge that autonomous weapons critics unduly anthropomorphize the technology could be inverted in relation to Heller\u2019s stance: he unduly technologizes the human in the process of killing. And this machinist logic weaves through the rest of the essay. In discussing the many challenges humans present to perfect compliance with IHL, Heller notes a series of biases: cognitive bias, availability bias, imaginability bias, base-rate bias, anchoring bias, overconfidence bias, object use bias, confirmation bias and stereotyping. He suggests, rightly, that these individual psychological traits make it difficult for the human to achieve compliance with IHL. In a section on \u201cdebiasing,\u201d he explains the difficulty in ameliorating these flaws: \u201c[t]raining people in statistical reasoning has had some success for simple tasks, but it \u2018has not typically been fully tested in complex environments using unfamiliar and abstract rules.\u2019\u201d[23]<\/sup> One might be forgiven for momentary confusion: is this a discussion of human capabilities, or the capabilities of a robotic system? The discussion he offers presses human capabilities into an analytical framework fit for the latter and analyzes warfare solely as an engineering problem.<\/p>\n\n\n\n Imposing this mechanistic model on human behaviour for a side-by-side comparison is not productive towards furthering the pressing moral and legal debate on autonomy in weapon systems. Elsewhere, Heller states that \u201csome of [the principle of distinction\u2019s] \u2026 central requirements, such as identifying combatants and recognising surrender, involve little more than object recognition<\/em>.\u201d[24]<\/sup> This narrow conception of IHL, and the role of the human within it, once again stacks the odds in favor of machine logics. <\/p>\n\n\n\n Heller\u2019s excessively technologized approach pathologizes the human agent in war as not merely flawed, but irredeemably so. The same approach also gives the autonomous weaponry tasked with our replacement too easy a ride. <\/p>\n\n\n\n Even if warfare were judged solely according to consequentialist standards for effectiveness, AWS would still be a flawed solution. Much of Heller\u2019s essay is dedicated to exposing the failings of humans in war. Measured against a set of technical standards, the irrational human is found wanting, permanently doomed to lack the necessary capacity and inclination to functionally adhere to IHL. Viewed in such terms by Heller, the age-old question of how to tame the destructive excesses of war has an obvious answer:<\/p>\n\n\n\n [G]iven the irrationality of human decision-making, particularly in combat, warfare that eliminates human control \u2013 that is fought autonomously \u2013 promises less<\/em> unnecessary death and destruction, not more<\/em>. Put more simply: in terms of IHL compliance, humans are the problem, not the solution.[25]<\/sup><\/p>\n\n\n\n We disagree. \u201cThe purpose of a system is what it does\u201d is a systems thinking heuristic first coined by Stafford Beer, who argues that there is \u201cno point in claiming that the purpose of a system is to do what it constantly fails to do.\u201d[26]<\/sup> This applies equally to autonomous weapons systems. Heller\u2019s position, we argue, cannot be sustained once the human is compared, as it must be, to autonomous violence as it actually<\/em> manifests, rather than to an imagined system of perfect rationality. <\/p>\n\n\n\n Pioneering programs that combine sophisticated and dynamic target nomination systems with weapons platforms, such as the U.S. developed Project Maven, have the potential for fully autonomous use.[27]<\/sup> These systems are specifically designed not to merely identify a torpedo or a tank, but to be used in a dynamic process. They sift through a massive volume of data to nominate and execute targets autonomously at an accelerated pace. At this stage, a human remains in the loop to sign off on suggested targets, but this sign off also becomes a mechanized task where \u201cdecision-makers\u201d are empowered, and required, to authorise \u201cas many as 80 targets in an hour of work\u201d \u2013\u2013 a \u201crapid staccato of \u2018Accept, Accept, Accept.\u2019\u201d[28]<\/sup> Moreover, AWS of this nature are typically connected to multiple AI systems which work in layers when performing targeting tasks. Speed and volume are clearly prioritized as key to greater lethality. <\/p>\n\n\n\n In order to take the critiques of AWS seriously, one must take the technologies the critics critique seriously. Much hinges on a system\u2019s actual capabilities, not its future promise, which is often wildly overstated and overhyped. Systems that employ AI for the full kill-chain are likely to be marred by incomplete, low-quality, incorrect, or discrepant data.[29]<\/sup>This, in turn, will lead to highly brittle systems and biased, harmful outcomes. Autonomous systems tend to be built and tested on rather limited samples of data. Sometimes it is synthetic data, and sometimes it is inappropriate data.[30]<\/sup> This is problematic enough to question the use of AWS, even before considering the messy complexities of the battlefield. <\/p>\n\n\n\n In addition to this, research in machine learning for AI is nowhere close to as scientifically robust as it might appear from media accounts. We know this, for example, from the healthcare industry, where AI systems have had mixed results in applications ranging from health predictions to diagnostics, often on account of overpromising the utility of AI for these tasks.[31]<\/sup> AI has value primarily for quite clearly delineated, narrow tasks, executed in closed systemic environments. Applications of AI elsewhere are highly likely to lead to faulty decisions, misapplications of force, and other harmful errors. System outcomes are inherently unpredictable, and the probabilistic nature of AI reasoning implicitly recognizes error and accident as a feature, not a bug, of the system.<\/p>\n\n\n\n Research in computer vision is also not nearly as robust as Heller suggests.[32]<\/sup> In fact, computer vision faces a number of unresolved challenges, especially when it comes to human action recognition. This includes issues around the detection of variations among human features, \u201cimage-quality and frame rate\u201d issues, \u201cmultiview variations\u201d especially in uncontrolled settings, and general environmental issues, like poor visibility through bad weather or other background \u201cnoise\u201d that may distort the relevant input data for an accurate and reliable result.[33]<\/sup> Detecting violent action is particularly challenging \u201cbecause the available violence dataset is insufficient for deep network training. Also, human behaviour contains high intra-class variations and inter-class similarities that make violence detection very challenging.\u201d[34]<\/sup> Some of these issues are likely to be mitigated as more data becomes available, hardware becomes more sophisticated, and technology generally advances. Others, critically, may not. We are dealing with an open question<\/em>, a fact that should not be forgotten in these debates. The insistence that autonomous weapons will morally outperform humans in battle, if not now then one day, should be seen for what it is: an article of faith. <\/p>\n\n\n\n As empirics on machine learning, computer vision, and the actual use of AWS reinforces, the perfectibility of this technology is a highly speculative assumption, not a given. This is important to repeat, given the determination of many AI developers to convince states and the public of the opposite. Here, it is worth further interrogating the examples provided by Heller to illustrate his confidence in such technology. Quoting Elliot Winter, Heller writes that \u201cnot only can machines \u2018observe at least as well as humans and, indeed, at higher resolution and with greater rapidity,\u2019 their recognition ability \u2018has now advanced to a point where it has reached parity with human recognition abilities.\u2019\u201d[35]<\/sup> Heller then substantiates this with reference to the company material of Malong Technologies, which boasts of a 94.78% accuracy in object recognition.[36]<\/sup> Elsewhere, Heller asserts that autonomous weapons systems will likely outmatch humans in differentiating between real and fake guns, an important ability in war, especially in complex urban operations. For evidence, Heller draws on the Patriot One\u2019s \u201cPatScan\u201d threat-detection product, a system able to identify weapons with a \u201cnearly 95% certainty.\u201d[37]<\/sup> This statistic stems from a company promotional video for a product designed specifically for use within a civilian context to recognize concealed weapons in shopping malls or hotels.<\/p>\n\n\n\n These industry claims deserve at least as much scrutiny as Heller gives the human. AI discourse is awash with exaggerated claims regarding the efficacy of such systems, pushed forward by private actors with a direct financial interest in normalizing the technology and lowering barriers for use. Even when the technical claims provided by companies are accurate, they are benchmarked and assessed in very limited settings.[38]<\/sup> Too often, these limitations are ignored or downplayed, with specific data about specific technologies uncritically extrapolated and misapplied to the radically different war context. <\/p>\n\n\n\n When it comes to the battlefield implementation of AI and machine learning, a high degree of caution is needed. As Paddy Walker notes, \u201c[r]esearch demonstrates that the introduction of new parameters or slightly heterogeneous data to the data under which the weapon has been trained will confound autonomous weapon ML processes for the task envisaged, particularly in the changing nature of a battlespace with its prevalence of hidden, partially observable or camouflaged states.\u201d[39]<\/sup> Autonomous weapon systems will need to be trained<\/em>, and not merely programmed \u2013 any \u201cmarginally different set up, discrepancies between training dataset and sensed information or data poisoning arising from adversarial intervention may all lead to substantial variation\u201d in the system\u2019s action.[40]<\/sup> Add to this the \u201ctechnical debt [that] arises from inappropriate architecture, shortcuts resulting from commercial pressures, poor testing protocols and poor holistic understanding of LAWS,\u201d[41]<\/sup> and the idealisation of AWS in Heller\u2019s piece becomes less and less tenable.<\/p>\n\n\n\n It is not our intention to deny the very real and enduring problems with human operators in war, nor do we wish to supress discussion of more humane alternatives to the military status-quo. But humility is needed. The drive to humanize war must begin with a recognition of how resistant many of the moral problems of the battlefield ultimately are to technical solution. We also need to avoid an uncritical embrace of industry claims that cynically or genuinely assume away the immutable complexities of war. <\/p>\n\n\n\n As depressing an admission as it is, war can always get worse. We differ fundamentally from Heller, in recognizing the potential for autonomous weapons to make warfare even more violent, brutal, and unstable. This view is grounded not in a quixotic and romanticised view of the human, but rather a deep and justified caution over the alleged promise of autonomous weapons. <\/p>\n\n\n\n By way of concluding, we would like to raise a question on the potential risks of autonomous weapons advocacy. Heller\u2019s concern, like ours, is with ensuring that war is neither commenced nor conducted unjustly. But does argumentation and advocacy in the mode of Heller ultimately help our efforts to 3and impulses of war? Our answer is no. Claiming, contrary to the interpretation and intuition of most, that \u201c[human] understanding is far less necessary to IHL than AWS critics assume,\u201d[42]<\/sup> is to abstract the letter of the law from the ethos. Heller seeks to provide a fair and, as he sees it, overdue critique of the human in war, and he encourages consideration of autonomous weapons as a potential moral and legal improvement. What he actually does, we fear, is wrongly elevate this technology to a status it is not close to deserving.<\/p>\n\n\n\n There are certainly grounds for pessimism about human qualities in warfare, but Heller\u2019s own pessimism in the human is supplemented by a somewhat unfounded optimism in technology and its perfectibility. He stacks the odds, placing the human agent into direct competition with an ideal machine entity that \u201cperceive[es] the world accurately, understand[s] rationally, quarantin[es] negative emotions and reliably translat[es] thought into action\u201d.[43]<\/sup> Missing is necessary reflection on the serious limits and dangers of such technological systems, and the functional embedding of humans within them. Also missing is an exploration of the assumptions and motivations of those who most forcefully push for this technology. There is a fetishization of speed underpinning both the design and use of AWS, a \u201cmove fast and break things\u201d logic driving innovation, and a spectre of great power competition invoked to silence criticism from those justifiably concerned with both the pace and character of this change. These factors matter immensely when considering how this technology will be used. <\/p>\n\n\n\n We need to think very seriously about the modes of war we should and should not countenance going forward, and the critical role of technology in fulfilling these visions. Morally and legally improving the battlefield is frustrating work. Positive change, when achievable at all, is rarely satisfying \u2013 too slow, too partial, too contingent. But beware the lure of the technological shortcut. History provides us with endless examples of human weakness and human cruelty in war. History also illustrates the acute moral danger when individual human agency in stripped away or surrendered to excessively systematic and anonymizing processes of violence. Such conditions open the way to battlefield misconduct or worse. Humanity is an endless disappointment in war, but we suspect that we will miss it when it is gone, especially if it is cleared away to make room for flawed technologies that fall short of their promise and deaden our moral imagination.<\/p>\n\n\n\n [1]<\/sup><\/a> Kevin Jon Heller, The Concept of \u201cThe Human\u201d in the Critique of Autonomous Weapons<\/em>, 15 Harv. Nat\u2019l Sec. J. 1, 6 (2023).<\/p>\n\n\n\n [2]<\/sup><\/a> See generally<\/em> Rob Sparrow, Robots and Respect: Assessing the Case Against Autonomous Weapon Systems<\/em>, 30 Ethics & Int\u2019l Affs. 93 (2016); Christian Nicholas Braun, LAWS and the Mala in Se Argument<\/em>, 33 Peace Rev. 237 (2022).<\/p>\n\n\n\n [3]<\/sup><\/a> Heller, supra<\/em> note 1, at 6.<\/p>\n\n\n\n [4]<\/sup><\/a> Id<\/em>. at 11. <\/p>\n\n\n\n [5]<\/sup><\/a> Id.<\/em> at 13.<\/p>\n\n\n\n [6]<\/sup><\/a> Id.<\/em> at 14.<\/p>\n\n\n\n [7]<\/sup><\/a> See generally<\/em> Neil Renic, Asymmetric Killing: Risk Avoidance, Just War, and the Warrior Ethos (2020).<\/p>\n\n\n\n [8]<\/sup><\/a> Heller, supra<\/em> note 1, at 7.<\/p>\n\n\n\n [9]<\/sup><\/a> Id<\/em>. at 8.<\/p>\n\n\n\n [10]<\/sup><\/a> Id.<\/em> at 9. <\/p>\n\n\n\n [11]<\/sup><\/a> Id. <\/em>at 9\u201310. <\/p>\n\n\n\n [12]<\/sup><\/a> See generally<\/em> Elke Schwarz, Autonomous Weapon Systems, Artificial Intelligence and the Problem of Meaningful Human Control<\/em>, 5 The Phil. J. of Conflict and Violence 53 (2021). <\/p>\n\n\n\n [13]<\/sup><\/a> Id.<\/em> at 10.<\/p>\n\n\n\n [14]<\/sup><\/a> See generally<\/em> Neil Renic & Elke Schwarz, Crimes of Dispassion: Autonomous Weapons and the Moral Challenge of Systematic Killing<\/em>, 37 Ethics & Int\u2019l Affs. 321 (2023). <\/p>\n\n\n\n [15]<\/sup><\/a> Heller, supra<\/em> note 1, at 9 (quoting Paul Scharre & Michael C. Horowitz, An Introduction To Autonomy In Weapon Systems 16 (2016)).<\/p>\n\n\n\n [16]<\/sup><\/a> See <\/em>Armin Krishnan, Killer Robots: Legality and Ethicality of Autonomous Weapons (2010); Elvira Rosert & Frank Sauer, Prohibiting Autonomous Weapons: Put Human Dignity First<\/em>, 10 Glob. Pol\u2019y 370, 370 (2019); Christof Heyns, Autonomous Weapons Systems: Living a Dignified Life and Dying a Dignified Death<\/em>, in<\/em> Autonomous Weapons Systems: Law, Ethics, Policy 3, 11 (Nehal Bhuta et al. eds.,<\/p>\n\n\n\n 2016).<\/p>\n\n\n\n [17]<\/sup><\/a> Christof Heyns, Autonomous Weapons in Armed Conflict and the Right to a Dignified Life: An African Perspective<\/em>, 33 S. Afr. J. on Hum. Rts., 51\u201352 (2017)<\/p>\n\n\n\n [18]<\/sup><\/a> Norbert Wiener knew this at the dawn of automated and autonomous weapons technology. See <\/em>Norbert Wiener, Some Moral and Technical Consequences of Automation<\/em>, 131 Science 1355, 1358 (1960).<\/p>\n\n\n\n [19]<\/sup><\/a> See id. <\/em><\/p>\n\n\n\n [20]<\/sup><\/a> Heller, supra<\/em> note 1, at 17.<\/p>\n\n\n\n [21]<\/sup><\/a> See generally<\/em> Elke Schwarz, Technology and Moral Vacuums in Just War Theorising<\/em>, 14 J. Int\u2019l Pol. Theory<\/em> 280 (2018); Elke Schwarz,Trolleyology: Algorithmic Ethics for Killer Robots, in <\/em>Handbook on the Ethics of Artificial Intelligence (David Gunkel ed., 2024).<\/em><\/p>\n\n\n\n [22]<\/sup><\/a> See generally<\/em> Christopher Coker, Ethics and War in the 21st<\/sup> Century (2008).<\/p>\n\n\n\n [23]<\/sup><\/a> Heller, supra<\/em> note 1, at 48.<\/p>\n\n\n\n [24]<\/sup><\/a> Id.<\/em> at 31 (emphasis added).<\/p>\n\n\n\n [25]<\/sup><\/a> Id.<\/em> at 54 (emphasis in original).<\/p>\n\n\n\n [26]<\/sup><\/a> David Komlos & David Benjamin, The Purpose of a System Is What it Does Not What It Claims To Do<\/em>, Forbes (Sept. 13, 2021), https:\/\/web.archive.org\/web\/20210913130445\/https:\/\/www.forbes.com\/sites\/benjaminkomlos\/2021\/09\/13\/the-purpose-of-a-system-is-what-it-does-not-what-it-claims-to-do\/.<\/p>\n\n\n\n [27]<\/sup><\/a> Katrina Manson, AI Warfare is Already Here,<\/em> Bloomberg Businessweek (Feb. 28, 2024), https:\/\/www.bloomberg.com\/features\/2024-ai-warfare-project-maven\/.<\/p>\n\n\n\n [28]<\/sup><\/a> Id.<\/em><\/p>\n\n\n\n [29]<\/sup><\/a> See <\/em>Arthur Holland Michel, Known Unknowns: Data Issues and Military Autonomous Systems 3\u20135 (2021), https:\/\/unidir.org\/publication\/known-unknowns-data-issues-and-military-autonomous-systems\/.<\/p>\n\n\n\n [30]<\/sup><\/a> Id.<\/em><\/p>\n\n\n\n [31]<\/sup><\/a> See generally<\/em> Hadyia Faheem & Sanjib Dutta, Artificial Intelligence Failure at IBM \u2018Watson for Oncology’<\/em>, 21 IUP J. Knowledge Mgmt. 47 (2023); David Kampmann, Venture Capital, the Fetish of Artificial Intelligence and the Contradictions of Making Intangible Assets<\/em>, 53 Econ. Soc. 39 (2024); Eliza Strickland, IBM Watson, Heal Thyself: How IBM Overpromised and Underdelivered on AI Health Care<\/em>, 56 IEEE Spectrum 24 (2019).<\/p>\n\n\n\n [32]<\/sup><\/a> Heller, supra<\/em> note 1, at 20.<\/p>\n\n\n\nI. Introduction<\/a><\/h1>\n\n\n\n
II. Pitfalls in Heller\u2019s Critique of the AWS Critics <\/a><\/h1>\n\n\n\n
A. Even If Machines Do Not Make Decisions Like a Human Does, They Deserve Scrutiny for Transforming the Role of Human Decision-Makers<\/a><\/h2>\n\n\n\n
B. AWS Have Greater Autonomy from Operator Intentions than Prior Weapon Systems<\/a><\/h2>\n\n\n\n
C. The Actions of AWS Cannot Be Held Accountable by Laws Intended to Regulate Morally Meaningful Human Action<\/a><\/h2>\n\n\n\n
D. Warfare Cannot Be Judged Solely by Standards of Technical Effectiveness<\/h2>\n\n\n\n
III. The Purpose of an Autonomous Weapon System Is What It Does <\/a><\/h1>\n\n\n\n
IV. Conclusion<\/a><\/h1>\n\n\n\n
\n\n\n\n