Public sector organizations increasingly use artificial intelligence to augment, support, and automate decision-making. However, such public AI can potentially infringe on citizens’ right to autonomy. Contestability is a system quality that protects against this by ensuring systems are open and responsive to disputes throughout their life cycle. While a growing body of work is investigating contestable AI by design, little of this knowledge has so far been evaluated with practitioners. To make explicit the guiding ideas underpinning contestable AI research, we construct the generative metaphor of the Agonistic Arena, inspired by the political theory of agonistic pluralism. Combining this metaphor and current contestable AI guidelines, we develop an infographic supporting the early-stage concept design of public AI system contestability mechanisms. We evaluate this infographic in five workshops paired with focus groups with a total of 18 practitioners, yielding ten concept designs. Our findings outline the mechanisms for contestability derived from these concept designs. Building on these findings, we subsequently evaluate the efficacy of the Agonistic Arena as a generative metaphor for the design of public AI and identify two competing metaphors at play in this space: the Black Box and the Sovereign.@en

Public sector organizations increasingly use artificial intelligence to augment, support, and automate decision-making. However, such public AI can potentially infringe on citizens’ right to autonomy. Contestability is a system quality that protects against this by ensuring systems are open and responsive to disputes throughout their life cycle. While a growing body of work is investigating contestable AI by design, little of this knowledge has so far been evaluated with practitioners. To make explicit the guiding ideas underpinning contestable AI research, we construct the generative metaphor of the Agonistic Arena, inspired by the political theory of agonistic pluralism. Combining this metaphor and current contestable AI guidelines, we develop an infographic supporting the early-stage concept design of public AI system contestability mechanisms. We evaluate this infographic in five workshops paired with focus groups with a total of 18 practitioners, yielding ten concept designs. Our findings outline the mechanisms for contestability derived from these concept designs. Building on these findings, we subsequently evaluate the efficacy of the Agonistic Arena as a generative metaphor for the design of public AI and identify two competing metaphors at play in this space: the Black Box and the Sovereign.

@en

Public sector organizations increasingly use artificial intelligence to augment, support, and automate decision-making. However, such public AI can potentially infringe on citizens’ right to autonomy. Contestability is a system quality that protects against this by ensuring systems are open and responsive to disputes throughout their life cycle. While a growing body of work is investigating contestable AI by design, little of this knowledge has so far been evaluated with practitioners. To make explicit the guiding ideas underpinning contestable AI research, we construct the generative metaphor of the Agonistic Arena, inspired by the political theory of agonistic pluralism. Combining this metaphor and current contestable AI guidelines, we develop an infographic supporting the early-stage concept design of public AI system contestability mechanisms. We evaluate this infographic in five workshops paired with focus groups with a total of 18 practitioners, yielding ten concept designs. Our findings outline the mechanisms for contestability derived from these concept designs. Building on these findings, we subsequently evaluate the efficacy of the Agonistic Arena as a generative metaphor for the design of public AI and identify two competing metaphors at play in this space: the Black Box and the Sovereign.@en

In Research through Design, design actions contribute to the method of research, to the way knowledge is developed. This brings out several tensions and confusions between what research and design are, what they produce, how the two are done together, and how the results can be shared with other researchers, practitioners, and stakeholders. This chapter draws lessons from two seminal PhD on how developing prototypes, and having a conceptual framework helps coherence in broad, phenomenon-led explorations.

@en

Local governments increasingly use artificial intelligence (AI) for automated decision-making. Contestability, making systems responsive to dispute, is a way to ensure they respect human rights to autonomy and dignity. We investigate the design of public urban AI systems for contestability through the example of camera cars: human-driven vehicles equipped with image sensors. Applying a provisional framework for contestable AI, we use speculative design to create a concept video of a contestable camera car. Using this concept video, we then conduct semi-structured interviews with 17 civil servants who work with AI employed by a large northwestern European city. The resulting data is analyzed using reflexive thematic analysis to identify the main challenges facing the implementation of contestability in public AI. We describe how civic participation faces issues of representation, public AI systems should integrate with existing democratic practices, and cities must expand capacities for responsible AI development and operation.@en

This chapter focuses on explorative research using research through design, because in these we have seen the strength of doing design as a part of doing research. It looks at one guided by a research prototype. The chapter outlines how both prototypes and frameworks can help guide explorative research: prototypes through giving a physical instantiation of a phenomenon, frameworks by placing a phenomenon in a conceptual perspective. The series of prototypes anchored the focus of the research, and determined the scope. The research progressed through a number of stages: after exploration, the prototype was developed and during its development it was continuously tested with members of the ID-StudioLab research community as test subjects and critical participants. The role that design skills, and design actions, can have in research has received growing attention. Prototypes force the researcher to confront theory, confront the world, they evoke discussion and reflection, change the world, and can be used to test a theory@en

The increasing use of artificial intelligence (AI) by public actors has led to a push for more transparency. Previous research has conceptualized AI transparency as knowledge that empowers citizens and experts to make informed choices about the use and governance of AI. Conversely, in this paper, we critically examine if transparency-as-knowledge is an appropriate concept for a public realm where private interests intersect with democratic concerns. We conduct a practice-based design research study in which we prototype and evaluate a transparent smart electric vehicle charge point, and investigate experts’ and citizens’ understanding of AI transparency. We find that citizens experience transparency as burdensome; experts hope transparency ensures acceptance, while citizens are mostly indifferent to AI; and with absent means of control, citizens question transparency’s relevance. The tensions we identify suggest transparency cannot be reduced to a product feature, but should be seen as a mediator of debate between experts and citizens.

@en

As the use of AI systems continues to increase, so do concerns over their lack of fairness, legitimacy and accountability. Such harmful automated decision-making can be guarded against by ensuring AI systems are contestable by design: responsive to human intervention throughout the system lifecycle. Contestable AI by design is a small but growing field of research. However, most available knowledge requires a significant amount of translation to be applicable in practice. A proven way of conveying intermediate-level, generative design knowledge is in the form of frameworks. In this article we use qualitative-interpretative methods and visual mapping techniques to extract from the literature sociotechnical features and practices that contribute to contestable AI, and synthesize these into a design framework@en

The increasing use of algorithmic decision-making systems in the public realm and in citieshas led to an urgent call for more transparencyand accountability. While recent work in algorith-mic fairness and human-centred ML has exploredways to include the concerns of people into thedesign of ML systems, the “street-level” expe-rience of algorithmic systems is not well under-stood. In this paper, we present a case study of a“transparent electric vehicle charge point” whichis designed to provide electric vehicle drivers withinsights of the operation of smart charging algo-rithms. Exploring limitations of the transparencyideal, we identify the need for contestability as acritical aspect of future public decision-makingsystems.@en

The increasing use of algorithms in cities has come under scrutiny. Transparency is widely seen as a way to ensure their fairness and accountability. We investigate how al- gorithmic transparency helps citizens understand smart electric vehicle charge points and how its conception differs between experts and citizens. Using a research-through- design approach we collaborated over a 10-month period with companies and Amsterdam municipality to prototype and evaluate a transparent smart electric vehicle charge point. We find that experts believe transparency is pro- duced by truthful information about inputs, processes and outcomes, that this information aids understanding and is actionable. We also find that citizens are indifferent to al-gorithmic decision-making when it serves common interests. Furthermore, transparency invites gaming, creates expectations of control, and adds to the cognitive burden of an already stressful task. Our findings suggest algorithmic transparency benefits professional stakeholders more than the citizens it is claimed to serve.@en