Public sector organizations increasingly use artificial intelligence to augment, support, and automate decision-making. However, such public AI can potentially infringe on citizens’ right to autonomy. Contestability is a system quality that protects against this by ensuring systems are open and responsive to disputes throughout their life cycle. While a growing body of work is investigating contestable AI by design, little of this knowledge has so far been evaluated with practitioners. To make explicit the guiding ideas underpinning contestable AI research, we construct the generative metaphor of the Agonistic Arena, inspired by the political theory of agonistic pluralism. Combining this metaphor and current contestable AI guidelines, we develop an infographic supporting the early-stage concept design of public AI system contestability mechanisms. We evaluate this infographic in five workshops paired with focus groups with a total of 18 practitioners, yielding ten concept designs. Our findings outline the mechanisms for contestability derived from these concept designs. Building on these findings, we subsequently evaluate the efficacy of the Agonistic Arena as a generative metaphor for the design of public AI and identify two competing metaphors at play in this space: the Black Box and the Sovereign.@en

Public sector organizations increasingly use artificial intelligence to augment, support, and automate decision-making. However, such public AI can potentially infringe on citizens’ right to autonomy. Contestability is a system quality that protects against this by ensuring systems are open and responsive to disputes throughout their life cycle. While a growing body of work is investigating contestable AI by design, little of this knowledge has so far been evaluated with practitioners. To make explicit the guiding ideas underpinning contestable AI research, we construct the generative metaphor of the Agonistic Arena, inspired by the political theory of agonistic pluralism. Combining this metaphor and current contestable AI guidelines, we develop an infographic supporting the early-stage concept design of public AI system contestability mechanisms. We evaluate this infographic in five workshops paired with focus groups with a total of 18 practitioners, yielding ten concept designs. Our findings outline the mechanisms for contestability derived from these concept designs. Building on these findings, we subsequently evaluate the efficacy of the Agonistic Arena as a generative metaphor for the design of public AI and identify two competing metaphors at play in this space: the Black Box and the Sovereign.@en

Public sector organizations increasingly use artificial intelligence to augment, support, and automate decision-making. However, such public AI can potentially infringe on citizens’ right to autonomy. Contestability is a system quality that protects against this by ensuring systems are open and responsive to disputes throughout their life cycle. While a growing body of work is investigating contestable AI by design, little of this knowledge has so far been evaluated with practitioners. To make explicit the guiding ideas underpinning contestable AI research, we construct the generative metaphor of the Agonistic Arena, inspired by the political theory of agonistic pluralism. Combining this metaphor and current contestable AI guidelines, we develop an infographic supporting the early-stage concept design of public AI system contestability mechanisms. We evaluate this infographic in five workshops paired with focus groups with a total of 18 practitioners, yielding ten concept designs. Our findings outline the mechanisms for contestability derived from these concept designs. Building on these findings, we subsequently evaluate the efficacy of the Agonistic Arena as a generative metaphor for the design of public AI and identify two competing metaphors at play in this space: the Black Box and the Sovereign.

@en

This thesis investigates the use of artificial intelligence (AI) in public policy execution. To contribute to preserving citizen autonomy, it introduces the concept of ‘contestability’—a system quality that ensures citizens retain control over their lives in the face of AI systems and can influence AI system development. The central research aim is to explore sociotechnical design interventions that enhance the contestability of public AI systems.

Utilizing constructive design research, the thesis reports on several studies in which researchers collaborate with design practitioners to create artifacts that function as data generation instruments. Methods encompass interaction design, speculative design, and information design, with case studies in smart electric vehicle charging, urban monitoring camera cars, and fraud risk models, all situated in Amsterdam.

Key findings include varying perceptions of AI transparency between citizens and experts, a design framework for contestable AI, challenges in local government implementation, and the metaphors designers use for public AI.

The research advocates for integrating citizen feedback into AI systems, promoting dialogue between citizens and system controllers, and enhancing democratic involvement in AI development. It also highlights the importance of design in AI implementation, emphasizing speculative design as a method for generating relevant data and guiding ideation and specification processes.

Concluding, the thesis calls for a greater engagement of design researchers and practitioners with political philosophy to understand the democratic implications of their work in AI and related fields.@en

Local governments increasingly use artificial intelligence (AI) for automated decision-making. Contestability, making systems responsive to dispute, is a way to ensure they respect human rights to autonomy and dignity. We investigate the design of public urban AI systems for contestability through the example of camera cars: human-driven vehicles equipped with image sensors. Applying a provisional framework for contestable AI, we use speculative design to create a concept video of a contestable camera car. Using this concept video, we then conduct semi-structured interviews with 17 civil servants who work with AI employed by a large northwestern European city. The resulting data is analyzed using reflexive thematic analysis to identify the main challenges facing the implementation of contestability in public AI. We describe how civic participation faces issues of representation, public AI systems should integrate with existing democratic practices, and cities must expand capacities for responsible AI development and operation.@en

As the use of AI systems continues to increase, so do concerns over their lack of fairness, legitimacy and accountability. Such harmful automated decision-making can be guarded against by ensuring AI systems are contestable by design: responsive to human intervention throughout the system lifecycle. Contestable AI by design is a small but growing field of research. However, most available knowledge requires a significant amount of translation to be applicable in practice. A proven way of conveying intermediate-level, generative design knowledge is in the form of frameworks. In this article we use qualitative-interpretative methods and visual mapping techniques to extract from the literature sociotechnical features and practices that contribute to contestable AI, and synthesize these into a design framework@en

The increasing use of artificial intelligence (AI) by public actors has led to a push for more transparency. Previous research has conceptualized AI transparency as knowledge that empowers citizens and experts to make informed choices about the use and governance of AI. Conversely, in this paper, we critically examine if transparency-as-knowledge is an appropriate concept for a public realm where private interests intersect with democratic concerns. We conduct a practice-based design research study in which we prototype and evaluate a transparent smart electric vehicle charge point, and investigate experts’ and citizens’ understanding of AI transparency. We find that citizens experience transparency as burdensome; experts hope transparency ensures acceptance, while citizens are mostly indifferent to AI; and with absent means of control, citizens question transparency’s relevance. The tensions we identify suggest transparency cannot be reduced to a product feature, but should be seen as a mediator of debate between experts and citizens.

@en

The increasing use of algorithmic decision-making systems in the public realm and in citieshas led to an urgent call for more transparencyand accountability. While recent work in algorith-mic fairness and human-centred ML has exploredways to include the concerns of people into thedesign of ML systems, the “street-level” expe-rience of algorithmic systems is not well under-stood. In this paper, we present a case study of a“transparent electric vehicle charge point” whichis designed to provide electric vehicle drivers withinsights of the operation of smart charging algo-rithms. Exploring limitations of the transparencyideal, we identify the need for contestability as acritical aspect of future public decision-makingsystems.@en

The increasing use of algorithms in cities has come under scrutiny. Transparency is widely seen as a way to ensure their fairness and accountability. We investigate how al- gorithmic transparency helps citizens understand smart electric vehicle charge points and how its conception differs between experts and citizens. Using a research-through- design approach we collaborated over a 10-month period with companies and Amsterdam municipality to prototype and evaluate a transparent smart electric vehicle charge point. We find that experts believe transparency is pro- duced by truthful information about inputs, processes and outcomes, that this information aids understanding and is actionable. We also find that citizens are indifferent to al-gorithmic decision-making when it serves common interests. Furthermore, transparency invites gaming, creates expectations of control, and adds to the cognitive burden of an already stressful task. Our findings suggest algorithmic transparency benefits professional stakeholders more than the citizens it is claimed to serve.@en