A Dual Perspective on Large Language Models for Code and Security

Doctoral Thesis (2026)
Author(s)

A. Al-Kaswan (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Contributor(s)

A. van Deursen – Promotor (TU Delft - Electrical Engineering, Mathematics and Computer Science)

M. Izadi – Copromotor (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Research Group
Software Engineering
DOI related publication
https://doi.org/10.4233/uuid:10c928de-d468-4ab6-bf12-ad661a93b5e4 Final published version
More Info
expand_more
Publication Year
2026
Language
English
Defense Date
16-12-2027
Awarding Institution
Delft University of Technology
Research Group
Software Engineering
ISBN (print)
978-94-6518-516-3
Page Views
15
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Artificial Intelligence (AI) systems that can read and write computer code, so-called Large Language Models (LLMs), are becoming a routine part of how software is built. Because these systems are good at reasoning about both code and natural language, they are not just useful for everyday programming, they are also relevant to security work, where understanding how a piece of software behaves, or how it might be attacked, is central. At the same time, the very properties that make these AI systems useful also create risks of their own: a model trained on huge amounts of code can accidentally leak private information from that training data, produce code with security flaws, or even help someone carry out a harmful task.

In this dissertation we study this two-sided relationship between AI language models and software security. The first part looks at how these AI systems can be adapted and properly tested for realistic security tasks, going beyond simple, artificial test problems toward settings that better resemble how security work is actually done in practice. This includes teaching an AI model to explain compiled, “black box” software without access to the original code, testing an AI system’s ability to work through a simulated hacking challenge on its own, and checking whether AI coding assistants can be led into helping with harmful requests.

The second part turns the question around, examining the risks that come with using these AI systems themselves. This includes showing that AI models can memorise and later leak fragments of the code they were trained on, including sensitive details that should never have been exposed, and demonstrating that these models can also “remember” and repeat buggy code they encountered during training, even when they would otherwise know better.

Throughout this work, the emphasis lies on testing these AI systems in ways that reflect real-world use, and on openly and responsibly sharing the datasets, tools, and testing methods that were developed, so that other researchers can verify and build on these findings.

With these results, we offer a dual perspective: Large Language Models as tools that can genuinely support security work, and as systems that must themselves be carefully evaluated and secured. With these findings, this dissertation contributes to a more grounded, realistic understanding of both the opportunities and the risks that these AI systems bring to software security.

Files

AliAlKaswanHR.pdf
(pdf | 41.2 Mb)
License info not available