While artificial intelligence (AI) has undeniably ushered numerous solutions across various fields, the growing belief that AI can solve all problems overshadows their lack of transparency that comes along. Understanding how decisions are made and what has led to the output is crucial in critical systems to ensure accountability and trust.

This research proposes a complementary method leveraging inter-host distances that localise the outlying hosts, logs and the time frames, which require more advanced analysis. By relying on a variant of a prominent statistical method in the field of authorship attribution - Burrows Delta - the approach enhances transparency in identifying deviating hosts, logs and time frames. Hence, the proposed solution offers an understandable complementary method that preserves integrability by being a log-based method while enabling understandable pinpointing of the specific hosts, logs and time frames that warrant further advanced analysis. By providing insights into the behaviour of the hosts over time, a temporal summarisation for security analysts is provided, relaxing their need to go through all the log files to understand the hosts' behaviour.

The results show that a complementary method based on the textual content of the metadata of the logs provides alternative insight into the activities of the hosts than the attributes. Moreover, the behaviour defined by the proposed method requires less extensive lookup than the behaviour defined by attributes. The inter-host distances based on the textual content allow understandable localisation of the host behaviour over time. Hence, this research provides an understandable method that will summarise the behaviour of the hosts over time, which enables the localisation of the logs requiring more advanced, in-depth analysis, and thereby reducing the amount of logs security analists need to consider during a compromise.

As our world has become increasingly digital and the number of tasks performed by software has grown, so too has the volume of software logs and the importance of cybersecurity. Anomaly detection on software logs is crucial for securing systems and identifying
the causes of past attacks. Extensive research has focused on developing effective log
parsers and anomaly detection methods. However, the process between obtaining parsed
logs and feeding them to the anomaly detection methods has remained unchanged for years.
Typically, logs are ordered by their generation time and split into fixed-length timeframes,
which are then analyzed for anomalies. This approach can group unrelated logs together
while splitting related logs across different timeframes, potentially missing critical context
for anomaly detection.
This research explores the concept of views, which are different perspectives used on input
data. For example, logs can be grouped by the computer they are generated on, allowing the comparison of computers instead of arbitrary timeframes. The study demonstrates
that the performance of anomaly detection methods can vary significantly depending on the
views used.
To validate this, two different anomaly detection methods were applied to two different
datasets, showing that the effect is not specific to any particular dataset or detection method.
The research discusses three methods for creating these views and proposes a method to
suggest the most useful view by using views that contain a lot of different log keys and log
values. Additionally, partial views, such as grouping logs by user, are examined. These
views, although missing some logs, prove valuable for detecting anomalous behaviour as
grouping all logs of a user together and comparing that to other users, helped find the
anomalous user within a dataset that contain more than 1400 users in total.
The study also investigates the cause of anomalous behavior in Windows event logs by analyzing the scores of individual logs to identify words present in anomalous logs but absent
in normal ones. While this approach has some false positives, it effectively highlighted the
timing and consequences of an attack.
Finally, the research explores various ways to combine scores from different views and their
effects. Combining all views of EVTX data provides a reliable middle ground, useful when
the most effective view is unknown. It was found that taking the highest score from all views
results in many false positives and should be avoided. However, using the highest x scores
from all views, as long as x is greater than one, does not significantly differ from taking the
average score.

The behavior of software systems can be modeled as state machines by looking at the log data from these systems. Conventional algorithms, such as L∗, however, require too much memory to process log data when it gets too large. These algorithms must first load all available data into memory, which is often way too much.
The approach investigated to deal with this large amount of data is to load the data instead in an on-disk database. The thesis researches the viability of this approach and proposes an algorithm for it.
When log data is present in a database, a state machine can be constructed iteratively by gradually querying more and more data from the database. Each time more data is gathered, partial data can be used to construct a hypothesis for the state machine in question. Such an approach is thus a combination of an active learning part that queries more data and an passive learning part that constructs the state machine from partial data.
Database technologies can contribute to what such an algorithm should look like. Given the vast landscape of different database technologies, some can shape what kinds of queries can be more efficiently executed. This should be taken into account when constructing an algorithm that queries data from a database to construct a state machine.
This thesis starts with an overview of the existing state machine learning algorithms and relevant database technologies. Then a new algorithm is roposed and it is tested on different datasets and compared with existing algorithms.

Most of the adversarial attacks suitable for attacking decision tree ensembles work by doing multiple local searches from randomly selected starting points, around the to be attacked victim. In this thesis we investigate the impact of these starting points on the performance of the attack, and find that the starting points significantly impact the performance: some do much better than others. However, we do find that this is not the case for all attacked points, as there are large differences between points in how difficult they are to attack and for all datasets some points are always optimally attacked.

We compare the baseline randomly selected points to three alternative strategies. First, we try alternate random distributions, playing with both the standard deviation, to create a more narrow cone around the victim point, and mean, creating bimodal distributions further away from the victim point. We find that for some datasets these can give up to $5$-$7\%$ improved performance on subsets of the dataset, but these improvements do not generalize to the remainder of the dataset. In general, as long as the distribution is wide enough to successfully find starting points we do not find a substantial performance change.

Secondly, we try to remove the randomness and attack from a fixed direction. For the simpler datasets we find it is possible for a starting direction to perform better than random starting points, but for larger datasets performance becomes much worse. We also try an attack from all main directions around the victim point, which we find performs much worse than $5$-$20$ times fewer random points.

Lastly, we create an attack strategy where we select the closest points that scored well on previously attacked victims. We find that on smaller test sets this gets outperformed by the baseline, but when we extend the attack and give more possible previously well performing starting points we match or outperform the baseline slightly.

This thesis offers a comprehensive exploration of log-based anomaly detection within the domain of cybersecurity incident response. The research describes a different approach and explores relevant log features for language model training, experimentation with different language models and training methodologies, and the investigation of the potential contribution of extra contextual features. The newly proposed approach is compared against an already implemented baseline in a finite-state classifier called FlexFringe, assessing their performance in detecting malicious anomalies across diverse datasets and hosts.

Key findings from this research underscore the importance of including human language for the generation of coherent clusters and a better performance of pretrained language models over models that were fine-tuned or built from scratch. Furthermore, the influence of clustering parameters on cluster quality proves to be crucial for cluster quality. Additionally, we gained insights into how extra contextual features are useful for log analysis.

In light of these findings, the study provides several recommendations for future research, including the expansion of the methodology to accommodate various log sources, the enhancement of preprocessing techniques, the integration of newer and more advanced language models, and the pursuit of efficient hyperparameter optimization. This work contributes to the continual advancement of log-based anomaly detection and its critical role in enhancing cybersecurity practices.

Current backdoor attacks against federated learning (FL) strongly rely on universal triggers or semantic patterns, which can be easily detected and filtered by certain defense mechanisms such as norm clipping, comparing parameter divergences among local updates. In this work, we propose a new stealthy and robust backdoor attack with flexible triggers against FL defenses. To achieve this, we build a generative trigger function that can learn to manipulate the benign samples with an imperceptible flexible trigger pattern and simultaneously make the trigger pattern include the most significant hidden features of the attacker-chosen label. Moreover, our trigger generator can keep learning and adapt across different rounds, allowing it to adjust to changes in the global model. By filling the distinguishable difference (the mapping between the trigger pattern and target label), we make our attack naturally stealthy. Extensive experiments on real-world datasets verify the effectiveness and stealthiness of our attack compared to prior attacks on decentralized learning framework with eight well-studied defenses.

Malware poses a serious security risk in today’s digital environment. The defense against malware mainly relies on proactive detection. However, antivirus products often fail to detect new malware when the signature is not yet available. In the event of a malware infection, the common remediation strategy is reinstalling the system. However, the user loses their personal data, and thus it is not an ideal solution.

The academic works on malware remediation focus on system replay and recovery-oriented computing, which relies on heavy monitoring and is not suitable for a normal user’s personal computer. The work from Paleari et al. [31] proposed a remediation methodology that can be used entirely after the infection. They run the malware sample in the sandbox to observe the behavior and generate a revert operation for each action that modifies the system state. However, the limitation of such an approach is unable to deal with the potentially different behaviors in the sandbox and on the real hosts.

In this work, we propose a system that can generate user-specific recovery procedures, without the need of any monitoring in advance. We extend the work from Paleari et al. [31] by combining information from the infected machine. We first extract the environment configuration from the infected computer and configure the same context to the sandbox virtual machine, in order to eliminate the environmental influence on the malware’s behavior. After getting the behavior from the sandbox, we combine forensic evidence to understand the exact actions that happened on the system and generate the user-specific recovery procedures.

We implement a prototype based on Windows 10 and CAPE sandbox and perform an evaluation on 894 malware samples. We are able to recover 51.3% of the changes made by malware, which doubles the recovery rate compared to directly matching the sandbox result. Additionally, our experiment result also demonstrates significantly different actual behavior from the user’s machine and sandbox result. Our system design maximizes the use of information displayed in the sandbox, but the unshown behavior still leads to the biggest limitation of behavior-based recovery.

Network Intrusion Detection Systems (NIDSs) defend our computer networks against malicious network attacks. Anomaly-based NIDSs use machine learning classifiers to categorise incoming traffic. Research has shown that classifiers are vulnerable to adversarial examples, perturbed inputs that lead the classifier into misclassifying the input. Existing work has shown weaknesses in classifiers for classifying network traffic, but none have shown the possibility of automatically recreating network attacks that can bypass existing anomaly-based NIDSs. Regular methods for generating black-box adversarial examples create packets that are invalid. We present AGONI, a Multi-Objective Genetic Algorithm for
generating network packets that are both valid packets and adversarial examples for NIDSs. AGONI can successfully generate valid adversarial examples in multiple attack scenarios. Against the NIDS Suricata, 99.93% of the generated adversarial examples can successfully bypass the defence. We compare the performance of AGONI against randomly generating network packets, the Boundary Attack and an adjusted version of the Boundary Attack which can better create valid adversarial examples. Only AGONI consistently generates valid adversarial examples when compared to the Random Attack (82%), the Boundary Attack (0%) and the Networking Boundary Attack (74%).

This research paper focuses on the complex domain of alert-driven attack graphs. SAGE is a tool which generates such attack graphs (AGs) by using a suffix-based probabilistic deterministic finite automaton (S-PDFA). One of the substantial properties of this algorithm is to detect infrequent severe alerts while maintaining the context of attacks via the help of sink states and state IDs. This is a modelling assumption that we validate by answering the question driving this research: How does allowing the sink states to merge with other sink states affect the generated alert-driven attack graphs? Our research used a transparent methodology to obtain size, complexity and completeness statistics of the two algorithms. Afterwards, outstanding values in size and complexity allow us to filter insightful attack graphs, subject to head-to-head comparisons concerning their interpretability. We discovered that many remarkable changes happen in the outputted attack graphs, leading to an evident decrease in interpretability and increased loss of context. Concurrently, we do not detect substantial changes in size, complexity and completeness, leading us to believe that it is possible to unlock SAGE's full potential by adding specific thresholds for merging sink states. One proposed constraint is allowing only the merges of sinks at equal distance to the victim node. This alteration leads to similar results in all metrics, including interpretability, where some AGs show improvements.

The interpretability of an attack graph is a key principle as it reflects the difficulty of a specialist to take insights into attacker strategies. However, the quantification of interpretability is considered to be a subjective manner and complex attack graphs can be challenging to read and interpret. In this research paper, we propose a new metric for quantifying the interpretability of attack graphs, aiming for comparable results between attack graphs regardless of the chosen drawing configuration or generation method. We address the gap in existing metrics by combining elements from the theory of cognitive chunks of information and user-experience-related fields to measure interpretability in terms of cognitive load. Our metric leverages Gestalt principles to formalize the quantification of interpretability based on cognitive overload. Compared to a similar approach, the proposed metric reveals a high level of similarity with the baseline, however, qualitative analysis revealed the proposed metric eliminates certain discrepancies with the expert's opinion that the baseline metric presented. Furthermore, a use case of the metric is presented and we evaluate our metric by comparing attack graphs generated using different methods, such as deterministic finite automaton (S-PDFA), Markov chain, and suffix tree. Finally, further work is proposed toward the goal of completing the metric by incorporating the remaining Gestalt principles.

SAGE is a deterministic and unsupervised learning pipeline that can generate attack graphs from intrusion alerts without input knowledge from a security analyst. Using a suffix-based probabilistic deterministic finite automaton (S-PDFA), the system compresses over 1 million alerts into less than 500 attack graphs (AGs), which are concise and manageable. Unlike other frequency analysis methods, SAGE does not discard infrequent high-severity alerts, which are crucial for learning the penetration strategies of attackers. This paper compares the baseline algorithm (i.e. S-PDFA) with a modelling assumption generated by swapping the S-PDFA with a PDFA. The aim is to validate the quality of SAGE and propose possible solutions for PDFA usage, allowing the algorithm to generate AGs in real-time. We compare them both quantitatively and qualitatively using size, complexity, completeness and interpretability metrics. Our findings show that AGs generated by the PDFA are more readable and as complete while being slightly larger (i.e. 16% larger) than the baseline S-PDFA. In certain cases, it can also better capture different attack strategies, proving that, if further optimized, it can perform better than the baseline.

Intrusion Detection Systems (IDSes) detect malicious traffic in computer networks and generate a large volume of alerts, which cannot be processed manually. SAGE is a deterministic algorithm that works without a priori network/expert knowledge and can compress these alerts into attack graphs (AGs), modelling intruders’ paths in the network. These AGs are too high in quantity/complexity for manual analysis, creating the necessity for prioritising individual attack stages (ASes). The existing prioritisation metric does not take into account graph
properties and is not granular enough to function on a node level. We propose PICA, an urgency metric inspired by the CIA triad (Confidentiality, Integrity and Availability) and the graph properties. It works on a node level and an attack-stage level. PICA is evaluated by comparison with the current implementation, based on AGs generated by SAGE using open-source intrusion alert datasets. The evaluation is based on the number and the type of the discovered attack stages. Results show that PICA manages to discover ASes that contain nodes with a high
in-degree but fails at discovering urgent ASes that contain many nodes with low in-degrees. Compared to the baseline, the ASes are distributed more evenly over the different urgency levels. Analysis of urgent node positioning revealed that sub-AGs lose information when objectives (final goal in a path) are also starting nodes. Changing the weights of the CIA triad showed a clear bias in results
towards the larger weights, as was intended. Finally, further work is proposed for PICA and in the generation process of SAGE’s AGs.

SAGE is an unsupervised sequence learning pipeline that generates alert-driven attack graphs (AGs) without the need for prior expert knowledge about existing vulnerabilities and network topology. Using a suffix-based probabilistic deterministic finite automaton (S-PDFA), it accentuates infrequent high-severity alerts without discarding frequent low-severity alerts. It also captures the context of the alerts with identical signatures and it is an interpretable model. In order to deal with infrequent data, SAGE utilises sink states which are not merged during the S-PDFA learning process. However, this could result in unnecessarily larger AGs. In this study, we have looked at the AGs resulting from merging sink states with other sinks and the core of the S-PDFA after the main merging process. Data from Collegiate Penetration Testing Competitions has been used to compare AGs based on the four metrics: size, complexity, interpretability and completeness. We have shown that the resulting graphs are, on average, slightly smaller, with about the same complexity and the same completeness, but with worse interpretability due to losses of context of attack episodes, which cannot be compensated by the slightly smaller size of the AGs.

Software is everywhere, and going back to a life without software is unimaginable. Unfortunately, software does not always behave as expected, even though during the development cycle, software is usually tested to verify its correctness. To aid in testing, methods such as fuzzing or symbolic execution are used for automatic verification software systems. These methods are able to quickly find inputs to the systems that cover large portions of the code base. However, both of these methods struggle to find inputs that cover code which requires many iterations through loops.

In symbolic execution, loops are a large contributing factor to the path explosion problem and therefore the overall runtime. For loops containing conditional branches, each iteration is a new decision point. This leads to an exponential number of possible paths through a loop in comparison to the number of iterations.

In this work, we investigate the use of loops in symbolic execution to reach portions of the code which require numerous iterations through loops with conditional branches. We propose a novel technique for symbolic execution that uses the effects of one or more iterations through a loop to reach new parts of the code. By implementing this technique and applying it to a set of challenges designed to stress current tools and methods for software verification, we show that our technique is able to efficiently reach new parts of these challenges. These areas are not reached by state-of-the-art methods within the same time budget.

Another method for verifying software behavior is active learning, where simple models are learned from a system. These models capture the behavior of the system at a high level, allowing easier analysis to verify the behavior of a system. During the automatic learning of these models, loops are not handled separately. This leads to models where the behavior of a system is not captured fully, leading to incomplete analysis. We propose new methods for finding changes in behavior after executing these loops numerous times. We have compared our techniques to existing methods and show that this produces more complete models of a system.

Federated learning allows a multitude of contributors to collaboratively build a deep learning model, all while keeping their individual training data private from one another. However, it is not immune to security flaws such as backdoor attacks in which malevolent adversaries manipulate the global model to trigger specific behaviors. In this paper, we investigate the impact of trigger intensity in backdoor attacks within the federated learning setting. We challenge the conventional requirement of training and testing on the same trigger intensity and propose a novel approach of training on a weak trigger and testing on a stronger trigger. Our experiments demonstrate that this technique proves capable of enhancing backdoor attack performance and robustness and might open the possibility for invisible backdoors during the testing phase. The ability to customize trigger visibility empowers attackers to craft stealthier and more potent attacks, making trigger detection challenging. Our findings complement existing state-of-the-art attacks, providing attackers with greater options to tailor their attack to their intended target. We discuss the implications of our research and highlight the importance of developing effective defense mechanisms to counter backdoor vulnerabilities in federated learning systems. Overall, our study contributes to the advancement of backdoor attack understanding in FL and provides valuable insights into trigger intensity as a critical factor for attack customization and resilience. By exploring new avenues for stronger and more stealthy attacks, we contribute to the ongoing efforts to safeguard AI systems’ privacy and reliability in real-world applications.

In an era where cyber threats evolve with alarming speed and sophistication, the role of Security Operation Centers (SOCs) has become increasingly pivotal in safeguarding digital infrastructures. SOCs serve as the frontline defence against malicious entities, where they continuously monitor and analyze network traffic, as well as the activity of users and systems for potential threats. The rapid growth of advanced cyber-attacks has amplified the reliance on Intrusion Detection Systems (IDS) to generate alerts for anomalous activities, and on SOC analysts to analyze those alerts. However, these systems often yield an overwhelming number of alerts, many of which are false positives, leading to alert fatigue among analysts. The scarcity of effective visualization tools, coupled with the analysts' dependence on manual investigation and correlation of events aggravates this issue, resulting in extended alert analysis times. Moreover, the number of attack scenarios keeps increasing daily, making it difficult to understand the possible next actions of an attacker and apply preventive measures.

This thesis introduces an innovative approach to aid SOC analysts in managing the large influx of alerts, mitigating alert fatigue, and enhancing the efficiency of threat identification and response. We present an attack prediction tool with alert visualization capabilities that produces real-time attack graphs, summarizing the alerts associated with a specific host. Our method utilizes a Suffix-based Probabilistic Deterministic Finite Automaton (SPDFA) to predict future attacker actions, promoting a proactive defence strategy, and achieving an accuracy of 33.71 %. We validate the practicality and relevance of our contributions through interviews with six security experts, confirming the utility of our methods in a live SOC context. Furthermore, we demonstrate the applicability of our approach by testing it with three datasets collected in the real world. Our work stands apart by simultaneously addressing alert correlation, attack visualization, and predictive modelling of attacker behaviour.

Every day, Intrusion Detection Systems around the world generate huge amounts of data. This data can be used to learn attacker behaviour, such as Techniques, Tactics, and Procedures (TTPs). Attack Graphs (AGs) provide a visual way of describing these attack patterns. They can be generated without expert knowledge and vulnerability reports. The goal of AGs is to reduce alert fatigue, and to give another perspective on attacker behaviour.
SAGE, the state-of-the-art method for generating AGs uses FlexFringe's state merging algorithms to learn the underlying state machine to model these attacks. A big challenge of these state merging algorithms is learning infrequent behaviour. Next to that, the underlying state machines cannot deal with noisy input data.
In this work, a sequence-automaton alignment algorithm is used to align sequences of states to a state machine. Our method iteratively aligns infrequent sequences to the model, effectively learning both frequent and infrequent behaviour.
The method is evaluated on a security competition dataset, where experiments show that the algorithm is able to recover from noise such as added or removed events. The learned models are also reduced to half its original size, while better fitting to the training data.
Last, we show how our method can be used to learn the model of an anomaly detection dataset. The test data is predicted using three anomaly conditions, and results in all anomalous data being labelled correctly. The F1-score is competitive compared to other state of the art methods.

The rise of alarming cyber breaches and cyber security attacks is causing the world to consider the security of our cyber space. A Security Operations Center (SOC) is a center where the security of a company is monitored to prevent cyber breaches. Security analysts in the SOC examine alerts that come from different devices and analyse what is causing these alerts. The SOC receives a high amount of false positive alerts and duplicates. Therefore security analysts will only react to alerts that seem critical. The problem is that analysts discard alerts that look like false positives but are actually genuine attacks. To tackle this problem of alert fatigue, related work has tried to implement machine learning models to reduce the number of alerts. However, the amount of workload that is reduced is still unsure. We argue that many machine learning models cluster the alerts that are easy for analysts to assess. This thesis compares traditional machine learning techniques with the state-of-the-art neural network DeepCASE and computes the amount of work that is reduced for the analysts. It also compares the machine learning models with a simple heuristic that reduces the duplicates in the dataset. We also enhance DeepCASE to find more sophisticated attacks. We show that using a simple heuristic is as good as using an advanced machine learning algorithm. We also show that using the enhanced version of a state-of-the-art neural network can find more sophisticated attacks.

With the amount of network connected devices every increasing, and many of them running the Secure Shell (SSH) protocol to facilitate remote management, research into SSH attacks is more important than ever. SSH honeypots can be used to act like vulnerable systems while gathering valuable data on the attacker and its methods in the meantime. The SSH handshake is a currently undervalued asset in these honeypots as a lot of data is already exchanged in this early part of the protocol. In this thesis we propose the MD-Honeypot-SSH framework that can be used to gather threat intelligence research data on the SSH handshake. We show the design choices made in the development of the framework and consider which data is useful to collect in the SSH handshake for future research. As part of the framework we modify an existing OpenSSH implementation to allow us to log any relevant branching decisions made in the server. We then use this logging data to create state machines of the server behaviour while handling a specific connection. We use these state machines to compare different connections and show, as a proof of concept, that we can group these connections based on the used client. The main contribution of this thesis is to provide the MD-Honeypot-SSH framework as a tool to future research, and we provide some recommendations for future research directions.