Training classifiers that are robust against adversarially modified examples is becoming increasingly important in practice. In the field of malware detection, adversaries modify malicious binary files to seem benign while preserving their malicious behavior. We report on the results of a recently held robust malware detection challenge. There were two tracks in which teams could participate: the attack track asked for adversarially modified malware samples and the defend track asked for trained neural network classifiers that are robust to such modifications. The teams were unaware of the attacks/defenses they had to detect/evade. Although only 9 teams participated, this unique setting allowed us to make several interesting observations. We also present the challenge winner: GRAMS, a family of novel techniques to train adversarially robust networks that preserve the intended (malicious) functionality and yield high-quality adversarial samples. These samples are used to iteratively train a robust classifier. We show that our techniques, based on discrete optimization techniques, beat purely gradient-based methods. GRAMS obtained first place in both the attack and defend tracks of the competition.@en

We propose new metrics drawing inspiration from the optimization domain that can be used to characterize the effectiveness of moving target defenses better. Besides that, we propose a Network Neighborhood Partitioning algorithm that can help to measure the influence of MTDs more precisely. The techniques proposed here are generic and could be combined with existing metrics. The obtained results demonstrate how additional information about the effectiveness of defenses can be obtained as well as how network neighborhood partitioning helps to improve the granularity of metrics.@en