This research revolves around measuring the quality of arguments. High-quality arguments help in improving political discussions, resulting in better decision-making. Wachsmuth et al. developed a taxonomy breaking down argument quality into several dimensions. This work makes use of that taxonomy and combines it with modern NLP models. A cross-dataset examination of argument quality models was conducted. In particular, models were investigated on their generalizability between dimensions. Overall results show that there is no large difference in accuracy and agreement when models predict data of a quality dimension they were trained on, over dimensions they were not trained on. One can conclude that generalizations of argument quality dimensions with language models were not found. Nevertheless, qualitative analysis highlights findings that indicate some generalization to other dimensions.