Schedule design in the transportation and logistics sector is a widely studied problem. Transport service providers, such as the train industry and aviation, aim for schedules to be on-time according to the planning (i.e., on-time performance or OTP) in order to increase the service level by ensuring that passengers actually make their connections and to reduce costs. Transportation services also aim for schedules that serve a high variety of destinations and frequency of connections (i.e., connectivity). OTP and connectivity are both highly dependent on buffer time: more lucrative connections can often be offered by reducing the buffer time in the schedule, while more delay can be absorbed by more buffer time. Given strict constraints on the minimum turnaround time of aircraft and minimum (and maximum acceptable) transfer times of passengers, assigning buffer time in an already tightly planned schedule to optimize OTP and connectivity simultaneously is a big challenge. This research presents a novel multi-objective formulation of a daily flight schedule where buffer scheduling is used to ensure the optimal balance between OTP of the schedule and the passenger connections as connectivity, given the tight restrictions. This problem formulation is solved using a simulation–optimization framework. Specifically, we use the Multi-Objective Evolutionary Algorithm (MOEA) BORG. As a proof of concept, a daily European flight schedule of a large international airline is optimized on both OTP and connectivity. The results demonstrate that the presented multi-objective formulation and associated solving through simulation–optimization can result in candidate schedules with both better on-time performance and a higher connectivity.

@en

Simplifications of the real world affect the validity and reliability of gaming simulations. This challenges the application of gaming simulations as an instrument for experiential learning, reflective practices and data collection. This study investigates the effects of simplification on extracting tacit knowledge from human behavior by answering the research question: Can tacit knowledge in a simplified design of a gaming simulation be transferred without compromising the validity and reliability corresponding to the real-world complexity? By applying a participatory design a gaming simulation is tested as an instrument to extract tacit knowledge. To test and evaluate the validity of this application, simulation sessions have been performed with experts from the field. In simplifying reality, participants' participation emphasized that the most accurate representation of reality is a prerequisite for capturing tacit knowledge. This in turn contributes again to the validity of the simulation design. The results show that simplification of the real world didn't affect participants' perspective on the use of the gaming simulation as an experiential tool to enable learning processes or create awareness. And that a simplified simulation design, is still valid in addressing the real-world complexity, with minimization of the level of abstraction and maximization of the truthfulness.

@en

This paper provides an overview of the Workshop on Affective Computing for Mental Wellbeing (mWELL) hosted at the 11th International Conference on Affective Computing and Intelligent Interaction (ACII) in 2023. The workshop aims to bring together researchers, practitioners, and experts from multiple disciplines, to explore how affective computing can contribute to addressing mental health challenges and promoting mental wellbeing, identify key challenges and solutions, and find the most appropriate ways to move the field forward. The paper highlights the workshop’s motivation, objectives, and the contributions made by the participants.@en

Affective aggression is a form of aggression characterized by impulsive reactions driven by strong negative emotions. Despite the extensive research in the area of automatic emotion recognition, affective aggression is a phenomenon that has received less attention. This study investigates the use of head motion as a potential indicator of affective aggression and negative affect. It provides an analysis of head movement patterns associated with various levels of aggression, valence, arousal and dominance, and compares behaviors and recognition performance under speaking and listening conditions. The study was conducted on the Negative Affect and Aggression database - a multimodal corpus of dyadic interactions between aggression regulation training actors and non-actors, annotated for levels of aggression, valence, arousal, and dominance. Results demonstrate that head motion features can serve as promising indicators of affect during both speaking and listening. Valence and arousal prediction achieved better performance during speaking, while aggression and dominance were better predicted during listening. Significant increases in the magnitude of pitch angular acceleration were associated with escalation along all four annotated dimensions. Interestingly, higher escalation was accompanied by a significant increase in the total number of movements during speaking, but a significant decrease of the number of movements was observed as escalation increased along listening intervals. These findings are particularly relevant as head motion can be used solely or potentially as a supplementary modality when other modalities such as speech or facial expressions are unavailable or altered.

@en

The monitoring of an escalating negative interaction has several benefits, particularly in security, (mental) health, and group management. The speech signal is particularly suited to this, as aspects of escalation, including emotional arousal, are proven to easily be captured by the audio signal. A challenge of applying trained systems in real-life applications is their strong dependence on the training material and limited generalization abilities. For this reason, in this contribution, we perform an extensive analysis of three corpora in the Dutch language. All three corpora are high in escalation behavior content and are annotated on alternative dimensions related to escalation. A process of label mapping resulted in two possible ground truth estimations for the three datasets as low, medium, and high escalation levels. To observe class behavior and inter-corpus differences more closely, we perform acoustic analysis of the audio samples, finding that derived labels perform similarly across each corpus, with escalation interaction increasing in pitch (F0) and intensity (dB). We explore the suitability of different speech features, data augmentation, merging corpora for training, and testing on actor and non-actor speech through our experiments. We find that the extent to which merging corpora is successful depends greatly on the similarities between label definitions before label mapping. Finally, we see that the escalation recognition task can be performed in a cross-corpus setup with hand-crafted speech features, obtaining up to 63.8% unweighted average recall (UAR) at best for a cross-corpus analysis, an increase from the inter-corpus results of 59.4% UAR.@en

This study investigates whether an agent-based Negotiation Training System (NTS) can teach women Strategic Empathy - a recently introduced negotiation strategy based on perspective taking - and whether this can improve their negotiation performance. Developed and tested through an interaction-based real-time experiment was a NTS that integrated instructions on how to utilize Strategic Empathy. Women in the experimental group showed significantly higher levels of perspective-taking compared to the control group, and their understanding and use of Strategic Empathy increased over time. Also, a significant positive effect was found of Strategic Empathy on women's self-efficacy. No significant positive effect was found of Strategic Empathy on persistence. The high cognitive load of the experiment and a lack of intrinsic motivation may have caused this finding. Overall, this work demonstrates the applicability of using NTS to teach Strategic Empathy, and its effectiveness for enhancing women's self-efficacy in salary negotiations. @en

Whereas entertainment games are capable of creating deeply rewarding and emotional experiences, applied game projects often result in products that, while potentially effective, are lacking in many other aspects of the user experience. This may be due to the fact that the focus of most design approaches for applied games lies primarily on the use of game mechanics, neglecting other aspects of design that aim to shape and influence the player’s emotional journey. This article provides an exploratory effort in a different approach to creating applied games, namely through the design of user attention and by integrating the theory of attention into applied game design practice. This approach is tested in two ongoing applied game projects, from which preliminary guidelines for applied game researchers and practitioners are proposed.@en

Background: Generalized anxiety disorder (GAD) refers to extreme, uncontrollable, and persistent worry and anxiety. The disorder is known to affect the social functioning and well-being of millions of people, but despite its prevalence and burden to society, it has proven difficult to identify unique behavioral markers. Interestingly, the worrying behavior observed in GAD is argued to stem from a verbal linguistic process. Therefore, the aim of the present study was to investigate if GAD can be predicted from the language people use to put their anxious worries into words. Given the importance of avoidance sensitivity (a higher likelihood to respond anxiously to novel or unexpected triggers) in GAD, this study also explored if prediction accuracy increases when individual differences in behavioral avoidance and approach sensitivity are taken into account.

Method: An expressive writing exercise was used to explore whether GAD can be predicted from linguistic characteristics of written narratives. Specifically, 144 undergraduate student participants were asked to recall an anxious experience during their university life, and describe this experience in written form. Clinically validated behavioral measures for GAD and self-reported sensitivity in behavioral avoidance/inhibition (BIS) and behavioral approach (BAS), were collected. A set of classification experiments was performed to evaluate GAD predictability based on linguistic features, BIS/BAS scores, and a concatenation of the two.

Results: The classification results show that GAD can, indeed, be successfully predicted from anxiety-focused written narratives. Prediction accuracy increased when differences in BIS and BAS were included, which suggests that, under those conditions, negatively valenced emotion words and words relating to social processes could be sufficient for recognition of GAD.

Conclusions: Undergraduate students with a high GAD score can be identified based on their written recollection of an anxious experience during university life. This insight is an important first step toward development of text-based digital health applications and technologies aimed at remote screening for GAD. Future work should investigate the extent to which these results uniquely apply to university campus populations or generalize to other demographics.@en

Although games are frequently described as ‘engaging’, what this means exactly continues to be subject of debate in game literature. Engagement is often defined through related concepts like immersion and positive emotions. However, this neglects the fact that applied games aim to provide more than an entertaining experience, and that engagement with the applied purpose can exist separately from engagement with the game's systems. To make this differentiation more apparent, this article introduces the Applied Games Engagement Model (AGEM), a theoretical model that distinguishes between an applied game's systems and its non-entertainment purpose. It poses that game systems and purpose can overlap in varying amounts, both from game to game, and from moment to moment within a single game. The value of the model is in the explicit acknowledgement that the attention necessary for engaging with content is a limited resource, and that measures for engagement in applied games need to consider that not all engagement is purposeful. The article lays the conceptual foundation for the study of engagement in applied games, and provides a framework for how to design for an applied purpose. It illustrates its use in analysing applied games and their designs through three case studies.@en

Validation of serious games tends to focus on evaluating their design as a whole. While this helps to assess whether a particular combination of game mechanics is successful, it provides little insight into how individual mechanics contribute or detract from a serious game's purpose or a player's game experience. This study analyses the effect of game mechanics commonly used in casual games for engagement, measured as a combination of player behaviour and reported game experience. Secondly, it examines the role of a serious game's purpose on those same measures. An experimental study was conducted with 204 participants playing several versions of a serious game to explore these points. The results show that adding additional game mechanics to a core gameplay loop did not lead to participants playing more or longer, nor did it improve their game experience. Players who were aware of the game's purpose, however, perceived the game as more beneficial, scored their game experience higher, and progressed further. The results show that game mechanics on their own do not necessarily improve engagement, while the effect of perceived value deserves further study. @en

Manufacturing companies are confronted with challenges due to increasing flexibility requirements and skill gaps. Augmented Reality applications offer an efficient way to overcome these tensions by enhancing the interaction between people and technology. The positive effects of Augmented Reality solutions are often described in individual models in the scientific literature. This research-in-progress aims to aggregate the empirical findings in the usage of Augmented Reality solutions in manufacturing environments. A meta-analysis is conducted to synthesise several small studies into one large study to achieve this. In particular, the meta-analysis will focus on the impact of Augmented Reality applications on cognitive load levels. Furthermore, the effect on processing time and error rates will be evaluated. Initial results of the meta-analysis will be expected and reported at this year’s NeuroIS Retreat.@en

Urban energy consumption is expected to continuously increase alongside rapid urbanization. The building sector represents a key area for curbing the consumption trend and reducing energy-related emissions by adopting energy efficiency strategies. Building age acts as a proxy for building insulation properties and is an important parameter for energy models that facilitate decision making. The present study explores the potential of predicting residential building age at a large geographical scale from open spatial data sources in eight municipalities in the German federal state of North-Rhine Westphalia. The proposed framework combines building attributes with street and block metrics as classification features in a Random Forest model. Results show that the addition of urban fabric metrics improves the accuracy of building age prediction in specific training scenarios. Furthermore, the findings highlight the way in which the spatial disposition of training and test samples influences classification accuracy. Additionally, the paper investigates the impact of age misclassification on residential building heat demand estimation. The age classification model leads to reasonable errors in energy estimates, in various scenarios of training, which suggests that the proposed method is a promising addition to the urban energy modelling toolkit.@en

The INTERSPEECH 2021 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the COVID-19 Cough and COVID-19 Speech Sub-Challenges, a binary classification on COVID-19 infection has to be made based on coughing sounds and speech; in the Escalation Sub- Challenge, a three-way assessment of the level of escalation in a dialogue is featured; and in the Primates Sub-Challenge, four species vs background need to be classified. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the 'usual' COMPARE and BoAW features as well as deep unsupervised representation learning using the AUDEEP toolkit, and deep feature extraction from pre-trained CNNs using the DEEP SPECTRUM toolkit; in addition, we add deep end-to-end sequential modelling, and partially linguistic analysis.

@en

Multimodal Sentiment Analysis in Real-life Media (MuSe) 2020 is a Challenge-based Workshop focusing on the tasks of sentiment recognition, as well as emotion-target engagement and trustworthiness detection by means of more comprehensively integrating the audio-visual and language modalities. The purpose of MuSe 2020 is to bring together communities from different disciplines; mainly, the audio-visual emotion recognition community (signal-based), and the sentiment analysis community (symbol-based). We present three distinct sub-challenges: MuSe-Wild, which focuses on continuous emotion (arousal and valence) prediction; MuSe-Topic, in which participants recognise 10 domain-specific topics as the target of 3-class (low, medium, high) emotions; and MuSe-Trust, in which the novel aspect of trustworthiness is to be predicted. In this paper, we provide detailed information on MuSe-CAR, the first of its kind in-the-wild database, which is utilised for the challenge, as well as the state-of-the-art features and modelling approaches applied. For each sub-challenge, a competitive baseline for participants is set; namely, on test we report for MuSe-Wild a combined (valence and arousal) CCC of .2568, for MuSe-Topic a score (computed as 0.34∗UAR + 0.66∗F1) of 76.78 % on the 10-class topic and 40.64 % on the 3-class emotion prediction, and for MuSe-Trust a CCC of .4359.

@en

Background. Revealing tacit knowledge often is seen as very valuable for organizations, although it is usually challenging to enunciate and share this type of knowledge. Methods. This study uses a participatory design and the application of a board gaming simulation as instruments to extract tacit knowledge. To illustrate this application, the gaming simulation is played with entrepreneurs from horticulture. Horticulture represents a complex social system where tacit knowledge plays a major role in the trade process. A participatory design process is used to explore whether the design and play of gaming simulations enable participants to explicate their tacit knowledge. Participants’ participation in designing the gaming simulation explicated that reconstructing reality was a prerequisite for their commitment. Results. The results from playing simulation sessions show that participants were able to: (1) narrow down the anecdotic behaviour to a few factors; (2) to structure these factors; (3) explore how these factors relate to trade barriers and (4) to explain which tactics are applied to foster trade. Conclusion. The educational value of this study is that it helped entrepreneurs in understanding complex real-life situations.

@en

Within the affective computing and social signal processing communities, increasing efforts are being made in order to collect data with genuine (emotional) content. When it comes to negative emotions and even aggression, ethical and privacy related issues prevent the usage of many emotion elicitation methods, and most often actors are employed to act out different scenarios. Moreover, for most databases, emotional arousal is not explicitly checked, and the footage is annotated by external raters based on observable behavior. In the attempt to gather data a step closer to real-life, previous work proposed an elicitation method for collecting the database of negative affect and aggression that involved unscripted role-plays between aggression regulation training actors (actors) and naive participants (students), where only short role descriptions and goals are given to the participants. In this paper we present a validation study for the database of negative affect and aggression by investigating whether the actors' behavior (e.g. becoming more aggressive) had a real impact on the students' emotional arousal. We found significant changes in the students' heart rate variability (HRV) parameters corresponding to changes in aggression level and emotional states of the actors, and therefore conclude that this method can be considered as a good candidate for emotion elicitation.@en

We present the collection and annotation of a multi-modal database with negative human-human interactions. The work is part of supporting behavior recognition in the context of a virtual reality aggression prevention training system. The data consist of dyadic interactions between professional aggression training actors (actors) and naive participants (students). In addition to audio and video, we have recorded motion capture data with kinect, head tracking, and physiological data: heart rate (ECG), galvanic skin response (GSR) and electromyography (EMG) of biceps, triceps and trapezius muscles. Aggression levels, fear, valence, arousal and dominance have been rated separately for actors and students. We observe higher inter-rater agreement for rating the actors than for rating the students, consistently for each annotated dimension, and a higher inter-rater agreement for speaking behavior than for listening behavior. The data can be used among others for research on affect recognition, multimodal fusion and the relation between different bodily manifestation.@en

Negative emotions and stress can impact human-human interactions and eventually lead to aggression. From the perspective of surveillance systems, it is of high importance to recognize as soon as an interaction escalates and human intervention is needed. One of the limitations of deploying a system in real life is that in practice it can only be trained on a limited number of situations. In this paper we examined the generalization capabilities of a trained system given context change. For this purpose we developed scenarios and made audio-visual recordings in four different contexts in which negative interactions might occur. To obtain a quantification of cross-context performance we kept the test context fixed and performed training on itself (cross-validation) and on all the other contexts. To explore whether multiple examples in the training set are beneficial, we also trained the classifier on a merged corpus of the three contexts that were not used for testing. These experiments were done with audio features, video features and audio-visual feature level fusion to investigate which modality generalizes best. We found that context change generates a decrease in performance that is varying with within-contexts similarities. Merging multiple contexts for training in most cases results in performance just below the best predictive single context. Audio is the most robust modality and in most cases the performance of audiovisual fusion is very close to the one of audio. @en