Large-scale survey tools enable the collection of citizen feedback in opinion corpora. Extracting the key arguments from a large and noisy set of opinions helps in understanding the opinions quickly and accurately. Fully automated methods can extract arguments but (1) require large labeled datasets that induce large annotation costs and (2) work well for known viewpoints, but not for novel points of view. We propose HyEnA, a hybrid (human + AI) method for extracting arguments from opinionated texts, combining the speed of automated processing with the understanding and reasoning capabilities of humans. We evaluate HyEnA on three citizen feedback corpora. We find that, on the one hand, HyEnA achieves higher coverage and precision than a state-of-The-Art automated method when compared to a common set of diverse opinions, justifying the need for human insight. On the other hand, HyEnA requires less human effort and does not compromise quality compared to (fully manual) expert analysis, demonstrating the benefit of combining human and artificial intelligence.

@en

Sequential decision making, commonly formalized as Markov Decision Process (MDP) optimization, is an important challenge in artificial intelligence. Two key approaches to this problem are reinforcement learning (RL) and planning. This survey is an integration of both fields, better known as model-based reinforcement learning. Model-based RL has two main steps. First, we systematically cover approaches to dynamics model learning, including challenges like dealing with stochasticity, uncertainty, partial observability, and temporal abstraction. Second, we present a systematic categorization of planning-learning integration, including aspects like: where to start planning, what budgets to allocate to planning and real data collection, how to plan, and how to integrate planning in the learning and acting loop. After these two sections, we also discuss implicit model-based RL as an end-to-end alternative for model learning and planning, and we cover the potential benefits of model-based RL. Along the way, the survey also draws connections to several related RL fields, like hierarchical RL and transfer learning. Altogether, the survey presents a broad conceptual overview of the combination of planning and learning for MDP optimization.

@en

The key arguments underlying a large and noisy set of opinions help understand the opinions quickly and accurately. Fully automated methods can extract arguments but (1) require large labeled datasets and (2) work well for known viewpoints, but not for novel points of view. We propose HyEnA, a hybrid (human + AI) method for extracting arguments from opinionated texts, combining the speed of automated processing with the understanding and reasoning capabilities of humans. We evaluate HyEnA on three feedback corpora. We find that, on the one hand, HyEnA achieves higher coverage and precision than a state-of-the-art automated method, when compared on a common set of diverse opinions, justifying the need for human insight. On the other hand, HyEnA requires less human effort and does not compromise quality compared to (fully manual) expert analysis, demonstrating the benefit of combining human and machine intelligence.

@en

Sequential decision making, commonly formalized as optimization of a Markov Decision Process, is a key challenge in artificial intelligence. Two successful approaches to MDP optimization are reinforcement learning and planning, which both largely have their own research communities. However, if both research fields solve the same problem, then we might be able to disentangle the common factors in their solution approaches. Therefore, this paper presents a unifying algorithmic framework for reinforcement learning and planning (FRAP), which identifies underlying dimensions on which MDP planning and learning algorithms have to decide. At the end of the paper, we compare a variety of well-known planning, model-free and model-based RL algorithms along these dimensions. Altogether, the framework may help provide deeper insight in the algorithmic design space of planning and reinforcement learning.

@en

Video games frequently feature 'open world' environments, designed to motivate exploration. Level design patterns are implemented to invoke curiosity and to guide player behavior. However, evidence of the efficacy of such patterns has remained theoretical. This study presents an empirical study of how level design patterns impact curiosity-driven exploration in a 3D open-world video game. 254 participants played a game in an empirical study using a between-subjects factorial design, testing 4 variables: presence or absence of patterns, goal or open-ended, nature and alien aesthetic, and assured or unassured compensation. Data collection consisted of in-game metrics and emotion word prompts as well as post-game questionnaires. Results show that design patterns invoke heightened exploration, but this effect is influenced by the presence of an explicit goal or monetary compensation. There appear to be many motivations behind exploratory behavior in games, with patterns raising expectations in players. A disposition for curiosity (i.e. 'trait curiosity') was not found to influence exploration. We interpret and discuss the impact of the conditions, individual patterns, and player motivations. @en

Curiosity is a strong motivator for human action, but the circumstances under which one becomes curious are not clear. This paper builds on the assumption that video games can be used as a stimulus for the experimental study of curiosity, and forms a basis in examining the type of curiosity motivated by spatial exploration. A video game was created that incorporates five proposed ‘game design patterns’ that may induce curiosity in players. The game, Shinobi Valley, was tested in a pilot study with 24 participants. Participants responded positively to the game and exhibited exploratory behaviour while playing without specifically being prompted to do so. The presented results suggest which of the patterns are most promising in inducing curiosity, and show that the game is of sufficient quality to be used in larger studies.

@en