Reinforcement Learning (RL) shows great potential for robotic manipulation tasks, yet it suffers from low sample efficiency and needs extensive exploration of state-action spaces. Some recent methods leverage the commonsense knowledge and reasoning abilities of Large Language Models (LLMs) to guide RL exploration toward more meaningful states. However, LLMs may generate semantically correct but physically infeasible plans, leading to unreliable solutions. In this paper, we propose \textit{Language-Guided exploration for Reinforcement Learning} (LGRL), a novel framework that utilizes LLMs' planning capability to directly guide RL exploration. This approach utilizes LLM planning at both the task and affordance levels, enhancing learning efficiency by directing RL agents toward semantically meaningful actions. Unlike previous methods that rely on the optimality of LLM-generated plans or rewards, LGRL corrects sub-optimality and explores multimodal affordance-level plans without human intervention.
We evaluated LGRL on pick-and-place tasks within standard RL benchmark environments, demonstrating significant improvements in both sample efficiency and success rates.

Previous work has shown that state abstraction can be an efficient way to plan in robotic environments with continuous actions and long task horizons. Although some of these works learn predicates for state abstractions, they often neglect an important part needed for generalization. Namely objects and their properties, our work shows that object properties can be exploited to increase the generalizability of learned models. We do this by extending a framework based on predicates for state abstraction and introducing affordances. Affordances allow us to group objects in a way that also considers possible actions. In order to learn affordance interpretations we make use of querying. First, the agent is tasked with solving a task made to test a specific affordance, then based on its uncertainty about an object's affordance interpretation it may query an expert. For example, when presented with a task where it needs to pick up an object it may query the expert on whether the object is actually grippable. We compare our approach with a baseline without affordances and show that our approach needs fewer operators to plan and that our approach is able to generalize to novel objects.

Autonomous vehicles rely on prediction modules, in order to plan collision-free trajectories. Vehicle trajectory prediction models are multimodal, to account for the multiple route options and the inherent uncertainty in human behavior. The state-of-the-art prediction models are deep-learning models, which are susceptible to mode collapse, a phenomenon in which the model fails to output the full distribution of modes and only predicts the most likely one. Mode collapsing poses safety concerns for autonomous driving, as missed predictions could result in collisions. Most works have focused on addressing this issue by generating diverse predictions that cover various route options at the environmental level. However, there are no metrics for mode-collapse. Furthermore, little attention has been given to generating diversity in the interaction modes among agent trajectories. Additionally, the traditional distance-based metrics are heavily dependent on datasets and do not evaluate interactions between agents. To this end, we propose a novel evaluation framework that assesses the interaction modes of joint trajectory predictions, focusing only on the safety-critical interactions in a dataset. We introduce a metric for mode-collapse and time-based metrics for mode correctness and coverage, shedding light on the temporal dimension of the predictions. We test four multi-agent trajectory prediction models on the widely used nuScenes dataset and conclude that mode collapse happens. While the rate of correctly predicted interaction modes increases closer to the interaction event, there are still cases where the models are unable to predict the interaction mode even right before the interaction happens. With the introduction of our novel framework, researchers can now benchmark their models’ performance in predicting critical interactions. This provides new insights and perspectives, helping the holistic evaluation and interpretation of a model’s performance. Additionally, our work offers a new developmental direction for prediction models, aiming for greater consistency and accuracy in predicting agent interactions, thereby advancing the safety of autonomous driving systems. Our evaluation framework is available online at: https://github.com/MaartenHugenholtz/InteractionEval

Intelligent manufacturing has become increasingly important in the food packaging industry due to the growing demand for enhanced productivity and flexibility while minimizing waste and lead times. This work explores the integration of such manufacturing in automated secondary robotic food packaging solutions that transfer food products into containers using pick-and-place robots. A major problem in these solutions is varying product supply caused by prior machinery. As a result productivity drops drastically when conveyor belt speeds are not optimally controlled. Conventional heuristic-based engineered approaches are used to address this issue but are inadequate, leading to noncompliance with industry's requirements. Reinforcement learning, on the other hand, has the potential of solving this problem by learning quick and predictive decision-making behavior based on experience. However, the lack of research in reinforcement learning for complex industrial robotic problems, limits its adoption in industry. Therefore, this work aims to investigate the feasibility of reinforcement learning in the robotic packaging industry. We propose a reinforcement learning framework, with policy inference in a highly complex control scheme, designed to optimize the conveyor belt speed of the secondary robotic packaging solution using real-world product supply data. The framework exceeds the 99.8 percent performance requirement and maintains quality at the required 100 percent when tested on real-world data. Compared to the current heuristic-based solution, our proposed framework improves productivity, has smoother control and reduces code execution time.