Soft robots are typically intended to operate in highly unpredictable and unstructured environments. Although their soft bodies help them to passively conform to their environment, the execution of specific tasks within such environments often requires the help of an operator that supervises the interaction between the robot and its environment and adjusts the actuation inputs in order to successfully execute the task. However, direct observation of the soft robot is often impeded by the environment in which it operates. Therefore, the operator has to depend on a real-time simulation of the soft robot based on the signals from proprioceptive sensors. However, the complicated three-dimensional (3D) configurations of the soft robot can be difficult to interpret using traditional visualization techniques. In this work, we present an open-source framework for real-time 3D reconstruction of soft robots in eXtended Reality (Augmented and Virtual Reality), based on signals from their proprioceptive sensors. This framework has a Robot Operating System (ROS) backbone, allowing for easy integration with existing soft robot control algorithms for intuitive and real-time teleoperation. This approach is demonstrated in Augmented Reality using a Microsoft Hololens device and runs at up to 60 FPS. We explore the influence that system parameters such as mesh density and armature complexity have on the reconstruction's key performance metrics (i.e., speed, scalability). The open-source framework is expected to function as a platform for future research and developments on real-time remote control of soft robots operating in environments that impede direct observation of the robot.@en

5G promises to shift Industry 4.0 to the next level by allowing flexible production. However, many communication standards are used throughout a production site, which will stay so in the foreseeable future. Furthermore, localization of assets will be equally valuable in order to get to a higher level of automation. This paper proposes a reference architecture for a convergent localization and communication network for smart manufacturing that combines 5G with other existing technologies and focuses on high-mix, low-volume applications, particularly at small and medium-sized enterprises. The architecture is derived from a set of functional requirements, and we describe different views on this architecture to show how the requirements can be fulfilled. It connects private and public mobile networks with local networking technologies to achieve a flexible setup addressing many industrial use cases.

@en

For designing large-scale products like an airplane, engaging end-users in the concept phase is difficult. However, early user evaluation is important to choose the path which fits the user’s needs best. In particular, comfort-related assessments are difficult to conduct with digital models that are shown on a desktop PC application. Digital Human Modelling (DHM) plays a role in postural comfort analysis, while the subjective comfort feedback still largely relied on consulting with end-users. This paper applies a human-centered design process and analyses the advantages and disadvantages of using VR prototypes for involving users during concept design. This study focused on using VR prototypes for concept selection and verification based on comfort assessment with potential end-users. The design process started with an online questionnaire for identifying the quality of the design elements (Step 1 online study). Then, alternative concepts were implemented in VR, and users evaluated these concepts via a VR headset (Step 2 Selection study). Finally, the research team redesigned the final concept and assessed it with potential users via a VR headset (Step 3 Experience study). Every design element contributed positively to the long-haul flight comfort, especially tap-basin height, storage, and facilities. The male and female participants had different preferences on posture, lighting, storage, and facilities. The final prototype showed a significantly higher comfort rate than the original prototypes. The first-person immersion in VR headsets helps to identify the nuances between concepts, thus supports better decision-making via collecting richer and more reliable user feedback to make faster and more satisfying improvements.

@en

Maintaining a complex system, such as a modern production line, is a knowledge-intensive task. Many firms use maintenance reports as a decision support tool. However, reports are often poor quality and tedious to compile. A Conversational User Interface (CUI) could streamline the reporting process by validating the user's input, eliciting more valuable information, and reducing the time needed. In this paper, we use a Technology Probe to explore the potential of a CUI to create instructional maintenance reports. We conducted a between-groups study (N = 24) in which participants had to replace the inner tube of a bicycle tire. One group documented the procedure using a CUI while replacing the inner tube, whereas the other group compiled a paper report afterward. The CUI was enacted by a researcher according to a set of rules. Our results indicate that using a CUI for maintenance reports saves a significant amount of time, is no more cognitively demanding than writing a report, and results in maintenance reports of higher quality.

@en

Higher levels of vehicle automation come with new challenges for designing safe systems. The Human Machine-Interface (HMI) plays a key role in mediating the interaction between the human driver and vehicle automation. By providing the driver with appropriate feedback, the HMI has the potential to increase mode awareness and situational awareness. For the development of appropriate HMI solutions, usability assessments are essential. Immersive Virtual Reality (VR) technology enables researchers and designers to construct realistic virtual prototypes and immersive evaluation scenarios with less time and resources. The current study presents a VR evaluation tool called VRHEAD, which is designed to facilitate an iterative design process and support the rapid implementation of virtual prototypes to evaluate of an automated vehicle's HMI. Initial results indicate that VRHEAD is a promising approach for the rapid implementation and evaluation of design concepts. The use of VR tools, like VRHEAD, can reduce the time and costs associated with developing high-fidelity prototypes and provide more flexibility in modifying a design according to new research findings, thus broadening the exploration of the HMI design space.

@en

Manufacturing machines need to be retooled approximately 15 times per week and in the future even more often because of decreasing batch sizes and increasing short-cyclic demands. Collaborative robots promise to offer a versatile automation approach for priorly manual tasks in small and medium-sized enterprises. However, their configuration needs to change at least as often as the retooling rate because different parts are produced by the machines or might require different handling in general. Therefore, it would be great if robots and autonomous factory systems, in general, would automatically adjust to these changes in an intelligent way. In our approach, we propose a context-aware and location-based approach for agile manufacturing, in which the manufacturing plant parts, especially the collaborative robots, store i) their constellation, ii) their configuration, and iii) their adaptation strategy, and can react to retooling changes and even re-location changes adaptively. For example, moving one collaborative robot to a different location next to the plant will automatically load its new configuration and consult the operator on the adaptation strategy (i. e. the safety requirements). To realize the localization and the network capabilities, we propose to use a multichannel 5G-enabled communication base station and an intelligent asset management strategy.

@en

This paper analyzes the effective accuracy for close-range operations for the first and the second generation of Microsoft HoloLens in combination with Vuforia Image Targets in a black-box approach. The implementation of Augmented Reality (AR) on optical see-through (OST), head-mounted devices (HMDs) has been proven viable for a variety of tasks, such as assembly, maintenance, or educational purposes. For most of these applications, minor localization errors are tolerated since no accurate alignment between the artificial and the real parts is required. For other potential applications, these accuracy errors represent a major obstacle. The "realistically achievable"accuracy remains largely unknown for close-range usages (e.g. within "arms-reach"of a user) for both generations of Microsoft HoloLens.Thus, the authors developed a method to benchmark and compare the applicability of these devices for tasks that demand a higher accuracy like composite manufacturing or medical surgery assistance. Furthermore, the method can be used for a broad variety of devices, establishing a platform for bench-marking and comparing these and future devices. This paper analyzes the performance of test users, which were asked to pinpoint the perceived location of holographic cones. The image recognition software package "Vuforia"was used to determine the spatial transform of the predefined ImageTarget. By comparing the user-markings with the algorithmic locations, a mean deviation of 2.59 ±1.79 [mm] (HL 1) and 1.11 ±0.98 [mm] (HL 2) has been found, which means that the mean accuracy improved by 57.1% and precision by 45.4%. The highest mean accuracy of a single test user has been measured with 0.47 ±1.683 [mm] (HL 1) and 0.085 ± 0.567 [mm] (HL 2).

@en

Bicycle production has not changed much over the last 100 years, it is still performed mainly by manual labor in mass production. During the global pandemic, the demand for ecologically friendly and customized transport has increased. Hence, customers start to impose the same requirements on bikes as on cars: they want more customized products and short delivery time. This publication describes an approach to transform bicycle manufacturing towards human-robot co-production to enable smaller batch sizes and production on-shoring. We list the challenges of this transformation, our applied methods, and presents preliminary results of the cobot-driven prototypes.

@en

This paper proposes to use Virtual Reality scenarios to explore the reaction of stakeholders within an innovation process in the context of the introduction of robots working in close collaboration with users. The goal is to design the system upfront in such a way, that it is not perceived as a threat to the worker or his/her job. Within the responsible research and innovation approach, the introduction of new technology needs to be accompanied by a careful investigation of the thoughts and feelings of all stakeholders. Especially workers who are currently not working with robots but their workspace is currently undergoing an Industry 4.0 driven transformation, experience fear, that this new technology will make their jobs redundant. On the other hand, it can be observed, that successful robot interaction processes, on the one hand, increase the overall productivity, but also can enhance human well-being. The feeling of 'teamwork' with the artificial intelligence entity can develop to be equally positive and motivating. To be able to design future workspaces which will result in a 'teamwork' perception instead of the 'fear' perception, the use of VR can be applied.

@en

How to visualize recorded production data in Virtual Reality? How to use state of the art Augmented Reality displays that can show robot data? This paper introduces an opensource ICT framework approach for combining Unity-based Mixed Reality applications with robotic production equipment using ROS Industrial. This publication gives details on the implementation and demonstrates the use as a data analysis tool in the context of scientific exchange within the area of Mixed Reality enabled human-robot co-production.

@en

Voice-enabled assistants, such as Alexa and Google Assistant, are among the fastest-growing information technology applications. Their technological foundation matured over the last years and reached a point where new application areas in challenging business environments become a certainty. Maintenance in manufacturing is one of these areas. This paper presents expectations, requirements, and a concept for a voice-enabled digital intelligent assistant that supports maintenance activities. We identified process monitoring, task execution, reporting, problem-solving, and maintenance planning as the key functional modules for an assistant. Realizing them depends on basic, utility, and maintenance functions. Our discussion states that all fundamental technologies and tools to realize an assistant for maintenance exist, but they have constraints. For instance, Speech-to-Text mechanisms lack transparent and performant solutions, and natural language understanding must rely on small datasets, which is challenging. We argue that continuous improvement and systematic evaluation of an assistant prototype is important to create high-quality training data. Trial-and-error is common because some technologies still mature, and conversation designers lack design patterns for the maintenance domain. Challenges for system adoption include providing an outstanding user experience, handling factory-specific jargon, and the limited availability of easy-to-use data exchange interfaces for machines and business applications. We conclude that further efforts on interoperability, technology stack management, AI-focused change management, and education programs are necessary. Furthermore, the accountability of AI systems is a cost factor for the assistant’s service providers and the client companies in manufacturing – AI insurance services, human-in-the-loop functions, user training, and professional education are actions to address this issue.@en

This design paper describes the development of custom built interface between a force-replicating virtual reality (VR) haptic interface glove, and a user. The ability to convey haptic information – both kinematic and tactile – is a critical barrier in creating comprehensive simulations. Haptic interface gloves can convey haptic information, but often the haptic “signal” is diluted by sensory “noise,” miscuing the user’s brain. Our goal is to convey compelling interactions – such as grasping, squeezing, and pressing – with virtual objects by improving one such haptic interface glove, the SenseGlove, through a redesign of the user-glove interface, soft glove. The redesign revolves around three critical design factors – comfort, realism, and performance – and three critical design areas – thimble/fingertip, palm, and haptic feedback. This paper introduces the redesign method and compares the two designs with a quantitative user study. The benefit of the improved soft glove can be shown by a significant improvement of the design factors, quantified through QUESI, NASA-TLX, and comfort questionnaires.

@en

In order to support the decision-making process of industry on how to implement Augmented Reality (AR) in production, this article wants to provide guidance through a set of comparative user studies. The results are obtained from the feedback of 160 participants who performed the same repair task on a switch cabinet of an industrial robot. The studies compare several AR instruction applications on different display devices (head-mounted display, handheld tablet PC and projection-based spatial AR) with baseline conditions (paper instructions and phone support), both in a single-user and a collaborative setting. Next to insights on the performance of the individual device types for the single mode operation, the study is able to show significant indications on AR techniques are being especially helpful in a collaborative setting.

@en

This paper describes the development of custom built interface between a force-replicating Virtual Reality (VR) haptic glove, and a user. The ability to convey haptic information - both kinesthetic and tactile - is a critical barrier in creating comprehensive simulations. Haptic interface gloves can convey haptic information, but often the haptic signal is diluted by sensory noise, miscuing the users brain. Our goal is to convey compelling interactions with virtual objects, such as grasping, squeezing, and pressing by improving one such haptic interface glove - the Sense Glove - through a redesign of the user-glove interface - Soft Glove. The redesign revolves around three critical design factors - comfort, realism, and performance - and three critical design areas - thimble/fingertip, palm, and haptic feedback. This paper introduces the redesign method and compares the two designs with a quantitative user study. The benefit of the Improved Soft Glove can be shown by a significant improvement of the design factors.

@en

The causes and treatment solutions of congenital heart defects are difficult to address and discuss between patient and doctor. This is mainly due to the complex spatial nature of congenital cardiac defects, which makes it difficult for the patients to envision the defect without prior anatomical knowledge and renders the comprehension largely dependent on doctors' (variable) skills to describe the anomaly. To improve communication, 3D printed hearts have been developed, yet these are expensive, difficult to manage for the large collection of defects, and require substantial oral explanation. In addition, the correlation with cardiac function remains rather abstract. Instead, we propose an augmented reality solution, involving a see-through head-mounted display (HMD) extended with a built-in heart rate monitor. In order to increase the presence and the conversational power, the heartbeat of the patient is used to drive an animation of a supersized, floating heart visualisation; enabling the user to inspect a specific heart condition from all sides. To enable this, a universal add-on casing was developed for the HoloLens. Heuristic analysis and pilot tests with $6+15$ participants reveal limitations of the implementation and show that the solution does increase comprehension, although more has to be done to enable a robust system.

@en

The project 'Factory-in-A-day' aims at reducing the installation time of a new hybrid robot-human production line, from weeks or months that current industrial systems now take, down to one day. The ability to rapidly install (and reconfigure) production lines where robots work alongside humans will strongly reduce operating cost and open a range of new opportunities for industry. In this paper, we explore a method of collaborative fabrication planning with the help of Augmented Reality as part of the concept Augmented Fabrication. In order to plan a new production line, two co-located workers at the factory wear a Microsoft Hololens head-mounted display and thus share a common visual context on the planed position of the robots and the production machines. They are assisted by an external remote expert connected via the Internet who is virtually co-located. We developed three different visualizations of the state of the local collaboration and plan to compare them in a user study.

@en

How does individualized production look like? This paper proposes a generic vision of a smart factory which is able to produce so-called ultra personalized products(UPPS) in a batch size of one. This means that each customer is able to configure his or her personal product perfectly adjusted to the needs on base of personal data like bodyscans. As we are convinced that there will always be humans involved in planning and conducting manufacturing processes, we want to highlight the human-to-human interaction and the computer supported collaborative work, which is required in order to set up such a production line.@en

Digitization and the growing capabilities of data networks enable companies to perform tasks via remote support, which previously required service personnel to travel. But which mixed reality method leads to better results regarding human factors, grounding and performance criteria? This paper reports on a collaborative user study, in which a local worker is guided by a remote expert with the help of different augmented reality methods. The performed task is an controller exchange in a switch cabinet of an industrial robot, a task rather typical for failure detection within the field. Our study was conducted in collaboration with a technician school of which 50 technician apprentices participated in our study. Our results show clear advantages of using augmented reality (AR) to enable remote support. It further gives significant indications for using a projection based AR method.@en

Within the European Factory-in-a-day project, the aim is to improve communication between automation integrator and factory owner, in their analysis of feasibility and appropriateness of automating a manual task. A visualisation tool with preconfigured workflows and working principles, with specific focus on efficient human-robot coproduction workflows can improve this process. This paper describes the Workflow Simulation Tool, which is part of the Human-Robot Coproduction Methodology, currently in development. The tool encompasses a portable tablet PC, which runs a visual modelling environment combined with a handheld 3D scanning solution. The tool also features pre-modelled template layouts, implementation of a checklist of persistent notes and portable visual documentation. The tool's appropriateness was iteratively validated in collaboration with automation integrators. This evaluation showed that offering an interactive visual simulation enriches the dialogue during conceptual design and helps in revealing requirements that otherwise only appear during or after implementation.

@en