Structural electronics has garnered significant attention in the past decade. However, there remains a lack of a systematic approach in designing and manufacturing sensors that leverage both mechanical and electronic properties of materials for different applications. In this paper, we introduce a method for designing piezoresistive force sensors utilizing structural electronics and 3D printing techniques. Based on the principles of piezoresistive force sensing, we defined the geometric profile of the sensor by simultaneously maximizing strain and ensuring as uniform as possible stress distribution across the geometry. CAD models of the sensors were then formulated based on the optimized profile and fabricated using conductive filaments and the material extrusion 3D printing technique. Subsequently, we evaluated the accuracy, the sensitivity, and part-to-part variations of the sensors during loading and unloading. The influence of environmental temperature and humidity on the sensor's response were also investigated and compensated. Experiment results demonstrated the feasibility of the proposed method and revealed potential application domains, as well as limitations of the sensors.

@en

In this paper, we propose a method to reconstruct a digital 3D model of a stolen/damaged statue using photogrammetric methods. This task is challenging because the number of available photos for a stolen statue is in general very limited – especially the side/back view photos. Besides using standard structure-from-motion and multi-view stereo methods, we match image pairs with low overlap using sliding windows and maximize the normalized cross-correlation (NCC) based patch-consistency so that the image pairs can be well aligned into a complete model to build the 3D mesh surface. Our method is based on the prior of the planar side on the statue’s pedestal, which can cover a large range of statues. We hope this work will motivate more research efforts for the reconstruction of those stolen/damaged statues and heritage preservation.@en

This article presents an efficient learning-based method to solve the <italic>inverse kinematic</italic> (IK) problem on soft robots with highly nonlinear deformation. The major challenge of efficiently computing IK for such robots is due to the lack of analytical formulation for either forward or inverse kinematics. To address this challenge, we employ neural networks to learn both the mapping function of forward kinematics and also the Jacobian of this function. As a result, Jacobian-based iteration can be applied to solve the IK problem. A sim-to-real training transfer strategy is conducted to make this approach more practical. We first generate a large number of samples in a simulation environment for learning both the kinematic and the Jacobian networks of a soft robot design. Thereafter, a sim-to-real layer of differentiable neurons is employed to map the results of simulation to the physical hardware, where this sim-to-real layer can be learned from a very limited number of training samples generated on the hardware.

@en

The geometric shapes and the relative position of coils influence the performance of a three-dimensional (3D) inductive power transfer system. In this paper, we propose a coil design method for specifying the positions and the 3D shapes of a pair of coils to transmit the desired power. Given region of interests (ROIs) for designing the transmitter and the receiver coils on two surfaces, the transmitter coil is generated around the center of its ROI. The center of the receiver coil is estimated as a random seed position in the corresponding 3D surface. At this position, we use the heatmap method with electromagnetic constraints to iteratively extend the coil until the desired power can be transferred via the set of coils. In each step, the shape of the extension, i.e., a new turn of the receiver coil, is found as a spiral curve based on the convex hulls of the 2D projected adjacent turns along their normal direction. Then, the optimal position of the receiver coil is found by maximizing the efficiency of the system. In the next step, the position and the shape of the transmitter coil are optimized based on the fixed receiver coil using the same method. This optimization process iterates until an optimum is reached. Simulations and experiments with digitally fabricated prototypes were conducted and the effectiveness of the proposed 3D coil design method was verified.

@en

Real-time proprioception is a challenging problem for soft robots, which have virtually infinite degrees of freedom in body deformation. When multiple actuators are used, it becomes more difficult as deformation can also occur on actuators caused by interaction between each other. To tackle this problem, we present a method in this article to sense and reconstruct 3-D deformation on pneumatic soft robots by first integrating multiple low-cost sensors inside the chambers of pneumatic actuators and then using machine learning to convert the captured signals into shape parameters of soft robots. An exterior motion capture system is employed to generate the datasets for both training and testing. With the help of good shape parameterization, the 3-D shape of a soft robot can be accurately reconstructed from signals obtained from multiple sensors. We demonstrate the effectiveness of this approach on two soft robot designs - a robotic joint and a deformable membrane. After parameterizing the deformation of these soft robots into compact shape parameters, we can effectively train the neural networks to reconstruct the 3-D deformation from the sensor signals. The sensing and shape prediction pipeline can run at 50 Hz in real time on a consumer-level device.

@en

The geometric shapes of coils influence the performance of a 3D IPT system. In this paper, we proposed a 3D coil design method based on (3D) printing electronics. Given a 3D transmitter coil, the center position of the receiver coil is estimated as a random seed position in the corresponding 3D surface first. At this position, we use the heatmap method with electromagnetic constraints to iteratively extend the coil until the desired power can be transferred via the coil. For each extension of the coil, i.e. a new turn, the shape of the coil is optimized by calculating the convex hull of the new turn in the 2D projection plane. Using this method, we are able to generate a receiver coil to transmit “just enough” power at a given seed position. Then, by fixing the receiver coil, the 3D shape of the transmitter coil can be optimized as well. This zig-zag optimization process iterates until there are few changes of the position and 3D shapes in the iteration. Experiment results with Ansys Maxwell verified the effectiveness of the proposed 3D coil design method, and highlighted possible future research directions as well.

@en

We present a method for effectively planning the motion trajectory of robots in manufacturing tasks, the tool paths of which are usually complex and have a large number of discrete time constraints as waypoints. Kinematic redundancy also exists in these robotic systems. The jerk of motion is optimized in our trajectory planning method at the meanwhile of fabrication process to improve the quality of fabrication. Our method is based on a sampling strategy and consists of two major parts. After determining an initial path by graph search, a greedy algorithm is adopted to optimize a path by locally applying adaptive filers in the regions with large jerks. The filtered result is obtained by numerical optimization. In order to achieve efficient computation, an adaptive sampling method is developed for learning a collision-indication function that is represented as a support-vector machine. Applications in robot-Assisted 3-D printing are given in this article to demonstrate the functionality of our approach. Note to Practitioners-In robot-Assisted manufacturing applications, robotic arms are employed to realize the motion of workpieces (or machining tools) specified as a sequence of waypoints with the positions of tool tip and the tool orientations constrained. The required degree of freedom (DOF) is often less than the robotic hardware system (e.g., a robotic arm has six-DOF). Specifically, rotations of the workpiece around the axis of a tool can be arbitrary (see Fig. 1 for an example). By using this redundancy, i.e., there are many possible poses of a robotic arm to realize a given waypoint, the trajectory of robots can be optimized to consider the performance of motion in velocity, acceleration, and jerk in the joint space. In addition, when fabricating complex models, each tool path can have a large amount of waypoints. It is crucial for a motion planning algorithm to compute a smooth and collision-free trajectory of robot to improve the fabrication quality. The time taken by the planning algorithm should not significantly lengthen the total manufacturing time; ideally, it would remain hidden as computing motions for a layer can be done while the previous layer is printing. The method presented in this article provides an efficient framework to tackle this problem. The framework has been well tested on our robot-Assisted additive manufacturing system to demonstrate its effectiveness and can be generally applied to other robot-Assisted manufacturing systems.

@en

High fidelity reproductions of paintings provide new opportunities to museums in preserving and providing access to cultural heritage. This paper presents an integrated system which is able to capture and fabricate color, topography and gloss of a painting, of which gloss capturing forms the most important contribution. A 3D imaging system, utilizing stereo imaging combined with fringe projection, is extended to capture spatially-varying gloss, based on the effect of specular reflectance polarization. The gloss is measured by sampling the specular reflection around Brewster’s angle, where these reflections are effectively polarized, and can be separated from the unpolarized, diffuse reflectance. Off-center gloss measurements are calibrated relative to the center measurement. Off-specular gloss measurements, following from local variation of the surface
normal, are masked based on the height map and corrected. Shadowed regions, caused by the 3D relief, are treated similarly. The area of a single capture is approximately 180x90mm at a resolution of 25x25μm. Aligned color, height, and gloss tiles are stitched together off-line, registering overlapping color regions. The resulting color, height and gloss maps are inputs for the poly-jet 3D printer. Two paintings were reproduced to verify the effectiveness and efficiency of the proposed system. One painting was scanned four times, consecutively rotated by 90 degrees, to evaluate the influence of the scanning system geometric configuration on the gloss measurement. Experimental results show that the method is sufficiently fast for practical application, i.e. to scan a whole painting within eight hours, during closing hours of a museum. The results can well be
used for the purpose of physical reproduction and other applications needing first-order estimates of the appearance (e.g. conservation diagnostics and condition reports). Our method to extend appearance scanning with gloss measurements is a valuable addition in the quest for realistic reproductions, in terms of its practical applicability - number of images needed for reconstruction and speed - and its perceptual added value, when added to color and topography reproduction.@en

For grasping (unknown) objects, soft pneumatic actuators are primarily designed to bend towards a specific direction. Due to the flexibility of material and structure, soft actuators are also prone to out-of-plane deformations including twisting and sidewards bending, especially if the loading is asymmetric. In this paper, we demonstrate the negative effects of out-of-plane deformation on grasping. A structural design is proposed to reduce this type of deformation and thus improve grasping stability. Comparisons are first performed on soft pneumatic actuators with the same bending stiffness but different resistances to out-of-plane deformation, which is realized by changing the cross-section of the inextensible layer. To reduce out-of-plane deformation, a stiffening structure inspired by spatial flexures is integrated into the soft actuator. The integrated design is 3D printed using a single material. Physical experiments have been conducted to verify the improved grasping stability.

@en

A seventeenth-century canvas painting is usually comprised of varnish and (translucent) paint layers on a substrate. A viewer’s perception of a work of art can be affected by changes in and damages to these layers. Crack formation in the multi-layered stratigraphy of the painting is visible in the surface topology. Furthermore, the impact of mechanical abrasion, (photo)chemical processes and treatments can affect the topography of the surface and thereby its appearance. New technological advancements in non-invasive imaging allow for the documentation and visualisation of a painting’s 3D shape across larger segments or even the complete surface. In this manuscript we compare three 3D scanning techniques, which have been used to capture the surface topology of Girl with a Pearl Earring by Johannes Vermeer (c. 1665): a painting in the collection of the Mauritshuis, the Hague. These three techniques are: multi-scale optical coherence tomography, 3D scanning based on fringe-encoded stereo imaging (at two resolutions), and 3D digital microscopy. Additionally, scans were made of a reference target and compared to 3D data obtained with white-light confocal profilometry. The 3D data sets were aligned using a scale-invariant template matching algorithm, and compared on their ability to visualise topographical details of interest. Also the merits and limitations for the individual imaging techniques are discussed in-depth. We find that the 3D digital microscopy and the multi-scale optical coherence tomography offer the highest measurement accuracy and precision. However, the small field-of-view of these techniques, makes them relatively slow and thereby less viable solutions for capturing larger (areas of) paintings. For Girl with a Pearl Earring we find that the 3D data provides an unparalleled insight into the surface features of this painting, specifically related to ‘moating’ around impasto, the effects of paint consolidation in earlier restoration campaigns and aging, through visualisation of the crack pattern. Furthermore, the data sets provide a starting point for future documentation and monitoring of the surface topology changes over time. These scans were carried out as part of the research project ‘The Girl in the Spotlight’.

@en

Monocular image based 3D object retrieval is a novel and challenging research topic in the field of 3D object retrieval. Given a RGB image captured in real world, it aims to search for relevant 3D objects from a dataset. To advance this promising research, we organize this SHREC track and build the first monocular image based 3D object retrieval benchmark by collecting 2D images from ImageNet and 3D objects from popular 3D datasets such as NTU, PSB, ModelNet40 and ShapeNet. The benchmark contains classified 21,000 2D images and 7,690 3D objects of 21 categories. This track attracted 9 groups from 4 countries and the submission of 20 runs. To have a comprehensive comparison, 7 commonly-used retrieval performance metrics have been used to evaluate their retrieval performance. The evaluation results show that the supervised cross domain learning get the superior retrieval performance (Best NN is 97.4 %) by bridging the domain gap with label information. However, there is still a big challenge for unsupervised cross domain learning (Best NN is 61.2%), which is more practical for the real application. Although we provided both view images and OBJ file for each 3D model, all the participants use the view images to represent the 3D model. One of the interesting work in the future is directly using the 3D information and 2D RGB information to solve the task of monocular Image based 3D model retrieval.@en

2D coil design limits the use of wireless power transfer (WPT) in many products with freeform outer shapes. In this paper, enabled by 3D printed electronics, we propose a systematic approach to design and fabricate 3D coils for WPT. Based on the circular spiral and rectangular spiral patterns, 3D receiver and transmitter coils can be generated on an arbitrarily selected region of a product and its offset, respectively. Mathematical models are proposed to estimate the self-inductance and the mutual-inductance of the 3D arbitrarily shaped coils for 3D WPT. This leads to a new design approach of a 3D WPT system. Several sets of 3D printed WPT systems were designed, simulated, and prototyped to demonstrate the effectiveness of the proposed design approach as well as the mathematical models. The calculation speed of the proposed mathematical models is 30 times faster than the simulation, and compared with the measurement results, the calculation results have mean absolute errors of 2.63% and 4.45% regarding the self- and the mutual-inductance, where the simulation results have mean absolute errors of 1.20% and 2.38%, respectively. Measurements also indicate that with a 5V input, the prototypes are able to deliver 1-watt power at an efficiency ranging between 20.9% and 25.3%. It was concluded that the proposed approach is feasible and promising for designing and manufacturing WPT using 3D printed electronics.

@en

Actuators using soft materials feature a large number of degrees of freedom. This tremendous flexibility allows a soft actuator to passively adapt its shape to the objects under interaction. In this paper, we propose a novel proprioception method for soft actuators during real-time interaction with previously unknown objects. First, we design a color-based sensing structure that instantly translates the inflation of a bellow into changes in color, which are subsequently detected by a miniaturized color sensor. The color sensor is small and, thus, multiple of them can be integrated into soft pneumatic actuators to reflect local deformations. Second, we make use of a feed-forward neural network to reconstruct a multivariate global shape deformation from local color signals. Our results demonstrate that deformations of the actuator during interaction, including sigmoid-like shapes, can be accurately reconstructed. The accurate shape sensing represents a significant step toward closed-loop control of soft robots in unstructured environments.

@en

In current bipolar electrosurgical instruments, a high frequency electrical sinusoidal wave is passed through the patient's body from an active electrode to the return electrode to cut, coagulate, or desiccate tissues. Even though current bipolar electrosurgical instruments have proven effective in minimizing blood loss, advancement is needed to allow for improved dexterity and adaptability. With current advances in threedimensional (3D)-print processes and its integration in the medical field, it has become possible to manufacture patient-and operation-specific instruments. In this study, we introduce the first 3D-printed steerable bipolar grasper (□ 5 mm) for use in minimal invasive surgery. The grasper significantly improves dexterity by the addition of two planar joints allowing for ±65 deg for sideways and ±85 deg for up- and downward movement. The joints enable a significantly higher bending stiffness, 4.0 N/mm for joint 1 and 4.4 N/mm for joint 2, than that of currently available steerable instruments. The tip consists of two metallic movable jaws that can be opened and closed with angles up to 170 deg and allows for grasping and coagulating of tissues; reaching tissue temperatures of over 75 °C for an activation time of ~5 s, respectively. In order to actuate the joint, tip, and electrosurgical system, as well as to tension the steering cables, a ring handle was designed. In summary, the 3D-printed steerable bipolar grasper provides the surgeon with electrosurgical capabilities, improved dexterity, improved stiffness, and the versatility that is needed to provide patient- and operation-specific care.

@en

In the past decade, wireless power transfer enabled many new applications. However, most designs of transmitter and receiver coils are 2D, which limits the possibilities of integrating them to more applications. In this paper, based on the Qi standard, we proposed a design of 3D coils by focusing on the 3D geometry of the coil, modelling, simulation and manufacturing. Using the 3D spiral transmitter coil and the receiver coil as an example, we present the mathematical model for WPT first. The performance of the model is then verified by FEM simulation. Three sets of 3D coils with different design parameters were prototyped by 3D printed electronics and measured against the model and the simulation results. Compared to the measurements, the model has average errors of 1.6%, 8.4% 8.5% regarding self-inductance, mutual-inductance and the coupling factor, where the simulation has average errors of 0.5%, 9.9% and 6.6%, respectively. Measurements also indicate that with a 5V input, the prototype is able to deliver 1-watt power at an average 20.5% efficiency without electromagnetic shielding. It was concluded that the proposed 3D coils designs are feasible and promising for WPT.@en

In recent years, mass-customization and on-demand production have spread to larger ground. To accommodate these developments, manufacturing systems are being transformed to allow more flexibility and agility. One of the technologies that allow flexibility and agility is Collaborative Robots. The design and implementation of Intelligent Manufacturing Systems is a complex activity that requires bridging between disciplines. With the introduction of Collaborative Robots, new disciplines are added to this activity, that need to be linked to existing design methods and procedures. Currently, the lack of these links is a bottleneck for Small and Medium sized Enterprises. These have limited resources for implementation. In this article, we introduce a Human-Robot Coproduction Design Methodology, with the aim of raising the capacity of Intelligent Manufacturing System designers for reasoning on collaboration between humans and robots in manufacturing. The methodology comprises four procedural steps: analysis, modeling, simulation, and evaluation, with specific methods, tools and instruments. The methodology has been evaluated in a laboratory environment by performing a pilot study with designers. While the current implementation of the methodology and its instrumentation is limited, it has been shown that the methodology enables quick design iterations during the conceptual design phase of Human-Robot Coproduction, thanks to procedures that have been tailored for this novel form of organizing and structuring production processes in Intelligent Manufacturing Systems.

@en

Being able to link captured material characteristics and fabricable material appearance attributes is important for creating life-like reproductions. In this paper we propose a method for gloss calibration, and an approach for gloss gamut mapping, as part of an integrated approach for color, topography and gloss reproduction. For gloss calibration, gloss calibration targets were printed in the primary printing colors (CMYK and White), with uniform gloss in equal distant gloss levels. These targets were scanned using the proposed gloss scanner. To create the gloss gamut map, a monotonic curve was fitted to the mean gloss scan values at different gloss levels. Analysis of fitted curves indicated that the gloss mapping is independent of the diffuse colors. As a case study, the painting ‘Fruit Still Life’ by Cornelis de Heem was scanned, and the measured gloss was mapped to printable gloss levels using the relation described by the fitted curve. The printed result shows good correspondence to the painting’s appearance, with clearly distinguishable gloss features for the in-gamut glass values.@en

How does individualized production look like? This paper proposes a generic vision of a smart factory which is able to produce so-called ultra personalized products(UPPS) in a batch size of one. This means that each customer is able to configure his or her personal product perfectly adjusted to the needs on base of personal data like bodyscans. As we are convinced that there will always be humans involved in planning and conducting manufacturing processes, we want to highlight the human-to-human interaction and the computer supported collaborative work, which is required in order to set up such a production line.@en

This paper introduces a novel approach for sensing the bending deformation on soft robots by leveraging multicolor 3D printing. The measurement of deformation enables to complete the feedback loop of deformation control on soft actuators. The working principle of our approach is based on using compact color sensors to detect deformation that is visualized by the change of color ratios. Two novel designs are presented to generate color signals on 3D printed objects, which we call an external signal generator and an internal signal generator. Signal processing and calibration methods are developed to transform the raw RGB-data into a meaningful deformation metric. Our experimental tests taken on soft pneumatic actuators verify that color signals can be stably generated and captured to indicate the bending deformation. The results also demonstrate the usability of this sensing approach in deformation control.@en