This Review discusses the main results obtained in training end-to-end neural architectures for guidance and control of interplanetary transfers, planetary landings, and close-proximity operations, highlighting the successful learning of optimality principles by the underlying neural models. Spacecraft and drones aimed at exploring our solar system are designed to operate in conditions where the smart use of onboard resources is vital to the success or failure of the mission. Sensorimotor actions are thus often derived from high-level, quantifiable, optimality principles assigned to each task, using consolidated tools in optimal control theory. The planned actions are derived on the ground and transferred on board, where controllers have the task of tracking the uploaded guidance profile. Here, we review recent trends based on the use of end-to-end networks, called guidance and control networks (G&CNets), which allow spacecraft to depart from such an architecture and to embrace the onboard computation of optimal actions. In this way, the sensor information is transformed in real time into optimal plans, thus increasing mission autonomy and robustness. We then analyze drone racing as an ideal gym environment to test these architectures on real robotic platforms and thus increase confidence in their use in future space exploration missions. Drone racing not only shares with spacecraft missions both limited onboard computational capabilities and similar control structures induced from the optimality principle sought but also entails different levels of uncertainties and unmodeled effects and a very different dynamical timescale.

@en

Navigation is an essential capability for autonomous robots. In particular, visual navigation has been a major research topic in robotics because cameras are lightweight, power-efficient sensors that provide rich information on the environment. However, the main challenge of visual navigation is that it requires substantial computational power and memory for visual processing and storage of the results. As of yet, this has precluded its use on small, extremely resource-constrained robots such as lightweight drones. Inspired by the parsimony of natural intelligence, we propose an insect-inspired approach toward visual navigation that is specifically aimed at extremely resource-restricted robots. It is a route-following approach in which a robot's outbound trajectory is stored as a collection of highly compressed panoramic images together with their spatial relationships as measured with odometry. During the inbound journey, the robot uses a combination of odometry and visual homing to return to the stored locations, with visual homing preventing the buildup of odometric drift. A main advancement of the proposed strategy is that the number of stored compressed images is minimized by spacing them apart as far as the accuracy of odometry allows. To demonstrate the suitability for small systems, we implemented the strategy on a tiny 56-gram drone. The drone could successfully follow routes up to 100 meters with a trajectory representation that consumed less than 20 bytes per meter. The presented method forms a substantial step toward the autonomous visual navigation of tiny robots, facilitating their more widespread application.

@en

Developing optimal controllers for aggressive high-speed quadcopter flight poses significant challenges in robotics. Recent trends in the field involve utilizing neural network controllers trained through supervised or reinforcement learning. However, the sim-to-real transfer introduces a reality gap, requiring the use of robust inner loop controllers during real flights, which limits the network's control authority and flight performance. In this paper, we investigate for the first time, an end-to-end neural network controller, addressing the reality gap issue without being restricted by an inner-loop controller. The networks, referred to as G&CNets, are trained to learn an energy-optimal policy mapping the quadcopter's state to rpm commands using an optimal trajectory dataset. In hover-to-hover flights, we identified the unmodeled moments as a significant contributor to the reality gap. To mitigate this, we propose an adaptive control strategy that works by learning from optimal trajectories of a system affected by constant external pitch, roll and yaw moments. In real test flights, this model mismatch is estimated onboard and fed to the network to obtain the optimal rpm command. We demonstrate the effectiveness of our method by performing energy-optimal hover-to-hover flights with and without moment feedback. Finally, we compare the adaptive controller to a state-of-the-art differential-flatness-based controller in a consecutive waypoint flight and demonstrate the advantages of our method in terms of energy optimality and robustness.

@en

ROVIO is one of the state-of-the-art monocular visual inertial odometry algorithms. It uses an Iterative Extended Kalman Filter (IEKF) to align visual features and update the vehicle state simultaneously by including the feature locations in the state vector of the IEKF. This algorithm is single-core intensive, which allows the other cores to be used for other algorithms, such as object detection and path optimization. However, the computational cost of the algorithm grows rapidly with the maximum number of features to track. Each feature adds three new states (a 2D bearing vector and inverse depth), leading to bigger matrix multiplications that are computationally expensive. The main computational load of ROVIO is the iterative update step of the IEKF. In this work, we reduce the average computational cost of ROVIO by 40% on an NVIDIA Jetson TX2, without affecting the accuracy of the algorithm. This computational gain is mainly achieved by exploiting the sparse matrices in ROVIO. Furthermore, we reduce the computational peaks by pre-selecting new features based on their already calculated FAST score. The combination of both modifications allows us to run ROVIO on the computationally restricted Raspberry Pi Zero 2W.

@en

Aggressive time-optimal control of quadcopters poses a significant challenge in the field of robotics. The state-of-the-art approach leverages reinforcement learning (RL) to train optimal neural policies. However, a critical hurdle is the sim-to-real gap, often addressed by employing a robust inner loop controller-an abstraction that, in theory, constrains the optimality of the trained controller, necessitating margins to counter potential disturbances. In contrast, our novel approach introduces high-speed quadcopter control using end-to-end RL (E2E) that gives direct motor commands. To bridge the reality gap, we incorporate a learned residual model and an adaptive method that can compensate for modeling errors in thrust and moments. We compare our E2E approach against a state-of-the-art network that commands thrust and body rates to an INDI inner loop controller, both in simulated and real-world flight. E2E showcases a significant 1.39-second advantage in simulation and a 0.17-second edge in real-world testing, highlighting end-to-end reinforcement learning's potential. The performance drop observed from simulation to reality shows potential for further improvement, including refining strategies to address the reality gap or exploring offline reinforcement learning with real flight data.

@en

Event cameras have recently gained significant traction since they open up new avenues for low-latency and low-power solutions to complex computer vision problems. To unlock these solutions, it is necessary to develop algorithms that can leverage the unique nature of event data. However, the current state-of-the-art is still highly influenced by the frame-based literature, and usually fails to deliver on these promises. In this work, we take this into consideration and propose a novel self-supervised learning pipeline for the sequential estimation of event-based optical flow that allows for the scaling of the models to high inference frequencies. At its core, we have a continuously-running stateful neural model that is trained using a novel formulation of contrast maximization that makes it robust to nonlinearities and varying statistics in the input events. Results across multiple datasets confirm the effectiveness of our method, which establishes a new state of the art in terms of accuracy for approaches trained or optimized without ground truth.

@en

Neuromorphic processing promises high energy efficiency and rapid response rates, making it an ideal candidate for achieving autonomous flight of resource-constrained robots. It can be especially beneficial for complex neural networks as are used for high-level visual perception. However, fully neuromorphic solutions also need to tackle low-level control tasks. Remarkably, it is currently still challenging to replicate even basic low-level controllers such as proportional-integral-derivative (PID) controllers. Specifically, it is difficult to incorporate the integral and derivative parts. To address this problem, we propose a neuromorphic controller that incorporates proportional, integral, and derivative pathways during learning. Our approach includes a novel input threshold adaptation mechanism for the integral pathway. This Input-Weighted Threshold Adaptation (IWTA) introduces an additional weight per synaptic connection, which is used to adapt the threshold of the post-synaptic neuron. We tackle the derivative term by employing neurons with different time constants. We first analyze the performance and limits of the proposed mechanisms and then put our controller to the test by implementing it on a microcontroller connected to the open-source tiny Crazyflie quadrotor, replacing the innermost rate controller. We demonstrate the stability of our bio-inspired algorithm with flights in the presence of disturbances. The current work represents a substantial step towards controlling highly dynamic systems with neuromorphic algorithms, thus advancing neuromorphic processing and robotics. In addition, integration is an important part of any temporal task, so the proposed Input-Weighted Threshold Adaptation (IWTA) mechanism may have implications well beyond control tasks.

@en

Moving Horizon Estimation (MHE) offers multiple advantages over Kalman Filters when it comes to the localization of drones. However, due to the high computational cost, they can not be used on Micro Air Vehicles (MAVs) with limited computational power. We have previously shown, that with a few assumptions and simplifications, MHE can be made more efficient while retaining good localization performance. In this paper, we present two additional improvements: the introduction of dynamic step sizes to the gradient descent algorithm, which leads to a significant increase in robustness, and the use of switching variables for outlier rejection, which further reduces the computational load. Both improvements are implemented and assessed in simulation and experiments. Using dynamic step sizes makes it possible to reliably use the estimator on board of a real drone, and the use of Newton’s method specifically opens the option to add different types of measurements. The new outlier rejection method on the other hand is shown to reduce the computational load significantly while having no big impact on accuracy. @en

Obstacle avoidance is an essential topic in the field of autonomous drone research. When choosing an avoidance algorithm, many different options are available, each with their advantages and disadvantages. As there is currently no consensus on testing methods, it is quite challenging to compare the performance between algorithms. In this paper, we propose AvoidBench, a benchmarking suite which can evaluate the performance of vision-based obstacle avoidance algorithms by subjecting them to a series of tasks. Thanks to the high fidelity of multi-rotors dynamics from RotorS and virtual scenes of Unity3D, AvoidBench can realize realistic simulated flight experiments. Compared to current drone simulators, we propose and implement both performance and environment metrics to reveal the suitability of obstacle avoidance algorithms for environments of different complexity. To illustrate AvoidBench's usage, we compare three algorithms: Ego-planner, MBPlanner, and Agile-autonomy. The trends observed are validated with real-world obstacle avoidance experiments. Code is available at: https://github.com/tudelft/AvoidBench

@en

Obstacle avoidance is an important capability for flying robots. But for robots with limited resources, such as small drones this becomes particularly challenging. Bug algorithms have been proposed to solve path planning with only minimal resources. And stereo vision provides a rich description of the world for limited weight but typically has a limited Field of View (FoV) and is fixed to the drone frame to further reduce weight. Based on these, a computationally light 3D path-planning algorithm is proposed. The proposed algorithm is called Frustumbug and is based on the Wedgebug algorithm since this algorithm copes well with a limited FoV. Since Wedgebug only addresses 2D problems, the Local-ϵ-Tangent-Graph (LETG) is used to extend the path planning to 3D. Disparity images are obtained through an optimized stereo block matching algorithm. Frustumbug copes well with noisy range sensor data and includes 3D trajectories like reversing, climbing and descending maneuvers to avoid or escape local minima. The algorithm has been tested with 225 flights in two challenging simulated environments and achieved a success rate of 96%. Here, 3.6% did not reach the goal and 0.4% collided. Frustumbug has been implemented on a 20-gram stereo vision system and was tested in the real world on a MAV. This shows the potential for small drones to reach their targets fully autonomously based on very limited resources. @en

Attitude control is an essential flight capability. Whereas flying robots commonly rely on accelerometers¹ for estimating attitude, flying insects lack an unambiguous sense of gravity^2,3. Despite the established role of several sense organs in attitude stabilization^3–5, the dependence of flying insects on an internal gravity direction estimate remains unclear. Here we show how attitude can be extracted from optic flow when combined with a motion model that relates attitude to acceleration direction. Although there are conditions such as hover in which the attitude is unobservable, we prove that the ensuing control system is still stable, continuously moving into and out of these conditions. Flying robot experiments confirm that accommodating unobservability in this manner leads to stable, but slightly oscillatory, attitude control. Moreover, experiments with a bio-inspired flapping-wing robot show that residual, high-frequency attitude oscillations from flapping motion improve observability. The presented approach holds a promise for robotics, with accelerometer-less autopilots paving the road for insect-scale autonomous flying robots⁶. Finally, it forms a hypothesis on insect attitude estimation and control, with the potential to provide further insight into known biological phenomena^5,7,8 and to generate new predictions such as reduced head and body attitude variance at higher flight speeds⁹.

@en

Highly automated Unmanned Aerial Vehicles (UAVs) or "flying robots" are rapidly becoming an important asset to society. The last decade has seen the advent of an impressive number of new UAV types and applications. For many applications, the UAVs need to be safe, highly automated, and versatile. Safety is a prerequisite to allowing their use in society. While flight safety comprises many aspects, one important safety factor is the total system mass. The common thread through this research is therefore to minimize the system mass while maintaining mission capabilities to increase safety. Flight automation is required to reach many applications' full potential by addressing operational labor costs and scalability. But despite great advances in ground-based robotics, the weight and power constraints of flying robots still constitute important challenges. Last but not least, many applications also require versatile aircraft that combine the ability to hover and fly fast efficiently. Hover is required for precision take-off & landing in confined areas at a growing number of locations and for the close-up inspection of assets. Fast and efficient flight is needed to reach distant locations, perform large surveys, cope with high headwind conditions, or simply reach destinations quickly. Unfortunately, the requirements for hover and fast flight are conflicting, and this drives the search for solutions to ``combine hover with fast flight in mission-capable flying robots while cost-effectively minimizing their size and maximizing their safety.'' To investigate the minimal feasible mass of mission-capable robots, in this thesis, a novel 20 g tailed flapping-wing robot called DelFly Explorer is presented that can autonomously explore unknown unprepared rooms. It was equipped with a 4 g micro stereo-vision system which necessitated algorithms that were optimized for tiny microcontrollers with low memory. Combined with a navigation strategy that keeps the area in front of the robot free of obstacles, a 0.9 g autopilot, and DelFly's novel stable slow hovering flight regime, this led to the lightest flying indoor exploration robot that could navigate in unknown environments. But to combine passive dynamic longitudinal stability at slow hover and fast flight in tailed ornithopters, a shift in the center of gravity location was shown to be needed. Moreover, the aerodynamically stabilizing tail also causes sensitivity to turbulence. Therefore, by using four pairs of flapping wings, a new tail-less flapping-wing concept called Quad-thopter was created which can hover precisely and transition to fast forward flight. The cranked-rocker-based mechanism contains no expensive parts and by re-using the main propulsion motors for attitude control, powerful control moments can be created which are very important in disturbance rejection. This design represents one of the first tailless flapping wing designs that was sufficiently light and agile for performing real missions while featuring a mechanism simple enough to permit large-scale production. Versatility of flight is also an asset for outdoor flight. Theory predicts that the most efficient hover is achieved by using a single large rotor while the most efficient forward flight is performed by using high aspect ratio fixed wings to generate lift. The combination of both has led to a novel helicopter-with-wings concept called DelftaCopter. The control of this platform yields unique challenges such as the inertia of the large fixed-wing interferes with the dynamics of the helicopter rotor. A controller was derived, and the dynamics were identified in hover and forward flight. The real-world performance of this flying robot is presented by analyzing the results of its participation in the outback medical challenge, showing that large single-rotor-equipped fixed-wing aircraft combine powerful attitude control, efficient hover, and efficient forward flight. Since efficiency in forward flight is not sufficient to achieve a very long endurance in electrically powered flying robots, a novel platform was developed around a hydrogen pressure cylinder and a fuel cell. The concept focuses on versatility, minimal weight, good control, and redundancy. A 12-motor tail-sitter is presented that re-uses all its motors for attitude control, hover, and forward flight and uses the wing structure to carry the propulsion. A dual automotive CAN-bus control network and dual flight modes remove the most critical single points of failure. The platform is called the NederDrone and is shown to fly 3h38 in a test departing from a moving ship at sea in 5 Beaufort wind conditions. While reaching fast flight in large free blocks of air is mainly a challenge for the design of the airframe and its energy source, as soon as obstacles are introduced, new bottlenecks appear and the weight and power consumption of sensing and processing become driving design considerations. Increasing the flight speed of flying robots in obstacle-packed or GPS-denied environments highlights the need for very lightweight fast but intelligent systems, as the processing weight and power not only reduce the flight times but also reduce the maneuvering capabilities and the maximum speed. Therefore, in this work, an extreme example is studied in the context of autonomous drone racing. A computationally light Artificial Intelligence (AI) based monocular navigation system is presented for indoor flight through obstacles. It enabled the flying robot to fly at higher speeds than what was possible with state-of-the-art visual-inertial odometry solutions. Overall, aerospace platforms require extreme optimization as every gram kept in the air requires constant energy. The consequence is that different missions will require vastly different platforms, while traditionally a lot of flying robot applications are still performed by multicopters. This thesis contributes to the design of intelligent flying robots that can both hover and fly fast, by solving several fundamental problems in novel concepts optimized around the five key requirements of mass, agility, efficiency, range, and speed-near-obstacles. These concepts are expected to contribute to the improvement of the mission capabilities of minimal-size flying robots to address the needs of society.@en

ROVIO is one of the state-of-the-art mono visual inertial odometry algorithms. It uses an Iterative Extended Kalman Filter (IEKF) to align features and update the vehicle state simultaneously by including the feature locations in the state vector of the IEKF. This algorithm is single core intensive, which allows using the other cores for other algorithms, such as object detection and path optimization. However, the computational cost of the algorithm grows rapidly with the total number of features. Each feature adds three new states (a 2D bearing vector and inverse depth), leading to bigger matrix multiplications which are computationally expensive. The main computational load of ROVIO is the iterative part of the IEKF. In this work, we reduce the average computational cost of ROVIO by 40% on an NVIDIA Jetson TX2, without affecting the accuracy of the algorithm. This computational gain is mainly achieved by utilizing the sparse matrices in ROVIO. @en

Relative localization is an important ability for multiple robots to perform cooperative tasks in GPS-denied environments. This paper presents a novel autonomous positioning framework for monocular relative localization of multiple tiny flying robots. This approach does not require any groundtruth data from external systems or manual labeling. Instead, the proposed framework is able to label real-world images with 3D relative positions between robots based on another onboard relative estimation technology, using ultra-wideband (UWB). After training in this self-supervised manner, the proposed deep neural network (DNN) can predict relative positions of peer robots by purely using a monocular camera. This deep learning-based visual relative localization is scalable, distributed, and autonomous. We also built an open-source and lightweight simulation pipeline by using Blender for 3D rendering, which allows synthetic image generation of other robots, and generalized training of the neural network. The proposed localization framework is tested on two real-world Crazyflie2 quadrotors by running the DNN on the onboard AIdeck (a tiny AI chip and monocular camera). All results demonstrate the effectiveness of the self-supervised multi-robot localization method. Video: https://youtu.be/7arkaIblPps@en

Time-optimal model predictive control is important for achieving fast racing drones but is computationally intensive and thereby rarely used onboard small quadcopters with limited computational resources. In this work, we simplify the optimal control problem (OCP) of the position loop for several maneuvers by exploiting the fact that the solution resembles a so-called ‘bang-bang’ in the critical direction, where only the switching time needs to be found. The noncritical direction uses a ‘minimum effort’ approach. The control parameters are obtained through bisection search schemes on an analytical path prediction model. The approach is compared with a classical PID controller and theoretical time-optimal trajectories in simulations. We explain the effects of the OCP simplifications and introduce a method of mitigating one of these effects. Finally, we have implemented the ‘bang-bang’ controller as a model predictive controller (MPC) onboard a Parrot Bebop and performed indoor flights to compare the controller’s performance to a PID controller. We show that the light novel controller outperforms the PID controller in waypoint-to-waypoint flight while requiring only minimal knowledge of the quadcopter’s dynamics.

@en

Hybrid UAVs have gained a lot of interest for their combined vertical take-off & landing (VTOL) and efficient forward flight capabilities. But their control is facing challenges in over-actuation and conflicting requirements depending on the flight phase which can easily lead to actuator saturation. Incremental Non-linear Control Allocation (INCA) has been proposed to solve the platform’s control allocation problem in the case of saturation or over-actuation by minimizing a set of objective functions. This work demonstrates INCA on quadplanes, an in-plane combination between a quadrotor and a conventional fixed-wing, and proposes an extension to control the outer loop. The novel controller is called Extended INCA (XINCA) and adds the wing orientation as a force-generating actuator in the outerloop control optimization. This leads to a single controller for all flight phases that avoids placing the wing at negative angles of attack and minimizes the load on hover motors. XINCA has low dependence on accurate vehicle models and requires only several optimization parameters. Flight simulations and experimental flights are performed to demonstrate the performance.

@en

This paper discusses a low-cost, open-source and open-hardware design and performance evaluation of a low-speed, multi-fan wind system dedicated to micro air vehicle (MAV) testing. In addition, a set of experiments with a flapping wing MAV and rotorcraft is presented, demonstrating the capabilities of the system and the properties of these different types of drones in response to various types of wind. We performed two sets of experiments where a MAV is flying into the wake of the fan system, gathering data about states, battery voltage and current. Firstly, we focus on steady wind conditions with wind speeds ranging from 0.5 m S-1 to 3.4 m S-1. During the second set of experiments, we introduce wind gusts, by periodically modulating the wind speed from 1.3 m S−1 to 3.4 m S−1 with wind gust oscillations of 0.5 Hz, 0.25 Hz and 0.125 Hz. The “Flapper” flapping wing MAV requires much larger pitch angles to counter wind than the “CrazyFlie” quadrotor. This is due to the Flapper's larger wing surface. In forward flight, its wings do provide extra lift, considerably reducing the power consumption. In contrast, the CrazyFlie's power consumption stays more constant for different wind speeds. The experiments with the varying wind show a quicker gust response by the CrazyFlie compared with the Flapper drone, but both their responses could be further improved. We expect that the proposed wind gust system will provide a useful tool to the community to achieve such improvements.@en

Wireless ranging measurements have been proposed for enabling multiple Micro Air Vehicles (MAVs) to localize with respect to each other. However, the high-dimensional relative states are weakly observable due to the scalar distance measurement. Hence, the MAVs have degraded relative localization and control performance under unobservable conditions as can be deduced by the Lie derivatives. This paper presents a nonlinear model predictive control (NMPC) by maximizing the determinant of the observability matrix to generate optimal control inputs, which also satisfy constraints including multi-robot tasks, input limitation, and state bounds. Simulation results validate the localization and control efficacy of the proposed MPC method for range-based multi-MAV systems with weak observability, which has faster convergence time and more accurate localization compared to previously proposed random motions. A real-world experiment on two Crazyflies indicates the optimal states and control behaviours generated by the proposed NMPC.

@en