We propose a new reinforcement learning method in the framework of Recursive Least Squares-Temporal Difference (RLS-TD). Instead of using the standard mechanism of eligibility traces (resulting in RLS-TD((Formula presented.))), we propose to use the forgetting factor commonly used in gradient-based or least-square estimation, and we show that it has a similar role as eligibility traces. An instrumental variable perspective is adopted to formulate the new algorithm, referred to as RLS-TD with forgetting factor (RLS-TD-f). An interesting aspect of the proposed algorithm is that it has an interpretation of a minimizer of an appropriate cost function. We test the effectiveness of the algorithm in a Policy Iteration setting, meaning that we aim to improve the performance of an initially stabilizing control policy (over large portion of the state space). We take a cart-pole benchmark and an adaptive cruise control benchmark as experimental platforms.

@en

The operation of Unmanned Aerial Vehicles (UAVs) is often subject to state-dependent alterations and unstructured uncertainty factors, such as unmodelled dynamics, environmental weather disturbances, aerodynamics gradients, or changes in inertia and mass due to payloads. While a large number of autopilot solutions have been proposed to operate UAVs, none of these solutions is able to counteract the effects of state-dependent and unstructured uncertainties online by parameter estimation and adaptive control techniques. This work presents a systematic integration of adaptive control into ArduPilot, a popular open-source autopilot suite maintained by a large community of UAV developers. Adaptation features are embedded in the ArduPilot control structure without altering the original architecture, to allow users to use the autopilot suite as usual. Tests show that the proposed adaptive ArduPilot provides consistent improved performance in several uncertain flight conditions. The source code of the proposed adaptive ArduPilot is released at https://github.com/Friend-Peng/Adaptive-ArduPilot-Autopilot.@en

In recent years, platooning solutions like cooperative adaptive cruise control (CACC) have been deeply studied. It is common in such platooning literature to assume that the vehicles drive on the same lane (longitudinal platooning). At the same time, lateral control during merging maneuvers is commonly addressed as a path planning problem, in which the ego vehicle changes the lane during merging without necessarily cooperating with its neighboring vehicles (i.e. without considering gap closing). The primary objective of this article is to develop a control strategy which involves both longitudinal and lateral vehicle dynamics, where the vehicles merge and form a platoon in a cooperative way without a priori path planning. Appropriately designed bi-dimensional artificial potential fields are used to achieve this goal and the proposed protocol is verified through simulations with CarSim.

@en

This article presents an adaptive method for ArduPilot-based autopilots of fixed-wing unmanned aerial vehicles (UAVs). ArduPilot is a popular open-source unmanned vehicle software suite. We explore how to augment the PID loops embedded inside ArduPilot with a model-free adaptive control method. The adaptive augmentation, adopted for both attitude and total energy control, uses input/output data without requiring an explicit model of the UAV. The augmented architecture is tested in a software-in-The-loop UAV platform in the presence of several uncertainties (unmodeled low-level dynamics, different payloads, time-varying wind, and changing mass). The performance is measured in terms of tracking errors and control efforts of the attitude and total energy control loops. Extensive experiments with the original ArduPilot, the proposed augmentation, and alternative autopilot strategies show that the augmentation can significantly improve the performance for all payloads and wind conditions: The UAV is less affected by wind and exhibits more than 70% improved tracking, with more than 7% reduced control effort.

@en

Despite the progress in the field of longitudinal formations of automated vehicles, only recently an interpretation of longitudinal platooning has been given in the framework of disturbance decoupling, i.e. the problem of making a controlled output independent of a disturbance. The appealing feature of this interpretation is that the disturbance decoupling approach naturally yields a decentralized controller that guarantees stability and string stability. In this work, we further exploit the disturbance decoupling framework and we show that convergence to a stable, string stable and disturbance decoupled behavior can be achieved even in the presence of parametric uncertainty of the engine time constant. We refer to this framework as adaptive disturbance decoupling.

@en

Effective design of autopilots for fixed-wing unmanned aerial vehicles (UAVs) is still a great challenge, due to unmodeled effects and uncertainties that these vehicles exhibit during flight. Unmodeled effects and uncertainties comprise longitudinal/lateral cross-couplings, as well as poor knowledge of equilibrium points (trimming points) of the UAV dynamics. The main contribution of this article is a new adaptive autopilot design, based on uncertain Euler-Lagrange dynamics of the UAV and where the control can explicitly take into account under-actuation in the dynamics, reduced structural knowledge of cross-couplings and trimming points. This system uncertainty is handled via appropriately designed adaptive laws: stability of the controlled UAV is analyzed. Hardware-in-the-loop tests, comparisons with an Ardupilot autopilot and with a robustified autopilot validate the effectiveness of the control design, even in the presence of strong saturation of the UAV actuators.

@en

Adaptive integral sliding mode control (AISMC) is an extension of adaptive sliding mode control which is a way to ensure sliding motion while handling system uncertainties. However, conventional AISMC formulations require to different extent a priori knowledge of the system uncertainty: either the upper bound of the uncertainty or of its time derivative are assumed to be bounded a priori, or the uncertainty is assumed to be parametrized by some structure-dependent factorization. This work proposes a variant of AISMC with reduced a priori knowledge of the system uncertainty: it is shown that Euler-Lagrange dynamics typical of sliding mode literature admit a structure-independent parametrization of the system uncertainty. This parametrization is not the result of structural knowledge, but it comes from basic properties of Euler-Lagrange dynamics, valid independently on the structure of the system. The AISMC control method arising from this parametrization is analyzed in the Lyapunov stability framework, and validated in systems with different structures: a surface vessel and an aerial vehicle.

@en

The broad learning system (BLS) paradigm has recently emerged as a computationally efficient approach to supervised learning. Its efficiency arises from a learning mechanism based on the method of least-squares. However, the need for storing and inverting large matrices can put the efficiency of such mechanism at risk in big-data scenarios. In this work, we propose a new implementation of BLS in which the need for storing and inverting large matrices is avoided. The distinguishing features of the designed learning mechanism are as follows: 1) the training process can balance between efficient usage of memory and required iterations (hybrid recursive learning) and 2) retraining is avoided when the network is expanded (incremental learning). It is shown that, while the proposed framework is equivalent to the standard BLS in terms of trained network weights,much larger networks than the standard BLS can be smoothly trained by the proposed solution, projecting BLS toward the big-data frontier.

@en

Recently proposed adaptive platooning strategies for connected automated vehicles are able to cope with uncertain vehicle parameters (uncertain driveline time constants), but can handle only acyclic graphs like look-ahead graphs. This prevents from enhancing platooning protocols with synchronized merging maneuvers, where cyclic communication is needed and creates algebraic loops that require well posedness of the inputs. We propose an adaptive platooning strategy for synchronized merging in the cyclic communication scenario. The protocol adopts a set of adaptive control laws, designed via Lyapunov stability theory to cope with uncertain driveline time constants. Well-posedness of the inputs is proven in a distributed way (using information from neighboring vehicles) in spite of uncertainty and cyclic communication. The proposed strategy is shown in a benchmark merging scenario.

@en

While several synchronization-based protocols have been provided for formation-keeping of cooperative vehicles, the problem of synchronized merging is more challenging. Challenges associated to the merging scenario include the need for establishing bidirectional interaction (in place of unidirectional look-ahead interaction), and the need for considering different engine dynamics (in place of homogeneous engine dynamics). This work shows how such challenges can be tackled via a newly proposed strategy based on adaptive control with bidirectional error: the adaptive control framework autonomously adapts to different engine dynamics, while the bidirectional error seamlessly allows the vehicle that wants to merge to interact with both the front and the rear vehicles, in a similar way as humans do.

@en

In adaptive platooning strategies proposed in literature to handle uncertain and nonidentical uncertain vehicle dynamics (uncertain heterogeneous platoons) two aspects requiring proper design are neglected: bidirectional interaction among vehicles which might lead to loss of string stability, and engine saturation constraints which might lead to loss of cohesiveness. This work proposes a novel adaptive platooning strategy handling these two crucial aspects. Specifically, bidirectional interaction is handled by designing bidirectional reference dynamics with proven string stability properties, to which the uncertain heterogeneous platoon should homogenize; engine constraints are handled via a proposed a mechanism that makes such reference dynamics 'not too demanding', by properly saturating their action. The saturation action will allow all vehicles in the platoon to not hit their engine limits, preserving cohesiveness. Simulations are conducted to validate the theoretical analysis and show the effectiveness of the method in retaining cohesiveness of the platoon.

@en