Actor-critic (AC) cooperative multiagent reinforcement learning (MARL) over directed graphs is studied in this article. The goal of the agents in MARL is to maximize the globally averaged return in a distributed way, i.e., each agent can only exchange information with its neighboring agents. AC methods proposed in the literature require the communication graphs to be undirected and the weight matrices to be doubly stochastic (more precisely, the weight matrices are row stochastic and their expectation are column stochastic). Differently from these methods, we propose a distributed AC algorithm for MARL over directed graph with fixed topology that only requires the weight matrix to be row stochastic. Then, we also study the MARL over directed graphs (possibly not connected) with changing topologies, proposing a different distributed AC algorithm based on the push-sum protocol that only requires the weight matrices to be column stochastic. Convergence of the proposed algorithms is proven for linear function approximation of the action value function. Simulations are presented to demonstrate the effectiveness of the proposed algorithms.

In recent years, platooning solutions like cooperative adaptive cruise control (CACC) have been deeply studied. It is common in such platooning literature to assume that the vehicles drive on the same lane (longitudinal platooning). At the same time, lateral control during merging maneuvers is commonly addressed as a path planning problem, in which the ego vehicle changes the lane during merging without necessarily cooperating with its neighboring vehicles (i.e. without considering gap closing). The primary objective of this article is to develop a control strategy which involves both longitudinal and lateral vehicle dynamics, where the vehicles merge and form a platoon in a cooperative way without a priori path planning. Appropriately designed bi-dimensional artificial potential fields are used to achieve this goal and the proposed protocol is verified through simulations with CarSim.

The broad learning system (BLS) paradigm has recently emerged as a computationally efficient approach to supervised learning. Its efficiency arises from a learning mechanism based on the method of least-squares. However, the need for storing and inverting large matrices can put the efficiency of such mechanism at risk in big-data scenarios. In this work, we propose a new implementation of BLS in which the need for storing and inverting large matrices is avoided. The distinguishing features of the designed learning mechanism are as follows: 1) the training process can balance between efficient usage of memory and required iterations (hybrid recursive learning) and 2) retraining is avoided when the network is expanded (incremental learning). It is shown that, while the proposed framework is equivalent to the standard BLS in terms of trained network weights,much larger networks than the standard BLS can be smoothly trained by the proposed solution, projecting BLS toward the big-data frontier.

This work investigates a reduced-complexity adaptive methodology to consensus tracking for a team of uncertain high-order nonlinear systems with switched (possibly asynchronous) dynamics. It is well known that high-order nonlinear systems are intrinsically challenging as feedback linearization and backstepping methods successfully developed for low-order systems fail to work. Even the adding-one-power-integrator methodology, well explored for the single-agent high-order case, presents some complexity issues and is unsuited for distributed control. At the core of the proposed distributed methodology is a newly proposed definition for separable functions: this definition allows the formulation of a separation-based lemma to handle the high-order terms with reduced complexity in the control design. Complexity is reduced in a twofold sense: the control gain of each virtual control law does not have to be incorporated in the next virtual control law iteratively, thus leading to a simpler expression of the control laws; the power of the virtual and actual control laws increases only proportionally (rather than exponentially) with the order of the systems, dramatically reducing high-gain issues.

This article investigates the distributed time-varying optimization problem for second-order multiagent systems (MASs) under limited interaction ranges. The goal is to seek the minimum of the sum of local time-varying cost functions (CFs), where each CF is only available to the corresponding agent. Limited communication range refers to the scenario where the agents have limited sensing and communication capabilities, that is, a pair of agents can communicate with each other only if their distance is within a certain range. To handle such a problem, a new continuous connectivity-preserving mechanism is presented to preserve the connectivity of the considered network. Then, two distributed optimization algorithms are presented to solve the optimization problem with time-varying CFs and time-invariant CFs, respectively. Theoretical analysis and two numerical examples are provided to verify the effectiveness of the methods.

Despite the progress in the field of longitudinal formations of automated vehicles, only recently an interpretation of longitudinal platooning has been given in the framework of disturbance decoupling, i.e. the problem of making a controlled output independent of a disturbance. The appealing feature of this interpretation is that the disturbance decoupling approach naturally yields a decentralized controller that guarantees stability and string stability. In this work, we further exploit the disturbance decoupling framework and we show that convergence to a stable, string stable and disturbance decoupled behavior can be achieved even in the presence of parametric uncertainty of the engine time constant. We refer to this framework as adaptive disturbance decoupling.

This paper proposes a novel set-invariance adaptive dynamic surface control (DSC) design for a larger class of uncertain large-scale nonlinear input-saturated systems. The peculiarity of this class is that no a priori bound on the continuous control gain functions is assumed (i.e., their boundedness cannot be assumed before obtaining system stability). This requires a new design. Differently from the available methods, the proposed design involves the construction of appropriate invariant sets for the closed-loop trajectories, which allows to remove the restrictive assumption of a priori bounds of the control gain functions. Furthermore, we show that such set-invariance design can handle input constraints in the form of input saturation. In line with the DSC methodology, semi-globally uniformly ultimate boundedness is proven: however, differently from the standard methodology, stability analysis requires the combination of Lyapunov and invariant set theories.

A challenging task in network synchronization is steering the network toward a coherent solution, when the dynamics of the constituent systems are heterogeneous and uncertain. In this situation, synchronization can be achieved via adaptive protocols (with adaptive feedback gains or adaptive coupling gains, or both). However, as state-of-the-art synchronization methods adopt a distributed observer architecture, they require to communicate extra observer variables among neighbors, in addition to the neighbors' states (or outputs). The distinguishing feature of this article is to show that for heterogeneous and uncertain networks of some classes of linear systems, synchronization is possible without the need for any distributed observer. Such classes are in line with those in model reference adaptive control literature. Lyapunov analysis is used to derive a new adaptive synchronization protocol with the simplest communication architecture, in which both feedback and coupling gains are adapted without any extra communication other than neighbors' states (in the full-state information case) or neighbors' outputs (in the partial-state information case).

In adaptive platooning strategies proposed in literature to handle uncertain and nonidentical uncertain vehicle dynamics (uncertain heterogeneous platoons) two aspects requiring proper design are neglected: bidirectional interaction among vehicles which might lead to loss of string stability, and engine saturation constraints which might lead to loss of cohesiveness. This work proposes a novel adaptive platooning strategy handling these two crucial aspects. Specifically, bidirectional interaction is handled by designing bidirectional reference dynamics with proven string stability properties, to which the uncertain heterogeneous platoon should homogenize; engine constraints are handled via a proposed a mechanism that makes such reference dynamics 'not too demanding', by properly saturating their action. The saturation action will allow all vehicles in the platoon to not hit their engine limits, preserving cohesiveness. Simulations are conducted to validate the theoretical analysis and show the effectiveness of the method in retaining cohesiveness of the platoon.

This article addresses and solves the adaptive asymptotic tracking for a class of uncertain switched positive linear dynamics (also known in the literature as compartmental models) subject to dwell-time constraints. Compared to the state-of-the-art, the innovative feature of this method is to attain for the first time asymptotic set-point tracking, while guaranteeing non-negativity of the systems states. To achieve asymptotic tracking, an interpolated Lyapunov function is adopted, which is nonincreasing at the switching instants and decreasing in two consecutive switching instants. Such Lyapunov function results in a novel adaptive law with time-varying adaptive gains, as opposed to state-of-the-art laws with fixed positive adaptive gains. The developed design is applicable to classes of compartmental systems compatible with those proposed in the literature: an example involving the infusion of anesthesia is conducted to show that the proposed method can achieve better performance than existing methods.

Recently proposed adaptive platooning strategies for connected automated vehicles are able to cope with uncertain vehicle parameters (uncertain driveline time constants), but can handle only acyclic graphs like look-ahead graphs. This prevents from enhancing platooning protocols with synchronized merging maneuvers, where cyclic communication is needed and creates algebraic loops that require well posedness of the inputs. We propose an adaptive platooning strategy for synchronized merging in the cyclic communication scenario. The protocol adopts a set of adaptive control laws, designed via Lyapunov stability theory to cope with uncertain driveline time constants. Well-posedness of the inputs is proven in a distributed way (using information from neighboring vehicles) in spite of uncertainty and cyclic communication. The proposed strategy is shown in a benchmark merging scenario.

This paper proposes a novel adaptive fuzzy dynamic surface control (DSC) method for an extended class of periodically disturbed strict-feedback nonlinear systems. The peculiarity of this extended class is that the control gain functions are not bounded a priori but simply taken to be continuous and with a known sign. In contrast with existing strategies, controllability must be guaranteed by constructing appropriate compact sets ensuring that all trajectories in the closed-loop system never leave these sets. We manage to do this by means of invariant set theory in combination with the Lyapunov theory. In other words, boundedness is achieved a posteriori as a result of stability analysis. The approximator composed of fuzzy logic systems and Fourier series expansion is constructed to deal with the unknown periodic disturbance terms.

In this article, the dynamic economic dispatch (DED) problem for smart grid is solved under the assumption that no knowledge of the mathematical formulation of the actual generation cost functions is available. The objective of the DED problem is to find the optimal power output of each unit at each time so as to minimize the total generation cost. To address the lack of a priori knowledge, a new distributed reinforcement learning optimization algorithm is proposed. The algorithm combines the state-action-value function approximation with a distributed optimization based on multiplier splitting. Theoretical analysis of the proposed algorithm is provided to prove the feasibility of the algorithm, and several case studies are presented to demonstrate its effectiveness.

This brief proposes a neuro-adaptive method for the unsolved problem of cooperative tracking rendezvous of nonholonomic mobile robots (NMRs) subject to uncertain and unmodelled dynamics. A hierarchical cooperative control framework is proposed, which consists of a novel distributed estimator along with local neuro-adaptive tracking controllers. Rigorous stability analysis as well as simulation experiments illustrate the proposed method.

The work presented in this paper concerns a switching-based control formulation for multi-intersection and multiphase traffic light systems. A macroscopic traffic flow modeling approach is first presented, which is instrumental to the development of a model-based and switching-based optimization method for traffic signal operation, in the framework of adaptive dynamic programming (ADP). The main advantage of the switching-based formulation is its capability to determine both 'when' to switch and 'which' mode to switch on without the need to use the cycle-based average flow approximation typical of state-of-the-art formulations. In addition, the framework can handle different cycle times across intersections without the need for synchronization constraints and, moreover, minimum dwell-time constraints can be directly enforced to comply with minimum green/red times in each phase. The simulation experiments on a multi-intersection and multiphase traffic light systems are presented to show the effectiveness of the method.

This work investigates the consensus tracking problem for high-power nonlinear multiagent systems with partially unknown control directions. The main challenge of considering such dynamics lies in the fact that their linearized dynamics contain uncontrollable modes, making the standard backstepping technique fail; also, the presence of mixed unknown control directions (some being known and some being unknown) requires a piecewise Nussbaum function that exploits the a priori knowledge of the known control directions. The piecewise Nussbaum function technique leaves some open problems, such as Can the technique handle multiagent dynamics beyond the standard backstepping procedure? and Can the technique handle more than one control direction for each agent? In this work, we propose a hybrid Nussbaum technique that can handle uncertain agents with high-power dynamics where the backstepping procedure fails, with nonsmooth behaviors (switching and quantization), and with multiple unknown control directions for each agent.

This work proposes a Nussbaum function-based adaptive control method for high-order nonlinear systems with mixed control directions (some being known, some being unknown) and dead-zone input. State-of-the-art techniques based on a piecewise Nussbaum function leave open the question if the technique handles high-order nonlinear dynamics. In this work, we address and solve such open problem in the framework of adding-one-power-integrator procedure, and using a variable-separable lemma to handle dead-zone nonlinearity. It is shown that all signals in the resulting closed-loop systems remain bounded.

In this work, we explore the use of a Broad Learning System (BLS) as a way to replace deep learning architectures for traffic flow prediction. BLS is shown to not only outperforms standard learning algorithms (Least absolute shrinkage and selection operator (LASSO), shallow and deep neural networks, stacked autoencoders) in terms of training time, but also in terms of testing accuracy.

This work proposes for the first time a model reference adaptive asymptotic tracking method for a class of uncertain switched linear systems with nonnegative orthant state constraints and dwell time switching. In contrast with the state-of-the-art switched adaptive methods, the main characteristic of this method consists in guaranteeing that the systems states remain in the nonnegative orthant of the state space. To achieve asymptotic tracking, an interpolated Lyapunov function is adopted. To ensure the positivity of the system states, a parameter projection law is needed, because some knowledge of the bounds of the parameter estimates is required to guarantee that the closed-loop system is a switched positive system. A benchmark traffic flow example (a switched positive linear system commonly used in literature) is adopted to verify the effectiveness of the proposed approach.

This paper establishes a novel adaptive hierarchical formation control method for uncertain heterogeneous nonlinear agents described by Euler–Lagrange (EL) dynamics. Formation control is framed as a synchronization problem where a distributed model reference adaptive control is used to synchronize the EL systems. The idea behind the proposed adaptive formation algorithm is that each agent must converge to the model defined by its hierarchically superior neighbors. Using a distributed inverse dynamics structure, we prove that distributed nonlinear matching conditions between connected agents hold, so that matching gains exist to make the entire formation converge to same homogeneous dynamics: to compensate for the presence of uncertainties, estimation laws are devised for such matching gains, leading to adaptive synchronization. An appropriately designed distributed Lyapunov function is used to derive asymptotic convergence of the synchronization error. The effectiveness of the proposed methodology is supported by simulations of a formation of Unmanned Aerial Vehicles (UAVs).