We tackle the anomaly detection problem within a given set of binary processes through a learning-based controlled sensing approach. This problem is particularly pertinent to applications related to the Internet of Things (IoT) that monitor multiple related processes. Each process is defined by a binary random variable indicating its anomalous status. To pinpoint anomalies, a decision-making agent observes a subset of processes at each time point, with each observation incurring an associated cost. We formulate a sequential selection policy that dynamically determines which processes to observe at each moment, aiming to minimize both decision delay and sensing cost. The conventional solution using model-based active hypothesis testing algorithms overlooks the joint statistics of the processes and fails to consider unequal sensing costs or errors in observations. To solve this problem, we, for the first time, pose it as a sequential hypothesis testing problem within the framework of Markov decision processes (MDPs), leveraging both a Bayesian log-likelihood ratio-based reward and an entropy-based reward. We address this problem through two approaches: 1) a deep reinforcement learning-based method involving the design of both deep Q-learning (DQN) and policy gradient actor-critic (AC) algorithms and 2) a deep active inference (AI)-based approach. Our model-based posterior updates to tackle the uncertainties in the observations, combined with the data-driven neural networks to handle the underlying statistical dependence between the processes, strike a balance between the model-based and the data-driven approaches. Our numerical experiments showcase the effectiveness of our algorithms, illustrating their capability to adapt to any unknown statistical dependence among the processes.

@en

We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes a subset of the processes at any given time instant and obtains a noisy binary indicator of whether or not the corresponding process is anomalous. We develop an anomaly detection algorithm that chooses the processes to be observed at a given time instant, decides when to stop taking observations, and declares the decision on anomalous processes. The objective of the detection algorithm is to identify the anomalies with an accuracy exceeding the desired value while minimizing the delay in decision making. We devise a centralized algorithm where the processes are jointly selected by a common agent as well as a decentralized algorithm where the decision of whether to select a process is made independently for each process. Our algorithms rely on a Markov decision process defined using the marginal probability of each process being normal or anomalous, conditioned on the observations. We implement the detection algorithms using the deep actor-critic reinforcement learning framework. Unlike prior work on this topic that has exponential complexity in the number of processes, our algorithms have computational and memory requirements that are both polynomial in the number of processes. We demonstrate the efficacy of these algorithms using numerical experiments by comparing them with state-of-the-art methods.@en

In this paper, we consider the problem of estimating the states of a linear dynamical system whose inputs are jointly sparse and outputs at a few unknown time instants are missing. We model the missing data mechanism using a Markov chain with two states representing the missing and non-missing data. This mechanism with memory governed by the Markov chain models intermittent outages due to communication channels and occlusions corresponding to moving objects. We rely on the sparse Bayesian learning framework to derive an estimation algorithm that uses Kalman smoothing to handle temporal correlation and the Viterbi algorithm to handle missing data. Further, we demonstrate the utility of our algorithm by applying it to the frequency division duplexed multiple input multiple output downlink channel estimation problem.

@en

In this paper, we consider the problem of sensor selection for parameter estimation with correlated measurement noise. We seek optimal sensor activations by formulating an optimization problem, in which the estimation error, given by the trace of the inverse of the Bayesian Fisher information matrix, is minimized subject to energy constraints. Fisher information has been widely used as an effective sensor selection criterion. However, existing information-based sensor selection methods are limited to the case of uncorrelated noise or weakly correlated noise due to the use of approximate metrics. By contrast, here we derive the closed form of the Fisher information matrix with respect to sensor selection variables that is valid for any arbitrary noise correlation regime and develop both a convex relaxation approach and a greedy algorithm to find near-optimal solutions. We further extend our framework of sensor selection to solve the problem of sensor scheduling, where a greedy algorithm is proposed to determine non-myopic (multi-time step ahead) sensor schedules. Lastly, numerical results are provided to illustrate the effectiveness of our approach, and to reveal the effect of noise correlation on estimation performance.@en