As one of the crucial sensors for environment sensing, frequency modulated continuous wave (FMCW) radars are widely used in modern vehicles for driving assistance/autonomous driving. However, the limited frequency bandwidth and the increasing number of equipped radar sensors would inevitably cause mutual interference, degrading target detection and producing safety hazards. In this paper, a deep learning-based interference mitigation (IM) approach is proposed for FMCW radars by using the dilated convolution for network construction and a designated contrast learning strategy for training. The dilated convolution enlarges the receptive field of the neural network, and the designated contrastive learning strategy enforces to distinguish better between interferences and desired signals. The results of numerical simulation and experimental data processing show that the dilated convolution-based IM network, compared to the traditional convolution-based ones, can achieve a higher Signal-to-Interference-plus-Noise-Ratio (SINR) and target detection rate. Moreover, the designated contrastive learning strategy enables a better and more stable IM performance without increasing the complexity of the network, which can facilitate faster signal processing.

@en

Autonomous driving systems rely on precise trajectory prediction for safe and efficient motion planning. Despite considerable efforts to enhance prediction accuracy, inherent uncertainties persist due to data noise and incomplete observations. Many strategies entail formalizing prediction outcomes into distributions and utilizing variance to represent uncertainty. However, our experimental investigation reveals that existing trajectory prediction models yield unreliable uncertainty estimates, necessitating additional customized calibration processes. On the other hand, directly applying current calibration techniques to prediction outputs may yield suboptimal results due to using a universal scaler for all predictions and neglecting informative data cues. In this paper, we propose Customized Calibration Temperature with Regularizer (CCTR), a generic framework that calibrates the output distribution. Specifically, CCTR 1) employs a calibration-based regularizer to align output variance with the discrepancy between prediction and ground truth and 2) generates a tailor-made temperature scaler for each prediction using a post-processing network guided by context and historical information. Extensive evaluation involving multiple prediction and planning methods demonstrates the superiority of CCTR over existing calibration algorithms and uncertainty-aware methods, with significant improvements of 11%-22% in calibration quality and 17%-46% in motion planning.

@en

Driver Behavior Modeling (DBM) aims to predict and model human driving behaviors, which is typically incorporated into the Advanced Driver Assistance System to enhance transportation safety and improve driving experience. Inverse reinforcement learning (IRL) is a prevailing DBM technique with the goal of modeling the driving policy by recovering an unknown internal reward function from human driver demonstrations. However, the latest IRL-based design is inefficient due to the laborious manual feature engineering processes. Besides, the reward function usually experiences increased prediction errors when deployed for unseen vehicles. In this paper, we propose a novel deep learning-based reward function for IRL-based DBM with efficient model personalization via machine unlearning. We evaluate our approach on a highway simulation constructed using the realistic human driving dataset NGSIM. We deploy our approach on both a server GPU and an embedded GPU. The evaluation results show that our approach achieves a higher prediction accuracy compared with the latest IRL-based DBM approach that uses a weighted sum of trajectory features as the reward function. Our model personalization method obtains the highest accuracy and lowest latency compared with the baselines.

@en

We present a deep learning-based approach called DipSAR for reconstructing millimeter-wave synthetic aperture radar (SAR) images from sparse samples. The primary challenge lies in the requirement of a large training dataset for deep learning schemes. To overcome this issue, we employ the deep image prior (DIP) technique, which eliminates the need for a large dataset and instead utilizes only the sparse sample itself. Our proposed DipSAR model recovers missing samples from sparse data and reconstructs the SAR image using a conventional method. In this study, we utilize an existing SAR dataset and create fourteen different patterns to generate additional sparse samples by removing certain data points. We then evaluate the performance of DipSAR in comparison to the conventional method. The results show that DipSAR outperforms the conventional method in terms of the intersection over union (IoU) score.

@en

Multi-modal fusion has shown initial promising results for object detection of autonomous driving perception. However, many existing fusion schemes do not consider the quality of each fusion input and may suffer from adverse conditions on one or more sensors. While predictive uncertainty has been applied to characterize single-modal object detection performance at run time, incorporating uncertainties into the multi-modal fusion still lacks effective solutions due primarily to the uncertainty's cross-modal incomparability and distinct sensitivities to various adverse conditions. To fill this gap, this paper proposes Uncertainty-Encoded Mixture-of-Experts (UMoE) that explicitly incorporates single-modal uncertainties into LiDAR-camera fusion. UMoE uses individual expert network to process each sensor's detection result together with encoded uncertainty. Then, the expert networks' outputs are analyzed by a gating network to determine the fusion weights. The proposed UMoE module can be integrated into any proposal fusion pipeline. Evaluation shows that UMoE achieves a maximum of 10.67%, 3.17%, and 5.40% performance gain compared with the state-of-the-art proposal-level multi-modal object detectors under extreme weather, adversarial, and blinding attack scenarios.

@en

Nowadays deep learning-based weather radar echo extrapolation methods have competently improved nowcasting quality. Current pure convolutional or convolutional recurrent neural network-based extrapolation pipelines inherently struggle in capturing both global and local spatiotemporal interactions simultaneously, thereby limiting nowcasting performances, e.g., they not only tend to underestimate heavy rainfalls' spatial coverage and intensity but also fail to precisely predict nonlinear motion patterns. Furthermore, the usually adopted pixel-wise objective functions lead to blurry predictions. To this end, we propose a novel motion-guided global-local aggregation Transformer network for effectively combining spatiotemporal cues at different time scales, thereby strengthening global-local spatiotemporal aggregation urgently required by the extrapolation task. First, we divide existing observations into both short- and long-term sequences to represent echo dynamics at different time scales. Then, to introduce reasonable motion guidance to Transformer, we customize an end-to-end module for jointly extracting motion representation of short- and long-term echo sequences (MRS, MRL), while estimating optical flow. Subsequently, based on Transformer architecture, MRS is used as queries to retrospect the most useful information from MRL for an effective aggregation of global long-term and local short-term cues. Finally, the fused feature is employed for future echo prediction. Additionally, for the blurry prediction problem, predictions from our model trained with an adversarial regularization achieve superior performances not only in nowcasting skill scores but also in precipitation details and image clarity over existing methods. Extensive experiments on two challenging radar echo datasets demonstrate the effectiveness of our proposed method.

@en

In this article, the interference mitigation (IM) problem is tackled as a regression problem. A prior-guided deep learning (DL)-based IM approach is proposed for frequency-modulated continuous-wave (FMCW) radars. Considering the complex-valued nature of radar signals, a complex-valued convolutional neural network, which is different from the conventional real-valued counterparts, is utilized as an architecture for implementation. Meanwhile, as the desired beat signals of FMCW radars and interferences exhibit different distributions in the time–frequency domain, this prior feature is exploited as a regularization term to avoid overfitting of the learned representation. The effectiveness and accuracy of our proposed complex-valued fully convolutional network (CV-FCN)-based IM approach are verified and analyzed through both simulated and measured radar signals. Compared with the real-valued counterparts, the CV-FCN shows a better IM performance with a potential of half memory reduction in low signal-to-interference-plus-noise ratio (SINR) scenarios. The average SINR of interfered signals has been improved from −9.13 to 10.46 dB. Moreover, the CV-FCN trained using only simulated data can be directly utilized for IM in various measured radar signals and shows a superior generalization capability. Furthermore, by incorporating the prior feature, the CV-FCN trained on only 1/8 of the full data achieves comparable performance as that on the full dataset in low SINR scenarios, and the training procedure converges faster.@en

Recently, frequency-modulated continuous-wave (FMCW) radar-based hand gesture recognition (HGR) using deep learning has achieved favorable performance. However, many existing methods use extracted features separately, i.e., using one of the range, Doppler, azimuth, or elevation angle information, or a combination of any two, to train convolutional neural networks (CNNs), which ignore the interrelation among the 5-D time-varying-range-Doppler-azimuth-elevation feature space. Although there have been methods using the 5-D information, their mining of the interrelation among the 5-D feature space is not sufficient, and there is still room for improvements. This article proposes a new processing scheme of HGR based on 5-D feature cubes that are jointly encoded by a 3-D fast Fourier transform (3-D-FFT)-based method. Then, a CNN is proposed by building two novel blocks, i.e., the spatiotemporal deformable convolution (STDC) block and the adaptive spatiotemporal context-aware convolution (ASTCAC) block. Concretely, STDC is designed to cope with hand gestures' large spatiotemporal geometric transformations in the 5-D feature space. Moreover, ASTCAC is designed for modeling long-distance global relationships, e.g., relationships between pixels of the feature at the upper left corner and lower right corner, and exploring the global spatiotemporal context, in order to enhance the target feature representation and suppress interference. Finally, our presented method is verified on a large radar dataset, including 19 760 sets of 16 common hand gestures, collected by 19 subjects. Our method obtains a recognition rate of 99.53% on the validation dataset and that of 97.22% on the test dataset, which is significantly better than state-of-the-art methods.

@en

High angular resolution is in high demand in automotive radar. To achieve a high azimuth resolution a large aperture antenna array is required. Although MIMO technique can be used to form larger virtual apertures, a large number of transmitter-receiver channels are needed, which is still technologically challenging and costly. To circumvent this problem, we propose a high-resolution Direction of Arrival (DoA) estimation by using multiple small radar sensors distributed on the fascia of the automobile. To exploit the diversity gain due to different target observation angles by different radars, a block Focal Under determined System Solver based approach is proposed to incoherently fuse the data from multiple small MIMO sensors. This method significantly improves the DoA estimation compared to single sensor, decreases probability of false alarm and increases probability of multiple target detection. Its performance is demonstrated through both numerical simulations and experimental results.@en

In this paper, constant false alarm rate (CFAR) detector-based approaches are proposed for interference mitigation of Frequency modulated continuous wave (FMCW) radars. The proposed methods exploit the fact that after dechirping and low-pass filtering operations the targets' beat signals of FMCW radars are composed of exponential sinusoidal components while interferences exhibit short chirp waves within a sweep. The spectra of interferences in the time-frequency (t-f) domain are detected by employing a 1-D CFAR detector along each frequency bin and then the detected map is dilated as a mask for interference suppression. The proposed approaches are applicable to the scenarios in the presence of multiple interferences. Compared to the existing methods, the proposed methods reduce the power loss of useful signals and are very computationally efficient. Their interference mitigation performances are demonstrated through both numerical simulations and experimental results.

@en

Elevation resolution is an important indicator in tomographic SAR imaging as it represents the ability to discriminate closed targets in elevation. In general, the elevation resolution is proportional to the length of the elevation aperture. However, as the elevation aperture increases, the geometric consistency of the image will undesirably deteriorate and hence fails the image coregistration approach required by the traditional super-resolution tomographic imaging. In this paper, a new super-resolution tomographic imaging method is proposed to overcome the inconsistency problem caused by the large elevation aperture. The core strategy is to get rid of two-dimensional image coregistration by applying a three-dimensional (3D) back projection like imaging manner: the 3D space is firstly divided into a 3D imaging grid, each of which is individually imaged via compressive sensing for super-resolution. The effectiveness of the proposed approach is evaluated by both computer simulations and real P-band UAV SAR data.

@en

In this paper, the interference mitigation for Frequency Modulated Continuous Wave (FMCW) radar system with a dechirping receiver is investigated. After dechirping operation, the scattered signals from targets result in beat signals, i.e., the sum of complex exponentials while the interferences lead to chirp-like short pulses. Taking advantage of these different time and frequency features between the useful signals and the interferences, the interference mitigation is formulated as an optimization problem: a sparse and low-rank decomposition of a Hankel matrix constructed by lifting the measurements. Then, an iterative optimization algorithm is proposed to tackle it by exploiting the Alternating Direction of Multipliers (ADMM) scheme. Compared to the existing methods, the proposed approach does not need to detect the interference and also improves the estimation accuracy of the separated useful signals. Both numerical simulations with point-like targets and experiment results with distributed targets (i.e., raindrops) are presented to demonstrate and verify its performance. The results show that the proposed approach is generally applicable for interference mitigation in both stationary and moving target scenarios.

@en

A novel matrix-pencil (MP)-based interference mitigation approach for frequency-modulated continuous-wave (FMCW) radars is proposed in this article. The interference-contaminated segment of the beat signal is first cut out, and then, the signal samples in the cutout region are reconstructed by modeling the beat signal as a sum of complex exponentials and using the MP method to estimate their parameters. The efficiency of the proposed approach for the interference with different parameters (i.e., interference duration, signal-to-noise ratio (SNR), and different target scenarios) is investigated by means of numerical simulations. The proposed interference mitigation approach is intensively verified on experimental data. Comparisons of the proposed approach with the zeroing and other beat-frequency interpolation techniques are presented. The results indicate the broad applicability and superiority of the proposed approach, especially in low SNR and long interference duration situations.

@en

Radio frequency interference, which makes it difficult to produce high-quality radar spectrograms, is a major issue for micro-Doppler-based human activity recognition (HAR). In this paper, we propose a deep-learning-based method to detect and cut out the interference in spectrograms. Then, we restore the spectrograms in the cut-out region. First, a fully convolutional neural network (FCN) is employed to detect and remove the interference. Then, a coarse-to-fine generative adversarial network (GAN) is proposed to restore the part of the spectrogram that is affected by the interferences. The simulated motion capture (MOCAP) spectrograms and the measured radar spectrograms with interference are used to verify the proposed method. Experimental results from both qualitative and quantitative perspectives show that the proposed method can mitigate the interference and restore high-quality radar spectrograms. Furthermore, the comparison experiments also demonstrate the efficiency of the proposed approach.

@en

3-D imaging with irregular planar multiple-input-multiple-output (MIMO) arrays is discussed. Due to signal acquisition on irregular spatial sampling grids by using these antenna arrays, the fast Fourier transform (FFT)-based imaging algorithms cannot readily be used for image formation. To avoid the application of computationally intensive coherent summation algorithms such as filtered backprojection or Kirchhoff migration, we propose a nonuniform FFT (NUFFT)-based MIMO Range Migration Algorithm (i.e., NUFFT-based MIMO-RMA) for efficient microwave imaging. The algorithm exploits NUFFT to reconstruct the wavenumber-domain spectra related to each Fourier frequency. It is generic and applicable to 3-D imaging with irregular planar MIMO arrays. The effects of irregular spatial sampling and signal bandwidth on the imaging performance and computational efficiency of the proposed algorithm are analyzed. Finally, some numerical simulations and experimental results are presented to demonstrate the performance of the proposed imaging algorithm.

@en

In this paper, we propose an Elevation-Radial scanned Synthetic Aperture Radar (E-RadSAR) for forward-looking ground penetrating radar (GPR) imaging. The E-RadSAR exploits the advantages of both RadSAR and Elevation-Circular SAR (E-CSAR) by utilizing the SAR technique in the cross- and down-range directions for signal acquisition. It could be implemented with fewer antennas compared to the RadSAR but provides higher spatial resolutions than that of E-CSAR. These features make it very attractive for space-and/or cost-constrained imaging applications, for instance, the GPR systems used for tunnel boring machines (TBM). However, the E-RadSAR synthesizes a three-dimensional (3-D) array by taking measurements in a volume, which makes the traditional sampling criterion no longer applicable for its sampling strategy design. To tackle 3-D (synthetic) array sampling/design, we formulate it as a sensor selection problem and suggest an efficient selection algorithm, i.e., modified clustered FrameSense (modified CFS). Then it is used for 3-D array sampling design. The imaging performances of the resultant near-optimal 3-D arrays are demonstrated through numerical simulations.

@en

In this paper, a Nonuniform fast Fourier transform (NUFFT)-based range migration algorithm (RMA) is proposed for three-dimensional (3-D) imaging with irregular planar arrays. The proposed imaging algorithm takes advantage of NUFFT to reconstruct the signal spectra in the wavenumber domain, which overcomes the constraint of irregular spatial sampling on the utilization of Fourier-based imaging algorithms. Compared to the conventional coherent summation algorithms, i.e., backprojection and Kirchhoff migration, it is a very efficient method for 3-D image formation with nonuniform spatial sampling. To demonstrate its imaging performance, some numerical simulations are presented and discussed.@en

In this communication, a novel approach to rotated antenna array topology design is proposed for fully polarimetric short-range imaging. The rotated antenna array proposed acquires two co-pol and one cross-pol signals in terms of the “local” polarization basis by means of three antenna pairs. Then, the fully polarimetric signals in a global polarization basis are retrieved via simple polarization basis transformation, which overcomes the spatially varied polarizations of the signals acquired at different positions and makes valid the assumption of traditional imaging algorithms that the polarization is constant within the aperture. The proposed rotated antenna array takes advantage of the synthetic aperture technique to synthesize a 2-D array for 3-D full-pol imaging. It utilizes a significantly smaller number of antennas in comparison to traditional fully polarimetric imaging arrays and also provides sufficiently accurate estimation to the full-pol electromagnetic signals scattered from targets. Both numerical simulations and experimental measurements have been performed and the results show the tolerance of the antenna array topology proposed to the quasi-monostatic measurements, its effectiveness, and accuracy for full-pol short-range microwave imaging.@en

In this paper, sampling design of three-dimensional (3-D) synthetic array (i.e., synthetic volume array) for microwave imaging is considered. Generally, the spatial sampling criteria for one- or two-dimensional arrays can be determined based on some narrowband/ultrawideband array theories. However, for 3-D arrays, where antennas are located in a volume instead of over a surface, these existing array theories are no longer straightforwardly applicable. To address the spatial sampling problem of 3-D arrays, we formulate it as a sensor/observation selection problem in this paper. Although some selection approaches exist and are conveniently applicable to small-scale problems, they are either less efficient or provide less optimal results for selection problems with data dimensions of hundreds or even thousands which is typical for microwave imaging. To get the (near-) optimal spatial sampling scheme for 3-D arrays, a greedy algorithm named clustered maximal projection on minimal eigenspace (CMPME) is proposed to select the most informative sampling positions based on some optimality criteria. This algorithm attempts to select the fewest sampling positions by considering an error threshold for the estimated images. Moreover, it has higher computational efficiency compared to the existing approaches. Finally, its effectiveness and selection performances are demonstrated through some imaging examples.@en

A matrix-pencil based wavenumber-domain multiband fusion approach is proposed for improved microwave imaging. The suggested fusion approach is based on the fact from the Born approximation that the wavenumber-domain signal in a given scattering direction can be represented as a sum of the same number of contributions over the whole bandwidth. Utilizing this fact, the wavenumber-domain signal in each radial direction can be modeled as a superposition of damped/undamped exponential functions, where measured multiband data are part of the observations of the whole-band signal. Then the multiband signal fusion is addressed by estimating a unified signal model over the while bandwidth with the matrix pencil approach in the wavenumber domain. By using the estimated signal model, the missing data in the frequency gap can be extrapolated, thus synthesizing an equivalent wideband signal spectrum. After an inverse Fourier transform, the synthesize spectrum leads to a focused image with enhanced resolution. Compared to the existing frequency-domain method, the proposed fusion approach can be used for radar imaging with the signals acquired by either collocated or non-collocated arrays in different frequency bands. The effectiveness and accuracy of the proposed fusion approach are demonstrated through some numerical simulations.@en