Autonomous driving systems rely on precise trajectory prediction for safe and efficient motion planning. Despite considerable efforts to enhance prediction accuracy, inherent uncertainties persist due to data noise and incomplete observations. Many strategies entail formalizing prediction outcomes into distributions and utilizing variance to represent uncertainty. However, our experimental investigation reveals that existing trajectory prediction models yield unreliable uncertainty estimates, necessitating additional customized calibration processes. On the other hand, directly applying current calibration techniques to prediction outputs may yield suboptimal results due to using a universal scaler for all predictions and neglecting informative data cues. In this paper, we propose Customized Calibration Temperature with Regularizer (CCTR), a generic framework that calibrates the output distribution. Specifically, CCTR 1) employs a calibration-based regularizer to align output variance with the discrepancy between prediction and ground truth and 2) generates a tailor-made temperature scaler for each prediction using a post-processing network guided by context and historical information. Extensive evaluation involving multiple prediction and planning methods demonstrates the superiority of CCTR over existing calibration algorithms and uncertainty-aware methods, with significant improvements of 11%-22% in calibration quality and 17%-46% in motion planning.

@en

As one of the crucial sensors for environment sensing, frequency modulated continuous wave (FMCW) radars are widely used in modern vehicles for driving assistance/autonomous driving. However, the limited frequency bandwidth and the increasing number of equipped radar sensors would inevitably cause mutual interference, degrading target detection and producing safety hazards. In this paper, a deep learning-based interference mitigation (IM) approach is proposed for FMCW radars by using the dilated convolution for network construction and a designated contrast learning strategy for training. The dilated convolution enlarges the receptive field of the neural network, and the designated contrastive learning strategy enforces to distinguish better between interferences and desired signals. The results of numerical simulation and experimental data processing show that the dilated convolution-based IM network, compared to the traditional convolution-based ones, can achieve a higher Signal-to-Interference-plus-Noise-Ratio (SINR) and target detection rate. Moreover, the designated contrastive learning strategy enables a better and more stable IM performance without increasing the complexity of the network, which can facilitate faster signal processing.

@en

Driver Behavior Modeling (DBM) aims to predict and model human driving behaviors, which is typically incorporated into the Advanced Driver Assistance System to enhance transportation safety and improve driving experience. Inverse reinforcement learning (IRL) is a prevailing DBM technique with the goal of modeling the driving policy by recovering an unknown internal reward function from human driver demonstrations. However, the latest IRL-based design is inefficient due to the laborious manual feature engineering processes. Besides, the reward function usually experiences increased prediction errors when deployed for unseen vehicles. In this paper, we propose a novel deep learning-based reward function for IRL-based DBM with efficient model personalization via machine unlearning. We evaluate our approach on a highway simulation constructed using the realistic human driving dataset NGSIM. We deploy our approach on both a server GPU and an embedded GPU. The evaluation results show that our approach achieves a higher prediction accuracy compared with the latest IRL-based DBM approach that uses a weighted sum of trajectory features as the reward function. Our model personalization method obtains the highest accuracy and lowest latency compared with the baselines.

@en

We present a deep learning-based approach called DipSAR for reconstructing millimeter-wave synthetic aperture radar (SAR) images from sparse samples. The primary challenge lies in the requirement of a large training dataset for deep learning schemes. To overcome this issue, we employ the deep image prior (DIP) technique, which eliminates the need for a large dataset and instead utilizes only the sparse sample itself. Our proposed DipSAR model recovers missing samples from sparse data and reconstructs the SAR image using a conventional method. In this study, we utilize an existing SAR dataset and create fourteen different patterns to generate additional sparse samples by removing certain data points. We then evaluate the performance of DipSAR in comparison to the conventional method. The results show that DipSAR outperforms the conventional method in terms of the intersection over union (IoU) score.

@en

Multi-modal fusion has shown initial promising results for object detection of autonomous driving perception. However, many existing fusion schemes do not consider the quality of each fusion input and may suffer from adverse conditions on one or more sensors. While predictive uncertainty has been applied to characterize single-modal object detection performance at run time, incorporating uncertainties into the multi-modal fusion still lacks effective solutions due primarily to the uncertainty's cross-modal incomparability and distinct sensitivities to various adverse conditions. To fill this gap, this paper proposes Uncertainty-Encoded Mixture-of-Experts (UMoE) that explicitly incorporates single-modal uncertainties into LiDAR-camera fusion. UMoE uses individual expert network to process each sensor's detection result together with encoded uncertainty. Then, the expert networks' outputs are analyzed by a gating network to determine the fusion weights. The proposed UMoE module can be integrated into any proposal fusion pipeline. Evaluation shows that UMoE achieves a maximum of 10.67%, 3.17%, and 5.40% performance gain compared with the state-of-the-art proposal-level multi-modal object detectors under extreme weather, adversarial, and blinding attack scenarios.

@en

In this paper, the interference mitigation for Frequency Modulated Continuous Wave (FMCW) radar system with a dechirping receiver is investigated. After dechirping operation, the scattered signals from targets result in beat signals, i.e., the sum of complex exponentials while the interferences lead to chirp-like short pulses. Taking advantage of these different time and frequency features between the useful signals and the interferences, the interference mitigation is formulated as an optimization problem: a sparse and low-rank decomposition of a Hankel matrix constructed by lifting the measurements. Then, an iterative optimization algorithm is proposed to tackle it by exploiting the Alternating Direction of Multipliers (ADMM) scheme. Compared to the existing methods, the proposed approach does not need to detect the interference and also improves the estimation accuracy of the separated useful signals. Both numerical simulations with point-like targets and experiment results with distributed targets (i.e., raindrops) are presented to demonstrate and verify its performance. The results show that the proposed approach is generally applicable for interference mitigation in both stationary and moving target scenarios.

@en

Nowadays deep learning-based weather radar echo extrapolation methods have competently improved nowcasting quality. Current pure convolutional or convolutional recurrent neural network-based extrapolation pipelines inherently struggle in capturing both global and local spatiotemporal interactions simultaneously, thereby limiting nowcasting performances, e.g., they not only tend to underestimate heavy rainfalls' spatial coverage and intensity but also fail to precisely predict nonlinear motion patterns. Furthermore, the usually adopted pixel-wise objective functions lead to blurry predictions. To this end, we propose a novel motion-guided global-local aggregation Transformer network for effectively combining spatiotemporal cues at different time scales, thereby strengthening global-local spatiotemporal aggregation urgently required by the extrapolation task. First, we divide existing observations into both short- and long-term sequences to represent echo dynamics at different time scales. Then, to introduce reasonable motion guidance to Transformer, we customize an end-to-end module for jointly extracting motion representation of short- and long-term echo sequences (MRS, MRL), while estimating optical flow. Subsequently, based on Transformer architecture, MRS is used as queries to retrospect the most useful information from MRL for an effective aggregation of global long-term and local short-term cues. Finally, the fused feature is employed for future echo prediction. Additionally, for the blurry prediction problem, predictions from our model trained with an adversarial regularization achieve superior performances not only in nowcasting skill scores but also in precipitation details and image clarity over existing methods. Extensive experiments on two challenging radar echo datasets demonstrate the effectiveness of our proposed method.

@en

Recently, frequency-modulated continuous-wave (FMCW) radar-based hand gesture recognition (HGR) using deep learning has achieved favorable performance. However, many existing methods use extracted features separately, i.e., using one of the range, Doppler, azimuth, or elevation angle information, or a combination of any two, to train convolutional neural networks (CNNs), which ignore the interrelation among the 5-D time-varying-range-Doppler-azimuth-elevation feature space. Although there have been methods using the 5-D information, their mining of the interrelation among the 5-D feature space is not sufficient, and there is still room for improvements. This article proposes a new processing scheme of HGR based on 5-D feature cubes that are jointly encoded by a 3-D fast Fourier transform (3-D-FFT)-based method. Then, a CNN is proposed by building two novel blocks, i.e., the spatiotemporal deformable convolution (STDC) block and the adaptive spatiotemporal context-aware convolution (ASTCAC) block. Concretely, STDC is designed to cope with hand gestures' large spatiotemporal geometric transformations in the 5-D feature space. Moreover, ASTCAC is designed for modeling long-distance global relationships, e.g., relationships between pixels of the feature at the upper left corner and lower right corner, and exploring the global spatiotemporal context, in order to enhance the target feature representation and suppress interference. Finally, our presented method is verified on a large radar dataset, including 19 760 sets of 16 common hand gestures, collected by 19 subjects. Our method obtains a recognition rate of 99.53% on the validation dataset and that of 97.22% on the test dataset, which is significantly better than state-of-the-art methods.

@en

In this article, the interference mitigation (IM) problem is tackled as a regression problem. A prior-guided deep learning (DL)-based IM approach is proposed for frequency-modulated continuous-wave (FMCW) radars. Considering the complex-valued nature of radar signals, a complex-valued convolutional neural network, which is different from the conventional real-valued counterparts, is utilized as an architecture for implementation. Meanwhile, as the desired beat signals of FMCW radars and interferences exhibit different distributions in the time–frequency domain, this prior feature is exploited as a regularization term to avoid overfitting of the learned representation. The effectiveness and accuracy of our proposed complex-valued fully convolutional network (CV-FCN)-based IM approach are verified and analyzed through both simulated and measured radar signals. Compared with the real-valued counterparts, the CV-FCN shows a better IM performance with a potential of half memory reduction in low signal-to-interference-plus-noise ratio (SINR) scenarios. The average SINR of interfered signals has been improved from −9.13 to 10.46 dB. Moreover, the CV-FCN trained using only simulated data can be directly utilized for IM in various measured radar signals and shows a superior generalization capability. Furthermore, by incorporating the prior feature, the CV-FCN trained on only 1/8 of the full data achieves comparable performance as that on the full dataset in low SINR scenarios, and the training procedure converges faster.@en

High angular resolution is in high demand in automotive radar. To achieve a high azimuth resolution a large aperture antenna array is required. Although MIMO technique can be used to form larger virtual apertures, a large number of transmitter-receiver channels are needed, which is still technologically challenging and costly. To circumvent this problem, we propose a high-resolution Direction of Arrival (DoA) estimation by using multiple small radar sensors distributed on the fascia of the automobile. To exploit the diversity gain due to different target observation angles by different radars, a block Focal Under determined System Solver based approach is proposed to incoherently fuse the data from multiple small MIMO sensors. This method significantly improves the DoA estimation compared to single sensor, decreases probability of false alarm and increases probability of multiple target detection. Its performance is demonstrated through both numerical simulations and experimental results.@en

Elevation resolution is an important indicator in tomographic SAR imaging as it represents the ability to discriminate closed targets in elevation. In general, the elevation resolution is proportional to the length of the elevation aperture. However, as the elevation aperture increases, the geometric consistency of the image will undesirably deteriorate and hence fails the image coregistration approach required by the traditional super-resolution tomographic imaging. In this paper, a new super-resolution tomographic imaging method is proposed to overcome the inconsistency problem caused by the large elevation aperture. The core strategy is to get rid of two-dimensional image coregistration by applying a three-dimensional (3D) back projection like imaging manner: the 3D space is firstly divided into a 3D imaging grid, each of which is individually imaged via compressive sensing for super-resolution. The effectiveness of the proposed approach is evaluated by both computer simulations and real P-band UAV SAR data.

@en

In this paper, constant false alarm rate (CFAR) detector-based approaches are proposed for interference mitigation of Frequency modulated continuous wave (FMCW) radars. The proposed methods exploit the fact that after dechirping and low-pass filtering operations the targets' beat signals of FMCW radars are composed of exponential sinusoidal components while interferences exhibit short chirp waves within a sweep. The spectra of interferences in the time-frequency (t-f) domain are detected by employing a 1-D CFAR detector along each frequency bin and then the detected map is dilated as a mask for interference suppression. The proposed approaches are applicable to the scenarios in the presence of multiple interferences. Compared to the existing methods, the proposed methods reduce the power loss of useful signals and are very computationally efficient. Their interference mitigation performances are demonstrated through both numerical simulations and experimental results.

@en

A novel matrix-pencil (MP)-based interference mitigation approach for frequency-modulated continuous-wave (FMCW) radars is proposed in this article. The interference-contaminated segment of the beat signal is first cut out, and then, the signal samples in the cutout region are reconstructed by modeling the beat signal as a sum of complex exponentials and using the MP method to estimate their parameters. The efficiency of the proposed approach for the interference with different parameters (i.e., interference duration, signal-to-noise ratio (SNR), and different target scenarios) is investigated by means of numerical simulations. The proposed interference mitigation approach is intensively verified on experimental data. Comparisons of the proposed approach with the zeroing and other beat-frequency interpolation techniques are presented. The results indicate the broad applicability and superiority of the proposed approach, especially in low SNR and long interference duration situations.

@en

3-D imaging with irregular planar multiple-input-multiple-output (MIMO) arrays is discussed. Due to signal acquisition on irregular spatial sampling grids by using these antenna arrays, the fast Fourier transform (FFT)-based imaging algorithms cannot readily be used for image formation. To avoid the application of computationally intensive coherent summation algorithms such as filtered backprojection or Kirchhoff migration, we propose a nonuniform FFT (NUFFT)-based MIMO Range Migration Algorithm (i.e., NUFFT-based MIMO-RMA) for efficient microwave imaging. The algorithm exploits NUFFT to reconstruct the wavenumber-domain spectra related to each Fourier frequency. It is generic and applicable to 3-D imaging with irregular planar MIMO arrays. The effects of irregular spatial sampling and signal bandwidth on the imaging performance and computational efficiency of the proposed algorithm are analyzed. Finally, some numerical simulations and experimental results are presented to demonstrate the performance of the proposed imaging algorithm.

@en

Radio frequency interference, which makes it difficult to produce high-quality radar spectrograms, is a major issue for micro-Doppler-based human activity recognition (HAR). In this paper, we propose a deep-learning-based method to detect and cut out the interference in spectrograms. Then, we restore the spectrograms in the cut-out region. First, a fully convolutional neural network (FCN) is employed to detect and remove the interference. Then, a coarse-to-fine generative adversarial network (GAN) is proposed to restore the part of the spectrogram that is affected by the interferences. The simulated motion capture (MOCAP) spectrograms and the measured radar spectrograms with interference are used to verify the proposed method. Experimental results from both qualitative and quantitative perspectives show that the proposed method can mitigate the interference and restore high-quality radar spectrograms. Furthermore, the comparison experiments also demonstrate the efficiency of the proposed approach.

@en

In this paper, we propose an Elevation-Radial scanned Synthetic Aperture Radar (E-RadSAR) for forward-looking ground penetrating radar (GPR) imaging. The E-RadSAR exploits the advantages of both RadSAR and Elevation-Circular SAR (E-CSAR) by utilizing the SAR technique in the cross- and down-range directions for signal acquisition. It could be implemented with fewer antennas compared to the RadSAR but provides higher spatial resolutions than that of E-CSAR. These features make it very attractive for space-and/or cost-constrained imaging applications, for instance, the GPR systems used for tunnel boring machines (TBM). However, the E-RadSAR synthesizes a three-dimensional (3-D) array by taking measurements in a volume, which makes the traditional sampling criterion no longer applicable for its sampling strategy design. To tackle 3-D (synthetic) array sampling/design, we formulate it as a sensor selection problem and suggest an efficient selection algorithm, i.e., modified clustered FrameSense (modified CFS). Then it is used for 3-D array sampling design. The imaging performances of the resultant near-optimal 3-D arrays are demonstrated through numerical simulations.

@en

In this paper, a Nonuniform fast Fourier transform (NUFFT)-based range migration algorithm (RMA) is proposed for three-dimensional (3-D) imaging with irregular planar arrays. The proposed imaging algorithm takes advantage of NUFFT to reconstruct the signal spectra in the wavenumber domain, which overcomes the constraint of irregular spatial sampling on the utilization of Fourier-based imaging algorithms. Compared to the conventional coherent summation algorithms, i.e., backprojection and Kirchhoff migration, it is a very efficient method for 3-D image formation with nonuniform spatial sampling. To demonstrate its imaging performance, some numerical simulations are presented and discussed.@en

In this paper, a wavenumber-domain matrix-pencil-based multiband signal fusion approach was proposed for multiband microwave imaging. The approach proposed is based on the Born approximation of the field scattered from a target resulting in the fact that in a given scattering direction, the scattered field can be represented over the whole frequency band as a sum of the same number of contributions. Exploiting the measured multiband data and taking advantage of the parametric modeling for the signals in a radial direction, a unified signal model can be estimated for a large bandwidth in the wavenumber domain. It can be used to fuse the signals at different subbands by extrapolating the missing data in the frequency gaps between them or coherently integrating the overlaps between the adjacent subbands, thus synthesizing an equivalent wideband signal spectrum. Taking an inverse Fourier transform, the synthesized spectrum results in a focused image with improved resolution. Compared with the space-time domain fusion methods, the proposed approach is applicable for radar imaging with the signals collected by either collocated or noncollocated arrays in different frequency bands. Its effectiveness and accuracy are demonstrated through both numerical simulations and experimental imaging results.

@en

Radar imaging using ultra-wideband (UWB) automotive radar is investigated. To overcome poor cross-range resolution of traditional beamforming (BF) algorithms, super resolution technique is applied for the Direction of Arrival (DOA) estimation. Both super-resolution algorithm and conventional BF method are used to process experimental data collected with UWB frequency modulated continuous wave (FMCW) automo- tive radar. Algorithm performances are thoroughly compared.

@en

Ground penetrating radar imaging from the data acquired with arbitrarily oriented dipole-like antennas is considered. To take into account variations of antenna orientations resulting in spatial rotation of antenna radiation patterns and polarizations of transmitted fields, the full-wave method that accounts for the near-, intermediate-, and far-field contributions to the radiation patterns is applied for image reconstruction, which is formulated as a linear inversion problem. Two approaches, namely, an interpolation-based method and a nonuniform fast Fourier transform-based method, are suggested to efficiently implement the full-wave method by computing exact Green's functions. The effectiveness and accuracy of the method proposed have been verified via both numerical simulations and experimental measurements, and significant improvement of the reconstructed image quality compared with the traditional scalar-wave-based migration algorithms is demonstrated. The results can be directly utilized by forward-looking microwave imaging sensors such as installed at tunnel boring machine or can be used for the observation matrix computation in regularization-based inversion algorithms.

@en