We present an electron backscattered diffraction (EBSD)-trained deep learning (DL) method integrating traditional material characterization informatics and artificial intelligence for a more accurate classification and quantification of complex microstructures using only regular scanning electron microscope (SEM) images. In this method, EBSD analysis is applied to produce accurate ground truth data for guiding the DL model training. An U-Net architecture is used to establish the correlation between SEM input images and EBSD ground truth data using only small experimental datasets. The proposed method is successfully applied to two engineering steels with complex microstructures, i.e., a dual-phase (DP) steel and a quenching and partitioning (Q&P) steel, to segment different phases and quantify phase content and grain size. Alternatively, once properly trained the method can also produce quasi-EBSD maps by inputting regular SEM images. The good generality of the trained models is demonstrated by using DP and Q&P steels not associated with the model training. Finally, the method is applied to SEM images with various states, i.e., different imaging modes, image qualities and magnifications, demonstrating its good robustness and strong application ability. Furthermore, the visualization of feature maps during the segmenting process is utilised to explain the mechanism of this method's good performance.