This study investigates the effects of nanofillers on the interfacial transition zone (ITZ) between aggregate and cement paste by using nanoindentation and statistical nanoindentation techniques. Moreover, the underlying mechanisms are revealed through micromechanical modeling. The nanoindentation results indicate that incorporating nanofillers increases the degree of hydration in the ITZ, reduces the content of micropores and low-density calcium silicate hydrate (LD C–S–H), and increases the content of high-density C–S–H (HD C–S–H) and ultrahigh-density C–S–H (UHD C–S–H). In particular, a new phase, namely nano-core-induced low-density C–S–H (NCILD C–S–H), with a superior hardness of 2.50 GPa and an indentation modulus similar to those of HD C–S–H or UHD C–S–H was identified in this study. The modeling results revealed that the presence of nanofillers increased the packing density of LD C–S–H and significantly enhanced the interaction (adhesion and friction) among the basic building blocks of C–S–H gels owing to the formation of nano-core–shell elements, thereby facilitating the formation of NCILD C–S–H and further improving the performance of the ITZ. This study provides insight into the effects of nanofillers on the ITZ in concrete at the nanoscale.