The creation and manipulation of optical vortices, both in free space and in two-dimensional systems such as surface plasmon polaritons (SPPs), has attracted widespread attention in nano-optics due to their robust topological structure. Coupled with strong spatial confinement in the case of SPPs, these plasmonic vortices and their underlying orbital angular momentum (OAM) have promise in novel light-matter interactions on the nanoscale with applications ranging from on-chip particle manipulation to tailored control of plasmonic quasiparticles. Until now, predominantly integer OAM values have been investigated. Here, we measure and analyze the time evolution of fractional OAM SPPs using time-resolved two-photon photoemission electron microscopy and near-field optical microscopy. We experimentally show the field’s complex rotational dynamics and observe the beating of integer OAM eigenmodes at fractional OAM excitations. With our ability to access the ultrafast time dynamics of the electric field, we can follow the buildup of the plasmonic fractional OAM during the interference of the converging surface plasmons. By adiabatically increasing the phase discontinuity at the excitation boundary, we track the total OAM, leading to plateaus around integer OAM values that arise from the interplay between intrinsic and extrinsic OAM.