We report on the fabrication and characterization of a Fabry-Perot microcavity enclosing a thin diamond membrane at cryogenic temperatures. The cavity is designed to enhance resonant emission of single nitrogen-vacancy centers by allowing spectral and spatial tuning while preserving the optical properties observed in bulk diamond. We demonstrate cavity finesse at cryogenic temperatures within the range of F ¼ 4000–12 000 and find a sub-nanometer cavity stability. Modeling shows that coupling nitrogen-vacancy centers to these cavities could lead to an increase in remote entanglement success rates by three orders of magnitude.@en