Many challenges to operate digital twins remain, hindering their design and implementation, and are rarely discussed. Furthermore, issues of social and legal nature are often overlooked. We identify the challenges of operating digital twins in the urban context through a bifurcated and multi-dimensional approach: a systematic literature review and an expert survey. The review organises the identified challenges across technical and non-technical dimensions. As the topic is novel, the corpus is rather small and lacking the contextualisation of challenges. Thus, we complement it with a survey based on the Delphi method, involving a diverse panel of domain experts covering academia, industry and government organisations. Combining the results, we identify 14 technical and 9 non-technical challenges and map them to phases of the digital twin's life cycle. The most severe challenges appear to be related to interoperability (e.g. disparate semantic standards) and practical value (e.g. lack of business models).