The vision and mission of research under the banner of Ubiquitous Computing has increasingly moved from focusing on the realm of “artifacts” to the realm of “environments”. We seek to scrutinize this very transition, and raise questions that relate to the specific attributes of built environments that set them inherently apart from artifacts. How does an interactive environment differ from an interactive artifact, a collection of artifacts, or an integrated suite of artifacts? Consequently, we ask what are the new user experience dimensions that HCI researchers should merge into their considerations, for example, by supplementing usability and engagement with occupants' comfort across multiple dimensions, and shifting attention from (often) short lifespan and discretionary to durable and immersive experiences? In this contribution, we bring arguments from the literature of environmental psychology and architecture that highlight the points of divergence between artifacts and architecture, and then translate them into challenges for Human-Computer Interaction, and particularly for the emerging domain of Human-Building Interaction.