Enhancers control the establishment of spatiotemporal gene expression patterns throughout development. Over the past decade, the development of new technologies has improved our capacity to link enhancers with their target genes based on their colocalization within the same topological domains. However, the mechanisms that regulate how enhancers specifically activate some genes but not others within a given domain remain unclear. In this Review, we discuss recent insights into the factors controlling enhancer specificity, including the genetic composition of enhancers and promoters, the linear and 3D distance between enhancers and their target genes, and cell-type specific chromatin landscapes. We also discuss how elucidating the molecular principles of enhancer specificity might help us to better understand and predict the pathological consequences of human genetic, epigenetic and structural variants.