During gastrulation, a vertebrate embryo re-organises into a structure with three germ layers and three distinct body axes in a manner that relies on cues from extra-embryonic tissues. Gastruloids – formed from aggregated stem cells – serve as powerful in vitro models to probe gastrulation in the absence of external cues. In mouse embryonic stem cell-derived gastruloids, the first symmetry-breaking event and development of the anteroposterior (AP) axis, marked by the polarised posterior expression of Brachyury (T), occurs autonomously. However, the extent to which T expression and axial patterning progress without extrinsic cues, such as Wnt stimulation, remains unknown. Here, Vikas Trivedi and colleagues use long-term imaging and single-cell RNA sequencing analyses to show that the spatial patterning of germ layers occurs in gastruloids even without external patterning cues. Interestingly, AP axial patterning and the generation of all germ layer progenitors remain robust to variations in gastruloid size. By comparing gastruloid and embryo transcriptomes, the authors show that gastruloid cells adopt a mesenchymal-like state and converge onto germ layer lineages through a distinct trajectory from that in vivo. Overall, this work reports the earliest polarity establishment and lineage specification events in a mammalian embryo-like system.