During development, gene expression is tightly controlled to facilitate the generation of the diverse cell types that form the central nervous system. Brahma-related gene 1 (Brg1, also known as Smarca4) is the catalytic subunit of the SWItch/sucrose nonfermentable (SWI/SNF) chromatin remodeling complex that regulates transcription. We investigated the role of Brg1 between embryonic day 6.5 (E6.5) and E14.5 in Sox2-positive neural stem cells (NSCs). Being without major consequences at E6.5 and E14.5, loss of Brg1 between E7.5 and E12.5 resulted in the formation of rosette-like structures in the subventricular zone, as well as morphological alterations and enlargement of neural retina (NR). Additionally, Brg1-deficient cells showed decreased survival in vitro and in vivo. Furthermore, we uncovered distinct changes in gene expression upon Brg1 loss, pointing towards impaired neuron functions, especially those involving synaptic communication and altered composition of the extracellular matrix. Comparison with mice deficient for integrase interactor 1 (Ini1, also known as Smarcb1) revealed that the enlarged NR was Brg1 specific and was not caused by a general dysfunction of the SWI/SNF complex. These results suggest a crucial role for Brg1 in NSCs during brain and eye development.