Stable silencing of the inactive X chromosome (Xi) in female mammals is crucial for the development of embryos and their postnatal health. SmcHD1 is essential for stable silencing of the Xi, and its functional deficiency results in derepression of many X-inactivated genes. Although SmcHD1 has been suggested to play an important role in the formation of higher-order chromatin structure of the Xi, the underlying mechanism is largely unknown. Here, we explore the epigenetic state of the Xi in SmcHD1-deficient epiblast stem cells and mouse embryonic fibroblasts in comparison with their wild-type counterparts. The results suggest that SmcHD1 underlies the formation of H3K9me3-enriched blocks on the Xi, which, although the importance of H3K9me3 has been largely overlooked in mice, play a crucial role in the establishment of the stably silenced state. We propose that the H3K9me3 blocks formed on the Xi facilitate robust heterochromatin formation in combination with H3K27me3, and that the substantial loss of H3K9me3 caused by SmcHD1 deficiency leads to aberrant distribution of H3K27me3 on the Xi and derepression of X-inactivated genes.