Non-genetic heterogeneity of cells in development: more than just noise

Hardly a phenomenon in metazoan biology is more ubiquitous, yet as neglected, as the inevitable heterogeneity of cell phenotypes within a population of cells of the same cell type. Nevertheless, the implications of such cell population heterogeneity are far reaching, and semantic questions are inevitable: if an apparently ‘clonal’ population of cells, which are, by definition, genetically identical, is heterogeneous and exhibits enduring cell-to-cell variations with distinct cell individuality, why then do we not talk about distinct cell types, or subtypes, in the population? Stem cell researchers will easily recognize the ontological (see Glossary, Box 1) problem (Orkin and Zon, 2002): if a culture of stem cells, even if clonal, is actually heterogeneous to start with, be it owing to impurities introduced by unavoidable technical limitations (Enver et al., 2009) or to intrinsic diversity, as discussed here, what then is the meaning of plasticity and multipotency, as defined as the ability of cells to generate a variety of cell types? Perhaps the process of generating and/or acquiring heterogeneity eo ipso could lie at the core of multipotency.

The problem of population heterogeneity is not purely academic. Just consider the experiment illustrated in Fig. 1. A western blot analysis of stem cell differentiation shows how c-Kit, a stem cell marker in many tissues, is downregulated upon the induction of differentiation (Fig. 1A) (H. Chang and S.H., unpublished). However, flow cytometry analysis for surface expression of c-Kit on individual cells, displayed as a histogram, reveals a different picture. As cells downregulate c-Kit, their distribution does not simply shift to the left of the histogram (to lower intensities of fluorescence signal) as one might expect. Instead, the initial population consistently splits into two subpopulations (‘peaks’). The majority of cells do indeed downregulate c-Kit surface expression, but a small subpopulation arises that contains cells with even higher c-Kit surface expression than the original median. The ‘average’ c-Kit expression (i.e. the normalized total) of the whole cell population, as measured in lysed cells by western blotting, still decreases. This splitting of the cell population reflects a heterogeneity of the population in terms of the response to the differentiating signal and is most aptly accounted for by what is termed ‘pitchfork bifurcation’. This is a general phenomenon that arises in some gene-regulatory circuits with feedback loops and forces cells to make a choice between two alternative stable states as their current state is destabilized (Huang et al., 2007).

This example illustrates how we have sidelined the notion of cell individuality in the quest to determine regulatory pathways by investigating which transcript or protein levels go up or down in response to a manipulation. Such investigations rely mostly on population-averaging lysate-based methods, such as immunoblots, PCR or microarrays. In using these methods, we tacitly assume cell populations to be uniform. However, heterogeneity in cell populations is not a new concept. It has been widely described in clonal populations of bacteria (Balaban et al., 2004; Novick and Weiner, 1957; Spudich and Koshland, 1976) and has also been referred to as ‘non-genetic individuality’. In the theory of evolutionary dynamics, the role of ‘phenotype variation’ at the organismal level that does not necessarily imply a genetic variation is becoming increasingly appreciated (Kaneko, 2007).

Studies of gene expression or protein levels in mammalian tissues using in situ hybridization or immunohistochemistry often reveal dramatic cell-to-cell variability within one cell type in the same tissue. Heterogeneity was more recently shown in single-cell analysis using fluorescently labeled probes for hybridization (Levsky et al., 2002) or visual gene-expression reporters (Sigal et al., 2006; Takasuka et al., 1998), as well as in single-cell PCR (Hayashi et al., 2008; Warren et al., 2006; Diercks et al., 2009). Apart from a few exceptions (Aird, 2004; Grundel and Rubin, 1988; Rubin, 1992), the biological significance of non-genetic cell population heterogeneity in mammals has not been explicitly articulated until recently, when it has been described in the context of stem cells and fate decision (Stockholm et al., 2007; Chambers et al., 2007; Chang et al., 2008; Dietrich and Hiiragi, 2007; Kalmar et al., 2009; Singh et al., 2007; Spencer et al., 2009; Kobayashi et al., 2009).

To facilitate the discourse on heterogeneity, we need clarity of terminology and a good grasp of the underlying physics. This Primer aims to provide an introduction to the phenomenon of heterogeneity through an exposition at three levels: first, terminology; second, how heterogeneity is measured experimentally; and third, the dynamics of heterogeneity (including its source). The goal is to equip experimental biologists with a solid set of vocabulary and with a conceptual framework for organizing observations, rather than with a comprehensive review of the literature. For in-depth discussions of the biological significance of non-genetic heterogeneity in cell fate control and for an alternative perspective from the related field of ‘gene expression noise’, we refer the reader to several excellent recent reviews (see Enver et al., 2009; Graf and Stadtfeld, 2008; Losick and Desplan, 2008; Raj and van Oudenaarden, 2008).

Heterogeneity is a property of a cell population, not of individual cells. Heterogeneity of a population implies the presence of cell-to-cell variability with respect to one (or more than one) measurable trait X, where X can be the cellular level of a given molecule, such as a protein, or any quantifiable morphological or functional parameter. A closer analysis of the term ‘heterogeneity’ reveals many facets and forms that are rarely explicitly articulated. Thus, an attempt at creating a taxonomy and at defining terms and naming newly described phenomena represents a linguistic problem that reaches into the field of onomasiology (see Glossary, Box 1). We propose the following scheme (shown in Fig. 2) that operationally organizes heterogeneity into a hierarchy of dichotomies (see Glossary, Box 1).

Heterogeneity of cell populations is only non-trivial if one considers heterogeneity within a clonal, that is, genetically identical population of cells, for all too often in biology, any variation in phenotype is explained by a genetic variation. Such ‘genetic determinism’, which has dominated thinking in biology for decades (Strohman, 1997), leaves little room for non-genetic variations (Jablonka, 1994; Lewontin, 2000; Morange, 2001; Strohman, 1994). Genetic variability as a source of heterogeneity in cells is a straightforward, but nonetheless relevant, concept; in cancer biology, for instance, heterogeneity among tumor cells is commonly explained by genetic mutations. This genetic heterogeneity (see Glossary, Box 1), in turn, is thought to be promoted by genomic instability in cancer cells (Lengauer et al., 1998; Loeb, 2001; Tomlinson et al., 2002). In other words, the heterogeneity with respect to trait X is genetic in that the difference in X between individual cells is due to the presence of distinct genes that control trait X in these cells.

Conversely, non-genetic heterogeneity (see Glossary, Box 1) in clonal cell populations defies genetic determinism and raises a simple question, the answer to which we might take for granted: how can the same set of genes generate vastly distinct, stable, and often inherited gene expression profiles and, thereby, distinct phenotypes? This question is at the core of multipotency, as discussed below.

To avoid ambiguities in further discussion, two key terms need further clarification. First, the general term ‘non-genetic’ is to be distinguished from the more specific term ‘epigenetic’, which is

In the former usage, increasingly encountered in systems biology, ‘epigenetic’ (as in ‘epigenetic landscape’) is employed, as originally coined by Conrad Waddington (Waddington, 1957), to refer to a multiplicity of stable gene expression states in systems that exhibit ‘multi-stability’, as explained below (see Box 1).

By contrast, molecular biologists use ‘epigenetic’ (as in ‘epigenetic mark’) to refer to DNA methylation and histone modifications (Goldberg et al., 2007; Kouzarides, 2007), which serve as a proximate explanation for relatively stable gene expression states. An ‘epigenetic mark’ is the conceptual cousin of a ‘genetic mutation’, both of which are used to explain lasting phenotypic changes by invoking a molecular event. However, epigenetic marks in the genome do not actually account for the existence of stable, inherited states, such as cell types, that originally inspired the term ‘epigenetic’, because covalent modifications are actually dynamic, reversible and lack locus specificity. Therefore, they are not, strictly speaking, explanatory, but represent a molecular mechanism for implementing stable gene expression patterns that first have to be orchestrated by the transcriptional network. It is this gene regulatory network that creates the ‘epigenetic landscape’ in Waddington's sense, as explained below (Bonifer et al., 2008; Huang, 2009; Ptashne, 2007; Slack, 2002).

Second, for a discourse on non-genetic heterogeneity in clonal (isogenic) populations (see Glossary, Box 1), a strict definition of what is a ‘clone’ is required. The term ‘clonal’ does not only mean ‘genetically identical’; if it were so, every developed multicellular organism being derived from a zygote would, in a trivial sense, be a clone. Yet, no one would call a mixture of (isogenic) bone marrow cells that include stem cells, white and red blood cell precursors and so forth, ‘clonal’. Thus, a stricter, non-trivial definition of clonality is that the cells of a clonal population (see Glossary, Box 1) are (1) recently derived from a single ancestor cell (‘colony’), (2) within the same uniform and constant micro-environment, and (3) have achieved, as a population, a certain stationarity.

In the trivial view of an entire organism as a clone, one could regard phenotypic differences between nominal cell types (e.g. liver cell versus neuron) as a sort of non-genetic heterogeneity. As the definition of what constitutes a cell type versus a subtype or a phenotypic variant is not at all clear, we propose the term macro-heterogeneity (see Glossary, Box 1) to describe evident variability between discrete (sub)populations, which can represent either cell types, subtypes, or just ‘variants’ with respect to a trait X. By contrast, we refer to the heterogeneity between cells within a nominally identical cell type in an apparently uniform population as micro-heterogeneity (see Glossary, Box 1). Crucially, despite the fuzziness that surrounds the definition of a ‘cell type’, a clear-cut delineation can be made based on the statistical distribution of a trait X observed in a population, such as in flow cytometry. Here, macro-heterogeneity is manifest as the presence of multiple discrete, but possibly overlapping, ‘peaks’ (i.e. a multi-modal distribution). By contrast, micro-heterogeneity is reflected in the spread (breadth) of one peak (see Fig. 2, inset). Multi-modality indicates multi-stability, i.e. the presence of multiple distinct attractor states, as discussed below.

Phenotype variation among cells of a single clonal population can be caused by extrinsic factors that do not act uniformly on the population and thus trigger a cellular response in only a fraction of the population. This is referred to as extrinsic heterogeneity (see Glossary, Box 1) and arises, for example, when a clonal population grows in a complex environment, such as tissue niches, where cells differ in their vicinity to blood vessels, other cells or structures, or when cells in culture differ with respect to their distance from neighboring cells and to the dish border, or with respect to oxygen gradients and so forth. The heterogeneity of the cell population would then simply mirror the heterogeneity of the environment. Such heterogeneity due to differential instruction from outside plays a key role in development. Notably, the very first diversifications of cell lineages in the early embryo, such as the lineage split into the trophectoderm and inner cell mass, appear to benefit from the physical asymmetries of the embryo (inside versus outside cells, polarity) (Zernicka-Goetz et al., 2009). Similarly, positional information, such as that provided by morphogen gradients, further drives (extrinsic) heterogeneity in later cell fate waves of diversification (Oates et al., 2009).

Intrinsic heterogeneity (see Glossary, Box 1), by contrast, cannot be explained in an obvious, straightforward way by an external cause. Instead, it denotes the spontaneous diversification of a clonal population into continuous or discretely distinct variants. Intrinsic diversification has a profound meaning in complex systems sciences, for it is essential for the spontaneous generation of complex patterns - the paradigm of an emergent property that is characteristic of a living system (Goodwin, 1993; Kauffman, 1993). Intrinsic diversification of an initially unbiased, symmetric state requires two processes: first, a symmetry-breaking event (as exemplified by the bifurcation in Fig. 1A) that creates discrete, alternative options; and second, a stochastic process (such as gene expression noise, as discussed below) that drives the choice of these options. In complexity theory, such intrinsic diversification has been linked to instabilities and irreversibility in non-equilibrium systems (Nicolis and Prigogine, 1989; Prigogine, 1997).

The introductory example (Fig. 1A) alerts us to the fact that population-averaging techniques that involve lysing entire populations of cells of a culture or tissue specimen mask substantial information on population distribution. However, even among single-cell analysis techniques, there is a broad spectrum of methods that differ fundamentally in the information they provide on the nature of heterogeneity. Rather than giving an overview of methods, we discuss three approaches that illustrate key distinctive principles and span a range of techniques.

Flow cytometry is, strictly speaking, not a single-cell analysis technique, but measures the property X of a cell, typically the cellular abundance of a specific protein, in an entire population at the resolution of individual cells. It offers the most lucid illustration of population heterogeneity. The histogram displays a roughly bell-shaped frequency distribution [most typically, a log-normal distribution for the frequency of X in the population, so that a Gaussian-like curve appears in the log-scale presentation (Kaneko, 2006; Mantzaris, 2006)]. The spread of cellular levels of a single protein in a clonal cell population typically covers a range of 10- to 1000-fold, substantially more than measurement error, which, as determined using standardized fluorescence beads, accounts for a less than 10-fold range of variability (Chang et al., 2008). Unlike single-cell analysis techniques, flow cytometry provides the statistical distribution of a trait X in a population - a layer of information that is crucial for studying gene expression noise and population dynamics, but is often ignored by biologists. Specifically, analyzing the distribution of X helps to detect the presence of multiple quasi-discrete subpopulations, as opposed to a continuous (smooth) distribution (Figs 1, 2) (Chang et al., 2006; Huang et al., 2009; Krutzik et al., 2004), and thus helps to distinguish between macro- and micro-heterogeneity (Fig. 2).

Flow cytometry provides population snapshots and cannot monitor temporal changes of a trait X within individual cells. At the other end of the spectrum of methods, single-cell monitoring of property X in real time using live video-microscopy tracks individual cells over time (Fig. 3). This permits the longitudinal monitoring of X; for example, the time course of the change in the cellular levels of a fluorescent protein. Such measurements deliver kinetic information on the temporal structure of the fluctuations of gene expression that evades flow cytometry (Austin et al., 2006). They also help to distinguish between ‘population noise’ and ‘temporal noise’, as discussed below. However, single-cell imaging captures only a handful to hundreds of cells at a time and hence barely produces sufficient data on the population as a statistical ensemble. Of biological interest is that single-cell monitoring allows the tracking of cell fate history and the construction of entire cell-lineage pedigrees (Kaufmann et al., 2007; Ravin et al., 2008; Schroeder, 2008).

The sorting of cell populations using FACS (fluorescence-activated cell sorting) allows for the physical isolation of any segment of the total cell population with respect to the distribution of property X. This can provide insight into the dynamics that underlie heterogeneity at the population level and thus complements single-cell monitoring. Two scenarios can be distinguished (Fig. 4A,B): the sorting and reculturing of either a distinct subpopulation (Fig. 4A) or of a population fraction (Fig. 4B). In both cases, flow cytometry can be used subsequently to monitor how the sorted cells behave over time, i.e. whether and at what rate they restore the original distribution of X. This is most readily achieved when X is the expression of a gene of interest and is reported by a fluorescent protein (Kalmar et al., 2009; Kashiwagi et al., 2006), but it can also be achieved through the direct immunolabeling of cells, provided that the antibody is removed before the sorted cells are recultured (Chang et al., 2008).

Sorting for a discrete subpopulation (Fig. 4A) and monitoring whether it repopulates the other subpopulations readily establishes whether a subpopulation is a separate, stable and irreversible cell (sub)type - the default assumption in cell biology - or is in a dynamic, reversible state (Fig. 5, see next section) (Chang et al., 2006; Kalmar et al., 2009). In the latter case, the bimodal macro-heterogeneity most likely represents a dynamic equilibrium, with cells transitioning back and forth between the two subpopulations.

By contrast, the utility of sorting out a fraction from a single-peaked distribution (Fig. 4B) is particularly counterintuitive because the dispersion is usually considered ‘noise’. However, recent studies demonstrate that the tail fractions exhibit distinct transcriptome-wide gene expression patterns of biological significance (Chang et al., 2008; Kobayashi et al., 2009).

A limitation of the physical sorting of populations that should be kept in mind is errors in sorting that can create ‘false-positive heterogeneity’, which needs to be separated from intrinsic population heterogeneity (Enver et al., 2009). Mis-sorted cells that have an inappropriate property X can contaminate a subpopulation and affect subsequent population behavior. This is particularly relevant if the sorted property X correlates with the rate of cell proliferation, such that a contaminating cell might proliferate much faster and ‘take over’ the sorted subpopulation with its progeny [discussed by Chang et al. (Chang et al., 2008)].

Among the various facets and forms of heterogeneity, non-genetic, intrinsic micro-heterogeneity emerges as the most intriguing in biological terms because there is no straightforward explanation of its source and because its biological implications are only beginning to be explicitly studied. This form of heterogeneity is the focus of the next sections.

Given the recent explosion in the study of ‘gene expression noise’ (Kaern et al., 2005; Raj and van Oudenaarden, 2008) we need, for a more rigorous discussion of the source of heterogeneity, to introduce another apparent dichotomy: temporal versus population noise (Fig. 3).

Temporal noise (see Glossary, Box 1) refers to fluctuations in a property X (e.g. the cellular content of a particular protein) in an individual cell over time. This is what is commonly referred to by the term ‘gene expression noise’ (Kaern et al., 2005; Raj and van Oudenaarden, 2008). Because cellular fluctuations of X are generally not synchronized between cells, they collectively give rise to a heterogeneous population at any time point.

The study of gene expression noise has been facilitated by the availability of fluorescent protein reporters that enable the monitoring of gene expression in individual living cells. Briefly, the temporal ‘noisy’ fluctuations in gene expression (Fig. 3A) are thought to stem from the random (i.e. stochastic), synthesis and breakdown of individual molecules, such as mRNA and proteins. They are ultimately a manifestation of the thermal nature of chemical reactions, which constitute probabilistic molecular events. As these biomolecules are present at relatively small copy numbers in the cell, their stochastic fluctuations are, unlike in classical test-tube chemistry, not averaged out. This is known as the ‘finite number effect’ (Kaern et al., 2005). Temporal noise is accentuated when the assembly and dissociation of transcriptional complexes follow slow kinetics and when the subsequent uneven transcript production is amplified by a high translation rate (Kaern et al., 2005; Kaufmann and van Oudenaarden, 2007). Most studies of gene expression rely on bacteria and yeast as model systems, and on the use of flow cytometry to measure the population distribution of the abundance of a trait X (generally, a fluorescent protein) in individual cells as a snapshot. The (normalized) standard deviation of the observed, typically bell-shaped histogram of X is then taken as a measure of temporal noise.

However, as mentioned above, a population of cells can, in principle, also exhibit a broad distribution of a trait X even if X does not fluctuate in time (Fig. 3A), i.e. if X is a time-invariant, stable trait of an individual cell (Fig. 3C). Such cell-to-cell variation is referred to here as population noise (see Glossary, Box 1) and is due to stable differences between individual members of a population, much in the same way as the measurement of shoe size in the adult human population will give rise to a bell-shaped histogram. In the context of the heterogeneity discussed here, however, the absence of genetic differences must be kept in mind.

Temporal and population noise are scenarios at opposing ends of a spectrum and are introduced here as general principles to allow one to place the real case somewhere within the spectrum that is flanked by these two extremes (Fig. 3). Importantly, this general framework does not exclude heterogeneity due to fluctuations at intermediate time scales. These can stem from a variety of complicated, possibly deterministic (i.e. not noise-generated), but regulated changes in the abundance of a molecule. An example of this would be gene circuit-generated oscillations of X in individual cells (Hirata et al., 2002; Lahav et al., 2004; Kobayashi et al., 2009) that, if randomly out of phase between the cells, would produce the cell-to-cell variability seen in snapshot population measurements (Fig. 3B). Clearly, in reality, the ‘net’ fluctuation will result from the combination of several sources of fluctuations that occur at various time scales, as at least the stochastic component is inevitable.

It is not possible to tell from a single population distribution snapshot (histogram) whether heterogeneity is mainly due to temporal or to population noise. However, the broad distribution is often taken by default as reflecting solely temporal fluctuations in individual cells. This inference makes the implicit assumption of ergodicity (see Glossary, Box 1) (Patrascioiu, 1987). The ergodic hypothesis states that one can determine the time average of the fluctuating X (Fig. 3A) of an individual cell by simply measuring N independent elements (i.e. cells) of an ensemble (i.e. cell population) just once (a snapshot histogram), instead of taking a number N repeated measurements of the same cell, provided that the fluctuations of an individual cell are so fast that the individual measurements at some typical time interval of the same cell give uncorrelated results (lack of autocorrelation) - the condition of ergodicity.

In brief, ergodicity implies that the time average equals the ensemble average. Ergodicity breaks down if the fluctuations are so slow that the length of time intervals needed to make the repeated measurements (avoiding autocorrelation) for determining a representative time average becomes impracticably long (approximating the case depicted in Fig. 3C).

Temporal noise, but not population noise, can satisfy the ergodic condition. Many of the early experimental studies on gene expression noise have implicitly assumed ergodicity and did not consider the time scale of fluctuations, which would entail the careful analysis of fluctuations of individual cells in real time. However, at least in mammalian cell populations, slow fluctuations of X (see below) that can span up to several cell division cycles appear to contribute substantially to population heterogeneity. This component of fluctuations is too slow for considering the distribution of flow cytometry histograms as a pure manifestation of noisy (thermal) fluctuations.

Although this Primer is not about gene expression noise, it is worth mentioning that in studies of gene expression noise the distinction between intrinsic and extrinsic noise has been prevalent (Elowitz et al., 2002; Volfson et al., 2006). Originally, these terms were defined operationally in conjunction with the experimental set-up used to distinguish between: (1) gene-intrinsic noise attributable to the probabilistic nature of the reactions mentioned above; and (2) cell environment-derived contributions (such as fluctuating upstream influences common to two genes) to the noisy expression of a particular protein. However, the semantics of the term ‘extrinsic noise’ have not been uniform and must be dealt with carefully, given that the definition of the system considered (a chemical reaction, a cellular process, a compartment or the entire cell) determines what is to be regarded as ‘intrinsic’ or ‘extrinsic’. Furthermore, the time scale of fluctuations, as discussed above, must also be considered. Thus, extrinsic noise has also been interpreted as being due to stochastic fluctuations that occur over a longer time scale than intrinsic noise and has been modeled as fluctuations in the ‘control parameters’ of gene circuit models (Lei, 2009). In this view, extrinsic noise might correspond to slowly changing cell-individual traits (e.g. nucleus size, ribosome content) and approaches population noise.

Finally, the drastic (but necessary) simplification used in mathematical models of noise that shrink the processes of gene expression from hundreds of chemical reactions that participate in transcription, splicing and protein synthesis and maturation, to the two-step ‘central dogma’ (transcription and translation), further blurs the formal interpretation of noise measurements. As such, consistency has thus far been achieved only for the basic principles of gene expression noise, not for the details (Kaufmann and van Oudenaarden, 2007; Paulsson, 2004). Real-time measurements of trait fluctuations in multiple individual cells in a population, as discussed above (see ‘Measuring heterogeneity’), will be crucial to address these problems and to determine the relative contributions of temporal versus population noise.

Recent kinetic studies of mammalian cells suggest that the wide dispersion of the expression level X of a given protein is due to fluctuations of X that have a significant component that occurs over several days, in addition to the fast temporal component caused by gene expression noise. Thus, heterogeneity is perhaps more aptly viewed as largely due to population noise when considering functional studies of mammalian developmental processes that typically extend over days to weeks. Nevertheless, brief bursts of protein expression due to temporal (gene expression) noise may also affect cell fate choice (Enver et al., 2009; Losick and Desplan, 2008; Raj and van Oudenaarden, 2008).

Slow fluctuations have been demonstrated by the real-time monitoring of protein expression in individual mammalian cells, using genomically integrated fluorescent protein reporters (Sigal et al., 2006), or by flow cytometry analysis of the time needed for FACS-sorted ‘outlier’ cells with ‘extreme’ levels of X to relax back and re-establish the native distribution of X (Fig. 4B) (Chang et al., 2008). The finding that heterogeneity is associated with the lasting, cell-specific expression of fate-determining proteins only in some, but not all, cells in early embryos (‘salt-and-pepper’ appearance) and the correlation of a cell's gene expression profile with its prospective fate (Graf and Stadtfeld, 2008; Hayashi et al., 2008; Kalmar et al., 2009; Silva and Smith, 2008; Zernicka-Goetz et al., 2009) support the view that heterogeneity is predominantly caused by slow changes that are substantially influenced by deterministic (non-stochastic) extrinsic factors. Thus, heterogeneity appears to have a developmental role, rather than simply reflecting epiphenomenal statistical blips due to temporal noise.

The source of these slow fluctuations is currently poorly understood (see Table 1 for specific explanations that have been proposed). Here, we summarize in qualitative terms a general conceptual framework from the field of non-equilibrium systems dynamics (Nicolis, 1986; Nicolis and Prigogine, 1989) that provides a different perspective for understanding non-genetic heterogeneity (see Box 2).

The expression level of a protein X is determined by a vastly complicated gene regulatory network (GRN) that encompasses virtually the entire genome. Thus, heterogeneity is actually multidimensional, not just with respect to a gene X as discussed so far. The integrated high-dimensional dynamics of a network can be formalized as a quasi-potential landscape (see Glossary, Box 1) (Bar-Yam et al., 2009; Huang and Kauffman, 2009; Wang et al., 2008), in which each point represents a network state (see Glossary, Box 1). This is, by definition, a gene expression pattern and approximately represents a cell phenotype (see Box 2 for an introduction to network dynamics). Thus, a movement in this landscape represents a change of the gene expression profile and, hence, of the cell phenotype. Cellular development, then, is represented by a flowing movement on the landscape, wherein the cell's state seeks the lower regions. This quasi-potential landscape

is the formal equivalent of Waddington's metaphoric ‘epigenetic landscape’ (Waddington, 1957) and captures in an intuitive manner the global behavior of complex GRNs. It is in this sense that the term ‘epigenetic’ was proposed by Waddington (Waddington, 1942) and is still used by physicists, as discussed above. Importantly, the specific landscape topography carries key information on the potential behaviors of a cell, as it is a direct visualization of how the GRN dictates the manner in which a gene expression pattern as a whole can change under the regulatory constraints (Box 2).

In the epigenetic landscape, the lowest point in a ‘valley’ (i.e. a ‘potential well’ in terms of a generalized quasi-potential) represents an attractor state (see Glossary, Box 1) of the network. An attractor is a stationary self-stabilizing network state, the associated gene expression pattern of which is interpreted as defining a cell type (Huang et al., 2005; Kauffman, 1993). Cells (or, more precisely, GRNs) tend towards the attractor states and stay there, resisting minor perturbations. However, external regulatory signals and large perturbations (e.g. strong gene expression noise) that alter the expression of typically multiple genes can induce transitions of network states across the ‘hills’ that separate the attractor states. Such quasi-discontinuous state transitions appear as the all-or-none cell-type switching in individual cells (Fig. 1B) (Huang and Kauffman, 2009).

The precise topography of the landscape, which is mathematically determined by the gene network architecture [for a more formal, accessible discussion, see Huang and Kauffman (Huang and Kauffman, 2009)], is not known in detail. Nevertheless, the general concept of an epigenetic landscape could explain non-genetic macro- and micro-heterogeneity, as well as temporal and population noise, within one formal framework. Because of cell-cell variation, a population of cells is represented not by a point, but by a ‘cloud’, i.e. by an ensemble of points distributed across a certain region in the landscape. With the incorporation of heterogeneous cell populations and of instabilities (‘hills’) that repel and partition the clouds, multicellular development can be thought of as the movement of a cloud in the landscape along the valleys, seeking their lowest points, akin to water flowing down the mountains and collecting into lakes. Thus, multicellular development could be formalized as the time-irreversible change of a complicated distribution (Petrosky and Prigogine, 1993; Prigogine, 1997).

The landscape of a complex network [at least, near the order-chaos regime boundary (Kauffman, 1993)] is generally ‘rugged’, that is, contains multiple, possibly nested valleys (attractors with ‘sub-attractors’) rather than a single dominating valley surrounded by smooth slopes. Thus, much as with real valleys, one can find a hierarchy of main valleys, sub-valleys and so forth, which would explain the distinct scales of heterogeneity encoded by the network. Macro-heterogeneity is then due to the simultaneous occupation by the cloud of multiple major valleys (attractors), each of which roughly corresponds to a distinct cell type (Huang et al., 2005; Kauffman, 1969b). They are separated by major ‘hills’ that are high enough to permit only rare noise-induced attractor transitions. Conversely, micro-heterogeneity is due to the dispersion of cells within a cell-type-specifying attractor into multiple sub-attractors that are separated by low energy barriers. The slow kinetics of the heterogeneity-producing fluctuations would be a manifestation of noise-induced transitions between these sub-attractors, which occur on a much slower time scale than thermal fluctuations between continuous expression levels (Fig. 5). Recent kinetic measurements of the non-exponential relaxation of outlier cells to the center of a main attractor indicate the presence of distinct intermediate states between the outlier and the ‘median’ states, which suggests the presence of sub-attractors (Chang et al., 2008).

Interestingly, this picture is reminiscent of an established concept employed to understand the folding of proteins, in which the coexistence of multiple energetic substates allows for many intermediate subconformations in addition to the main conformation of a protein. The dynamics of transitions between these (higher-energy) subconformations can be represented by a multi-well ‘washboard potential’ (akin to Fig. 5A) that traps rare outlier subconformations away from the central equilibrium state, so that molecules in the higher substates only slowly (possibly non-exponentially) relax back to the native state (Frauenfelder et al., 1991). The transient, but continual occupation of the substates corresponds to the presence of outlier cells at equilibrium and contributes to the spreading of the population histogram.

The general complex systems dynamics framework set out above can accommodate modifications to include alternative sources of slow dynamics and heterogeneity (see Table 1), such as strange attractors due to chaotic dynamics (Furusawa and Kaneko, 2001) or noise-induced action-potential-like excursions around an attractor state (Kalmar et al., 2009).

The epigenetic landscape offers a new vista on the dynamics of non-genetic heterogeneity and their biological consequences. In the case of micro-heterogeneity, the extreme expression values of protein X in outliers at the tail of a distribution could have biological consequences if X is a fate-determining protein, as the relatively long persistence of an outlier state is in the time scale (days) of gene regulation. In other words, non-ergodicity means that outliers, not just the mean, matter. Noise is more than just statistics that need to be averaged out. In the case of macro-heterogeneity, it should be kept in mind that two subpopulations (separate ‘peaks’ in the histogram) are not necessarily inconvertible cell types, but might transition into each other, depending on the relative height of the potential barrier (hill) that separates them. This is contrary to many biologists‘ deeply rooted intuition that discretely distinct subpopulations represent irreversibly committed cell types, and instead provides a conceptual framework for cell type plasticity (Graf, 2002; Joshi and Enver, 2002).

Evidence for the biological relevance of non-genetic diversity and slow dynamics is accumulating. The associated spontaneous and lasting phenotypic individuality of cells appears to play a central role in driving the diversification of cell types in metazoans, and hence could be a crucial phenomenon in multipotency.

However, the biological significance of non-genetic heterogeneity extends beyond multipotency and development. It also might be important for cell population survival by producing the phenotypic diversity for non-genetic selection (’mutationless evolution’), notably in tumors responding to chemotherapy (Brock et al., 2009; Cohen et al., 2008; Spencer et al., 2009). Almost no drug designed to alter a cell phenotype (e.g. by inducing apoptosis or differentiation) does so (at reasonable doses) in all of the cells of a population, perhaps because of the vast intrinsic non-genetic heterogeneity with regard to the responsiveness of the cells. This drastically limits the efficacy of even modern target-selective drugs or of reprogramming efforts (Brock et al., 2009; Huang, 2009).

Much as biologists have learned to appreciate random, inheritable genetic variability as a source of species diversification in evolution, they might need to embrace random non-genetic heterogeneity of inheritable traits in cell populations as a source of cell type diversification, instead of regarding it as a nuisance, in order to understand the essence of multicellular life.

We acknowledge the U.S. Air Force Office of Scientific Research and the National Institutes of Health for supporting this work and thank Alfonso Martinez Arias and Stuart Kauffman for valuable discussions. Deposited in PMC for release after 12 months.