The visual allure of microscopy makes it an intuitively powerful research tool. Intuition, however, can easily obscure or distort the reality of the information contained in an image. Common cognitive biases, combined with institutional pressures that reward positive research results, can quickly skew a microscopy project towards upholding, rather than rigorously challenging, a hypothesis. The impact of these biases on a variety of research topics is well known. What might be less appreciated are the many forms in which bias can permeate a microscopy experiment. Even well-intentioned researchers are susceptible to bias, which must therefore be actively recognized to be mitigated. Importantly, although image quantification has increasingly become an expectation, ostensibly to confront subtle biases, it is not a guarantee against bias and cannot alone shield an experiment from cognitive distortions. Here, we provide illustrative examples of the insidiously pervasive nature of bias in microscopy experiments – from initial experimental design to image acquisition, analysis and data interpretation. We then provide suggestions that can serve as guard rails against bias.

Humans are visual creatures (Kaas and Balaram, 2014). Microscopy, with its unique ability to combine our instinctive reliance on visual cues and our curiosity-driven discovery, is an exceptionally powerful research tool. It has gained such universal acceptance in the life sciences that ‘seeing is believing’ has become a banal mantra to promote microscopy. However, the true power of modern microscopy is its parallel ability to verify observational discoveries across large biological and temporal scales with quantitative analysis (Reiche et al., 2022; Wait et al., 2020). This has been made possible by rapid advancements in optical hardware, digital detectors, sample preparation techniques and computational power (Balasubramanian et al., 2023; Hickey et al., 2022; Huang et al., 2021). Consequently, the research power provided by these capabilities comes with an accompanying increase in responsibility to perform reliable and reproducible science (Munafò et al., 2017).

This imperative has led to an abundance of guides in the literature for every stage of a microscopy project from experimental conceptualization (Jost and Waters, 2019; Wait et al., 2020) to image acquisition (Jonkman et al., 2020; North, 2006) and quantification (Khater et al., 2020; Waters and Swedlow, 2008). Journals, reviewers and funding agencies can encourage the use of these best practices, but ultimately, implementation of these procedures falls on the observer. Although the foundation of science relies on the falsification of hypotheses (Platt, 1964), the modern scientific research enterprise – from publications to grant funding and subsequently promotion – tends to reward positive outcomes that support hypotheses. This tempts even well-intentioned observers to turn the intuitive appeal of microscopy on its head: believing becomes the impetus for seeing what was expected, providing multiple vulnerabilities for observer bias to take control of an experiment.

Observers are naturally susceptible to a wide variety of cognitive biases, which manifest in many forms during a microscopy project (see Table 1 for several examples). Our tendency to prefer information that supports existing beliefs and is most available to us affects our daily decisions (Kahneman, 2011); these same mental heuristics can also shape quantitative microscopy. At each stage where there is the opportunity for experimental choice, the impulse to support the hypothesis can lead an observer astray. Selection of a region of interest or image acquisition parameter can be misinformed by the subjective assessment of the observer. Likewise, identification of features in an image can be skewed by flaws in visual perception. To circumvent these problems, journals and funding organizations have begun to insist on quantitative image analysis. Unfortunately, it is a common misconception that quantitative analysis is a fail-safe measure against bias. Even more perilous is the belief that once imaging data have been quantitatively analyzed, the data interpretation is incontrovertible. In fact, quantitative analysis does not guarantee an accurate representation of reality. Furthermore, quantitative results might not even always be informative (Wait et al., 2020).

Table 1.

Examples of cognitive biases, which can impact all stages of a microscopy experiment

Examples of cognitive biases, which can impact all stages of a microscopy experiment
Examples of cognitive biases, which can impact all stages of a microscopy experiment

Acknowledging observer bias is essential for diminishing its impact. It requires intellectual humility (Hoekstra and Vazire, 2021) to rigorously test a hypothesis through falsification and the assessment of alternative hypotheses (Platt, 1964). Good intentions cannot cover for flawed reasoning (Bishop, 2020) and bias must be actively and consciously minimized. In this Opinion, we will illustrate how several common biases jeopardize the fidelity of microscopy-based experiments and how they derail accurate data analysis and interpretation. We explore how these errors can be recognized and addressed. Although the detailed manifestations of the many biases in microscopy are too numerous for the scope of any one article, the concepts illustrated here are chosen to cover a broad range of common yet often under-appreciated bias types. Most importantly, the proposed solutions are broadly applicable to a range of experimental designs and analysis approaches (see Box 1). The underlying principle of this paper is that, above all else, disciplined critical thinking is indispensable to rooting out subtle but grave experimental biases.

Box 1. Strategies against bias

Be thoughtful about experimental design

Thoughtful experimental design includes built-in checks on bias. In the case of hypothesis-testing experiments, each step of the project should be designed to challenge the hypothesis, rather than support it (Platt, 1964; Wait et al., 2020). Exploratory experiments can also benefit from careful experimental design – iterative experimental design and pilot studies can focus and refine hypotheses. Incorporating multiple techniques into a project, such as comparing two labeling strategies or comparing imaging results to biochemical assays, also strengthens data interpretation. Data masking (also known as blinding) is a simple approach to reduce the impact of preconceived notions, but it is woefully underutilized in life science (Holman et al., 2015; Jost and Waters, 2019; MacCoun and Perlmutter, 2015).

Use best practice guides – there are many!

Training and education can lessen acquisition and analysis mistakes that reinforce cognitive biases (Imreh et al., 2023). There is a wealth of guidance in the literature on experimental design (Jost and Waters, 2019; Wait et al., 2020), sample preparation (Reiche et al., 2022), image acquisition (Lee et al., 2018; North, 2006; Waters and Swedlow, 2008), image analysis (Aaron et al., 2018; Aaron et al., 2019; see 2019 article by Cimini at https://carpenter-singh-lab.broadinstitute.org/blog/when-to-say-good-enough; Khater et al., 2020) and statistical techniques (Bishop, 2020; Krzywinski and Altman, 2013; Makin and Orban de Xivry, 2019; Nuzzo, 2014; Pollard et al., 2019) – take advantage of these resources.

Take advantage of computational tools

Computational tools can minimize the hazards of choosing experimental parameters and challenge naïve perceptions. Using imaging acquisition software to automate ROI selection removes a source of user bias (Jost and Waters, 2019). Attention to visualization choices, such as brightness and contrast, can reveal underappreciated facets of a dataset. Quantitative analysis can also provide a check on assumptions, although a lack of critical discernment in the application of such tools can introduce new biases. When interpreting quantitative results, it is essential to understand the assumptions of the method by which they were produced.

Contribute as a member of the scientific community

Preconceptions must be mitigated by individual researchers, but the broader research community is also essential for addressing bias. Reproducible science is enabled by the combined efforts of researchers, reviewers, journals and institutions (Munafò et al., 2017) through practices such as proper methods reporting (Aaron and Chew, 2021; Heddleston et al., 2021; Montero Llopis et al., 2021), sharing data and metadata in appropriate repositories (Boehm et al., 2021; Linkert et al., 2010), and pre-registering studies (Kupferschmidt, 2018).

Include context and test alternative hypotheses

The cumulative effects of bias become most apparent during data interpretation. Always keep in mind the choices that were made at each step of the experiment – technical limitations and practical choices can lead to limited fields of view or confounding experimental variables. Automated analysis tools can hide intermediate calculation steps or be sensitive to chosen parameters. Consider what alternative hypotheses could explain the observed results, and design approaches to test them, which will strengthen conclusions.

Any imaging approach is fundamentally limited to revealing only a subset of features in a biological system. For example, the constraints of live-cell imaging limit the number of labels that are feasible (Reiche et al., 2022), and even highly multiplexed fixed-sample imaging techniques, for example, expansion-assisted iterative fluorescence in situ hybridization (EASI-FISH; Wang et al., 2021), cannot capture every facet of a sample. The abundance of cellular structures is more clearly revealed by electron microscopy (Xu et al., 2020), yet electron microscopy lacks molecular specificity and can still leave many sample features obscured (Peddie et al., 2022). Such overlooked features do not negate the usefulness of imaging experiments but must be recognized during data interpretation to avoid ill-supported conclusions.

Hidden biological components and confounding variables are inevitable in any experimental design, but poor image acquisition choices can further conceal important biological information. For example, adjusting detector offset during a confocal acquisition changes the intensity level at which pixels are set to a value of zero (i.e. black). Adjusting the offset thus may lead to visually ‘cleaner’ images, but can also obscure real features, such as excluding the diffuse pool of G-actin from images of cells expressing green fluorescent protein (GFP)-labeled actin (Fig. 1A,B). Rejecting this population might naïvely seem like a tidy way to simplify downstream analysis, but this imaging choice complicates the interpretation of the observed biology. Even if F-actin is the focus of study, adjusting the acquisition offset improperly can unintentionally end up excluding dimmer filamentous structures, especially if actin morphology changes (Fig. 1C). Concerningly, this imaging choice can even lead to different conclusions from a purportedly unbiased analysis of intensity over time. Excluding the contribution of diffuse G-actin might lead to the erroneous conclusion that overall actin levels are decreasing when, more accurately, it is cycling through different polymerization states (Fig. 1D). Attention to imaging best practices (Jost and Waters, 2019; Lee et al., 2018; North, 2006; Waters and Swedlow, 2008) can lessen, although never fully eliminate, detrimental loss of information. Deliberate reflection on the scientific goal (total actin or F-actin) and how this compares to what is actually recorded (what are the limitations of my experimental approach?) is essential for addressing bias during data interpretation.

Fig. 1.

Overlooked information biases data interpretability. (A) A contrast-adjusted summed intensity projection of a confocal microscopy image of a Ptk2 cell (a Potorous tridactylus epithelial kidney cell line) labeled with actin–GFP (LSM 980 microscope; Plan-Apochromat 63×/1.40 oil objective; 488 nm laser; 134 z-slices with an interval of 250 nm; 15-min intervals between timepoints). At time t=0, 15 µM Y-27632 (a Rho kinase inhibitor) was added. After 60 min of treatment, actin stress fibers have been disrupted. (B) The same cell was imaged at the same time points using a large negative offset. (C) Merged image comparing the 60-min timepoint of A and B. Information in the green regions has been overlooked by the imaging choices in B, including small filamentous structures. (D) Analysis of normalized actin intensity over time in the two imaging conditions gives differing impressions of the effect of Y–27632. Summed intensities within the cell boundary were normalized to the first time point (dashed line). Scale bars: 25 µm. Additional imaging and analysis details are provided in the supplementary section.

Fig. 1.

Overlooked information biases data interpretability. (A) A contrast-adjusted summed intensity projection of a confocal microscopy image of a Ptk2 cell (a Potorous tridactylus epithelial kidney cell line) labeled with actin–GFP (LSM 980 microscope; Plan-Apochromat 63×/1.40 oil objective; 488 nm laser; 134 z-slices with an interval of 250 nm; 15-min intervals between timepoints). At time t=0, 15 µM Y-27632 (a Rho kinase inhibitor) was added. After 60 min of treatment, actin stress fibers have been disrupted. (B) The same cell was imaged at the same time points using a large negative offset. (C) Merged image comparing the 60-min timepoint of A and B. Information in the green regions has been overlooked by the imaging choices in B, including small filamentous structures. (D) Analysis of normalized actin intensity over time in the two imaging conditions gives differing impressions of the effect of Y–27632. Summed intensities within the cell boundary were normalized to the first time point (dashed line). Scale bars: 25 µm. Additional imaging and analysis details are provided in the supplementary section.

The bias introduced by overlooked factors can be further alleviated by making a conscious choice to place imaging results in proper context. Context is important even at the earliest stages of experimental design, where different labeling approaches for the same protein can lead to different conclusions. For example, many labeling strategies have been developed for actin, each with their own strengths and weaknesses (Melak et al., 2017). Whereas actin–fluorescent protein fusions highlight both filamentous and diffuse actin (Fig. 1), phalloidin-based tags are specific for F-actin, leaving G-actin invisible. Similarly, specimens expressing Lifeact (Riedl et al., 2008) and GFP–actin, respectively, might initially exhibit similar actin morphologies but will produce vastly different outcomes during a fluorescence recovery after photobleaching (FRAP) experiment due to the different labeling mechanisms of each probe (Belin et al., 2014; Melak et al., 2017). In fact, the FRAP recovery of Lifeact does not represent actual actin monomer turnover but rather the transient exchange of Lifeact on the actin filament. In many cases, a single label will not provide enough context to sufficiently answer a research question. For example, immunostaining phosphorylated myosin species alone does not provide enough information to determine in which subcellular regions myosin is preferentially phosphorylated, whereas imaging total myosin alone does not provide any information on myosin activity. Only together can the two labels provide an accurate picture of myosin contractility in the cell (Chew et al., 2002).

Information loss is not limited to labeling choices or acquisition settings: all experimental parameters will exclude relevant information, which must then be consciously acknowledged during data interpretation. Even when images are appropriately acquired, the choice of analytical parameters can end up unintentionally excluding certain populations. For example, object-tracking algorithms use a maximum distance parameter to keep calculations computationally feasible, but this same parameter can end up inadvertently excluding fast moving objects when its value is too small (Aaron et al., 2019; Jaqaman et al., 2008). Proper interpretation of such tracking results requires careful consideration of whether populations might have been excluded owing to imaging or analysis limitations, and the conscious acknowledgement of this excluded information. Similarly, regions of interest should be interpreted in the context of the larger specimen, as described below. Throughout an experiment, continuing to place results in context will help minimize biases in data interpretation.

Despite the importance of placing results in context, the very nature of microscopy experiments requires an observer to select from a population of interest. During image acquisition, physical limitations lead to limited fields of view and thus require selection of a region of interest (ROI). Although microscopists are constantly confronted with the need for selection, it is worryingly easy to forget this context during data interpretation. Presenting work to others often similarly requires the selection of so-called ‘representative’ cells to illustrate a process. In each case, the mere act of selection creates a vulnerability for bias to steer results toward the assumptions of the experimenter, and away from rigorous hypothesis testing.

No single cell or ROI can fully capture the heterogeneity present across a biological population, yet such selection is frequently necessary. For example, showing every cell on a microscope slide might partially convey the extent of population heterogeneity (Fig. 2A) but it will obscure cellular details that can only be seen in a single cell image (Fig. 2B–D). Quantitative measures, such as median cell size (Fig. 2E), can help guide cell selection. However, a cell with representative size (Fig. 2B) might display other morphology (e.g. circularity) that is atypical (Fig. 2F,G). A valiant attempt can be made to find a cell that is representative across a wide range of features (Fig. 2C,D), yet there will always remain unexplored features in which any given cell may be an outlier, rather than the norm.

Fig. 2.

A representative cell cannot fully encapsulate a population. (A) A tiled widefield microscopy image of U2OS cells (Nikon Eclipse Ti inverted widefield microscope; Plan Apo λ 20×/0.75 air objective; Lumencor Spectra X LED light engine; 10×10 tiled image). ROIs for B–D are indicated. Scale bar: 1 mm. (B–D) Representative cells selected with properties described in E–G. Cells were stained for tubulin (green) and DAPI (magenta). Scale bars: 20 µm. (E) Histogram of the cell areas for the population shown in A, with the values of each of the cells in B–D indicated by colored lines. (F) Histogram of circularity, which measures how round each cell is (a perfect circle has a circularity value of 1). (G) Histogram of eccentricity, which measures the elongation of a cell shape (a perfect circle has an eccentricity of 0, while a line has a value of 1). A cell which has a median value for one property might have an extreme value for another property. Additional imaging and analysis details are provided in the supplementary section.

Fig. 2.

A representative cell cannot fully encapsulate a population. (A) A tiled widefield microscopy image of U2OS cells (Nikon Eclipse Ti inverted widefield microscope; Plan Apo λ 20×/0.75 air objective; Lumencor Spectra X LED light engine; 10×10 tiled image). ROIs for B–D are indicated. Scale bar: 1 mm. (B–D) Representative cells selected with properties described in E–G. Cells were stained for tubulin (green) and DAPI (magenta). Scale bars: 20 µm. (E) Histogram of the cell areas for the population shown in A, with the values of each of the cells in B–D indicated by colored lines. (F) Histogram of circularity, which measures how round each cell is (a perfect circle has a circularity value of 1). (G) Histogram of eccentricity, which measures the elongation of a cell shape (a perfect circle has an eccentricity of 0, while a line has a value of 1). A cell which has a median value for one property might have an extreme value for another property. Additional imaging and analysis details are provided in the supplementary section.

Although no one specimen can be truly representative of a population, a selected ROI can still provide a valuable illustration. Many imaging acquisition software packages include options to randomly select the fields of view for imaging, which removes user choice as a source of bias (Jost and Waters, 2019). Statistical measurements can be used to determine the number of ROIs appropriate for the questions of interest in the context of the variability in the model system (Krzywinski and Altman, 2013; Pollard et al., 2019). Similarly, software tools can be used to identify representative images (Markey et al., 1999), although this approach requires thoughtful definition of the quantitative metrics of interest. Inclusion of multiple representative images in a figure can help to illustrate population heterogeneity; however, the choice of heterogeneous features to illustrate will be as susceptible to selection bias (Table 1) as any individual image and thus the set of images should be chosen with care. In addition to these practical techniques for ROI selection, the illustrative nature of any single sample should be kept at the mental forefront, especially when a chosen sample appears to support an as-of-yet untested hypothesis. Illustrative samples are useful for hypothesis generation, but extrapolations derived from any single ROI should always be rigorously evaluated. Careful testing of the hypothesis should be ensured by a combination of masked image assessment (Holman et al., 2015; Jost and Waters, 2019; MacCoun and Perlmutter, 2015), automatic analysis techniques (discussed in a later section), and, whenever suitable, complementary approaches, such as biochemical assays.

The impact of bias on microscopy experiments is especially pernicious when visual assessment of an image appears to confirm preexisting hypotheses. Although humans excel at finding visual patterns (Treisman, 2002), this ability can be skewed or even utterly inaccurate, leading to perceptual illusions. This distortion can take the amusing and largely harmless form of seeing animals in the shapes of clouds or the face of a man on Mars (Sagan, 1995; Voss et al., 2012), but it can also lead to serious errors with severe consequences. The explosion of literature on SARS-CoV-2 in 2020 included many images of viral particles in presumptively infected tissues (Fig. 3A). Regrettably, biased by the expectation of infection, some researchers in fact misidentified endoplasmic reticulum (ER) or multivesicular bodies as viral particles (Dittmayer et al., 2020). This type of bias, which is exacerbated by our preconceived expectations, along with pressure to provide timely and novel results, can be reduced by testing alternative hypothesis (what other membrane-bound structures could be expected in this tissue?), and by preestablishing analysis approaches (e.g. identifying particles by their ribonucleoprotein density) even before image acquisition begins (Wait et al., 2020). Complementary experimental approaches, such as confirming the presence of viral particles with immunolabeling, can also be a check on visual intuition.

Fig. 3.

Visual perception can be misleading. (A) SARS-CoV-2 particles show electron-dense ribonucleoprotein (black arrows), are found inside membrane compartments (white arrows) and, in well-preserved cases, show fine surface protrusions. Other membrane-bound objects could easily be mistaken for viral particles, especially if a hypothesis encourages this conclusion. Figure reproduced from Dittmayer et al. (2020) with permission from Elsevier. (B) The eye is naturally drawn to large, bright nuclei (magenta arrow) in this H2B-labeled zebrafish embryo, but smaller, dimmer nuclei (green arrowhead) are also important for the development of the embryo. (C) A maximum intensity projection of a cell labeled for ER (green) and endosomes (labeled by transferrin, magenta). Scale bar: 15 µm. (D) 3D visualization of C reveals several instances of endosomes close to the ER (white arrowheads). Each panel is ∼2 µm by 2 µm. (E) Measured distances between endosomes and ER (pink line) are compared to the distances in simulations of randomly placed lysosomes (100 overlapping gray lines), revealing endosomes are further from the ER than would be expected by chance. (F) The left image is intuitively perceived as random but was generated using repulsive particles. The ‘clustering’ of particles in the right image is a natural effect of random placement. See supplementary section for more details.

Fig. 3.

Visual perception can be misleading. (A) SARS-CoV-2 particles show electron-dense ribonucleoprotein (black arrows), are found inside membrane compartments (white arrows) and, in well-preserved cases, show fine surface protrusions. Other membrane-bound objects could easily be mistaken for viral particles, especially if a hypothesis encourages this conclusion. Figure reproduced from Dittmayer et al. (2020) with permission from Elsevier. (B) The eye is naturally drawn to large, bright nuclei (magenta arrow) in this H2B-labeled zebrafish embryo, but smaller, dimmer nuclei (green arrowhead) are also important for the development of the embryo. (C) A maximum intensity projection of a cell labeled for ER (green) and endosomes (labeled by transferrin, magenta). Scale bar: 15 µm. (D) 3D visualization of C reveals several instances of endosomes close to the ER (white arrowheads). Each panel is ∼2 µm by 2 µm. (E) Measured distances between endosomes and ER (pink line) are compared to the distances in simulations of randomly placed lysosomes (100 overlapping gray lines), revealing endosomes are further from the ER than would be expected by chance. (F) The left image is intuitively perceived as random but was generated using repulsive particles. The ‘clustering’ of particles in the right image is a natural effect of random placement. See supplementary section for more details.

Despite such perceptual illusions, visualization remains a powerful tool for discovery. Visual inspection of microscopy data requires choosing various visualization parameters, providing an additional opening for perceptual bias. Changing image display contrast is a basic manipulation that can instantly change our perception of a dataset. Not only does overall image contrast impact our impression of the data, but also the relative contrast of objects draws our initial visual attention to larger or brighter objects, which can cause objects such as small, dim nuclei to be overlooked (Fig. 3B). Perhaps less commonly appreciated is the impact of temporal contrast on our assessment of moving objects. For instance, varying the playback speed of a microscopy time-series will influence whether fast- or slow-moving objects will appear more prominent by changing the magnitude of the associated temporal changes. In all cases, visual prominence can steer attention away from important features and reinforce preconceived notions about a dataset. Fortunately, all major image-processing software packages contain tools to adjust both spatial and temporal contrast, providing an easy visual check on the biased perception of a set of images. For example, saturated display contrast might reveal previously unappreciated fine protrusions, while an increased playback speed could expose subtle morphogenic movements. Similarly, most software packages allow for the adjustment of gamma, which adjusts the contrast of dim and bright objects unequally and thus can reveal visual features that might be underappreciated with simpler contrast adjustments. However, non-linear adjustments such as gamma should always be used with care and appropriately described in figure legends and methods sections (Jonkman et al., 2020). Technical limitations, such as data loading (Pietzsch et al., 2015) or limited screen resolution (Healey and Sawant, 2012; Ni et al., 2006), might limit perceptual understanding of large datasets; in these cases, it is helpful to intentionally ask how initial visual interpretations fit into the larger context of the data. Exploration and hypothesis generation are essential to scientific progress; hypotheses generated by visual inspection should be made with awareness of visualization choices and followed by rigorous testing.

Visual patterns include not only the perception of isolated objects, but also the relationships between them. Distortions of this perception can lead to assumed relationships that do not exist, such as a visual assessment of ‘colocalization’ in a maximum intensity projection or due to channel bleed-through (Jost and Waters, 2019). Limited resolution might likewise indicate relationships that would be rejected if higher resolution information was available (Aaron et al., 2018; Jost and Waters, 2019). Similarly, proximity promotes an intuitive visual argument for object interaction but can be treacherously misleading. Consider the distribution of ER and endosomes within a cell (Fig. 3C). Visualizing the three-dimensional distribution of these organelles reveals many instances of endosomes in close proximity to the ER (white arrowheads, Fig. 3D). Tasked with testing the hypothesis that endosomes and ER interact, it would be tempting to use these examples of proximity as supporting evidence. However, actively countering bias requires considering alternative hypotheses (congruence bias, Table 1). In this case, an alternative hypothesis is that cytoplasmic crowding will naturally result in the occasional proximity of any given pair of organelles. Such a hypothesis can be tested by comparing the measured distances between the organelles to those distances that would be measured if the organelle locations were computationally randomized (Campanello et al., 2021). This check on bias leads to the realization that the observed endosomes are indeed further from the ER than would be expected by random chance (Fig. 3E), contradicting the visual intuition of Fig. 3D. Although this result does not exclude the possibility of interactions between the ER and endosomes, it does indicate that proximity alone is insufficient to make this claim. Complimentary metrics, such as measuring the coordinated motion of the organelles over time (Wu et al., 2018), the use of contact labels such as split-GFP (Kakimoto et al., 2018), or the use of biochemical pulldown assays (Guillén-Samander et al., 2019), would provide additional support for the initial imaging results.

This example is not isolated. Human perception frequently overestimates the significance of relationships in both space and time. A key illustration is our assessment of randomness. Evenly spaced objects are intuitively perceived as random despite underlying non-random relationships, while true randomness will inevitably include clusters of objects (Fig. 3F). Similarly, clusters of events in time instinctively seem meaningful, but can be the natural result of random chance (Gilovich et al., 1985). Quantitative approaches for testing randomness, such as the use of Ripley's K function (Khater et al., 2020), can be used to minimize this perceptual distortion. In other cases, illusory relationships might appear due to global trends in the studied biological system. For example, co-alignment of cytoskeletal components, such as vimentin and microtubules, might indicate a direct relationship, or co-alignment might reflect that both vimentin and microtubules respond to the overall polarity of the cell. Careful quantification can help to disentangle the local and global factors that contribute to an observed phenotype (Zaritsky et al., 2017) and thus distinguish between competing hypotheses. Perceptual distortions will always be present in our observation; however, testing alternative hypotheses and using quantitative tools can provide strong checks on our visual assumptions.

Although quantification can be a strong check on intuition, the act of quantification alone does not guarantee bias-free data interpretation. As with any part of a microscopy protocol, quantification should be designed to challenge, rather than support, a hypothesis. In addition, alternative hypotheses for observed results should always be considered. This is notably true when presented with automated results that support the hypothesis, where there is a strong temptation to consider the results complete without further scrutiny. As an illustration of this pitfall, consider an analysis protocol designed to test the hypothesis ‘drug X increases cell area’. Given a misplaced trust in the impartial nature of automated analysis, it is tempting to accept quantitative results in support of the hypothesis at face value (Fig. 4A). However, it is essential to compare quantitative results to the underlying data (Schmied et al., 2023). In this case (Fig. 4B), the results suggest an over two-fold increase in cell area, whereas the images themselves do not noticeably indicate any such changes. Even though the lack of difference in the images is clear, misplaced faith in automatic analysis combined with confirmation bias can make it easy to overlook even the blatantly obvious. Such a discrepancy should prompt investigation into the analysis approach, which in this case would reveal that touching cells were misinterpreted as single cells (see supplementary section).

Fig. 4.

Automatic image analysis is shaped by user bias. (A) An automated image analysis tool is used to test the hypothesis that ‘drug X increases cell area’, leading to mean area results that appear to support the hypothesis. (B) Inspection of the underlying images reveals no clear visual difference in cell area, and thus implies underlying issues with the analysis technique in A, which are easily overlooked given the ‘encouraging’ area results. Scale bars: 100 µm. The images in B are regions selected from the image in Fig. 2A. (C) Spinning disk confocal images of MEF cells labeled for talin reveal distinct focal adhesion morphologies (Nikon CSU-W1 microscope; Plan Apo λD 100×/1.45 NA oil objective; 561 nm laser illumination; see supplementary section for more details). Scale bars: 20 µm. (D) The number of focal adhesions that are less than 0.5 µm2 in area (left) indicate that there are more ‘immature’ focal adhesions in cell 1. Focal adhesions that meet the area criterium are highlighted in magenta (right). (E) The number of focal adhesions within 2 µm of the cell edge (left) indicate that there are more ‘immature’ focal adhesions in cell 2. Focal adhesions near the cell edge are highlighted in magenta (right). The choice of metrics for focal adhesion characterization provides an opening for bias; different metrics can lead to opposing results.

Fig. 4.

Automatic image analysis is shaped by user bias. (A) An automated image analysis tool is used to test the hypothesis that ‘drug X increases cell area’, leading to mean area results that appear to support the hypothesis. (B) Inspection of the underlying images reveals no clear visual difference in cell area, and thus implies underlying issues with the analysis technique in A, which are easily overlooked given the ‘encouraging’ area results. Scale bars: 100 µm. The images in B are regions selected from the image in Fig. 2A. (C) Spinning disk confocal images of MEF cells labeled for talin reveal distinct focal adhesion morphologies (Nikon CSU-W1 microscope; Plan Apo λD 100×/1.45 NA oil objective; 561 nm laser illumination; see supplementary section for more details). Scale bars: 20 µm. (D) The number of focal adhesions that are less than 0.5 µm2 in area (left) indicate that there are more ‘immature’ focal adhesions in cell 1. Focal adhesions that meet the area criterium are highlighted in magenta (right). (E) The number of focal adhesions within 2 µm of the cell edge (left) indicate that there are more ‘immature’ focal adhesions in cell 2. Focal adhesions near the cell edge are highlighted in magenta (right). The choice of metrics for focal adhesion characterization provides an opening for bias; different metrics can lead to opposing results.

Although all results should be viewed with a critical eye for potential discrepancies, confirmation bias (Table 1) pushes us to accept anticipated results at face value. Unexpected results, on the other hand, often prompt a round of troubleshooting. If the previous example featured the opposite hypothesis (e.g. drug X decreases cell area), it would be a natural response to consider alternative explanations for the results in Fig. 4A and thus further investigate the analysis details. Whether or not quantitative results support a hypothesis, any approach should be as rigorously designed and tested as any other step of a microscopy project. For example, visualizing the intermediate steps of analysis protocols (such as segmentation results in the previous example) are a key safeguard against bias, as well as unintentional computational errors.

The development of clear and testable hypotheses can further reduce bias introduced by preconceived expectations. Although broad concepts such as ‘dynamics’ or ‘microenvironment’ are biologically intuitive at a high level, they introduce ambiguity into an experiment (Wait et al., 2020). Specifying what it would mean to measure such concepts not only strengthens a quantitative approach, but also improves the development of the entire experiment. Consider the study of focal adhesion maturation, where quantification of immature versus mature focal adhesions is of interest. Given two cells with distinct morphologies (Fig. 4C), there are multiple viable approaches to characterize adhesions, and thus the careful definition of ‘immature’ versus ‘mature’ becomes an important factor for data interpretation. Focal adhesions are commonly assessed by their size (Dugina et al., 2001; Fokkelman et al., 2016), and, in this case, a size-based analysis finds that Cell 1 contains more immature adhesions (Fig. 4D). However, adhesions also are characterized by their distance from the cell edge (Parsons et al., 2010) and quantification of this property leads to the opposing conclusion that Cell 2 contains more immature adhesions (Fig. 4E). Although both metrics might be biologically justifiable and were carefully quantified, neither measure alone provides enough context to fully understand the focal adhesion phenotype in these cells. Immature adhesions tend to be smaller than large mature adhesions, but small objects could also indicate disassembling adhesions, especially when small adhesions are found away from the leading edge (Parsons et al., 2010). Similarly, although immature adhesions tend to form near the leading edge, mature adhesions can be found here as well, such as the those seen in the protrusions of Cell 2 (Fig. 4E, magenta adhesions). In neither case is the quantification of these features ‘incorrect’, but biased assumptions about an experimental system or academic pressure to support a hypothesis could easily push a researcher to select one metric rather than the other, at the expense of better understanding of the biological system.

Metrics should be chosen not because they give the expected results, but rather because they best reveal a feature of the biological system that can test the hypothesis. For the previous example, a combination of both focal adhesion size and location can provide a richer and more informative set of results than either value alone, a realization that is reflected in a wealth of multiparametric focal adhesion literature (Berginski et al., 2011; Kumari et al., 2023 preprint; Winograd-Katz et al., 2009). The process of focal adhesion maturation reflects changes over time, and thus complementary experimental approaches such as live-cell timelapse imaging can also provide a more nuanced view of focal adhesion classification. Previously published metrics (e.g. adhesion size) can provide a solid starting point for understanding an experimental system but must always be put into context. Data interpretation is a key entry point for bias, and assumptions must be actively checked when interpreting quantified results, regardless of how accurately the quantification was implemented.

Although each of the examples described above relied on automated analysis approaches (i.e. they did not require manual annotation or counting of objects by the user), observer bias was easily incorporated into each method. Here, we should note that the examples in Fig. 4B do not represent distinct experimental conditions, but rather are regions deliberately pulled from the image shown in Fig. 2A to illustrate that it is disturbingly easy to manipulate analysis results when focused on a specific goal. Similarly, the images in Fig. 4C are two identically treated cells imaged under the same conditions that were chosen to illustrate adhesion variability. When implemented subconsciously, this type of selection bias (Table 1) towards supporting, rather than rigorously testing, a hypothesis can completely undermine the validity of quantitative results. Mitigating quantitative bias requires judicious consideration of the chosen analysis approach along with validation methods appropriate to both the biological question and computational tool at hand (Chen et al., 2023; Lambert and Waters, 2023). Application-specific safeguards for common image processing tasks are available in several useful reviews, including discussions of object segmentation, colocalization (Aaron et al., 2018) and particle tracking (Aaron et al., 2019). Similarly, accounting for figure design (Lord et al., 2020) and statistical best practices (Bishop, 2020; Curran-Everett, 2013; Makin and Orban de Xivry, 2019; Nuzzo, 2014; Wasserstein and Lazar, 2016) both strengthens experimental design and helps counteract biases in quantification, although it should be noted that misuse of statistical principles can add to, rather than diminish bias.

The many ways in which preconceptions can impact a microscopy project (Table 1) highlight the importance of explicitly addressing bias. An experiment cannot be fully disentangled from the experimenter, and good intentions alone cannot overcome the predisposition to see what is already believed. Although quantification is a powerful and necessary check on initial perceptions, like any tool, it can quickly become a double-edged sword. It is thus the responsibility of every researcher to purposely design experiments with built-in safeguards against bias and to always test alternative hypotheses for observed results.

The power of modern microscopy allows us to visualize a multitude of living systems across a wide range of spatial and temporal scales. And yet, even visually stunning datasets are limited in their impact without the ability to interpret those results in context. The presence of bias in science, or more specifically in microscopy, is not a new concept (Jost and Waters, 2019; Munafò et al., 2017). However, the diversity and subtlety of bias in microscopy experiments is often underappreciated. The types of bias discussed in this article are commonly encountered in many real-life research scenarios. Rather than deliberate misconduct, those situations are more commonly the result of well-intentioned researchers led astray by steadfast adherence to their hypothesis or misplaced faith in the absolute veracity of image quantification. It thus bears emphasizing that bias must be (1) actively acknowledged and (2) rigorously guarded against at all stages of a microscopy project.

Acknowledging bias is universally important but it is especially essential with concepts that are familiar. Although any microscopist will acknowledge that no single experiment can reveal all features of a specimen, it is just as easy to forget this critical context when interpreting imaging results. Similarly, although most researchers are aware of the importance of carefully selecting cells or regions of interest, tempting yet poorly supported extrapolations from a single ‘representative’ cell can quickly steer an experiment away from informative results. Indeed, data interpretation is the ultimate stage at which the cumulative effects of even subtle bias can lead to significantly distorted conclusions.

The cumulative impact of bias makes it all the more essential to emphasize that we cannot always trust our own visual perception when interpreting images. The bulk of life science research ultimately relies on observation (Wait et al., 2020), yet the risks posed by perceptual biases are not only pervasive, but insidious. Purposeful discipline is required to confront such distortions in our observations. Perceptual biases can often be alleviated with image analysis, but even reproducible quantification can equally perpetuate or even escalate bias if not implemented with care. The false sense of security provided by automated quantification must never substitute for rigorous scientific reasoning. The illustrative examples considered here cannot fully capture the multitude of ways in which bias can influence microscopy, but the solutions we discuss are broadly applicable to any experiment. Persistent vigilance in designing robust tests of hypotheses, including testing alternative explanations, is key to supporting impactful research results.

Although addressing biases at the earliest stages of hypothesis formulation and experimental design is crucial, the scientific enterprise also relies on peer evaluation to identify and challenge biased conclusions. However, this check on biases is only effective if the wider scientific community has access to complete information about the underlying experiments, especially the types of methodological details that are often omitted from biomedical research publications (Marques et al., 2020). Careful reporting of experimental details (Heddleston et al., 2021; Montero Llopis et al., 2021) and downstream analysis (Aaron and Chew, 2021; Schmied et al., 2023) are key to confronting bias. The growing movement to share imaging data in appropriate repositories can also support the assessment of research results (Linkert et al., 2010; Swedlow et al., 2021; Williams et al., 2017). Commensurate with this trend is the community effort to advocate for the disclosure of full metadata along with bioimages (Boehm et al., 2021). Similarly, various forms of preregistered studies can be used to create guardrails around the choices an observer makes (Kupferschmidt, 2018). Such open science publishing practices allow sunlight (i.e. public scrutiny) to be a disinfectant for bias.

Transparent open science practices are vital for scientific reproducibility. However, unthinking replication is more likely to propagate than eliminate the deeply concealed bias in the original work. The oft-repeated refrain of ‘seeing is believing’ notwithstanding, the notion of ‘belief’ should never be associated with scientific practice. It shackles scientists with tunnel vision rather than freeing their curiosity to drive discovery. No amount of quantification and transparency can replace scientific rigor. Put simply, the veneer of authority provided by undiscerning use of methodological best practices is never a substitute for critical thinking.

We gratefully acknowledge the shared resources and project teams at Howard Hughes Medical Institute Janelia Research Campus. Widefield and confocal imaging experiments were conducted at the Janelia Integrative Imaging Facility with support from Damien Alcor and Michael DeSantis. We thank Usha Kadiyala for sharing her zebrafish imaging data. We also extend our gratitude to Chad Hobson for his comments and imaging suggestions.

Funding

The Advanced Imaging Center at Janelia Research Campus is generously supported by the Howard Hughes Medical Institute and the Gordon and Betty Moore Foundation.

Data availability

New imaging data and custom analysis scripts described are available at doi:10.25378/janelia.c.6954477.

Aaron
,
J.
and
Chew
,
T.-L.
(
2021
).
A guide to accurate reporting in digital image processing – can anyone reproduce your quantitative analysis?
J. Cell Sci.
134
,
jcs254151
.
Aaron
,
J. S.
,
Taylor
,
A. B.
and
Chew
,
T.
(
2018
).
Image co-localization – co-occurrence versus correlation
.
J. Cell Sci.
131
,
jcs211847
.
Aaron
,
J.
,
Wait
,
E.
,
DeSantis
,
M.
and
Chew
,
T.-L.
(
2019
).
Practical considerations in particle and object tracking and analysis
.
Curr. Protoc. Cell Biol.
83
,
e88
.
Arnold
,
V.
,
Collier
,
P. A.
,
Leech
,
S. A.
and
Sutton
,
S. G.
(
2000
).
The effect of experience and complexity on order and recency bias in decision making by professional accountants
.
Account. Finance
40
,
109
-
134
.
Balasubramanian
,
H.
,
Hobson
,
C. M.
,
Chew
,
T.-L.
and
Aaron
,
J. S.
(
2023
).
Imagining the future of optical microscopy: everything, everywhere, all at once
.
Commun. Biol.
6
,
1096
.
Belin
,
B. J.
,
Goins
,
L. M.
and
Mullins
,
R. D.
(
2014
).
Comparative analysis of tools for live cell imaging of actin network architecture
.
BioArchitecture
4
,
189
-
202
.
Berginski
,
M. E.
,
Vitriol
,
E. A.
,
Hahn
,
K. M.
and
Gomez
,
S. M.
(
2011
).
High-resolution quantification of focal adhesion spatiotemporal dynamics in living cells
.
PLoS ONE
6
,
e22025
.
Bishop
,
D.
(
2020
).
How scientists can stop fooling themselves over statistics
.
Nature
584
,
9
-
9
.
Boehm
,
U.
,
Nelson
,
G.
,
Brown
,
C. M.
,
Bagley
,
S.
,
Bajcsy
,
P.
,
Bischof
,
J.
,
Dauphin
,
A.
,
Dobbie
,
I. M.
,
Eriksson
,
J. E.
,
Faklaris
,
O.
et al. 
(
2021
).
QUAREP-LiMi: a community endeavor to advance quality assessment and reproducibility in light microscopy
.
Nat. Methods
18
,
1423
-
1426
.
Bos
,
L. D. J.
,
Sinha
,
P.
and
Dickson
,
R. P.
(
2020
).
The perils of premature phenotyping in COVID-19: a call for caution
.
Eur. Respir. J.
56
,
2001768
.
Campanello
,
L.
,
Traver
,
M. K.
,
Shroff
,
H.
,
Schaefer
,
B. C.
and
Losert
,
W.
(
2021
).
Signaling through polymerization and degradation: Analysis and simulations of T cell activation mediated by Bcl10
.
PLoS Comput. Biol.
17
,
e1007986
.
Chapman
,
L. J.
(
1967
).
Illusory correlation in observational report
.
J. Verbal Learn. Verbal Behav.
6
,
151
-
155
.
Chen
,
J.
,
Viana
,
M. P.
and
Rafelski
,
S. M.
(
2023
).
When seeing is not believing: application-appropriate validation matters for quantitative bioimage analysis
.
Nat. Methods
20
,
968
-
970
.
Chew
,
T.-L.
,
Wolf
,
W. A.
,
Gallagher
,
P. J.
,
Matsumura
,
F.
and
Chisholm
,
R. L.
(
2002
).
A fluorescent resonant energy transfer–based biosensor reveals transient and regional myosin light chain kinase activation in lamella and cleavage furrows
.
J. Cell Biol.
156
,
543
-
553
.
Curran-Everett
,
D.
(
2013
).
Explorations in statistics: the analysis of ratios and normalized data
.
Adv. Physiol. Educ.
37
,
213
-
219
.
Dehaene
,
S.
(
2003
).
The neural basis of the Weber–Fechner law: a logarithmic mental number line
.
Trends Cogn. Sci.
7
,
145
-
147
.
Dittmayer
,
C.
,
Meinhardt
,
J.
,
Radbruch
,
H.
,
Radke
,
J.
,
Heppner
,
B. I.
,
Heppner
,
F. L.
,
Stenzel
,
W.
,
Holland
,
G.
and
Laue
,
M.
(
2020
).
Why misinterpretation of electron micrographs in SARS-CoV-2-infected tissue goes viral
.
The Lancet
396
,
e64
-
e65
.
Dugina
,
V.
,
Fontao
,
L.
,
Chaponnier
,
C.
,
Vasiliev
,
J.
and
Gabbiani
,
G.
(
2001
).
Focal adhesion features during myofibroblastic differentiation are controlled by intracellular and extracellular factors
.
J. Cell Sci.
114
,
3285
-
3296
.
Ellenberg
,
J. H.
(
1994
).
Selection bias in observational and experimental studies
.
Stat. Med.
13
,
557
-
567
.
Fokkelman
,
M.
,
Balcıoğlu
,
H. E.
,
Klip
,
J. E.
,
Yan
,
K.
,
Verbeek
,
F. J.
,
Danen
,
E. H. J.
and
van de Water
,
B.
(
2016
).
Cellular adhesome screen identifies critical modulators of focal adhesion dynamics, cellular traction forces and cell migration behaviour
.
Sci. Rep.
6
,
31707
.
Foye
,
P. M.
,
Koger
,
T. J.
and
Massey
,
H. R.
(
2021
).
Baby yoda: pareidolia and patternicity in sacral MRI and CT scans
.
PM R.
13
,
217
-
218
.
Gilovich
,
T.
,
Vallone
,
R.
and
Tversky
,
A.
(
1985
).
The hot hand in basketball: On the misperception of random sequences
.
Cognit. Psychol.
17
,
295
-
314
.
Guillén-Samander
,
A.
,
Bian
,
X.
and
De Camilli
,
P.
(
2019
).
PDZD8 mediates a Rab7-dependent interaction of the ER with late endosomes and lysosomes
.
Proc. Natl Acad. Sci. USA
116
,
22619
-
22623
.
Healey
,
C. G.
and
Sawant
,
A. P.
(
2012
).
On the limits of resolution and visual angle in visualization
.
ACM Trans. Appl. Percept
9
,
1
-
20
.
Heddleston
,
J. M.
,
Aaron
,
J. S.
,
Khuon
,
S.
and
Chew
,
T.-L.
(
2021
).
A guide to accurate reporting in digital image acquisition – can anyone replicate your microscopy data?
J. Cell Sci.
134
,
jcs254144
.
Hickey
,
S. M.
,
Ung
,
B.
,
Bader
,
C.
,
Brooks
,
R.
,
Lazniewska
,
J.
,
Johnson
,
I. R. D.
,
Sorvina
,
A.
,
Logan
,
J.
,
Martini
,
C.
,
Moore
,
C. R.
et al. 
(
2022
).
Fluorescence microscopy—An outline of hardware, biological handling, and fluorophore considerations
.
Cells
11
,
35
.
Hoekstra
,
R.
and
Vazire
,
S.
(
2021
).
Aspiring to greater intellectual humility in science
.
Nat. Hum. Behav.
5
,
1602
-
1607
.
Holman
,
L.
,
Head
,
M. L.
,
Lanfear
,
R.
and
Jennions
,
M. D.
(
2015
).
Evidence of experimental bias in the life sciences: why we need blind data recording
.
PLoS Biol.
13
,
e1002190
.
Huang
,
Q.
,
Garrett
,
A.
,
Bose
,
S.
,
Blocker
,
S.
,
Rios
,
A. C.
,
Clevers
,
H.
and
Shen
,
X.
(
2021
).
The frontier of live tissue imaging across space and time
.
Cell Stem Cell
28
,
603
-
622
.
Imreh
,
G.
,
Hu
,
J.
and
Le Guyader
,
S.
(
2023
).
Improving light microscopy training routines with evidence-based education
.
J. Microsc
.
Jaqaman
,
K.
,
Loerke
,
D.
,
Mettlen
,
M.
,
Kuwata
,
H.
,
Grinstein
,
S.
,
Schmid
,
S. L.
and
Danuser
,
G.
(
2008
).
Robust single-particle tracking in live-cell time-lapse sequences
.
Nat. Methods
5
,
695
-
702
.
Jonkman
,
J.
,
Brown
,
C. M.
,
Wright
,
G. D.
,
Anderson
,
K. I.
and
North
,
A. J.
(
2020
).
Tutorial: guidance for quantitative confocal microscopy
.
Nat. Protoc.
15
,
1585
-
1611
.
Joober
,
R.
,
Schmitz
,
N.
,
Annable
,
L.
and
Boksa
,
P.
(
2012
).
Publication bias: What are the challenges and can they be overcome?
J. Psychiatry Neurosci.
37
,
149
-
152
.
Jost
,
A. P.-T.
and
Waters
,
J. C.
(
2019
).
Designing a rigorous microscopy experiment: Validating methods and avoiding bias
.
J. Cell Biol.
218
,
1452
-
1466
.
Kaas
,
J. H.
and
Balaram
,
P.
(
2014
).
Current research on the organization and function of the visual system in primates
.
Eye Brain
6
,
1
-
4
.
Kahneman
,
D.
(
2011
).
Thinking, Fast and Slow
, 1st edn
New York
:
Farrar, Straus and Giroux
.
Kakimoto
,
Y.
,
Tashiro
,
S.
,
Kojima
,
R.
,
Morozumi
,
Y.
,
Endo
,
T.
and
Tamura
,
Y.
(
2018
).
Visualizing multiple inter-organelle contact sites using the organelle-targeted split-GFP system
.
Sci. Rep.
8
,
6175
.
Khater
,
I. M.
,
Nabi
,
I. R.
and
Hamarneh
,
G.
(
2020
).
A review of super-resolution single-molecule localization microscopy cluster analysis and quantification methods
.
Patterns
1
,
100038
.
Kovesi
,
P
. (
2015
).
Good Colour Maps: How to Design Them. arXiv
1509.03700
.
Krzywinski
,
M.
and
Altman
,
N.
(
2013
).
Points of significance: Power and sample size
.
Nat. Methods
10
,
1139
-
1140
.
Kumari
,
R.
,
Ven
,
K.
,
Chastney
,
M.
,
Peranen
,
J.
,
Aaron
,
J.
,
Almeida-Souza
,
L.
,
Kremneva
,
E.
,
Poincloux
,
R.
,
Chew
,
T. L.
,
Gunning
,
P. W.
et al. 
. (
2023
).
Specialized actin nanoscale layers control focal adhesion turnover
.
bioRxiv
2023.02.15.528622
.
Kupferschmidt
,
K.
(
2018
).
A recipe for rigor
.
Science
361
,
1192
-
1193
.
Lambert
,
T.
and
Waters
,
J.
(
2023
).
Towards effective adoption of novel image analysis methods
.
Nat. Methods
20
,
971
-
972
.
Lee
,
J.-Y.
,
Kitaoka
,
M.
and
Drubin
,
D. G.
(
2018
).
A beginner's guide to rigor and reproducibility in fluorescence imaging experiments
.
MBoC
29
,
1519
-
1525
.
Linkert
,
M.
,
Rueden
,
C. T.
,
Allan
,
C.
,
Burel
,
J.-M.
,
Moore
,
W.
,
Patterson
,
A.
,
Loranger
,
B.
,
Moore
,
J.
,
Neves
,
C.
,
MacDonald
,
D.
et al. 
(
2010
).
Metadata matters: access to image data in the real world
.
J. Cell Biol.
189
,
777
-
782
.
Lord
,
S. J.
,
Velle
,
K. B.
,
Mullins
,
R. D.
and
Fritz-Laylin
,
L. K.
(
2020
).
SuperPlots: Communicating reproducibility and variability in cell biology
.
J. Cell Biol.
219
,
e202001064
.
Luckhoff
,
C
. (
2021
).
Congruence bias
. In
Decision Making in Emergency Medicine: Biases, Errors and Solutions
(ed.
M.
Raz
and
P.
Pouryahya
), pp.
89
-
96
.
Singapore
:
Springer
.
MacCoun
,
R.
and
Perlmutter
,
S.
(
2015
).
Blind analysis: Hide results to seek the truth
.
Nature
526
,
187
-
189
.
Makin
,
T. R.
and
Orban de Xivry
,
J.-J.
(
2019
).
Science Forum: Ten common statistical mistakes to watch out for when writing or reviewing a manuscript
.
eLife
8
,
e48175
.
Markey
,
M. K.
,
Boland
,
M. V.
and
Murphy
,
R. F.
(
1999
).
Toward objective selection of representative microscope images
.
Biophys. J.
76
,
2230
-
2237
.
Marques
,
G.
,
Pengo
,
T.
and
Sanders
,
M. A.
(
2020
).
Science Forum: Imaging methods are vastly underreported in biomedical research
.
eLife
9
,
e55133
.
Melak
,
M.
,
Plessner
,
M.
and
Grosse
,
R.
(
2017
).
Actin visualization at a glance
.
J. Cell Sci.
130
,
525
-
530
.
Montero Llopis
,
P.
,
Senft
,
R. A.
,
Ross-Elliott
,
T. J.
,
Stephansky
,
R.
,
Keeley
,
D. P.
,
Koshar
,
P.
,
Marqués
,
G.
,
Gao
,
Y.-S.
,
Carlson
,
B. R.
,
Pengo
,
T.
et al. 
(
2021
).
Best practices and tools for reporting reproducible fluorescence microscopy methods
.
Nat. Methods
18
,
1463
-
1476
.
Munafò
,
M. R.
,
Nosek
,
B. A.
,
Bishop
,
D. V. M.
,
Button
,
K. S.
,
Chambers
,
C. D.
,
du Sert
,
P.
,
Simonsohn
,
N.
,
Wagenmakers
,
U.
,
Ware
,
E.-J.
,
and Ioannidis
,
J. J.
et al. 
(
2017
).
A manifesto for reproducible science
.
Nat. Hum. Behav.
1
,
0021
.
Mynatt
,
C. R.
,
Doherty
,
M. E.
and
Tweney
,
R. D.
(
1977
).
Confirmation bias in a simulated research environment: an experimental study of scientific inference
.
Q J. Exp. Psychol. (Colchester)
29
,
85
-
95
.
Ni
,
T.
,
Bowman
,
D. A.
and
Chen
,
J
. (
2006
).
Increased display size and resolution improve task performance in information-rich virtual environments
. In
Information-Rich Virtual Environments. Graphics Interface (GI) 2006
, pp.
139
-
146
.
Canadian Information Processing Society
.
North
,
A. J.
(
2006
).
Seeing is believing? A beginners’ guide to practical pitfalls in image acquisition
.
J. Cell Biol.
172
,
9
-
18
.
Nuzzo
,
R.
(
2014
).
Statistical errors
.
Nature
506
,
150
-
152
.
Nuzzo
,
R.
(
2015
).
How scientists fool themselves – and how they can stop
.
Nature News
526
,
182
.
Parsons
,
J. T.
,
Horwitz
,
A. R.
and
Schwartz
,
M. A.
(
2010
).
Cell adhesion: integrating cytoskeletal dynamics and cellular tension
.
Nat. Rev. Mol. Cell Biol.
11
,
633
-
643
.
Peddie
,
C. J.
,
Genoud
,
C.
,
Kreshuk
,
A.
,
Meechan
,
K.
,
Micheva
,
K. D.
,
Narayan
,
K.
,
Pape
,
C.
,
Parton
,
R. G.
,
Schieber
,
N. L.
,
Schwab
,
Y.
et al. 
(
2022
).
Volume electron microscopy
.
Nat. Rev. Methods Primers
2
,
51
.
Pietzsch
,
T.
,
Saalfeld
,
S.
,
Preibisch
,
S.
and
Tomancak
,
P.
(
2015
).
BigDataViewer: visualization and processing for large image data sets
.
Nat. Methods
12
,
481
-
483
.
Platt
,
J. R.
(
1964
).
Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others
.
Science
146
,
347
-
353
.
Pollard
,
D. A.
,
Pollard
,
T. D.
and
Pollard
,
K. S.
(
2019
).
Empowering statistical methods for cellular and molecular biologists
.
MBoC
30
,
1359
-
1368
.
Reiche
,
M. A.
,
Aaron
,
J. S.
,
Boehm
,
U.
,
DeSantis
,
M. C.
,
Hobson
,
C. M.
,
Khuon
,
S.
,
Lee
,
R. M.
and
Chew
,
T.-L.
(
2022
).
When light meets biology – how the specimen affects quantitative microscopy
.
J. Cell Sci.
135
,
jcs259656
.
Riedl
,
J.
,
Crevenna
,
A. H.
,
Kessenbrock
,
K.
,
Yu
,
J. H.
,
Neukirchen
,
D.
,
Bista
,
M.
,
Bradke
,
F.
,
Jenne
,
D.
,
Holak
,
T. A.
,
Werb
,
Z.
et al. 
(
2008
).
Lifeact: a versatile marker to visualize F-actin
.
Nat. Methods
5
,
605
-
607
.
Sagan
,
C.
(
1995
).
The Demon-Haunted World: Science as a Candle in the Dark
.
New York
:
Random House
.
Schmied
,
C.
,
Nelson
,
M. S.
,
Avilov
,
S.
,
Bakker
,
G.-J.
,
Bertocchi
,
C.
,
Bischof
,
J.
,
Boehm
,
U.
,
Brocher
,
J.
,
Carvalho
,
M. T.
,
Chiritescu
,
C.
et al. 
(
2023
).
Community-developed checklists for publishing images and image analyses
.
Nat. Methods
.
Swedlow
,
J. R.
,
Kankaanpää
,
P.
,
Sarkans
,
U.
,
Goscinski
,
W.
,
Galloway
,
G.
,
Malacrida
,
L.
,
Sullivan
,
R. P.
,
Härtel
,
S.
,
Brown
,
C. M.
,
Wood
,
C.
et al. 
(
2021
).
A global view of standards for open image data formats and repositories
.
Nat. Methods
18
,
1440
-
1446
.
Taylor
,
A. B.
,
Ioannou
,
M. S.
,
Watanabe
,
T.
,
Hahn
,
K.
and
Chew
,
T.-L.
(
2017
).
Perceptually accurate display of two greyscale images as a single colour image
.
J. Microsc.
268
,
73
-
83
.
Treisman
,
A
. (
2002
).
Chapter 16: features and objects in visual processing
. In
Foundations of Cognitive Psychology: Core Readings
(ed.
D. J.
Levitin
), pp.
399
-
413
.
Cambridge, Mass
:
A Bradford Book
.
Voss
,
J. L.
,
Federmeier
,
K. D.
and
Paller
,
K. A.
(
2012
).
The potato chip really does look like Elvis! neural hallmarks of conceptual processing associated with finding novel shapes subjectively meaningful
.
Cereb. Cortex
22
,
2354
-
2364
.
Wait
,
E. C.
,
Reiche
,
M. A.
and
Chew
,
T.-L.
(
2020
).
Hypothesis-driven quantitative fluorescence microscopy – the importance of reverse-thinking in experimental design
.
J. Cell Sci.
133
,
jcs250027
.
Wang
,
Y.
,
Eddison
,
M.
,
Fleishman
,
G.
,
Weigert
,
M.
,
Xu
,
S.
,
Wang
,
T.
,
Rokicki
,
K.
,
Goina
,
C.
,
Henry
,
F. E.
,
Lemire
,
A. L.
et al. 
(
2021
).
EASI-FISH for thick tissue defines lateral hypothalamus spatio-molecular organization
.
Cell
184
,
6361
-
6377.e24
.
Wason
,
P. C.
(
1960
).
On the failure to eliminate hypotheses in a conceptual task
.
Q J. Exp. Psychol. (Colchester)
12
,
129
-
140
.
Wasserstein
,
R. L.
and
Lazar
,
N. A.
(
2016
).
The ASA's statement on p-values: context, process, and purpose
.
Am. Stat.
70
,
129
-
133
.
Waters
,
J. C.
and
Swedlow
,
J. R
. (
2008
).
Interpreting fluorescence microscopy images and measurements
. In
Evaluating Techniques in Biochemical Research
, pp.
37
-
42
.
Cell Press
.
White
,
M.
(
1979
).
A new effect of pattern on perceived lightness
.
Perception
8
,
413
-
416
.
Williams
,
E.
,
Moore
,
J.
,
Li
,
S. W.
,
Rustici
,
G.
,
Tarkowska
,
A.
,
Chessel
,
A.
,
Leo
,
S.
,
Antal
,
B.
,
Ferguson
,
R. K.
,
Sarkans
,
U.
et al. 
(
2017
).
Image Data Resource: a bioimage data integration and publication platform
.
Nat. Methods
14
,
775
-
781
.
Winograd-Katz
,
S. E.
,
Itzkovitz
,
S.
,
Kam
,
Z.
and
Geiger
,
B.
(
2009
).
Multiparametric analysis of focal adhesion formation by RNAi-mediated gene knockdown
.
J. Cell Biol.
186
,
423
-
436
.
Wong
,
B.
(
2010
).
Points of view: color coding
.
Nat. Methods
7
,
573
-
573
.
Wu
,
H.
,
Carvalho
,
P.
and
Voeltz
,
G. K.
(
2018
).
Here, there, and everywhere: the importance of ER membrane contact sites
.
Science
361
,
eaan5835
.
Xu
,
C. S.
,
Pang
,
S.
,
Hayworth
,
K. J.
and
Hess
,
H. F
. (
2020
).
Transforming FIB-SEM systems for large-volume connectomics
. In
Volume Microscopy: Multiscale Imaging with Photons, Electrons, and Ions
(eds
I.
Wacker
,
E.
Hummel
,
S.
Burgold
and
R.
Schröder
), pp.
221
-
243
.
New York, NY
:
Springer US
.
Zaritsky
,
A.
,
Obolski
,
U.
,
Gan
,
Z.
,
Reis
,
C. R.
,
Kadlecova
,
Z.
,
Du
,
Y.
,
Schmid
,
S. L.
and
Danuser
,
G.
(
2017
).
Decoupling global biases and local interactions between cell biological variables
.
eLife
6
,
e22323
.

Competing interests

The authors declare no competing or financial interests.

Supplementary information