Estimating maximum performance: effects of intraindividual variation

Locomotor performance is a central measure of whole-organism function, and is considered a key link between fitness in the natural environment and underlying biochemical, morphological and physiological traits(Huey and Stevenson, 1979; Arnold, 1983; Arnold and Bennett, 1988; Garland and Losos, 1995; Norberg, 1995). Consequently,hundreds of studies have measured burst speed, endurance, jumping distance and other aspects of locomotor performance in the laboratory(Garland, 1994; Wainwright and Reilly, 1994; Garland and Losos,1995; Alexander,2002). Laboratory measurements of locomotor performance are used for diverse purposes, ranging from identifying its mechanistic basis (e.g. Garland, 1984; Gleeson and Harrison, 1988; Miles et al., 1995; Bonine and Garland, 1999) to examining the fitness consequences of among-individual variation in performance (Arnold and Bennett,1988; Jayne and Bennett,1990; Watkins,1996; Le Galliard et al.,2004; Miles, 2004; Husak et al., 2006; Peterson and Husak, 2006; Irschick and Meyers,2007).

Whereas the goals of locomotion studies are diverse, laboratory procedures often share a common feature: maximum performance [less commonly, mean performance (Jayne and Bennett,1990)] is estimated using a relatively small number of trials per individual. Researchers have long recognized that the performance of individual animals varies from one laboratory trial to the next. The presence of intraindividual variation, coupled with relatively small sample sizes per individual, guarantees that each individual's performance is estimated with some error, whether individual maximum or mean values are used for statistical analysis. In particular, maximum performance will always be underestimated, as individuals will rarely achieve their true maximum in a small number of laboratory trials. This problem was highlighted by Losos et al.(Losos et al., 2002), who also described the related problem of individual subjects who consistently perform submaximally in the laboratory.

In this study we examined how intraindividual variation and per individual sample size affect the statistical estimation of performance. To obtain an example data set we measured burst speed performance in western fence lizards(Sceloporus occidentalis) 20 times per individual at 20°C and 35°C. We describe the intra- and interindividual statistical distributions of performance, and estimate overall repeatability and whether it varies over time. We then ran statistical sampling experiments that simulated laboratory studies in which speed was measured 1, 2, 3,... 20 times for each individual,to evaluate how the accuracy and precision of performance estimates vary with per individual sample size. This analysis addressed several goals.

1. To quantify how observed maximum performance increases as a function of the number of trials per individual.

2. To determine the effect of within-individual variability on the above relationship, by performing the analysis on data sets adjusted to have increased and decreased variability.

3. To determine the effect of within-individual variability and number of trials on the estimated correlation between two traits. Although our study used speed measured at two different temperatures as the two traits, this analysis is more broadly applicable to any correlational analysis of two traits, such as whether performance covaries with a morphological or biochemical trait.

The western fence lizard (Sceloporus occidentalis Baird and Girard) is a small diurnal insectivorous lizard that lives in a wide variety of habitats in the western United States(Stebbins, 2003). It is a classic sit-and-wait predator with limited endurance capacity but can run swiftly over short distances (Bennett and Gleeson, 1976; Gleeson,1979; Schall and Sarni,1987). Locomotor performance of S. occidentalis has been studied in diverse contexts (e.g. Bennett,1980; Bennett and Gleeson,1976; Garland et al.,1990; Gleeson,1979; Holem et al.,2006; Marsh and Bennett,1986; Schall et al.,1982; Sinervo and Adolph,1989; Sinervo and Huey,1990; Sinervo et al.,1991; Sinervo and Losos,1991; Tsuji et al.,1989; van Berkum,1988; van Berkum and Tsuji,1987; van Berkum et al.,1989).

We collected adult and subadult male lizards (N=21) from two sites in Los Angeles County, California, in June 2004: Table Mountain (2 km northwest of Wrightwood) and Joshua (8 km east of Valyermo). Adolph(Adolph, 1990) and Sinervo and Adolph (Sinervo and Adolph,1994) provide further information on the ecology of S. occidentalis at these sites. Lizard body mass averaged 10.1 g (range 5.6–15.2 g) and snout–vent length averaged 68.1 mm (range 60–79 mm). Lizards were held in the laboratory individually in 38 l terraria with an incandescent light bulb (75 W) on 8 h per day. The air temperature of the room averaged 20.5°C at night, and during the day the light permitted the lizards to attain their preferred body temperature of 35°C. Lizards were fed crickets two to four times per week.

We measured sprint speed following the procedures of Hertz et al.(Hertz et al., 1983). Prior to each run, lizards were held individually in 1 l plastic containers within a constant-temperature chamber at either 35°C or 20°C for at least 1 h;35°C is the optimal temperature for sprint locomotion in this species, and is approximately the mean body temperature of field-active animals, whereas 20°C is at the lower end of the field body temperature distribution for these populations (Bennett,1980; Marsh and Bennett,1986; van Berkum,1988; Adolph,1990). We removed each lizard from the chamber and chased it along a horizontal racetrack (2.5 m long×28 cm wide) that had a rough particle board surface. Photocells spaced every 0.25 m were connected to a computer that recorded elapsed times (Huey et al.,1981; Hertz et al.,1983). We gave each lizard five training runs several days prior to the experiment (Bennett,1980). Trials were conducted between 10:00 h and 17:00 h. Each lizard ran five trials per day, with at least 1 h rest in the environmental chamber between trials. The fastest 0.75 m interval was used as the lizard's speed for a given trial. Lizards were run in haphazard order for each trial on a given day. We weighed and measured (snout–vent length) each lizard on the first day of racing. Lizards were given 1–2 days of rest between each set of five trials. All trials at 35°C were run first, followed by the trials at 20°C. On one trial day at 35°C, three subjects were run six times to replace data lost from previous runs.

We assessed the repeatability of sprint performance in several ways. Overall repeatability of speed within each of the two temperatures was examined by estimating intraclass correlation coefficients(r_i) following Haggard(Haggard, 1958) (see also Sokal and Rohlf, 1981; Lessells and Boag, 1987). Standard errors for intraclass correlation coefficients were calculated following Becker (Becker,1984). We also calculated pairwise Pearson product-moment correlations between mean, median, maximum and minimum speed both within and across temperatures as additional measures of individual consistency and repeatability.

We tested for temporal dependence of repeatability within each temperature by calculating all pairwise correlations r_ij between speeds on trials i and j, where i≠j and both i and j range from 1 to 20. We tested the hypothesis that the magnitude of r_ij should decrease with the separation between trials (|i–j|); i.e. trials that immediately follow one another should have more similar speeds than widely spaced trials (T. Garland, Jr, personal communication). We used Mantel tests (Mantel, 1967) to assess the statistical significance of the relationship between pairwise correlation of speeds and the spacing between trials because the number of pairwise correlations (190) exceeded the number of independent trials (20). We wrote a Matlab program using Manly's(Manly, 1986) algorithm to perform the Mantel tests.

To determine the effects of sample size (number of trials per individual)on estimates of performance parameters, we performed a sampling experiment in which we randomly chose N_trials speed measurements from each individual (without replacement), where N_trialsranged from 1 to 20. We chose the maximum and mean speeds from the random sample for each individual as performance estimates. This procedure was repeated 1000 times for each value of N_trials at each temperature, using programs written in Matlab. This sampling experiment allowed us to quantify the effect of N_trials on the precision and accuracy of estimating maximum speed, mean speed and the correlation between speeds at the two temperatures.

We used the same procedure to evaluate how the magnitude of intraindividual variation influences the sampling distribution of sprint speed statistics. To do this, we modified the data set by multiplying each individual's sprint speed residuals by a constant factor and adding these rescaled residuals back to the individual's mean sprint speed, thereby changing the within-individual variance but not the among-individual variance. We adjusted the data to achieve repeatabilities (r_i) of 0.25, 0.50 and 0.75 (for examining correlations of mean speeds across temperatures) and to achieve mean within-individual coefficients of variation (CV) of 10%, 20% and 30% (for examining maximum speed). We then repeated the statistical sampling experiments using these modified data sets. Mathieu et al.(Mathieu et al., 1981)performed a conceptually similar study involving the effects of measurement error and sampling variation in stereological analysis of microscope images.

Mean speed did not increase or decrease with time at either 35°C(linear regression, r²=0.103, P=0.168) or 20°C (r²=0.023, P=0.526; Fig. 1). The trial number on which a lizard attained its maximum speed ranged from 1 to 20 at both 35°C and 20°C; the median trial number at which maximum speed was attained was 6 at 35°C and 9 at 20°C. Thus, lizards did not show evidence of either fatigue or training during this study.

Both maximum and mean speeds were greater at 35°C than at 20°C(repeated measures ANOVA; Table 1, Fig. 1). Maximum speed at 20°C was 72% of maximum speed at 35°C. Mean speed at 20°C was 72% of maximum speed at 20°C, and mean speed at 35°C was 73% of maximum speed at 35°C. Neither maximum nor mean speed differed between lizards from the Joshua and Table Mountain populations (repeated measures ANOVA, P>0.70), and neither maximum nor mean speed covaried with body mass (regression, P>0.10).

Lizards showed substantial interindividual and intraindividual variation in speed at both temperatures (Fig. 2). Maximum speed of individual subjects ranged from 1.34 to 2.27 m s^–1 at 20°C and from 1.70 to 3.41 m s^–1 at 35°C. Likewise, mean speed of individual subjects ranged from 0.56 to 1.91 m s^–1 at 20°C and from 1.36 to 2.88 m s^–1 at 35°C. Two individuals were unusually fast outliers at 35°C (Figs 2, 3). CV for speeds of individual lizards were similar at the two temperatures: the mean CV was 22.8% at 20°C (range 10.0–56.2%) and mean CV was 20.3% at 35°C (range 13.0–26.6%). The distribution of residual speeds around each subject's average speed did not differ significantly from normality (Ryan–Joiner tests). A randomization test (1000 trials; Matlab program) revealed no significant variation among individuals in the magnitude of residual speeds either at 20°C (P=0.815) or at 35°C (P=0.579). The symmetry of residual speeds is also reflected by the fact that median and mean speeds for each individual were nearly identical at each temperature(Fig. 3).

Speed was repeatable among trials within each temperature: intraclass correlation coefficients (±s.e.) were 0.559 (±0.083) for speed at 20°C and 0.502 (±0.085) at 35°C (ANOVA, using all 20 trials per individual; P<0.0001 for each temperature). Thus, the total variance in speed was approximately equally partitioned among and within individuals. Repeatability in sprint performance both within and between temperatures was also indicated by significant positive correlations between minimum, maximum, median and mean speeds among individuals(Table 2). Correlations involving measurements of speed at two different temperatures were weaker than correlations involving speeds at the same temperature, and correlations involving means or medians were typically stronger than those involving maxima or minima (Table 2).

The magnitude of the pairwise correlations between sprint speeds for two different trials decreased with increasing separation between the trials(Fig. 4). These declines were significant for both 20°C and 35°C (Mantel tests, P=0.010 and P=0.002, respectively). The regression equations for these relationships predicted a decrease in the pairwise correlation from 0.62 for successive trials to 0.50 for trial 1 vs trial 20 at 20°C, and from 0.60 for successive trials to 0.35 for trial 1 vs trial 20 at 35°C. Thus, repeatability of sprint speed declined with time over a 1–2 week time frame. However, separation between trials explained relatively little of the overall variation in pairwise correlations(r²=0.123 for 35°C and r²=0.058 for 20°C). Mean pairwise correlations (±s.d.) were 0.579(±0.128) for 20°C and 0.516 (±0.176) for 35°C.

The standard deviation of speed for each individual was negatively correlated with individual mean speed at 20°C (r=–0.442, P=0.043) but positively correlated with mean speed at 35°C(r=0.565, P=0.007). The within-individual standard deviations of speed at 35°C and 20°C were not correlated(r=0.153, P=0.514). Likewise, the within-individual CV of speed at 35°C and 20°C were not correlated (r=0.331, P=0.144). Thus, while speed itself was repeatable across the two test temperatures, the amount of within-individual variability in speed was not.

Statistical resampling of the data sets showed that estimates of maximum sprint speed were biased when small samples were used: all values of N_trials<20 yielded underestimates on average(Fig. 5). Both the magnitude of the bias and variability of the estimate decreased as N_trials increased (Fig. 5), and the form of this relationship was virtually identical for the 20°C and 35°C data. For example, choosing the fastest speed of two trials per individual would underestimate maximum performance by 20% (on average), whereas using five trials per individual would reduce this bias to 11%.

In contrast, increasing N_trials did not provide a more accurate estimation of mean sprint speed. Overall, the estimated mean speed(average of all sample means) remained the same (71–72% of maximum speed) for all values of N_trials. Increasing N_trials also increased the precision of estimating both mean and maximum speeds.

The resampling experiment also showed that the correlation between individual mean speed at 20°C and individual mean speed at 35°C was underestimated; the magnitude of this bias was inversely related to N_trials (Fig. 6). For example, the correlation between mean speeds at 20°C and 35°C averaged 0.47 for N_trials=2 and increased to 0.59 for N_trials=5. The sample variance of the estimated correlation coefficient was also much higher for lower values of N_trials (Fig. 6). The correlation between maximum speeds at 20°C and 35°C likewise increased as a function of N_trials(Fig. 6). The correlation between maximum speeds was lower than the correlation between mean speeds for all values of N_trials except 1 (in which case the maximum and mean speed for an individual were the same).

The manipulated data sets illustrated how the amount of intraindividual variation affects the bias in estimating performance parameters. Maximum sprint performance was biased by the greatest amount (for a given value of N_trials) when the CV for each individual averaged 30%, and least biased when average CV was adjusted to 10%(Fig. 7A). Similarly, the bias in the estimated correlation between mean speed at 20°C and mean speed at 35°C was greatest when repeatability was low(r_i=0.25), and bias decreased as repeatability increased(Fig. 7B).

Burst sprint speed in S. occidentalis was repeatable both within and between temperatures. Statistically significant repeatability of locomotor performance has been demonstrated in numerous other species (e.g. Bennett, 1980; Bennett, 1987; Tolley et al., 1983; Huey and Hertz, 1984; Garland, 1985; Huey and Dunham, 1987; Jayne and Bennett, 1990; Austin and Shaffer, 1992; Reidy et al., 2000). Repeatability is considered to set the upper bound for heritability of quantitative traits, and therefore indicates the potential for traits to respond to directional selection (Falconer and Mackay, 1997) (but see Dohm, 2002; Nespolo and Franco, 2007). Previous studies have documented both repeatability and broad-sense heritability of sprint speed in S. occidentalis from Washington state(van Berkum and Tsuji, 1987; Tsuji et al., 1989; van Berkum et al., 1989). Bennett (Bennett, 1980) found that individual differences in sprint speed in S. occidentalis from southern California were consistent across temperatures. Our study confirms these earlier findings for this species.

However, a trait can be significantly repeatable and still exhibit substantial intraindividual variation. Intraindividual variation comprises about 50% of the overall phenotypic variation in sprint performance of fence lizards at both 20°C and 35°C. In contrast to interindividual variation, intraindividual variation is not informative for researchers; it is functionally equivalent to measurement error. Nevertheless, it is important to quantify intraindividual variation because it leads to biased estimates of maximum performance and of the correlations between traits. These biases can be substantial for the sample sizes often used in locomotion studies.

Studies of sprint locomotion in lizards typically use from two to five trials per individual per condition (e.g. different temperatures, slopes,perch diameters). If the magnitude of intraindividual variation is similar for different species, our analysis indicates that published maximum performance values are underestimated by 10–20% on average. This bias may not be a problem in some contexts; for example, the estimated optimal temperature for sprint performance would probably not be affected much. However, comparisons of maximum performance between species or populations could be affected if either the number of trials per individual or the amount of intraindividual variation differed between the samples. Therefore, we recommend that researchers report average CV for individuals in addition to reporting N_trials; future quantitative analyses may provide correction factors that rely on this information.

Few studies report information about the absolute amount of intraindividual variation in speed in lizards or other animals. One noteworthy exception is Bennett's (Bennett, 1980) study of burst sprint speed in S. occidentalis and several other lizard species [Aspidoscelis (Cnemidophorus) murinus,Dipsosaurus dorsalis, Plestiodon (Eumeces) obsoletus,Elgaria (Gerrhonotus) multicarinatus and Uma inornata]. In each of these six species the maximum speed from three trials was approximately 15% higher than the average speed. Our results for S. occidentalis were very similar: the maximum speed from three trials averaged 17.6% higher than the mean speed for 35°C and 17.9% higher for 20°C. This suggests that the CV of speeds for each individual is similar for these six lizard species, which represent six different families. Consequently, the statistical properties of the estimated maximum sprint speed(Fig. 5) might be similar for diverse groups of lizards.

Fuiman and Cowan (Fuiman and Cowan,2003) reported averages of individual CV for a variety of anti-predator performance traits in larval fish (Sciaenops ocellatus). Average CV varied widely among traits, ranging from 14.5% for visual response latency score to 91.3% for acoustic response score. Their results illustrate that CV of performance variables can be quite high,underscoring the importance of using multiple trials for estimating maximum values. Interestingly, routine swimming speed in S. ocellatus has a high CV (38.7%) but also a high repeatability (r_i=0.86),reflecting a large among-individual variance.

The underestimation of correlation coefficients(Fig. 6) due to intraindividual variation (or measurement error) has long been known to statisticians(Spearman, 1904; Fuller, 1987), but is not well known in organismal biology. There is a simple estimator that corrects for this bias, as long as the repeatabilities for both traits are known(Adolph and Hardin, 2007). Using this estimator yields an estimate of r=0.686 for the correlation between mean speeds at 20°C and 35°C, which is slightly higher than the correlation between mean speeds obtained using all 20 samples for each individual (0.657; Table 2). This indicates that even a large per individual sample will underestimate the correlation coefficient on average.

Attenuation of correlation coefficients can affect our ability to detect functional relationships between traits. For example, a number of studies have investigated whether individual variation in muscle fiber morphology, enzyme activity, and other lower-level physiological and biochemical traits is correlated with individual variation in whole-organism locomotor performance(Garland, 1984; Gleeson and Harrison, 1988; Bennett et al., 1989; Husak et al., 2006). Attenuation due to within-individual variation in either or both traits could reduce the sample correlation coefficient to a non-significant value. The degree of attenuation can be reduced by using the mean of multiple measurements for each individual, which has dual benefits: it increases statistical power and it permits an unbiased estimate of the correlation(Adolph and Hardin, 2007).

Whereas bias in correlation coefficients involving individual mean values is straightforward to correct, we do not know of a simple statistical remedy for the underestimation of maximal performance per se, or of the correlations involving maximum performance values. Statistical distributions of maxima and minima are generally more complicated than are distributions of mean values (Gumbel, 1958; Gaines and Denny, 1993), and are likely to differ among organisms and due to laboratory procedures.

While burst speed was significantly repeatable both within and between temperatures, the magnitude of repeatability declined with the temporal separation between trials (Fig. 4). Because we measured sprint speed over a relatively short time span (several weeks), it is unlikely that the decline in repeatability was due to changes in the physiological factors affecting speed, particularly given the lack of a decline in speed during this study(Fig. 1). Instead, it seems more likely that temporal fluctuations in labile behavioral factors such as motivation are responsible for the decline in repeatability over time. For example, two successive races may be more likely to be run under similar motivational levels, contributing to the greater similarity of sprint speeds measured close together in time.

Other researchers have reported decreases in repeatability of locomotor performance over time, particularly when measurements were separated by time spans of several months to several years(van Berkum et al., 1989; Shaffer et al., 1991; Watkins, 1997; Elnitsky and Claussen, 2006). For example, Jayne and Bennett (Jayne and Bennett, 1990) found that the magnitude of correlations involving speed and endurance in garter snakes decreased with increasing time separating the measurements. Similarly, Austin and Shaffer(Austin and Shaffer, 1992)found that repeatability of speed in tiger salamanders was lower over a 15 month period than over shorter time intervals. However, long-term declines in repeatability are not inevitable for locomotor performance or other physiological traits, as several other studies illustrate(Rønning et al., 2005; Vézina and Williams,2005; Elnitzsky and Claussen, 2006; Nespolo and Franco, 2007).

Several recent studies have combined laboratory and field measurements of locomotor performance by lizards (Irschick and Garland, 2001; Braña, 2003; Irschick, 2003; Irschick et al., 2005; Husak, 2006; Husak and Fox, 2006). These studies have shown that individual lizards in the field often do not use their maximum locomotor capacity during activities such as predator avoidance and foraging. These findings highlight the importance of accurately estimating maximum sprint performance in the laboratory, because realized performances in the field are evaluated by direct comparison to laboratory values. Bias and lack of precision in estimating laboratory performance values will result in reduced statistical power for detecting interesting patterns that involve individual field-to-laboratory comparisons.

Two individuals at 35°C were substantially faster than the rest of the sample (Figs 2 and 3). The mean speeds of each of these two individuals were 2.5 and 2.9 s.d. greater than the population mean speed at 35°C. These two individuals were likewise among the fastest individuals at 20°C, ranking 2nd and 6th out of 21 individuals. However,we did not observe any unusually fast outliers at 20°C (Figs 2 and 3). Other studies have sometimes identified unusually strong performances by a few individuals in a sample (Bennett and Huey,1990). For example, Huey et al.(Huey et al., 1990) measured endurance times in two Sceloporus merriami females that exceeded the population mean by more than 6 s.d. The physiological or behavioral bases for these exceptional performances by lizards are unknown. Intriguingly, Garland et al. (Garland et al., 2002)have discovered a discrete polymorphism in leg muscle structure and function within artificially selected mouse populations. These two muscle types differ in their biochemistry and contractile properties and exhibit a trade-off between endurance and power. The discrete polymorphism evidently results from variation at a single genetic locus(Garland et al., 2002; Houle-Leroy et al., 2003). Single-locus effects on muscle structure and running performance have also been described in whippets (Mosher et al.,2007).

In lizards, muscle fiber-type composition varies substantially among species, and is correlated interspecifically with locomotor performance capability [burst speed vs endurance(Bonine et al., 2001; Bonine et al., 2005)]. Similarly, Gleeson and Harrison (Gleeson and Harrison, 1988) found significant negative correlations between some measurements of muscle fiber size and sprint speed among individual desert iguanas, suggesting a possible causal relationship. Although Gleeson and Harrison (Gleeson and Harrison, 1988) did not mention exceptional individual performances, they did report approximately twofold variation in speed among individuals, which is typical for lizards in general. Populations that reveal exceptional individual performances, such as we observed in two individuals at 35°C, might serve as promising candidates for exploring whether genetic variants with discretely different locomotor capabilities exist in lizards,and for detecting discrete differences in morphological or biochemical traits associated with performance.

Whereas two individuals at 35°C were exceptionally fast runners, we did not observe any unusually slow individuals that were clearly performing submaximally. Instead, lizards had similar magnitudes of within-individual variation in speed, and the distribution of mean speeds among individuals did not show any discontinuities or outliers that would indicate submaximally performing individuals (Losos et al.,2002).

Intraindividual variation, especially when combined with small per individual sample sizes, leads to a biased estimation of maximum performance and of the correlation between two traits. Our analyses illustrate how these biases are affected by the per individual sample size and by the magnitude of intraindividual variation. Biased correlations between individual mean values of two performance traits can be corrected if the within- and among-individual components of variance are known. However, unbiased estimators for maximum performance, and for correlations involving maximum performance, have not yet been devised. Future studies that describe the statistical distributions of individual performance data will help researchers develop estimation procedures that correct for bias.

We thank Dee Asbury, Christine Buckley and Simon Stump for help with collecting lizards; Jesse Abrams, Chuck Matlack, Aileen Ng, Ken Dye, Tonya Icenogle and Andrew Yip for building the racetrack and writing the timing software, and Sam Tanenbaum for facilitating the construction of the racetrack as an engineering class project. T.P. was supported by a grant to Harvey Mudd College from the Howard Hughes Medical Institute (F. S. Wettack, PI). Manuscript preparation was supported by Mellon Foundation sabbatical enhancement funds and a grant from the W. M. Keck Foundation to Harvey Mudd College, via the Center for Quantitative Life Sciences. We thank Ted Garland, C. R. Tracy, Ray Huey, Duncan Irschick, Donna Folk, Peter Niewiarowski, Jo Hardin and an anonymous reviewer for their helpful suggestions. S.C.A. thanks Monica Geber and the Department of Ecology and Evolutionary Biology at Cornell University for providing space and assistance during his sabbatical.