Modelling colour constancy in fish: implications for vision and signalling in water

Fish are known for their bright colours, but how do these colours evolve and how can they work as signals? It is thought that land animals detect form and motion mostly by luminance, while colour serves object recognition. This is because the pattern of light and shade make it difficult to judge the overall reflectance (grey level) of a surface, whereas the spectral composition of reflected light is a relatively stable cue to material properties (e.g. pigmentation; Rubin and Richards, 1982; Livingstone and Hubel, 1988; Gegenfurtner and Kiper, 2003; Osorio and Vorobyev, 2005; Baddeley and Attewell, 2009). Nonetheless, terrestrial illumination spectra do vary, so that judgement of a reflectance spectrum – known as ‘object colour’ or ‘absolute colour’ – requires colour constancy: that is the ability to discount the effects of illumination on colour appearance. Colour vision can therefore be understood as a means to recover reflectance spectra from photoreceptor signals (Barlow, 1982; Buchsbaum and Gottschalk, 1983; Maloney, 1986; Osorio and Vorobyev, 2005).

At short ranges (<0.1 m) in shallow water, colour vision can operate much as it does on land, but natural waters scatter and absorb light far more than air, which makes colour constancy difficult (Figs 1 and 2; Jerlov, 1976; Mobley, 1994; Osorio et al., 1997; Johnsen, 2012; Cronin et al., 2014). Vorobyev (2001) and others (Marshall and Vorobyev, 2003) modelled colour constancy based on the von Kries transformation (see below), for the red and brown fish Scarus spinus and magenta and yellow fish Pseudochromis paccagnellae, and concluded that it failed to compensate for changes in the colour with varying distance. Consequently, aquatic animals have been thought to be less concerned with the representation of reflectance spectra (or object colour) than with the detection of visual contrast – either within the coloration pattern itself, or against the background. Notably, the chromatic offset hypothesis proposes that aquatic animals evolve multiple cone classes to enhance the visual contrast of objects seen in open water (McFarland and Munz, 1975; Lythgoe, 1979; Sabbah and Hawryshyn, 2013). Supporting this account, Marshall and others (2006) examined the colours used by several fish species as communication signals by comparing visual systems and their performance over depth in various marine light environments. The study did not consider colour constancy, but its conclusion that a fish's pattern could be a more reliable signal than its colour (Marshall et al., 2006), is consistent with evidence that cichlid cone sensitivities are well adapted for detecting patterns (Sabbah and Hawryshyn, 2013).

From the foregoing arguments it follows that where colour is used for communication over distances of greater than roughly 0.1 m (depending on turbidity) or at varying depths, it is the patterns rather than the colours themselves that are the primary signals (Marshall et al., 2006); a conclusion that contrasts with the emphasis on object colour as the primary signal for land animals (Hill and Montgomerie, 1994; Osorio and Vorobyev, 2008). Nonetheless, object colour is thought to be important to fish communication (Houde, 1997; Seehausen et al., 2008; Elmer et al., 2009; Maan and Sefc, 2013), so one can ask under what conditions it might be used: are some colours expected to offer more reliable signals with variable depth and/or viewing distance than others? Will the best set of receptors be general for all spectra in a given visual environment? Or will it depend on the specific reflectance spectra?

Perceptual constancies allow an observer to perceive the cause of a stimulus (e.g. an object), despite variation in the stimulus received by the sense organs. Human colour constancy involves both low-level (e.g. retinal) and high-level (e.g. cortical) mechanisms (Brainard and Freeman, 1997; Smithson, 2005; Foster, 2011), but it is logical to start with physiologically and mathematically the simplest colour constancy mechanism, namely the von Kries transformation, whereby each photoreceptor's response is normalised to the average for that receptor class across the image (Eqns 3,4; Worthey and Brill, 1986; Smithson, 2005; Foster, 2011).

The von Kries transformation can, at least formally, be attributed to light adaptation, which takes place in photoreceptors and other early stages of visual processing (Vanleeuwen et al., 2007; Sabbah et al., 2013), and given the universality of light adaptation, it is not surprising that all animals tested, including insects, terrestrial vertebrates and fish, have colour constancy (Dörr and Neumeyer, 1997, 2000; Chittka et al., 2014). It is, however, difficult to identify the specific mechanism; for example, Neumeyer and co-workers (2002) found that goldfish colour constancy is consistent with a von Kries transformation, but there is evidence that colour constancy in guppies improves with experience (Intskirveli et al., 2002), which is indicative of higher-level processes. Also, it is has been suggested that the spectral opponent responses of horizontal cells in teleost retinas have a role in colour constancy (Kamermans et al., 1998). As horizontal cells receive multiple, and often colour opponent, receptor inputs, their involvement implies a role for interactions between different spectral receptors, which is inconsistent with a von Kries mechanism (Vanleeuwen et al., 2007).

Here, we evaluate the potential and limitations of colour vision and colour signalling in water by modelling of the propagation of light in coral reef water to a depth of 30 m. We estimate the responses of fish photoreceptors viewing a set of 25 fish reflectance spectra over a range of depths and distances (Figs 1–3).

To implement the von Kries transformation the model receptor responses are normalised, either to the horizontal space light – i.e. the background radiance in open water with a horizontal line of sight – or to an achromatic background. These two idealised backgrounds are fundamentally different because the spectral composition of light from a reflecting surface changes with viewing distance, whereas the light from open water is fixed.

We consider three coral reef teleost fish (Fig. 2): a barracuda, Sphyraena helleri Jenkins 1901, which like many open-water fish is dichromatic (see the Materials and Methods), and two damselfishes, Chromis verater Jordan and Metz 1912 and Chromis hanui Randall and Swerdloff 1973. Both damselfishes are trichromatic, but they have markedly different photoreceptor spectral sensitivities, with that of C. hanui being more widely separated and extending into the UV. We do not model tetrachromatic fish vision (Neumeyer, 1992), but we expect this to be qualitatively similar to that for trichromats (Kelber and Osorio, 2010).

Our aim is not to predict any particular optimal system for colour communication, which would require details of the fish's vision, colours, behaviour and visual environment, but rather to understand the adaptive landscape on which fish colours and colour vision co-evolve (Seehausen et al., 2008; Miyagi et al., 2012). Specifically, we aim to: (1) compare trichromacy and dichromacy; (2) examine the effects of varying photoreceptor spectral tuning in trichromats; (3) model how the reflectance spectrum affects colour constancy; and (4) determine whether performance is sensitive to an open-water or a reflective surface background.

Light scatter and absorption mean that, in water, the illumination spectrum falling on a surface is dependent on its orientation (Figs 1 and 2; Johnsen, 2012). We assume here that the surface being viewed is Lambertian (matte) and oriented perpendicular to a horizontal line of sight. The background is either open water (‘space-light’; Johnsen, 2012) or a matte spectrally neutral surface (i.e. with equal reflectance across the spectrum) at the same distance as the object. The key difference is that light from a reflecting surface varies with distance, whereas space-light is constant. In fact, the reflectance spectra of natural backgrounds, such as sand or coral rubble, are probably not achromatic, but tend to increase linearly with wavelength (giving a brownish colour), but any difference would have minimal impact on our conclusions (Osorio et al., 1997).

where x is the distance from the subject and L(x) is the radiance. Constants α and S were calculated using Hydrolight (Sequoia Scientific) for a Case I bio-optical model, assuming a chlorophyll concentration of 0.5 mg m⁻³.

where L(x) is the radiance at distance x from the object, α is again the attenuation coefficient, which equals the sum of the absorption coefficient and the scattering coefficient. In this form, it is evident that the horizontal radiance is a mixture of the reflected radiance and the space-light, weighted by an exponentially decreasing function of distance.

We model receptor responses of three teleosts, S. helleri, C. verater and C. hanui (Fig. 2), which live in or around corals reefs. The fishes' photoreceptor sensitivities are derived from photopigment absorbances and the transmission of their ocular media (Losey et al., 2003). The 25 reflectance spectra are from freshly captured coral reef fish in Hawaii, which were measured with illumination normal to the surface, and the detector at 45 deg (Fig. 3; Marshall et al., 2003a).

where r_i is the rate at which photons activate the photopigment (assuming all photopigment molecules are available for transduction), and Λ represents the wavelength range over which the integral is performed, in this case 300 to 700 nm.

with n_i being ordered by the wavelength of peak sensitivity (λ_max) from short to long. L, M and S refer to the responses of the long, medium and short wavelength sensitive photoreceptor responses, respectively (Fig. 2), either before or after normalisation to the background (Eqn 4). Note that although scattered light in clear water generally looks blue to divers and objects become bluer with increasing distance, it is implicit in our model that object colours would move to the achromatic point with increasing distance.

A failure of colour constancy can be behaviourally significant only if the shift exceeds the colour discrimination threshold, or one just-noticeable difference (JND; here 1 JND will be detected 75% of the time from two alternatives). We consider only chromatic signals (i.e. changes in hue and saturation) and assume that colour thresholds are independent of light intensity (i.e. Weber's law holds; Kelber et al., 2003), with receptor noise equivalent to a contrast of 0.05 in each cone type (Figs 4,6,7; eqns 3,4 in Vorobyev and Osorio, 1998). This estimate of the JND is similar to a recent estimate for a bird (Olsson et al., 2015), although in reality, the effects of the ambient illumination – which changes with depth – on receptor photon catch are likely to affect the discrimination thresholds (Marshall and Vorobyev, 2003).

The terms hue, saturation and brightness refer to aspects of human colour perception (Wyszecki and Stiles, 1982), which cannot at present be defined for any animal (Kelber and Osorio, 2010). Here, we use geometric definitions that parallel the human terms. We decompose the space into a brightness axis, and an n−1 dimensional chromaticity space. The location in a chromaticity space is given by dividing the receptor catch coordinates by the sum of receptor values (nominally brightness). Saturation is the distance from the centre of the chomaticity space and hue is the remaining dimension(s). It follows that a dichromat does not distinguish hue, a trichromat has one dimension of hue and a tetrachromat has two. Note also that the n-chromacy (di-, tri- etc.) is defined not by the number of spectrally distinct cone photoreceptors in the eye but by the number of primaries needed to match any colour. Here, in the absence of direct behavioural evidence, we assume that S. helleri is a dichromat and the Chromis species are trichromats.

We model photoreceptor responses of three fish to fish reflectance spectra (Fig. 3) in coral reef water. The models predict how varying the viewing distance, depth and background (Fig. 1) will affect receptor responses and chromatic signals after photoreceptor adaptation to the background (Eqns 3 and 4). Modelled colours are plotted in chromaticity diagrams, which represent the colour based on photoreceptor quantum catches (Eqns 3-5, 7,8), in terms of the chromatic aspects of colour (i.e. hue and saturation for humans; Wyszecki and Stiles, 1982), independent of intensity (or brightness). A dichromat has a single chromatic dimension, so colours are represented on a line (Fig. 4), whereas trichromats have two dimensions, which are represented by a plane (Figs 5–7).

We modelled the effects of varying viewing distance from 0 to 100 m against open water and a spectrally neutral reflector at the same distance as the object. Visibility falls rapidly, so 100 m is in effect infinity (Figs 4 and 5; Loew and Lythgoe, 1975; Cronin et al., 2014). Light scatter and absorption (Fig. 1) cause colours to become less saturated with increasing distance, shifting them towards the achromatic point (Figs 4 and 5), which is by definition the background colour. For the trichromatic Chromis species, spectrally selective absorption has a slight effect, causing hue shifts, which are seen as ‘hooks’ on the plots in the chromaticity diagram (Fig. 5), evident at ranges exceeding 3 m.

An open-water background does not change with viewing distance, so that the photoreceptor adaptation state is fixed, and von Kries colour constancy can have no effect. By comparison, a reflecting background in the same plane as the object changes with distance in a similar manner to the object, which does allow the von Kries transform to take effect. However, the transform corrects for multiplicative effects (effects of illumination or absorption in most real-world cases), which do not apply to scattering and, in fact, the modelled colour changes for the open-water and solid backgrounds are qualitatively similar, with colours moving toward the achromatic point (Fig. 4A and Fig. 5). Thus, the model implies that receptor adaptation to the background will not affect colour changes caused by varying viewing distance, because scatter dominates light absorption by water (Figs 1 and 2).

We modelled receptor responses of the three fish species for depths of 0–30 m (Fig. 2), with a viewing distance of 0.3 m. Here, photoreceptor adaptation substantially offsets the effects of changing depth on the relative rates of photon absorption by the different spectral receptors (Fig. 4B, Figs 6 and 7). Nonetheless, residual changes (Fig. 4B, Figs 7 and 8) may exceed the colour discrimination threshold, and so might cause failures of colour constancy.

For the dichromat S. helleri, which has one chromatic dimension, all the residual changes are towards the achromatic point with increasing depth, but they vary in magnitude for different spectra (Fig. 4B), ranging from <1 to >3 JNDs. The larger shifts are for spectra that reflect strongly at long wavelengths, which lie to the right of the neutral point.

For the trichromatic Chromis species, residual shifts vary substantially in their magnitudes and their directions in colour space (Figs 7 and 8). For C. hanui, the shifts range from 0.005 to 0.05, with a mean of 0.025 units, in the x–y colour space, whereas for C. verater, shifts are smaller, ranging from 0.005 to 0.04, with a mean of 0.015 units. These values can be compared with the JND, which ranges from 0.01 to 0.02 units, depending on location and direction in the colour space (Figs 6 and 7). The difference between the two species is probably due mainly to the spectral sensitivities of C. verater photoreceptors being more closely spaced than those of C. hanui, but the particular spectral locations of the receptors is also relevant (Fig. 2; Worthey and Brill, 1986; Osorio et al., 1997) and it is evident that the direction and magnitude of shifts depend upon the specific set of photoreceptors, the spectra and the viewing conditions (illumination spectrum and adapting background). Also, there are examples of metamerism, where different spectra have the same colours, for instance, spectra 15 and 16 are almost identical for C. hanui, but not for C. verater (Fig. 3D and Fig. 7).

To examine how absorption and scattering of light might affect colour vision and communication in water (Fig. 1), we modelled chromatic signals for three species of fish viewing fish reflectance spectra. There are four scenarios: either the distance from the viewer to the object varies at a fixed depth (Fig. 4A and Fig. 5) or the depth varies at a fixed distance (Fig. 4B, Figs 6–8) and the background is either open water or a grey surface at the same location as the object. We assume that colour constancy is provided by normalisation of receptor responses to the background (Smithson, 2005; Foster, 2011; Neumeyer et al., 2002). Fish may have additional retinal (Kamermans et al., 1998, Vanleeuwen et al., 2007) and higher-level mechanisms (Intskirveli et al., 2002; Smithson, 2005; Foster, 2011) but is it logical to start with von Kries constancy.

As the distance to the object changes, scatter and absorption remove light and light is scattered into the path. Scatter is fairly spectrally neutral, but the absorption is spectrally selective, removing long and short wavelengths and leaving blue light (Fig. 1), which is then available to be scattered into the path. This moves the spectrum towards that of the open water so that the visibility of the fish declines to near zero over a few metres (Fig. 4A and Fig. 5). Furthermore, because an open-water background has a fixed spectrum (Figs 1, 4 and 5) colour constancy based on adaptation to the background is useless. When the background is a surface at the same distance as the object, von Kries constancy could theoretically have an effect, but in fact, because of the effects of scattered light, the modelled changes of colour are almost identical for open-water and reflective backgrounds, with spectral loci moving toward the achromatic point as distance increases (Fig. 4A and Fig. 5).

The model implies that the failure of colour constancy with varying distance could not be corrected unless the viewer takes account of both the distance to the object and the turbidity of the water, which is probably difficult (but see Schechner et al., 2003). These observations lead to two conclusions: first, that for trichromats an object's hue will be more constant than its saturation, and second that the range over which a colour can be detected will increase with increasing saturation (relative to the background). These considerations could account for the intense colours of some aquatic animals. Furthermore, at least for dichromats like S. helleri (Losey et al., 2003), which cannot distinguish hue from saturation (see Materials and Methods), this conclusion is consistent with the view that colour vision is concerned more with pattern recognition than object colour (Munz and McFarland, 1973; Marshall et al., 2006; Sabbah and Hawryshyn, 2013).

With varying depth but at a fixed viewing distance, spectrally selective light attenuation by water alters colours. In the absence of receptor adaptation – or some equivalent colour constancy mechanism – all colours shift towards the illumination locus with increasing depth (Fig. 6). However, as on land (Smithson, 2005; Foster, 2011), the von Kries transformation would be effective, so that a fish viewing an object from a fixed distance can achieve useful colour constancy over a range of depths. Sometimes, over tens of metres, the residual shifts – which correspond to failures of colour constancy – are negligible, falling below the discrimination threshold (Fig. 4B, Figs 7 and 8). As expected from theory (Worthey and Brill, 1986; Osorio et al., 1997), the more widely separated receptors of C. hanui suffer shifts that are, on average, 40% larger than C. verater. In theory, higher-level mechanisms might compensate for such failures, but the fact that the residual shifts vary in magnitude and direction (Figs 7 and 8) would complicate any such compensation.

Assuming that accurate judgement of colour over depth is relevant, what are the consequences of the evolution and co-evolution of fish photoreceptor spectral sensitivities and reflectance spectra? It is notable that colour changes for different spectra vary both in their magnitudes and in their directions in the trichromatic colour spaces (Figs 3, 7 and 8). Many spectra shift towards the short wavelength (bottom left) corner of the colour triangle, but blue spectra (e.g. spectra 8, 9 and 10) shift towards the long wavelength corner (bottom right). Similarly, the magnitudes of shifts in the trichromat colour spaces are not easily predictable, either from the location of the colours in their chromaticity diagrams (Figs 7 and 8) or from their grouping identified by the k-means clustering algorithm (Figs 3 and 8): the largest shifts tend to be for spectra with high reflectance at longer wavelengths, such as those in group V, and the smallest shifts being for those such as group III with high reflectance at short wavelengths, but there is much variation between related spectra, especially for the reddish colours in group I. Moreover, shifts can be different for spectra that have similar colour loci: for example, for S. helleri spectra 6 and 11 (Fig. 4B), and for C. hanui, spectra 11 and 12 (Fig. 7B); the latter difference probably arises because spectrum 12 is double peaked (Fig. 3). The unpredictability of these colour changes implies that it would be difficult to apply a simple rule to compensate for them and that the stability of the colour of a given spectrum is contingent upon the local visual environment and the colour vision of the receiver.

Communication depends on a receiver being able to discriminate different possible states of the signaller. Much work implies that object colour is important for fish communication, as it is on land (see Introduction; Osorio and Vorobyev, 2008), but the widespread occurrence of dichromacy in coral reef fish, coupled with recognition of the problem of colour constancy (Marshall et al., 2003b; Marshall and Vorobyev, 2003) suggests that this view may be simplistic. Instead, it is argued that receptor sensitivities evolve to benefit contrast with the background, as proposed by the offset hypothesis (Loew and Lythgoe, 1975; Sabbah and Hawryshyn, 2013) and likewise, that the displays of reef fish are adapted to produce conspicuous body patterns (Marshall et al., 2003b).

Despite the problems faced by colour vision in water, we find that, at least for trichromatic fish (and by implication for tetrachromatic species), colour constancy can effectively limit colour shifts associated spectral absorption of light at varying depths, but not light scattering with varying distance. It follows that if the level of pigmentation, which typically affects saturation, is an informative component of a colour signal (Milinski and Bakker, 1990; Hill and Montgomerie, 1994), decisions about object colour, for instance in mate choice, even in clear water should be made at fixed ranges of less than 1 m. Similarly, it follows that for colour variation in saturation, but not so much in hue, the opposite sex always looks better when nearer. Therefore, fish gain from coming closer – although signals are really only fairly compared if they originate from the same distance.

Trichromats can separate hue from saturation, and hue is affected little by veiling light (Fig. 5). Taking account of both scattering and absorption, this implies that the best colours for signalling in water should be saturated, with minimal hue shift following receptor adaptation to the background. Hue changes would then be robust and potentially informative. In general, von Kries colour constancy favours small photoreceptor separations (Figs 2, 7 and 8; Osorio et al., 1997) but beyond this, both the magnitudes and the directions of changes are variable, being dependent upon interactions between the photoreceptor spectral sensitivities, reflectance spectra and the visual environment. For example, in clear coastal seawater for C. verater, with its closely spaced photoreceptor spectral sensitivities, many of the bluish spectra would be satisfactory, (Figs 2, 3, 7 and 8). By comparison, the larger spectral separation of C. hanui photoreceptors increases chromatic signals (Figs 5 and 7), but this advantage may be negated by the failure of colour constancy (Figs 7 and 8; and also by reduced quantum catch). Saturated colours, such as 4, 15 and 18 (Fig 3A,D,E), where constancy failures with depth cause shifts in saturation would potentially be useful, because their hue can offer a reliable signal, whereas colour 23 (Fig. 3F), which has substantial hue shift would be less good.

Our prediction that there will be co-evolutionary interactions between the spectral sensitivities of photoreceptors used for colour vision by aquatic animals and the signalling colours directed at them (Osorio and Vorobyev, 2008; Cheney et al., 2009; Cheney and Marshall, 2009; Hofmann et al., 2009) contrasts with the sensitivity hypotheses, which proposes that fish photoreceptor spectral sensitivities tend to match the ambient illumination spectrum (Lythgoe, 1979; Bowmaker et al., 1994). It may therefore be worthwhile to take account of how colour constancy might affect the evolution and co-evolution of fish colours and of photoreceptor spectral sensitivities (Seehausen et al., 2008; Miyagi et al., 2012; Maan and Sefc, 2013).

We thank Tim Caro and Devi Stuart-Fox for comments on the manuscript.