The responses to bimodal stimuli consisting of odour and colour were recorded using calcium-sensitive optical imaging in the mushroom bodies of the hawkmoth Manduca sexta. The results show that the activity in the mushroom bodies is influenced by both olfaction and vision. The interaction between the two modalities depends on the odour and the colour of the visual stimulus. A blue stimulus suppressed the response to a general flower scent(phenylacetaldehyde). By contrast, the response to a green leaf scent(1-octanol) was enhanced by the presence of the blue stimulus. A green colour had no influence on these odours but caused a marked increase in the response to an odour component (benzaldehyde) of the hawkmoth-pollinated Petunia axillaris.
In the natural foraging situation, visual and olfactory cues always exist together. Although many studies have investigated how individual flower cues attract naïve insects, few have looked at the interaction between vision and olfaction. This is surprising considering that the use of multiple modalities has several advantages to the animal and improves the animal's ability to respond to a changing environment(Hebets and Papaj, 2005; Rowe, 1999). It can increase the detection of relevant objects against background noise and it can also potentially lead to a decreased reaction time and increased reliability.
Both odour and vision play an important role for hawkmoths in flower searching (Balkenius et al.,2006; Brantjes,1976; Brantjes,1978; Knoll,1925). It has been shown that the number of flowers that hawkmoths approach increases when colour is accompanied by a flower odour(Brantjes, 1978). In bumblebees, colour discrimination increases in the presence of odour and in honeybees an odour can enhance foraging efficiency(Giurfa et al., 1994; Kunze and Gumbert, 2001).
Studies of the relative importance of vision and olfaction in nocturnal and diurnal hawkmoths have shown that olfaction is the primary modality for nocturnal hawkmoths, whereas diurnal species mainly depend on vision(Balkenius et al., 2006). However, since flowers are multi-modal stimuli, we have to assume that insect pollinators use several sensory input channels to find them. This interaction has only recently started to be investigated(Balkenius and Kelber, 2006; Raguso and Willis, 2002). For example, the crepuscular hawkmoth Manduca sexta relies on odour at a distance and vision at close range to locate flowers(Raguso, 2001). Although, each modality on its own is sufficient to attract the moths, the combination is required to elicit foraging behaviour and for the moth to unroll the proboscis. This shows that it has a preference for a configuration of both cues (Raguso and Willis,2002). In another study, it was shown that the feeding behaviour also depends on the temporal and spatial pattern in which the olfactory and visual stimuli are perceived. Odour stimulation before take off enhanced the response to an odourless visual target(Goyret et al., 2007).
The main olfactory regions of the brain in insects are the antennal lobes(AL) and the mushroom bodies (MB). The AL are functional homologues of the vertebrate olfactory bulbs (Hildebrand and Shepherd, 1997). There are around 64 spheroidal glomeruli in the AL that receive axons from the antennae in M. sexta. The neurones in the AL are local inter-neurones or neurones projecting to different part of the brain (Kanzaki et al.,1989).
The projection neurones from the AL converge in the MB which is a higher centre for olfaction in insects (Laurent and Davidowitz, 1994; Strausfeld and Li, 1999). The MB consist of Kenyon cells, calyces and lobes. The number of Kenyon cells is taxon specific and the morphology of the calyces and lobes also differ between species. The calyces are the input zones of the mushroom body and receive collaterals from the AL. Insects with reduced antennae also have smaller calyces (Farris, 2005; Strausfeld et al., 1998).
The eyes of M. sexta are adapted to a nocturnal life style with enhanced light sensitivity. They have superposition compound eyes that can receive light from numerous facet lenses and this increases the chance of light capture (Land and Nilsson,2002; Stavenga and Arikawa,2006; Warrant et al.,2003). They also have colour vision and the three spectral types of photoreceptors are sensitive to ultraviolet (P357), blue (P450) and green(P520) wavelengths. The nocturnal hawkmoth Deilaphila elpenor can use colour vision under very dim light (Kelber et al., 2002). Both the crepscular M. sexta and the diurnal hawkmoth Macroglossum stellatarum has an innate preference for blue (Cutler et al., 1995; Goyret et al., 2008; Kelber, 1997; Kelber, 2003). Even though M. sexta have a preference for blue they learn to visit white or yellow night-blooming flowers in nature(Moré et al., 2003; Raguso et al., 2003; White et al., 1994).
The behavioural reactions to multimodal stimuli in the hawkmoths has been well studied (Balkenius and Kelber,2006; Raguso and Willis,2002; Raguso and Willis,2005). Much is also known about their olfactory processing(Christensen and Hildebrand,1987; Kanzaki et al.,1989; Kanzaki et al.,1991; Hansson et al.,1991; Hansson et al.,2003; Sun et al.,1993; Sun et al.,1996) and their visual system(Kelber, 1996; Kelber and Pfaff, 1997; White et al., 1983), but the neural processing of multimodal stimuli has never been investigated.
We have studied higher order bimodal processing in the hawkmoth, M. sexta using calcium-sensitive optical imaging to show neural activity. A probable site for multimodal processing in the hawkmoth is the MB. The MB is also believed to be where the integration of different modalities occurs(Liu et al., 1999; Schildberger, 1984). In addition, the MB is probably involved in learning and memory(Erber et al., 1980; Fan et al., 1997; Li and Strausfeld, 1997; Li and Strausfeld, 1999; Mauelshagen, 1993; Menzel, 1999; Pascual and Préat,2001). It has also been suggested that neural activity in the MB reflects attentional selection that facilitates the processing of a stimulus while filtering out irrelevant information(Xi et al., 2008).
Optical imaging has previously been used to reveal olfactory coding patterns in the AL of moths (Carlsson et al., 2002; Carlsson et al.,2005; Galizia and Menzel,2001; Hansson et al.,2003) and in the MB of honeybees and Drosophila melanogaster (Faber and Menzel,2001; Wang et al.,2004). The neuronal calcium activity can be measured in real time with this technique and both the spatial and the temporal aspects of the signal can be visualised.
By applying the optical imaging technique to the MB, we were able to address a number of questions about the processing of visual and olfactory stimuli. Does visual stimulation influence the MB? Is there an interaction between visual and olfactory responses in the MB? What does such an interaction look like? Are responses increased or decreased by bimodal stimuli? Does the spatial pattern of activity change? Is the interaction identical for different olfactory stimuli?
In experiment 1, we tested the interaction between the presentation of a blue colour and a general green leaf volatile (octanol) or a general flower odour (phenylacetaldehyde). We measured the magnitude of the response, the timing of the signals and the total area with a response and analysed pattern differences for the different stimulus combinations. In experiment 2, we used a green colour instead to see if the interaction would change depending on the colour. Finally in experiment 3, we tested a number of additional odours.
MATERIALS AND METHODS
The animals used were both male and female hawkmoths, Manduca sexta L. (Lepidoptera: Sphingidae). Larvae were reared on an artificial diet modified from that of Bell and Joachim(Bell and Joachim, 1976). The animals were kept under a 16 h:8 h light:dark cycle at 23–25°C and 40–50% relative humidity. Experiments were performed on 2–6 days post-emergent moths.
The moths were secured in a plastic tube and fixed by dental wax. The head capsule was cut open between the eyes and neck, and muscle, glands and trachea were removed to expose the MB. The eyes were covered by a flexible tube, which was fixed with dental wax. During recordings, light-guides were connected to the tubes. Calcium-dependent activation responses could be observed in 70 animals (35 females and 35 males) out of 170 tested animals.
A calcium green-2-AM dye (Molecular Probes, Eugene, OR, USA) was dissolved in 20% Pluronic F-127 in dimethyl sulfoxide (Molecular Probes) and diluted in moth Ringer solution to 30 μmol l–1.
The calcium dye was applied directly to the brain and the preparation was left in a dark and cold (13°C) place for 1–2 h. Recordings were made in vivo after incubation and washing.
The antennae were ventilated from a glass tube (7 mm internal diameter)with a continuous charcoal-filtered and moistened air stream (30 ml s–1). The glass tube ended 10 mm from the antennae. The odorant was applied on filter paper (5 mm315 mm) and inserted into a Pasteur pipette. The pipette was inserted through a small hole in the continuous airflow glass tube. Another air stream (5 m s–1) was blown through the pipette by an automatically triggered puffer device (Syntech,Hilversum, The Netherlands) for 1 s into the continuous air stream. During odour stimulation, the air stream was switched from an empty pipette to an odour-laden one to minimise the influence of added air volume.
Three plant-derived odours were used; phenylacetaldehyde (PAA),benzaldehyde and 1-octanol (OCT), which were dissolved in paraffin oil. These odours are known to elicit responses in the antennal lobes of moth(Carlsson et al., 2005; Hansson et al., 2003) and PAA elicits a feeding response (A.B., unpublished observation). Benzaldehyde is a strong component of the floral blend of Petunia axillaris which is visited by M. sexta (Hoballah et al., 2005). The dose was 50 μg. In addition, we used extract of honeysuckle (5 μl) and lavender (10 μl; Body Shop, Lund, Sweden).
The visual stimulus was generated by a 3 mm LED of 430 nm and an intensity of approximately 0.01 cd m–2 (full moon intensity). This blue colour (BLUE) is known to be attractive to the moths during foraging(Cutler et al., 1995). To produce a green colour (GREEN, 480 nm), a colours filter was placed in series with the LED (Rosco Supergel, London, UK; medium yellow).
The light source was controlled by a custom-made interface box, which controlled the intensity of the visual stimulus. A fibre-optic light guide was used to transfer the visual stimulus to the eyes of the moth. The optically isolated light guides where docked to the eyes using small rubber tubes that were kept in place using dental wax.
TILL Photonics imaging (Gräfling, Germany) was used to record the responses with an Olympus microscope with filter settings of dichroic 500 nm,emission LP 515 nm, and the preparation was illuminated at 475 nm. Sequences of 40 frames (4 Hz, 200 ms exposure time) were recorded through a 103 (NA 0.50; Olympus, Hamburg, Germany) air objective. The same regions of the calyces of the left and right mushroom bodies were recorded. These regions were selected because they consistently reacted to the stimuli.
Stimulus generation and data collection was fully automatic and controlled by the Till-vision 4.0 software (TILL Photonics). Three different protocols were used and presented in random order: one with only vision, one with only odour and one with both stimuli presented together. The stimulus presentation started at frame 12 after 3 s and lasted for 1 s.
The collected images where processed in ImageJ (National Institute of Health, Bethesda, MD, USA). A custom Java plug-in was written for ImageJ to automatically filter the images to detect activation. To compensate for image motion, recursive alignment using a rigid body transform was performed using the StackReg plug-in (Thévenaz et al., 1998). The images were subsequently spatially smoothed with a Gaussian filter (10 pixel radius) and time averaged using bins of 9 and 19 frames. A positive difference between the shorter and longer average was used as a indicator of calcium activity and was divided by activity just before the stimulus onset to give an estimate of the relative change in fluorescenceΔ F/F at each location.
To analyse the activity magnitude, the total response was calculated as the average over all animals of ΔF/F over all frames within a circular region with a diameter of 125 μm around the location with maximum response. The fraction of the imaged area with aΔ F/F above a threshold of 0.1% at frame 20 was also calculated to indicate the size of the activated region.
To analyse the response patterns in the MB, we selected the four sites(A–D) in each animal that were maximally responsive to each of the four stimulus combinations (PAA, OCT, PAA+BLUE and OCT+BLUE) and the response magnitude in regions with a radius of 7 μm were calculated by fitting a decay model to the data for the frames outside the stimulus presentation using the least-squares method. This resulted in a four-dimensional response pattern for each stimulus combination. To tests for different response patterns, we used a MANOVA to investigate the influence of the different stimuli on the patterns. Furthermore, to determine the consistency of the response patterns,we compared the responses between two presentations of the different stimulus combinations. The response vectors were first mapped onto a four-dimensional principal component space and Fisher's linear discriminant functions were constructed to distinguish between the different stimulus combinations. The discriminant functions were subsequently used to predict the stimulus combination on the second presentation from the elicited response patterns in the MB. The timing of the responses was measured by setting a threshold ofΔ F/F at 0.15% and the response was assumed to have started the first time the signal crossed this level after stimulus presentation.
Calcium-dependent activation could be observed in M. sextamushroom bodies after stimulation by odour and by the combined stimuli(Fig. 1).
Localisation of activity
The responses to the different stimulus combinations were located in the MB calyces (Fig. 1). Fig. 1A shows an overview of the location where the recordings were made in one representative female moth. The same area of the MB was activated for both odours(Fig. 1B). Without any stimulation, very little activity was seen(Fig. 1C). This was also the case for BLUE on its own (Fig. 1D). However, BLUE in combination with OCT(Fig. 1E) activated a larger region than OCT on its own (Fig. 1F). By contrast, PAA on its own(Fig. 1H) elicited a larger reaction than PAA+BLUE (Fig. 1G).
The quantitative analysis of the size of the area above threshold is shown in Fig. 2. The area of the MB that was activated by the bimodal stimulus was significantly different for OCT+BLUE compared with OCT alone (repeated measures ANOVA, N=23, P<0.001) and for PAA+BLUE compared with PAA on its own (repeated measures ANOVA, N=23, P<0.001). The change in the two cases has the same direction as for the magnitude of the response.
The effects of the different stimuli on the magnitude of the Ca2+ response are shown in Fig. 3. In the test sequence with OCT, a repeated measures two-way ANOVA showed main effects of stimulus on Ca2+ response magnitude(P<0.001, N=18). There was no effect of sex(P=0.34) or interaction between sex and the difference responses(P=0.58). A post-hoc Bonferroni test showed that there was a significant difference between the response magnitude to the multimodal stimulus (OCT+BLUE) and the response to the odour (OCT) on its own(P<0.01). The reaction to the visual stimulus on its own was not significantly different from the Ca2+ activity without any stimulus(Bonferroni test, P=1).
In the tests with PAA, a repeated measures two-way ANOVA showed the main effects of stimulus on Ca2+ response magnitude(P<0.001, N=18). There was no significant effect of sex on the different stimulus combinations (P=0.43) or interaction between stimulus type and sex (P=0.45). A Bonferroni test showed significant differences in the response to PAA together with BLUE compared with PAA on its own (P<0.001). Again, there was no significant difference between the response to BLUE alone and no stimulus (Bonferroni test, P=1). Note that the interaction between vision and olfaction for OCT is the opposite to that for PAA.
Fig. 4 shows the responses at the four selected sites (A, B, C and D) to the different stimulus combinations. A MANOVA on the four sites showed significant differences between the response patterns for the different stimulus combinations(P<0.001). However, this result is due to the biased selection of the measurement sites in the images. When comparing the response pattern to the same stimulus combination at a second presentation it was not possible to predict the stimulus from the response pattern. Using Fisher's linear discriminant in the four-dimensional principal component space of the patterns, the identities of the predicted stimuli were close to chance levels. The predicted odour was 50% correct, the predicted presence of the visual stimulus was 55% correct, and when attempting to predict the complete stimulus combination, the success rate was only 32%. This strongly indicates that there are no pattern differences for the different stimulus types.
The timing of the different responses for the different stimulus combinations are shown in Fig. 5. The timing of the response was on average 131 ms faster when the odour was presented together with the visual stimulus. A two-way ANOVA showed main effects of the visual stimulus on the timing of the responses(P<0.05), but no effect of the particular odour (P=0.34). There was no interaction between odours and the presence or not of the visual stimulus (P=0.45).
The effects of the different stimuli on the magnitude of the Ca2+ response are shown in Fig. 6. In the tests with PAA, a repeated measures two-way ANOVA showed the main effects of stimulus on Ca2+ response magnitude(P<0.001, N=25). There was no significant effect of sex on the different stimulus combinations (P=0.43) or interaction between stimulus type and sex (P=0.43). A Bonferroni test showed no significant differences of the response to PAA together with GREEN compared with PAA on its own (P=0.38).
In the test sequence with OCT, a repeated measures two-way ANOVA showed the main effects of stimulus on Ca2+ response magnitude(P<0.001, N=27). There was no effect of sex(P=0.35) or interaction between sex and the different responses(P=0.45). A post-hoc Bonferroni test showed that there was no significant difference between the response magnitude to the multimodal stimulus (OCT+GREEN) compared with the response to the odour (OCT) on its own(P=1).
The responses to the three additional odours, benzaldehyde, honeysuckle and lavender, were in the same range as the responses to OCT and PAA. The responses to all odours were increased when they were presented together with a green colour (Table 1). By contrast, the blue colour did not influence the response to benzaldehyde and lavender, but increased the response when presented together with honeysuckle.
Using optical imaging of the MB of M. sexta, we have been able to show for the first time that the activation in this higher level brain region reflects a bimodal interaction between vision and olfaction. The magnitude of activation caused by an odour depends on the presence of a visual stimulus. The activation caused by presentation of OCT, which is a green leaf odour, is enhanced by BLUE. By contrast, the activation caused by PAA, which is a general flower scent, is suppressed by BLUE. The reaction to these two odours was not influenced by the presentation of GREEN. On the other hand, GREEN caused a marked increase in the response to benzaldehyde. A similar interaction was seen for lavender, whereas both colours enhanced the response to honeysuckle.
These results give the first physiological evidence that visual as well as olfactory information influences activity in the MB. Moreover, the different interactions for different colours show that colour identity as well as odour influences the activity of the MB.
Colours and odours are among the most important floral signals, and nectar searching insects can use the combination as one cue to find flowers(Balkenius and Kelber, 2006). The combination is also sometimes required to start feeding behaviour(Raguso and Willis, 2002). It is therefore surprising that the presence of a colour stimulus suppresses the activity in the MB caused by a general flower scent whereas the activity related to green leaf odour is enhanced. One possible explanation for this is that a visual stimulus is more reliable than odour for flight control close to the flower and the suppression of odour activity could reflect a shift to visual control during the final flower approach and proboscis extension. The effect of multisensory integration could be a way to enhance or depress a modality that would interfere with the control of the current behaviour even if the signal is completely consistent with that behaviour.
Several studies have shown that Lepidoptera in general use the combination of different sensory inputs during foraging(Andersson and Dobson, 2003; Balkenius et al., 2006; Brantjes, 1978; Raguso and Willis, 2002), but different modalities have different hierarchical values and the responses could be additive or synergetic. The results presented here parallel those of our earlier behavioural studies of M. stellatarum where it was shown that a colour presented together with an odour would influence odour processing (Balkenius and Kelber,2006). In that case, different colours interacted differently with an odour. In the experiments presented here, this effect was seen also for different odours. The same colour interacted with different odours in different ways.
The result is consistent with intracellular recordings in male M. sexta, which showed that visual stimulation can elicit either excitatory or inhibitory responses in the olfactory descending neurones(Kanzaki et al., 1991). Interestingly, although we found no response in the MB to BLUE on its own, the response to the bimodal stimulus was faster than the response to the unimodal odour whether the magnitude of the response to the bimodal stimulus was larger or smaller. This shows that the difference in the timing is not an artefact of the magnitude of the signal. This also precludes that the difference in timing is due to feed-back from later processing stages, since this would mean that the visual response would be slower rather than faster. Instead, it suggests that the bimodal interaction takes place in or before the MB.
Although there are very few data on multisensory reactions in insects, it has been found in many other species that both neuronal and behavioural reactions to multimodal stimuli are faster than those to unimodal stimuli(Brett-Green et al., 2003; Molholm et al., 2006; Sakata et al., 2004; Stein and Stanford, 2008). Even though multimodal responses are faster, they may be depressed as well as enhanced (Avillac et al., 2007; Stein and Stanford, 2008). This is consistent with our finding that the bimodal response was faster even when the magnitude of the signal was lower. One explanation for this result could be that visual processing is faster than olfactory processing and that the visual signal would trigger a weak response in the bimodal neurones even before the olfactory signal appears. However, this would require a measurable unimodal visual response, and we did not see this in the experiments.
Another explanation that fits the results is that the observed responses are the result of two bimodal neural processes. The first is responsible for the initial response, the latency of which decreases for bimodal stimuli. The second process is slower and responsible for the control of the response magnitude. The summed effect of these two processes would account for the data.
The neural activity was found in the same general region of the MB for all stimuli. The spatial distribution of the response evoked by OCT is wider than that evoked by PAA, which reflects a difference in the magnitude of the responses. In D. melanogaster calyces, the response patterns were spatially organised for the different odours(Fiala et al., 2002). In the AL, different odours have their unique activity pattern(Hansson et al., 2003). Different response patterns to different odours have also been found in the MB of honeybee (Szyszka et al.,2005) and D. melanogaster(Wang et al., 2004) although the spatial patterns found were not as distinct as in the glomeruli in the AL. However, in this study, we were not able to find any consistent differences in the response patterns between presentations of the same stimulus.
The inability to predict the stimulus combinations based on the recorded response patterns in the MB strongly suggests that the Ca2+patterns in the MB do not carry any information about stimulus identity. Our procedure of selecting points that maximised the pattern difference at the first stimulus presentation contributes to this conclusion.
One possibility for this failure to detect patterns in the MB may be a too low image resolution or a too high noise level. Another possibility is that the MB has another function than to code for stimulus identity. It has been suggested that this region plays a role in fixation behaviour by modulating the selection of a single target stimulus among several distracters through inhibition of other brain regions (Xi et al., 2008).
There are several factors that could influence such a selection mechanism. One is the innate preferences. For example, blue is an innately preferred colour for M. sexta even though their natural food plant has white flowers (Goyret et al., 2008). There may also be an innate preference for benzaldehyde(Hoballah et al., 2005). Different activation levels could also reflect gating of different learning mechanisms as has been seen in earlier studies of hawkmoths(Balkenius, et al., 2006). The interaction effects seen in our experiments are reminiscent of overshadowing effects that occur during learning. It would be interesting to investigate whether a visual stimulus that suppresses odour activity in the MB would overshadow that odour during learning. The selection of different stimuli or stimulus modalities may also depend on factors such as the distance to the flower. For example, visual information is more reliable than odour to guide the proboscis close to the flower (Warrant et al., 1999).
This raises the question of the functional role of the responses revealed by the Ca2+ activity. If the responses in the MB reflect a selection mechanism, then there are many possibilities. The MB may select a particular stimulus or the spatial location of that stimulus. It is also possible that the role of the MB response is to enhance or suppress one or several modalities in the control of behaviour. However, further studies are necessary to distinguish between these possibilities. We will also continue to investigate if it is possible to find a pattern difference between different odours and colours in the MB of M. sexta.
To summarise, we have shown that the processing of odours in the MB is greatly influenced by the visual stimulus. However, we were not able to find any responses to visual stimuli on their own. The bimodal enhancement can thus not be explained simply by summation, but is a sign of true interaction. This is further supported by the fact that the interaction can be either positive or negative and depends on the colour of the visual stimulus as well as the particular odour.
Thanks to Mikael Carlsson for teaching us some of the methods, Elisabeth Marling for rearing the moths with all her patience and knowledge, Sabine Kaltofen for sending extra pupae, Anders J. Johansson and Christian Balkenius for technical support, fruitful discussions and critical reading of the manuscript. Research support from the Swedish Research Council is gratefully acknowledged.