Learning context modulates aversive taste strength in honey bees

Taste is the sense that allows animals to distinguish between chemical compounds and the sensations they produce based on contact chemoreceptors. It allows the discrimination of edible from non-edible items because the latter have usually a bitter taste (Yamamoto et al., 1994; Scott, 2004; Reilly and Schachtman, 2008; Yarmolinsky et al., 2009). Animals rapidly learn to avoid bitter substances and, as a consequence, numerous learning protocols use aversive tastes as reinforcers to promote robust learning and memory (Darling and Slotnick, 1994; Laska and Metzker, 1998; Ito et al., 1999; Gerber and Hendel, 2006; Kemenes et al., 2011; Salloum et al., 2011).

In the honey bee Apis mellifera, an insect that has a model status for the study of learning and memory (Menzel, 1999; Menzel and Giurfa, 2001; Giurfa, 2007; Avarguès-Weber et al., 2011; Giurfa and Sandoz, 2012), gustatory perception has been less well studied (de Brito Sanchez et al., 2007; de Brito Sanchez, 2011). Despite the biological relevance of taste for these insects given their contact with and collection of different types of nectar and pollens, resins, water and other natural products (de Brito Sanchez et al., 2007; de Brito Sanchez, 2011), less is known about their taste discrimination abilities. For instance, the capacity of bees to detect bitter substances is controversial (de Brito Sanchez et al., 2005; de Brito Sanchez et al., 2014). In fact, the use of the term ‘bitter’ can be questioned in the case of the honey bee because it is not known whether this insect perceives quinine and other chemical components as bitter. Here and henceforth, we use the term ‘bitter’ as in the insect gustatory literature, without making claims on perceptual sensations (e.g. Tanimura and Kikuchi, 1972; Glendinning et al., 2001, 2002; Weiss et al., 2011).

While harnessed bees in the laboratory ingest without reluctance different kinds of noxious substances, including bitter ones (e.g. quinine, salicin, amygdaline and l-canavanine) and even die as a consequence of the malaise induced by this ingestion (Ayestarán et al., 2010), free-flying bees avoid bitter substances used as a penalty in visual discrimination tasks (Chittka et al., 2003; Avarguès-Weber et al., 2010; Rodríguez-Gironés et al., 2013). The first scenario corresponds to the typical preparation used in the laboratory to study Pavlovian learning in bees: the olfactory conditioning of the proboscis extension reflex (PER), an appetitive reflex triggered by sucrose solution. Insects are harnessed individually in tubes, left for several hours in a resting situation and then exposed to an odorant (the conditioned stimulus or CS) paired with sucrose solution (the unconditioned stimulus or US) (Takeda, 1961; Bitterman et al., 1983; Giurfa and Sandoz, 2012). Harnessed bees learn to associate the odor with the appetitive sucrose solution and exhibit afterwards PER to the conditioned odor. It thus seems that under these harnessed conditions, bitter and other aversive substances (e.g. concentrated NaCl solution), which are actually toxic for the bees (Ayestarán et al., 2010), are either not detected or detected and ingested because of lower acceptance thresholds.

The second scenario corresponds to another typical preparation used to study visual learning in bees: free-flying bees are trained to choose a visual target associated with sucrose reward and to avoid a distracter associated with the absence of reward or with an aversive substance. The associations built in these contexts can be either operant, classical or both, i.e. they may link the response of the animal (e.g. landing) and the reward/punishment (US), the visual stimuli (CS) and the US, or both. The experimental framework is nevertheless mainly operant because the bee's behavior is determinant for either obtaining the sucrose reinforcement or not (Avarguès-Weber et al., 2011). In this context, quinine solution acts as a negative reinforcement, improving the performances of visual discrimination in protocols where a color is associated with sucrose, and a distracter with water (neutral) or quinine (bitter substance) (Chittka et al., 2003; Avarguès-Weber et al., 2010; Rodríguez-Gironés et al., 2013). Compared with water, quinine improves the bees' visual discrimination, leading to the suggestion that its penalizing nature increases attentional processes (Avarguès-Weber et al., 2010; Avarguès-Weber and Giurfa, 2014). It thus seems that under free-flying conditions, bitter substances are detected and rejected, contrary to what occurs in harnessed conditions.

Here, we raised the question of whether gustatory perception of honey bees changes with the behavioral context so that aversive tastes that are less effective as negative reinforcements in a given context become more effective in a different context. We trained bees to discriminate one odor (CS+) paired with sucrose from another odor (CS−) paired with either distilled water (a neutral substance), NaCl or quinine solution (both toxic substances). Conditioning was either Pavlovian (olfactory conditioning of PER), or mainly operant (training of free-walking bees to discriminate the same two odors within a small Y-maze). PER-trained bees were then tested both in their original context and in the Y-maze whereas maze-trained bees were tested both in their original maze context and in the Pavlovian PER context. In this way, we aimed to determine whether the evaluation of taste changes depending on the behavioral (Pavlovian, operant) context, i.e. whether aversive tastes are perceived differently by harnessed and free-walking bees.

Four experiments were performed (Fig. 1C). In experiment 1, harnessed bees were subjected to a differential olfactory PER conditioning (Fig. 1A) in which they had to learn to discriminate between an odor rewarded with sucrose and a non-rewarded odor paired with water, salt or quinine; retention was afterwards measured in the operant context of a small Y-maze in which they could freely walk and choose/avoid the two odors previously trained (Fig. 1B). In experiment 2, training was identical to that of experiment 1 but retention was measured in the same Pavlovian context and at the same time as the operant testing of experiment 1 (Fig. 1B). This experiment was conceived as a control of experiment 1, to verify that performance variation between contexts of experiment 1 was due to context change and not to an unspecific memory decay. It was thus important to determine whether 2 h after PER conditioning, and after being subjected to the same post-conditioning handling, bees conserved the memories acquired in the olfactory PER conditioning when tested in the same context. In experiment 3, training occurred in the Y-maze and retention was afterwards measured in the Pavlovian context as each bee was individually harnessed and tested for PER to the odors previously conditioned in the maze (Fig. 1C). Finally, in experiment 4, conceived as a control for experiment 3, both conditioning and testing occurred in the operant context of the Y-maze. As for experiment 2, the goal of experiment 4 was to determine whether performance variation in experiment 3 was due to a nonspecific memory decay or to the change of contexts. Importantly, in switching bees from one experimental context to the other, cooling was never used to immobilize them as it has amnesic effects, which could potentially impair the memories acquired in the first context (Menzel et al., 1974; Erber et al., 1980). Instead, bees were carefully handled with individual forceps whose tips were covered with soft foam rubber.

Bees were conditioned during 12 trials to discriminate two similar odors, 1-nonanol and 1-octanol, one of which (CS+) was paired with an appetitive reward of sucrose 1 mol l⁻¹ (6 CS+ trials) and the other (CS−) with distilled water (water group), quinine 60 mmol l⁻¹ (quinine group) or NaCl 3 mol l⁻¹ (NaCl group) (6 CS− trials). Within each of these groups (water, quinine and NaCl), there were two subgroups, one in which 1-nonanol was the CS+ and 1-octanol the CS−, and another in which the contingencies were reversed (1-nonanol was the CS− and 1-octanol the CS+). For all three groups (water, quinine and NaCl) there were no significant differences in acquisition according to which odor was the CS+ or the CS− (two-factor ANOVA for repeated measurements; factor odor; water group: F_1,52=0.43, P=0.51; quinine group: F_1,47=0.07, P=0.78; NaCl group: F_1,53=0.26, P=0.61) so that results were pooled within groups and presented in terms of a CS+ versus CS− discrimination irrespective of odorant identity.

Fig. 2A shows the learning performance of three conditioned groups of bees, which satisfied the criterion established for further testing (see the Materials and methods), namely that they responded correctly to the CS+ in the last conditioning trial. These bees also responded correctly to the CS+ in the previous trial, thus confirming previous findings showing that once bees start responding correctly to the CS+ in olfactory PER conditioning, they keep responding to it in further trials (Pamir et al., 2011). All three groups learned to discriminate the CS+ from the CS− during conditioning (Fig. 2A; water group: factor CS: F_1,53=107.14, P<0.0001; factor interaction: F_5,265=23.41, P<0.0001; quinine group: factor CS: F_1,48=113.79, P<0.0001; factor interaction: F_5,240=25.33, P<0.0001; NaCl group: factor CS: F_1,54=429.36, P<0.0001; factor interaction: F_5,270=42.21, P<0.0001). However, the discrimination performance varied depending on the US associated with the CS− (factor group: F_2,155=4.18, P<0.02). Specifically, although the acquisition curves for the rewarded odor (CS+) were identical (F_2,155=1.32, P=0.27), the acquisition curves for the non-rewarded odor (CS−) differed significantly (F_2,155=5.71, P<0.005). In other words, the three reinforcers, water, quinine and NaCl, had different capacities to inhibit responses to the CS−. Tukey tests showed that when taken globally, the CS− curves of the NaCl and water groups differed significantly (P<0.01) but the CS− curve of the quinine group did not differ from that of the water group (P=0.47) or the NaCl group (P=0.94).

After PER conditioning, bees were kept for 2 h in individual glass tubes and then transferred to the Y-maze for two transfer tests in the absence of reinforcement. In the learning test, the CS+ and the CS− were presented against each other in the new context; in the avoidance test, the CS− was presented against the novel odor eugenol (Nod) to determine to what extent the CS− learned in the Pavlovian context induced avoidance and thus choice of the novel odor.

In the learning test, all three groups showed a tendency to prefer the previously rewarded CS+ to the CS− because their choice of the CS+ was above 50% (Fig. 2B). A comparison between groups by means of a one-factor ANOVA for independent measurements yielded no differences between groups (F_2,155=1.87, P=0.16). However, comparing the proportion of choices of the CS+ to a theoretical value of 50% within each group showed that only the NaCl group exhibited a significant preference with 76% of the bees preferring the CS+ (t₅₄=4.56, P<0.0001), whereas the quinine and the water groups exhibited a similar non-significant performance with 61% of the bees preferring the CS+ in both cases (water group: t₅₃=1.66, P=0.10; quinine group: t₄₈=1.60, P=0.12). These results confirm the stronger inhibitory effect of 3 mol l⁻¹ NaCl as an aversive reinforcement and the weaker and comparable inhibitory strength of distilled water and 60 mmol l⁻¹ quinine solution in olfactory PER conditioning.

In the avoidance test, all three groups tended to avoid the CS− in favor of the novel odor as their choice of the CS− was below 50% (Fig. 2C). A comparison between groups showed no significant differences (F_2,155=0.90, P=0.41). However, comparing the proportion of choices of the CS− to a theoretical value of 50% within each group showed that the NaCl group exhibited a higher and significant avoidance of the CS− (t₅₄=−3.04, P<0.005) whereas the quinine and water groups showed a non-significant performance (water group: t₅₃=−1.09, P=0.28; quinine group: t₄₈=−1.30, P=0.20). These results thus reaffirm that in olfactory PER conditioning, NaCl induced stronger aversion when paired with a CS− whereas both water and quinine had a weaker and similar aversive effect. As a consequence, both discrimination learning and retention in the maze were better in the NaCl group owing to the stronger aversive nature of NaCl as negative reinforcement.

This experiment was conceived as a control for experiment 1. The level of correct choices in the Y-maze (CS+ choices in the learning test) was lower than that observed in PER conditioning. It was thus important to determine whether 2 h after conditioning, and after being subjected to the same post-conditioning handling, bees conserved the memories acquired in the olfactory PER conditioning when tested in that context.

The first phase (differential olfactory PER conditioning) was identical to that of experiment 1. For all three groups (water, quinine and NaCl) there were no significant differences in acquisition according to which odor was the CS+ or the CS− (two-factor ANOVA for repeated measurements; factor odor: water group, F_1,31=1.56, P=0.22; quinine group, F_1,32=0.64, P=0.43; NaCl group, F_1,29=0.27, P=0.61) so that results were pooled within groups.

As in the previous experiment, all three groups learned to discriminate the CS+ from the CS− during conditioning (Fig. 3A; water group, factor CS: F_1,32=25.06, P<0.0001, factor interaction: F_5,160=7.71, P<0.0001; quinine group, factor CS: F_1,33=48.67, P<0.0001, factor interaction: F_5,165=11.50, P<0.0001; NaCl group, factor CS: F_1,30=113.22, P<0.0001, factor interaction: F_5,150=24.61, P<0.0001). Performance varied significantly depending on the US associated with the CS− (F_2,95=8.89, P<0.001). Specifically, although the acquisition curves for the rewarded odor (CS+) were identical (F_2,95=0.76, P=0.47) between groups, the acquisition curves for the non-rewarded odor (CS−) differed significantly (F_2,95=9.93, P<0.001). When taken globally, the CS− curves of the NaCl and the water groups differed significantly (Tukey test; P<0.01) whereas the differences between the CS− curve of the quinine group and those of the water and NaCl groups were marginally non-significant (P=0.05 and P=0.08, respectively). The amount of discrimination reached at the end of conditioning (CS+ responses minus CS− responses) was the same in experiments 1 and 2 for the water group (F_1,85=1.75, P=0.19), the quinine group (F_1,81=0.61, P=0.44) and the NaCl group (F_1,84=1.65, P=0.20), thus showing that conditioning yielded the same results in experiments 1 and 2 for all three groups. In both experiments, NaCl was the strongest aversive reinforcer, followed by quinine and then by water.

After being placed in individual glass tubes for 2 h as for bees in experiment 1 and then being transferred to metal tubes for PER testing, bees were presented with the CS+, the CS− and the novel odor Eugenol (Nod) in a random sequence. Fig. 3B shows that all three groups responded mainly to the CS+, less to the CS− and practically not at all to the Nod, thus confirming the low generalization between both CS odors and eugenol. Importantly, the level of responses to the CS+ was similar to that reached at the end of conditioning in all three groups. Thus, the handling procedure to which bees were exposed after PER conditioning did not induce a loss of their memories.

No differences were found between groups (two-factor ANOVA for repeated measures; factor US group: F_2,95=0.4, P=0.66), but a highly significant effect was introduced by the odorant tested (factor odor: F_2,190=87.29, P<0.0001). Tukey tests showed that although the NaCl group exhibited significant retention as it discriminated the CS+ from the CS− (Fig. 3B, P<0.0001), the quinine group showed an intermediate significant preference for the CS+ over the CS− (P<0.05) and the water group a non-significant preference (P=0.74).

In order to refine the analysis of retention performances, we focused on bees showing a CS-specific memory, i.e. responding only to the CS+ and not to the CS− or Nod. Fig. 3C shows that the proportion of bees with CS-specific memory was higher in the NaCl group, intermediate in the quinine group and lower in the water group, consistent with the learning performance. Marascuilo tests for comparing multiple proportions showed that the CS specific memory of the NaCl group was significantly higher than that of the water group (P<0.05); no other comparisons were significant.

This experiment is the reversed version of experiment 1. Bees were conditioned to discriminate 1-octanol and 1-nonanol in the Y-maze and then tested under harnessing conditions.

As in experiments 1 and 2, bees were conditioned over 12 trials to discriminate the CS+ associated with sucrose from the CS− associated either with water, quinine or NaCl. Within each US group, there were no significant differences in acquisition according to which odor, 1-octanol or 1-nonanol, was the CS+ or the CS− (water group: F_1,9=0.007, P=0.94; quinine group: F_1,20=0.009, P=0.93; NaCl group: F_1,19=1.19, P=0.29) so that results were pooled within groups.

Fig. 4 represents the learning performance of the three groups of bees which efficiently responded to the CS+ in the last visit to the maze. All three groups learned the task as shown by the improvement in the percentage of correct choices at the end of conditioning (water group: F_11,110=2.57, P<0.01; quinine group: F_11,231=2.30, P<0.02; NaCl group: F_11,220=2.92, P<0.002). Responses in the maze show a typical variable pattern, which reflects both the information acquired through learning and the expression of exploratory tendencies that decrease the level of correct choices.

A comparison between groups revealed significant differences (factor group: F_2,51=3.75, P<0.05) and a global significant increase of correct responses (factor trial: F_11,561=5.83, P<0.0001) but no significant interaction (F_22,561=1.00, P=0.46). When taken globally, the curves of the NaCl and the water groups differed significantly (Tukey test; P<0.05) while the differences between the curves of the quinine group and those of the water and the NaCl groups were not significant (P=0.07 and P=0.89, respectively). Thus, in the operant context of the Y-maze, the effect of quinine is more similar to that of NaCl which remains the strongest aversive reinforcement.

After the last visit to the maze, bees were placed in individual glass tubes and then fixed in metal tubes for PER testing upon odor stimulation. Bees were presented with the CS+, the CS− and the novel odorant Eugenol (Nod). Surprisingly, no bee responded to either odorant in these transfer tests. The absence of response transfer may reflect a memory loss 2 h after conditioning or a refusal (or incapacity) to express the existing memories under the new restrictive, harnessing conditions. The latter option would be consistent with an increase of stress levels following immobilization, which would preclude bees responding appropriately to the learned odors. To test this hypothesis, bees were trained in the Y-maze and tested 2 h after conditioning in the same maze after similar handling and timing as in experiment 3.

Bees were conditioned in the Y-maze as in experiment 3. Within each US group, there were no significant differences in acquisition according to which odor, 1-octanol or 1-nonanol, was the CS+ or the CS− (water group: F_1,6=0.21, P=0.66; quinine group: F_1,11=3.62, P=0.08; NaCl group: F_1,12=2.00, P=0.18) so that results were pooled within groups. All three groups of bees, which efficiently responded to the CS+ in the last visit to the maze, learned the task (Fig. 5A; water group: F_11,77=2.25, P<0.02; quinine group: F_11,132=1.87, P<0.05; NaCl group: F_11,143=2.13, P<0.03) but differed significantly in their performance (factor group: F_2,32=14.77, P<0.001). Interestingly, the curves of the NaCl and quinine groups did not differ significantly (P=0.99), thus showing that both reinforcements induced similar learning performances when paired with the CS−. However, both groups differed significantly from the water group (quinine versus water: P<0.001; NaCl versus water: P<0.001), which induced less-efficient learning when paired with the CS−. These results thus confirm the tendencies observed in experiment 3 and show that in the Y-maze, the effect of quinine is similar to that of NaCl and distinct from that induced by water.

After conditioning, bees were placed in individual glass tubes where they were kept for 2 h. Thereafter, they were replaced individually in the Y-maze where they were subjected to a learning test opposing the CS+ and the CS− previously learned and to an avoidance test opposing the CS− and the Nod.

In the learning test, all three groups showed a tendency to prefer the previously rewarded CS+ to the CS− (Fig. 5B) as shown by the fact that their CS+-choice level was above of 50% in all cases. No differences between groups were detected (F_2,32=0.23, P=0.80). However, comparing within each group the proportion of CS+ choices to 50% showed that both the quinine and the NaCl groups exhibited a significant preference with 85% and 86%, respectively, of the bees preferring the CS+ (quinine group: t₁₂=3.32, P<0.01; NaCl group: t₁₃=3.68, P<0.005). On the contrary, the bias towards the CS+ observed in the water group was not significant (t₆=1.16, P=0.29). These results show that, consistently with their learning performances, the quinine and the NaCl groups had better retention than the water group. Such an effect confirms the stronger inhibitory effect of both quinine and NaCl 3 mol l⁻¹ as aversive reinforcements and the weaker strength of distilled water in the context of the Y-maze.

In the avoidance test, only the quinine and the NaCl groups tended to avoid the CS− in favor of the novel odor while the water group chose randomly between both odors (Fig. 5C). A comparison between groups was close to significance (F_2,32=2.70, P=0.08). Comparing within each group the proportion of choices of the CS− to a theoretical value of 50% showed that, while the water group had a non-significant performance (t₇=0, P=1), the quinine and the NaCl groups avoided significantly the CS− (quinine group: t₁₂=5.50, P<0.001; NaCl group: t₁₃=2.51, P<0.05). These results thus reaffirm that in the operant context of the Y-maze, quinine and NaCl had a strong and similar inhibitory effect.

Our results show that the impact of some compounds used as negative reinforcements in learning experiments with honey bee varies with the experimental context chosen to train the animals. In both the Pavlovian context and the operant context of the Y-maze, the solution of 3 mol l⁻¹ NaCl induced better olfactory discrimination, robust mid-term memories and efficient transfer of these memories from the Pavlovian to the operant context. Also, in both the Pavlovian and the operant contexts, distilled water induced less olfactory discrimination, less retention and no or less memory transfer from the Pavlovian to the operant context. However, a clear variation was found for the solution of 60 mmol l⁻¹ quinine. In olfactory PER conditioning, quinine induced intermediate acquisition and discrimination between the CS+ and the CS− (Figs 2A and 3A), as well as intermediate memory retention (Fig. 3B,C). After PER conditioning, it induced only a partial and non-significant memory transfer to the operant context of the Y-maze (Fig. 2B,C). Conversely, in the operant context of the Y-maze, quinine had a more powerful effect, similar to that of NaCl and definitely stronger than that of distilled water (Fig. 5). In the Y-maze, it supported efficient discrimination learning (Figs 4 and 5A) and retention (Fig. 5B,C). These results thus show that quinine does not have the same impact act as a negative reinforcement in two different experimental contexts in which the common task was to learn to discriminate between the same two odorants.

Experiments 1 and 2 allowed us to compare the impact of different substances as negative reinforcements in Pavlovian olfactory PER conditioning. In this protocol, pairing distilled water, quinine or NaCl with a CS− induced different discrimination learning from a CS+ paired with sucrose solution (Figs 2A and 3A). Whereas 3 mol l⁻¹ NaCl clearly improved odor discrimination thus revealing a stronger aversive effect, distilled water induced a lower level of discrimination consistent with its less-aversive nature; 60 mmol l⁻¹ quinine induced an intermediate discrimination level. The results of the transfer tests in the Y-maze (experiment 1; see Fig. 2B,C) reflected those obtained in olfactory PER conditioning because only NaCl induced significant CS+ preference and CS− avoidance, thus confirming its stronger aversive nature. Water and quinine induced non-significant biases, thus revealing that the bitter substance had a lower impact because its pairing with a CS− did not enhance olfactory discrimination from a CS+ associated with sucrose solution. These results were confirmed by experiment 2 where the pattern of acquisition was the same as in experiment 1 and where retention tests showed again a higher level of specific memory in the NaCl group, a significantly lower level of specific memory in the water group and an intermediate level of specific memory in the quinine group (Fig. 3C), consistent with the results obtained in the acquisition phase of the experiment. Note that in experiments 1 and 2, the stimulation time with the negative reinforcements was the same for all bees (6 s; see the Materials and methods) so that differences in performance cannot be attributed to differences in US exposure.

The fact that quinine was not experienced as an intense aversive reinforcement accounts for previous results showing that bees in the same conditions as those in experiments 1 and 2, i.e. harnessed in individual metal tubes, ingest without reluctance different kinds of noxious substances, including bitter ones, even if they die as a consequence of the malaise induced by this ingestion (Ayestarán et al., 2010). Clearly, for these bees, quinine solution (10 and 100 mmol l⁻¹) and other pure bitter substances (e.g. salicin, amygdaline, l-canavanine) are not penalizing enough to induce rejection when delivered to the mouth parts even if 2 h after ingestion, some of these substances (e.g. quinine) induce high levels of mortality (50% and 40% for quinine solutions of 10 and 100 mmol l⁻¹, respectively).

Taken together, experiments 1 and 2 show that different US induce different memories because of their different aversive effects. For harnessed bees subjected to olfactory PER conditioning, quinine solution was not the strongest aversive reinforcement because it yielded only intermediate levels of acquisition, retention and specific memory. NaCl, by contrast, was the most aversive US reinforcement as it induced better discrimination, retention and specific memory levels.

In the context of the Y-maze, the quinine solution used as a negative reinforcement was definitely a more-efficient aversive US than distilled water (Fig. 5A) and induced acquisition and retention levels comparable to those induced by the NaCl solution (Fig. 5B,C). In this case, the hypothesis that a bitter compound acts as a powerful aversive stimulus was proved, in contrast to results observed in olfactory PER conditioning.

Contrary to PER conditioning, and as a result of the operant framework of experiments 3 and 4, it was not possible to ensure that the time of exposure to the negative reinforcement was the same in all three groups. In the maze, the bees control their US exposure. Yet, the learning curve of the water group was always lower than those of the NaCl and the quinine groups which ran parallel to each other; this means that water bees made more errors that brought them more frequently to water than the quinine and the NaCl bees did with their respective negative reinforcements. This, however, did not make water a more powerful negative US. The fact that in this protocol quinine was as strong as NaCl cannot be explained by a higher number of errors in the quinine group (i.e. by more contacts with the quinine solution) because the NaCl and the quinine groups exhibited a similar number of errors throughout acquisition.

Our results thus confirm findings from experiments in which free-flying bees were trained to discriminate a visual target paired with sucrose solution from one or various visual distracters paired with quinine solution, under the assumption that quinine would act as a strong penalty favoring learning and discrimination (Chittka et al., 2003; Dyer and Chittka, 2004; Avarguès-Weber et al., 2010; Reser et al., 2012; Avarguès-Weber and Giurfa, 2014). In our experiments, bees did not fly within the small Y-maze, but as in the works just mentioned, they could move, explore the maze and compare the odor stimuli and their corresponding outcomes through sampling. This capacity to move and actively express choice behavior, not granted by olfactory PER conditioning, seems crucial to favor discrimination. However, it may also make the quantification of this discrimination difficult because bees could show a decrease of CS+ choices, not only because of a learning or memory deficit, but potentially because of exploratory behavior in the maze. This effect may be particularly strong in a transfer test from PER conditioning to the maze: if, after harnessing, the animal recovers the capacity to move and explore the novel environment of the maze, it may retrieve its olfactory memories, leading them to the CS+ but, at the same time, feel inclined to visit both arms of the maze. These opposing tendencies might explain why the level of correct choices decreases from PER conditioning to the test in the Y-maze, and affects both the appetitive and the aversive memories.

Our results show that memory transfer was possible from PER conditioning to the Y-maze, but not in the opposite direction. The main difference between both experimental contexts resides in the freedom to move granted by the Y-maze but not by the contention tubes. Olfactory PER conditioning is a Pavlovian protocol supporting CS−US (odor-gustatory reinforcement) associations, over which the behavior of the conditioned bee has no control (Bitterman et al., 1983). Olfactory discrimination learning in the Y-maze also supports Pavlovian associations as the trained bee learns associations between odors and gustatory reinforcements; yet, in this case, the choice behavior of the insect is crucial and determines the reinforcement outcome experienced effectively. Thus, while both Pavlovian and operant associations are formed in the Y-maze, the latter are determinant for the animal to perform correctly in this context.

In our experiments, only bees that succeeded in learning the CS+ effectively in their respective training contexts were subjected to transfer tests. Differences in transfer performance were not due, therefore, to differences in CS+ learning, but only to the way the bees evaluated the CS− penalty. Differences in the aversive strength of CS− reinforcers, together with exploratory behaviors (see above), account for the transfer performances observable in the Y-maze after PER conditioning. In this case, the bee that recovers the freedom to express the contents of its memory behaviorally shows odor discrimination whenever the CS− reinforcement is strong enough (as in the case of NaCl) to overcome the potential obscuring effect of exploratory behavior.

The situation is, however, different when the animal loses behavioral freedom by being transferred from the Y-maze to the individual harnesses for testing PER. In this case, no PER response could be recorded, irrespective of the CS− reinforcers used for conditioning. Olfactory memories were not lost during the 2 h period elapsed since the end of training in the maze and the transfer to the tubes for PER testing, as shown by experiment 4. However, they were not expressed in contention, in the form of PER. The fact that the same result was found for all three CS− reinforcers shows that the absence of responses was a general effect, due to the novel situation imposed on the bees. Harnessing bees in tubes implies a loss of motor freedom and requires the expression of a behavior, PER, that was not necessarily contingent with sucrose in the maze. Thus, while recovering freedom by being transferred from PER conditioning to the maze may be experienced as a positive situation, the reverse situation, from maze conditioning to immobilization in the tubes, may be experienced as a negative, stressful situation inhibiting memory expression.

Various studies have shown that the olfactory experience acquired by bees in a context can affect their subsequent behavior in a different context (e.g. Gerber et al., 1996; Sandoz et al., 2000; Chaffiol et al., 2005; Mc Cabe and Farina, 2009). However, none of these previous studies provided the precise control of experience and/or the bi-directional transfer in both contexts achieved in our work. Both features suggest that the constraints imposed by the Y-maze and by the contention tubes in our work constitute different challenges for a bee transferred from one context to the other. While a transfer from the contention tubes to the maze may be experienced as a positive situation as a result of the recovery of freedom of movement, the loss of this freedom in the reverse transfer from the maze to the contention tubes may be accompanied by high stress levels that need to be reduced before observing the expression of memories acquired in the operant context. These higher levels of stress may trigger a series of physiological processes underlying a change in perception of quinine strength. For instance, stress activates an opioid-like system in the bee (Núñez et al., 1997; Balderrama et al., 2002), which leads to a potential increase of tolerance to harmful stimuli, without affecting the response to appetitive stimuli (Balderrama et al., 2002). Under these conditions, tolerance to the effects of quinine, either pre- (taste) or post-ingestive (malaise) might be increased.

The concentrated NaCl solution was the strongest aversive reinforcement in both contexts. This was probably due to the fact that concentrated saline solutions disrupt osmotic equilibrium and lead rapidly to death. In the case of the NaCl solution used in our experiments (3 mol l⁻¹), ingestion of 20 µl induces 80% mortality 2 h after ingestion (Ayestarán et al., 2010), thus showing its lethal effects. Distilled water, by contrast, was the weakest aversive reinforcement in both contexts. This is not surprising because water is a neutral substance with no dramatic impact on the physiology of the bees. Quinine solution was the reinforcement whose strength varied between the Pavlovian and the operant contexts. What, besides stress-associated mechanisms, could underlie such variation?

Another possible mechanism of aversive-strength regulation relies on NPY-type peptides, which play a prominent role in management of stress responses and emotion in mammals (Bannon et al., 2000; Thorsell et al., 2000; Thorsell and Heilig, 2002). In Drosophila, neuropeptide F (NPF), the counterpart of the mammalian neuropeptide Y (NPY), also promotes resilience to diverse stressors and prevents aggressive behavior (Wu et al., 2003, Wu et al., 2005a,b). In particular, NPFR1, a G-protein-coupled NPF receptor, exerts an inhibitory effect on larval aversion to diverse stressful stimuli. This system also regulates the propensity to feed on potentially toxic food in Drosophila larvae (Wu et al., 2005a); larvae are more prone to feed on foods adulterated with 0.5% quinine with longer deprivation and stress periods (Wu et al., 2005a) and this risk-prone feeding is associated with higher levels of expression of NPFR1, whose up-regulation is sufficient to trigger intake of noxious food in non-deprived larvae. Conversely, disruption of neural NPFR1 signaling in deprived animals leads to a decrease in noxious-food feeding (Wu et al., 2005a). In the honey bee, two NPY-related genes were identified, npf and snpf (Hummon et al., 2006), but a receptor for only the short (s) peptide (sNPFR) was found (Hauser et al., 2006); sNPFR is up-regulated by food deprivation, which may constitute a form of stress (Ament et al., 2011). We thus suggest that the experimental constraints imposed by the contention tubes of the Pavlovian preparation may result in a stress-induced up-regulation of sNPFR, which in turn will promote intake of quinine solution.

So far, we have no proof concerning the existence of specific receptors for bitter taste at the peripheral level in the gustative system of the bee (de Brito Sanchez et al., 2005, 2014; de Brito Sanchez, 2011). The present results suggest that a peripheral detection of quinine during learning in the Y-maze is possible, and would be a fruitful line of investigation for further research. Indeed, odor discrimination is fast in this context so that post-ingestive malaise would not have time to develop. Low mortality was observed in our experiments (<5%), thus suggesting that the bees of the quinine group did not ingest the quinine but based their avoidance on brief taste contacts, as has previously been observed in experiments with free-flying honeybees (Avarguès-Weber et al., 2010). These contacts could have occurred at the beginning of the training phase and may be sufficient to prevent the bees from choosing the CS− odorant. The honey bee genome (Honeybee Genome Sequencing Consortium, 2006) showed that in the reduced set of gustatory receptor genes of this insect, no homolog of the bitter-taste receptor genes of Drosophila could be found (Robertson and Wanner, 2006). Thus, the mechanisms underlying peripheral detection of pure quinine solution remain to be determined.

Our study shows that some gustatory processes can be modulated by experience. This is an important conclusion that not only has practical consequences for the design of experiments involving taste input, but also has more general consequences, as it underlines that perception is not absolute but may also be subjected to top-down modifications depending upon experience and context. Further studies should focus on the mechanisms underlying such modulation by exploring, among others, the hypotheses raised in our work.

Honey bee (Apis mellifera Linnaeus 1758) foragers, from a hive located 50 m from the laboratory, were caught at the hive in the morning. They were placed in glass vials, and cooled on ice until they stopped moving.

In both the olfactory conditioning of PER and in the Y-maze, bees were trained using a differential conditioning procedure to discriminate two odorants, one of which (CS+) was always paired with 1 mol l⁻¹sucrose solution, while the other (CS−) was paired either with distilled water (control group), 60 mmol l⁻¹ quinine solution (bitter group) or 3 mol l⁻¹ NaCl (NaCl group). We thus aimed at determining whether the presence of a putative bitter reinforcement (quinine) on the CS− enhanced olfactory discrimination. Quinine (60 mmol l⁻¹) was chosen as it is highly concentrated and improves visual discrimination of small color differences in free-flying bees (Avarguès-Weber et al., 2010). Quinine solutions of lower (10 mmol l⁻¹) and higher concentrations (100 mmol l⁻¹) induce high mortality when ingested by harnessed bees (Ayestarán et al., 2010) so the solution used in our experiments was potentially noxious to the bees if imbibed. NaCl (3 mol l⁻¹) was chosen because its association with a CS− increases discrimination performances in olfactory PER conditioning (Getz et al., 1986).

The odorants used for conditioning were 1-nonanol and 1-octanol, which are hardly discriminable from each other in olfactory PER conditioning, when one of them is paired with sucrose and the other with absence of it (Guerrieri et al., 2005). Eugenol, was used as a novel odorant in the retention tests. It was chosen because it is easily discriminable from 1-octanol and 1-nonanol and induces low generalization from the conditioned odors (Guerrieri et al., 2005). All chemicals were obtained from Sigma-Aldrich (Saint Quentin Fallavier, France).

In both the olfactory conditioning of PER and in the Y-maze, the CS+ was always paired with sucrose solution 1 M. The CS− was paired either with distilled water (control group), 60 mmol l⁻¹ quinine solution (bitter group) or 3 mol l⁻¹NaCl (NaCl group). We thus aimed to determine whether the presence of a putative bitter reinforcement (quinine) on the CS– enhanced olfactory discrimination. Experiments were fully balanced, i.e. independent groups were trained either with 1-nonanol as CS+ and 1-octanol as CS− or with the reverse contingency. An independent group was used for each negative reinforcement (water, quinine, NaCl) so that six groups were conditioned in total.

Bees were individually harnessed in small tubes so that they could only move their antennae and mouthparts, including the proboscis (Fig. 1A). They were then fed with 2 µl of 1 mol l⁻¹ sucrose solution and kept in the dark and in high humidity for approximately 2 h (Matsumoto et al., 2012). Fifteen minutes before conditioning, each subject was checked for intact PER by lightly touching the antennae with a toothpick imbibed with 1 mol l⁻¹ sucrose solution without subsequent feeding. Hungry, motivated bees respond to this stimulation by extending reflexively the proboscis to lick the sucrose. Extension of the proboscis beyond the virtual line between the open mandibles was counted as a response (Fig. 1A). Animals that did not exhibit a PER at this stage were not used in the experiments (<5%). Odors were delivered by means of an olfactometer (Fig. 1A), which sent a constant clean-air stream in which odor pulses of known duration could be introduced. Four µl of each odor were applied onto a filter paper placed within a syringe connected to the odor-delivery setup. The air stream was produced by an air pump (Rena Air 400, Annecy, France) and directed to the relevant syringes by means of electronic valves (Lee Company S.A., Voisins-le-Bretonneux, France) controlled by a computer. In the absence of olfactory stimulation, the air stream passed through a syringe containing a clean piece of filter paper (clean air stream). During olfactory stimulation, the air stream was directed to a syringe containing a filter paper loaded with odor. After 6 s stimulation, the air stream was again redirected to the odorless syringe until the next olfactory stimulation. The whole setup was placed in front of an air extractor, which impeded the accumulation of residual odors after delivery of an olfactory stimulus.

Conditioning (experiments 1 and 2) consisted of 6 CS+ and 6 CS− trials, presented in a pseudo-random sequence starting with the CS+ or the CS− odor in a balanced way. The intertrial interval was 10 min. Each conditioning trial lasted 30 s. A trial started when a harnessed bee was placed between the olfactometer and the air extractor for 10 s to allow familiarization with the training situation. Thereafter, the CS was delivered for 6 s. Three seconds after CS onset, the antennae and the proboscis were stimulated for 6 s with a toothpick soaked in the US solution (sucrose in CS+ trials, and water, quinine or NaCl in CS− trials). If PER occurred, the bee was allowed to feed during this stimulation period. Otherwise, the proboscis was extended by means of the toothpick, and the solution was brought into direct contact with the proboscis for the same period. In all cases we observed that the drop at the tip of the toothpick disappeared, suggesting that the amount imbibed was similar for all solutions. The interstimulus interval was 3 s. The bee was left in the conditioning place for 11 s and then removed.

During each trial, we recorded whether the bee extended its proboscis after CS onset and before US onset (conditioned response). We then established and represented the percentage of bees exhibiting a conditioned response to the CS+ and the CS− during conditioning trials.

For retention tests under harnessing conditions (experiments 2 and 3), bees were presented with the CS+ and the CS− odorants used during olfactory PER conditioning as well as with the novel odor eugenol in order to assess the specificity of the retrieved olfactory memories (Matsumoto et al., 2012). Eugenol is easily discriminable from 1-octanol and 1-nonanol used for conditioning so that generalization between conditioned odors and the novel odor was negligible (Guerrieri et al., 2005). Odorant stimulation was identical to that of conditioning trials (6 s) but no US was given. Odor presentations were spaced by 10 min. The sequence of odors varied randomly between bees. No changes were detected depending on odor sequence. After the tests, PER to the sucrose US was checked once again. Animals unable to show PER at this point (<1%) were not considered for the analyses.

Each bee was confined in an individual glass tube where it could move and placed afterwards twice in the Y-maze deprived of odors (Fig. 1B) to familiarize it with the set-up (pre-training) (Carcaud et al., 2009). A small drop of sucrose solution was placed in the decision zone of the maze, at the intersection of both arms, to incite the bee to further explore the set-up. After getting the sucrose reward, the bee was returned to its tube, the filter paper covering the floor of the maze replaced with a clean one.

The odorants and reinforcements used to condition the bees were the same as for olfactory PER conditioning (see above). Originally, the acrylic Y-maze was positioned under homogeneous red light (Carcaud et al., 2009), provided by a cold light source in a dark room, which prevented the bees from using visual cues for orientation and from trying to fly. However, preliminary experiments (not shown) demonstrated that identical performances were obtained in the laboratory under normal room light. Thus, to avoid a change from dark to light between the Y-maze and the PER conditioning, experiments were performed under room light.

The entrance channel and the arms of the maze were 1.9 cm high, and 8 cm and 6 cm long, respectively. The arms were at a 90 deg angle, each at 135 deg from the entrance channel. The maze was placed on a rectangular base (13.5×14.5 cm) from which it could be removed to be cleaned with ethanol. The maze was covered by a glass plate (10×15 cm). The floor of the maze was covered by a piece of filter paper, which was replaced by a clean one after each visit of a honeybee to the maze.

The entrance to each arm was defined at its narrowest point, connecting the arms to the entrance channel (see dotted lines, Fig. 1B). In each arm, a micropipette tip containing a piece of filter paper (1×20 mm) loaded with 4 μl odor substance was inserted into a hole in the floor. The tips were sealed at their bottom and covered with a plastic net hood at their top to avoid direct contact with the chemicals. Each tip was placed 1.5 cm from the arm entrance, so that honeybees entering an arm experienced the odor emanating from it. An air stream (15 ml min⁻¹) filtered by active charcoal was humidified and driven from the back of each arm by means of plastic tubes. This allowed the odors to be driven towards the decision area of the maze. The tips were renewed during the experiment approximately every hour, or if the bee walked on one of them. The glass plate covering the maze impeded bees from escaping and allowed better concentration of odors.

One and a half µl of each US solution was placed on a plastic disc positioned close to the back wall of each arm and at 3 cm of the odor tips. In this way, a bee entering an arm of the maze perceived first the odor and then the reinforcing solution, thus creating the conditions for a forward pairing between odor and its associated reinforcement.

Conditioning consisted of 12 training visits (12 trials) to the Y-maze presenting the conditioned odors. The bee selected for training could freely walk and choose between the odorants presented in the arms of the maze. Between visits, the bee was returned to its glass tube, the Y-shaped filter paper covering the floor of the maze was changed and the glass plate and the maze cleaned with alcohol. Special care was taken to always eliminate all possible traces of alcohol that could affect the bee's choice.

As in the olfactory PER conditioning, the CS+ was always paired with 1 mol l⁻¹ sucrose solution whilst the CS− was paired either with distilled water (control group), 60 mmol l⁻¹ quinine solution (bitter group) or 3 mol l⁻¹ NaCl (NaCl group). The position of the CS+ and the CS− and of their associated reinforcers was interchanged between arms following a random sequence in order to avoid the development of side biases.

During each trial, we noted the first arm chosen by the bee. The first choice could be correct, i.e. choice of the arm with the CS+ leading to sucrose solution or incorrect, i.e. choice of the arm with the CS− leading to the negative reinforcement. If the choice was correct, the entrance of the negative arm was immediately blocked by means of a plastic wall to impede the bee experiencing the negative reinforcement. If the choice was incorrect, the bee was free to move to the arm with the CS+, and the negative arm was then blocked as explained above. Once the bee drank the sucrose solution and went back to the decision chamber, it was captured and brought back again to its glass tube until the next conditioning trial.

Two retention tests separated by 10 min were performed: in one test (learning test), the CS+ and the CS− odors previously conditioned were presented against each other; in the other test (avoidance test), the CS− odor was presented against a novel odor which was eugenol as in retention tests following PER conditioning (see above). The sequence of tests varied randomly between bees and no changes were detected depending on test order. No reinforcement was provided during the tests. The learning test allowed measuring retention and transfer of the information learned in the Pavlovian context to the operant context; the avoidance test allowed to determine to what extent the CS− learned in the Pavlovian context induced avoidance and thus choice of the novel odor eugenol.

From one bee to the next, the placement of the test odors was swapped between arms, so that no effect of the sides could influence the results. Between tests, the maze was cleaned with ethanol and a new filter paper was placed to cover the floor. For every bee and test, we recorded the first choice within the Y-maze. Afterwards, the bee was removed from the maze.

At the end of the acquisition phase of olfactory PER conditioning, bees were left in their contention tubes for 23 h in an oven at 30°C and then released into individual glass tubes where they could move. From there, they were transferred, one by one, into the Y-maze for olfactory testing (experiment 1). In the case of experiment 2, which implied testing in harnessing conditions, bees were handled in the same way except that they were maintained in the contention tubes all the time.

At the end of the acquisition phase in the Y-maze, bees were individually harnessed in contention tubes and placed in a warmed oven at 30°C for 2 h. The bees were subsequently brought individually to the olfactometer (Fig. 1A) for PER testing (experiment 3). In the case of experiment 4, which implied retesting in the Y-maze, bees were handled in the same manner except that at the end of the 2 h rest period, they were released into individual glass tubes from which they could be re-transferred to the Y-maze. The 2 h period between the two phases of each experiment was chosen as it corresponds to the minimal resting period that is commonly used in PER conditioning to habituate bees to the harnessing conditions before the start of conditioning (Matsumoto et al., 2012).

PER to odors (conditioned response) was quantified as a dichotomous variable (1: yes; 0: no) and expressed as the percentage of animals exhibiting PER to an olfactory stimulation. For the retention/transfer tests after Pavlovian PER conditioning, we used only bees that responded correctly to the CS+ in the last conditioning trial, irrespective of their response to the CS−. In a differential conditioning, learning results from the excitatory response induced by the CS+ (here the odor paired with sucrose solution) and the inhibitory response induced by the CS− (here the odor paired with either water, quinine or NaCl), both leading to discrimination of the CS+ from the CS−. As our question was how the US associated with the CS− (NaCl, water or quinine) modulates learning (i.e. if a certain US was more effective to induce inhibitory learning of the CS−), it was necessary to keep the CS+ response level fixed to focus exclusively on the impact of the US paired with the CS−. This is why only efficient CS+ learners with potentially different CS− experiences were brought to the retention/transfer tests.

Acquisition performances were analyzed by means of analyses of variance (ANOVAs) for repeated measurements both for within and between-group comparisons. Monte Carlo studies have shown that it is permissible to use ANOVA on dichotomous data under controlled conditions, which are met by our experiments (equal cell frequencies and at least 20 degrees of freedom of the error) (Lunney, 1970). Post hoc comparisons were performed using Tukey tests.

For retention tests, the proportions of responses to the three test odors presented (CS+, CS− and novel odor eugenol) were analyzed by means of ANOVA of repeated measures. In these retention tests, besides quantifying the proportion of responses/choices, we established the proportion of bees exhibiting a CS-specific memory, which was defined as the proportion of bees responding only to the CS+ and neither to the CS− nor the Nod. Multiple comparisons between CS specific memory levels were performed by means of Marascuilo tests, which allow comparisons of multiple proportions. All statistical analyses were carried out using Statistica 5.5 (StatSoft, France), and differences were considered significant if the P value was smaller than 0.05.

Odor choice in the Y-maze during training was quantified as a dichotomous variable (1: correct; 0: incorrect) and represented as the percentage of animals exhibiting correct responses along the 12 conditioning trials. For retention/transfer tests, we used only bees that chose the CS+ correctly and not the CS− in the last visit to the maze. In this way, only efficient CS+ learners with potentially different CS− experiences were brought to the retention/transfer tests.

Acquisition performances in the Y-maze were analyzed by means of analyses of variance (ANOVAs) for repeated measurements both for within and between-group comparisons. For retention tests in the Y-maze (experiments 1 and 4), we quantified the proportion of choices in each dual choice test (CS+ vs CS− and CS− vs Nod). Thus, a single value per bee per test was obtained. To avoid pseudo-replication, a one-sample t-test was used to test the null hypothesis of this proportion not being different from the theoretical value of 50% (random choice). Between-group comparisons were performed by means of a one-factor ANOVA for independent measures.

We thank two anonymous reviewers for useful comments and corrections to previous versions of the manuscript. We also thank Lucie Hotier for valuable beekeeping support during our experiments.