Rules and mechanisms of punishment learning in honey bees: the aversive conditioning of the sting extension response

Honeybees (Apis mellifera) constitute a traditional invertebrate model for the study of associative learning at the behavioral, cellular and molecular level (Menzel, 1999; Menzel, 2001; Giurfa, 2003; Giurfa, 2007). For almost a century, research on honeybee learning and memory has focused almost exclusively on appetitive learning, exploiting the fact that bees can learn about a variety of sensory stimuli or to perform certain behaviors if these are rewarded with sucrose solution, the equivalent of nectar collected in flowers (Giurfa, 2007; Avarguès-Weber et al., 2011). Since the discovery of the immense potential of this appetitive behavior by Karl von Frisch (von Frisch, 1914), researchers interested in honeybee learning have concentrated on appetitive learning. Following its establishment in 1961 (Takeda, 1961), a single Pavlovian conditioning protocol – the olfactory conditioning of the proboscis extension reflex (PER) – has been used as the unique tool to access the neural and molecular bases of learning and memory in honeybees (Bitterman et al., 1983; Giurfa and Sandoz, 2012; Matsumoto et al., 2012). This protocol relies on an appetitive response exhibited by a harnessed honeybee to the unconditioned stimulus (US) of sucrose solution delivered to its antennae and mouth parts (Bitterman et al., 1983): upon such appetitive stimulation, a hungry bee reflexively extends its proboscis searching for a food reward. After pairing a neutral odorant (the conditioned stimulus, CS) and sucrose, the bee learns the association between odorant and food and extends its proboscis in response to the odorant alone (Takeda, 1961; Bitterman et al., 1983; Giurfa and Sandoz, 2012; Matsumoto et al., 2012). This protocol has been used extensively in the last five decades and has provided valuable information about the behavioral, cellular and molecular mechanisms underlying appetitive learning (Menzel, 1999; Giurfa and Sandoz, 2012).

In contrast, not much was known about the capacity of honeybees to learn aversive events in their environment. In the fruit fly Drosophila melanogaster, the other insect that has emerged as a powerful model for the study of learning and memory, aversive learning has been the dominant framework (Heisenberg, 2003; Davis, 2005; Margulies et al., 2005; Keene and Waddell, 2007; Busto et al., 2010). In the fruit fly, olfactory aversive conditioning consists of training groups of flies in a T-maze which allows alternated presentation of two different odors, one (CS+) paired with the US of an electric shock, and another (CS−) non-paired with the shock (Tully and Quinn, 1985). Retention is tested afterwards in a dual-choice situation as flies have to choose between the CS+ and the CS− without aversive reinforcement. Successful learning and retention result in CS+ avoidance. This behavioral protocol has allowed the dissection of aversive learning at the cellular and molecular level and identification of the cellular location of different aversive memory traces (Heisenberg, 2003; Davis, 2005; Margulies et al., 2005; Keene and Waddell, 2007; Busto et al., 2010).

Because of obvious differences in behavioral and motivational contexts, in addition to the impracticality of equating the nature and strength of the US, it has been difficult to compare appetitive and aversive learning in bees and flies, respectively, despite their fundamental contribution to the understanding of learning and memory at multiple levels. As a consequence, the question of whether the mechanisms underlying learning and memory in these two insect models are general or rather specific has remained unanswered. Yet, in the last 5 years, a new conditioning protocol has been established in honeybees, which was conceived to fill this gap (Vergoz et al., 2007a). This protocol is the aversive conditioning of the sting extension response (SER), which is a defensive response to potentially noxious stimuli (Breed et al., 2004). This unconditioned response can be elicited by means of electric-shock delivery to a harnessed bee (Núñez et al., 1997). As no appetitive responses are involved in this experimental context, true punishment (aversive) learning could be studied for the first time in harnessed honeybees. Here we review the multiple aspects of honeybee aversive learning that have been uncovered using Pavlovian conditioning of the SER. From its behavioral principles and sensory variants to its cellular bases and implications for understanding social organization, we present the latest advancements in the study of punishment learning in bees and discuss its perspectives in order to define future research avenues and necessary improvements.

The first attempts to study aversive learning in bees used free-flying honeybee or bumblebee foragers and avoidance learning protocols (Gould, 1986; Dukas, 2001; Chittka et al., 2003; Dukas and Morse, 2003; Avarguès-Weber et al., 2010). Even though these studies have contributed to the understanding of learning and memory in bees, they all preserved an appetitive framework. Indeed, bees were trained to find food on artificial feeders and were, afterwards, confronted with different kinds of disturbance. Their behavioral responses to these disturbances were then measured. For instance, free-flying bees foraging for sucrose solution learned to avoid flower patches infested with real or robotic crab spiders (Dukas, 2001; Dukas and Morse, 2003; Ings and Chittka, 2008; Ings and Chittka, 2009), or artificial flowers when penalized either with quinine (Chittka et al., 2003; Avarguès-Weber et al., 2010) or a puff of compressed air (Gould, 1986). Aversive treatments given in a context in which a sucrose reward is also delivered induce an increase in choice accuracy in difficult perceptual discriminations (Chittka et al., 2003; Ings and Chittka, 2008; Avarguès-Weber et al., 2010). This improvement is usually achieved via a decrease in the speed of inspection flights, which results in accurate stimulus detection and recognition.

Electric shock had also been used, although seldom, to generate avoidance of visited food sources in free-flying honeybees (Núñez and Denti, 1970; Abramson, 1986). In a pioneer study, Núñez and Denti (Núñez and Denti, 1970) delivered an electric shock to individual bees trained to collect sucrose solution at a food source and landing on a metal plate covering the feeder. Bees reduced drinking attempts upon shock delivery and showed evidence of learning the disturbance program (i.e. the timing of the electric shock) established by the experimenters. Comparable results were later obtained by Abramson (Abramson, 1986), who showed that free-flying bees quickly learned to avoid a feeder paired with an electric shock delivered upon feeding.

Despite using aversive stimulations, all these studies have in common the impossibility of accessing the nervous system in parallel with behavioral recording because they used free-flying bees. Furthermore, they all maintain an appetitive framework as they aim to inhibit the appetitive response of food search. Thus, they pose the problem of potentially confounding frameworks in the study of associative learning. This problem is also present in a variant of olfactory PER conditioning in which, after pairing an odorant and sucrose, an electric shock is delivered to the proboscis of the bee so that it learns to retract it in response to the odorant (Smith et al., 1991).

Eluding the appetitive context was first possible when a non-appetitive reflex was chosen as the behavior to be conditioned. Inspired by the work of Núñez and co-workers, who used the SER to study the presence of an opioid-like system in honeybees (Núñez et al., 1997), and by the well-established protocol of olfactory PER conditioning (Takeda, 1961; Bitterman et al., 1983; Giurfa and Sandoz, 2012), the protocol of olfactory conditioning of the SER was successfully established (Vergoz et al., 2007a). This protocol allowed punishment learning in bees to be studied for the first time independently of appetitive stimulations, as forward-pairing an odor with an electric shock results in bees learning this contingency and therefore extending their sting in response to the previously punished odor (Vergoz et al., 2007a).

To this end, bees are fixed individually on a metallic holder so that they build a bridge between two brass plates through which a 2 s mild electric shock (7.5 V) is delivered by a stimulator (60 Hz, AC current) (Fig. 1A). Bees treated in this way extend their sting reflexively in response to the electric shock (Burrell and Smith, 1994; Núñez et al., 1997) (Fig. 1B). Bees of a ‘paired group’ are trained with explicitly paired presentations of an odor (the CS) and the electric shock (the US) following an absolute-conditioning design (a single odorant reinforced). As a control for this kind of conditioning, an ‘explicitly unpaired group’ of bees is presented with unpaired presentations of odor and shock. Fig. 1C,D shows that bees from the paired group learn the odor–shock association and increase the conditioned SER to the punished odor during trials. In contrast, bees in the explicitly unpaired group show no significant change in responsiveness to the odor during trials. Thus, the increase of the SER observed in the paired group is due to associative learning and not to the simple experience with the odor and the shock. One hour after conditioning, bees of the paired group still remember the conditioned odor while bees of the unpaired group do not respond to the odor (Fig. 1C). Therefore, an aversive memory retrievable 1 h after learning is established in the paired but not in the explicitly unpaired group (Vergoz et al., 2007a).

Moreover, in a differential-conditioning design in which each bee acts as its own control, bees learn to extend their sting in response to an odor paired with an electric shock and not to respond to another non-reinforced odor. Bees are conditioned during six reinforced and six non-reinforced trials, presented in a pseudo-random sequence. The resulting learning curves (Fig. 1D) show that bees learn to discriminate between odors as a result of conditioning. Thus, olfactory conditioning of the SER is truly associative and does not rely on the simple exposure to the training stimuli, independently of their outcome (Vergoz et al., 2007a).

The capacity to learn about aversive outcomes of olfactory cues can enable bees to overcome hardwired appetitive responses. For instance, immobilized bees were trained to discriminate two odorants in a differential conditioning procedure in which one odor was neutral and the other was either an attractive pheromone (geraniol or citral) or an attractive floral odorant (phenylacetaldehyde). In all cases, bees developed a conditioned aversive response to the punished odor and efficiently retrieved this information 1 h later. No learning asymmetries between neutral odors and pheromones were found (Roussel et al., 2012). Thus, associative aversive learning in bees overcomes pre-programmed responses as bees learn to develop a conditioned aversive response to attractant pheromones and to an attractive floral odor, thereby uncovering an impressive behavioral flexibility.

Pairing an odor with the electric shock results in the odor gradually gaining control over the SER. Because the animals are restrained in individual holders, their eventual avoidance of the punished odor cannot be assessed. In Drosophila, the aversive nature of differential conditioning (CS+ versus CS−) is clear, because after a successful conditioning the flies avoid the odor paired with the shock (CS+) and choose the safe odor (CS−) in dual-choice tests (see above). The flies learn the Pavlovian association between odor and shock, but at the same time their active choice determines whether or not they receive the shock, which suggests that operant associations are also learned in this protocol, which is presented as being exclusively Pavlovian.

In the case of olfactory SER conditioning, the term ‘aversive’ could be considered inappropriate given that no response inhibition or avoidance is observed after successful conditioning and that the orientation behavior of bees towards the CS was never evaluated. In order to determine whether conditioned bees explicitly avoid the CS as a consequence of the odor having acquired an aversive value, Carcaud and colleagues (Carcaud et al., 2009) conditioned bees in a differential conditioning protocol (one odor punished with shock and the other not) and then released them individually in a mini Y-maze under red light (i.e. in the dark for bees) in which the two odors used as CS+ and CS− were presented (Carcaud et al., 2009). To provide a comparison with appetitive conditioning, they also conditioned bees following differential conditioning but using the appetitive PER protocol (one odor rewarded with sucrose solution and the other not). These bees were also released individually in the Y-maze and their orientation behavior towards the two odors used as CS was evaluated (Carcaud et al., 2009). The question raised was whether SER-conditioned bees would avoid the CS+ in accordance with the aversive punishment associated with it, while PER-conditioned bees would approach it in accordance with the appetitive sucrose reward associated with it.

Fig. 2 shows that both groups of bees (PER- and SER-conditioned bees) efficiently learned to discriminate between two odorants with different valence. PER-conditioned bees significantly increased the PER to the CS+ and decreased it to the CS−. SER-conditioned bees also learned to differentiate the CS+ from the CS− in the course of training and significantly increased the SER to the former and decreased it to the latter. In both cases, bees that performed correctly in the last two blocks of trials, responding only to the CS+ and not to the CS−, were tested 1 h after the end of conditioning in the Y-maze.

Once in the maze, bees that learned the appetitive discrimination preferred the odor previously paired with sucrose (Fig. 2C). In contrast, bees that learned the aversive discrimination avoided the odor previously paired with the shock, thus preferring the previously non-reinforced odor (Fig. 2C). The inhibitory, aversive nature of SER conditioning was, therefore, revealed by this avoidance behavior, which was expressed when the bees had the opportunity to freely choose between CS+ and CS− (Carcaud et al., 2009).

In these experiments, foragers captured at the hive entrance when departing from the hive were used. The possibility cannot be excluded that the same experiments performed with guards would yield a different result (i.e. bees orienting towards the odor paired with shock and exhibiting the SER) or that providing contextual stimuli such as odors from the hive or social pheromones within the Y-maze may also change the response of the bees towards the odor previously punished. Despite these particularities, the results obtained so far demonstrate that SER conditioning in honeybees is a true case of aversive conditioning (Carcaud et al., 2009).

Restrained honeybees showed the presence of aversive memories in retention tests performed 1 h after conditioning (Vergoz et al., 2007a; Carcaud et al., 2009). This period corresponds, in appetitive PER conditioning, to mid-term memory, which is independent of protein synthesis and thus relatively labile (Menzel, 1999). The appetitive protocol, however, leads to the formation of more stable memories, including long-term memories, which can last the entire lifetime of a bee (i.e. 2–3 weeks in the case of an active forager) if no other odors are learned that may interfere with the original learning (Menzel, 1999). One pairing of an odorant with sucrose (i.e. one conditioning trial) leads to an early long-term memory (e-LTM) that can be retrieved 24–48 h after conditioning. This e-LTM depends on translation but not on gene transcription and is not, therefore, affected by transcription inhibitors such as actinomycin D. Three conditioning trials, however, lead to a stable late long-term memory (l-LTM) that can be retrieved 72 h or more after conditioning. Unlike e-LTM, l-LTM requires gene transcription and can therefore be inhibited by actinomycin D (Menzel, 1999; Menzel, 2001; Eisenhardt, 2006; Schwärzel and Müller, 2006; Giurfa and Sandoz, 2012). Does olfactory SER conditioning also lead to the formation of different memories with different stability and persistence?

To answer this question, Giurfa and colleagues (Giurfa et al., 2009) conditioned bees in a differential conditioning protocol with spaced trials (intertrial interval of 10 min), and performed retention tests 1, 24, 48 and 72 h after training. An independent group of bees was used for each retention time. All groups learned to discriminate the CS+ from the CS− and reached comparable levels of discrimination at the end of training. After conditioning, bees responded more to the CS+ than to the CS− in all retention intervals assayed (Fig. 3A). These results show that SER conditioning leads to a robust memory that is retrievable even 3 days after training (Giurfa et al., 2009).

Such a LTM was studied with respect to its molecular basis. Specifically, the possible dependency of 3 day LTM on de novo protein synthesis was analyzed. The conditioning procedure was identical to that of the previous experiment. In the 2 h following conditioning, bees were injected in the brain through the ocellar tract with PBS (control group), anisomycin (a translation inhibitor) or actinomycin D (a transcription inhibitor). Retention performance measured 72 h after conditioning varied depending on treatment (Fig. 3B). Retention performance was significant in control bees injected with PBS but not in bees injected either with anisomycin or with actinomycin D, thus showing that both translation and transcription are essential events for LTM formation of the odor–shock association (Giurfa et al., 2009).

These results show that aversive learning can induce a robust and stable l-LTM that relies on protein synthesis as it depends on both translation and transcription. Bees have the capacity to remember aversive experiences long after they took place. The biological contexts in which such capacity could be applied are multiple. On the one hand, foragers could in this way avoid returning to food places in which negative experiences, or eventually unfulfilled expectations, occurred, thereby enhancing foraging efficiency. On the other hand, it may be adaptive to memorize and remember for long periods the smell of predators in order to exhibit appropriate defensive responses to them.

Odorants are processed at different stages in the bee brain (Fig. 4) (for a review, see Sandoz, 2011). Olfactory detection starts at the level of the antennae where olfactory receptor neurons are located within specialized hairs called sensilla. Sensory neurons endowed with molecular olfactory receptors convey information about odorants to the antennal lobe via the antennal nerve. Each antennal lobe consists of 165 globular structures called glomeruli (Galizia and Menzel, 2001). Glomeruli are synaptic interaction sites between olfactory receptor neurons, local inhibitory interneurons interconnecting glomeruli and projection neurons conveying processed olfactory information to higher order centers such as the lateral horn and the mushroom bodies (Kirschner et al., 2006). Mushroom bodies are considered to be higher order integration centers as they receive input from visual, gustatory and mechanosensory pathways in addition to the olfactory pathway (Strausfeld, 2002).

In naive bees, odorants are encoded at the level of the antennal lobe in terms of specific spatial patterns of glomerular activity (Joerges et al., 1997). These patterns can be visualized using optophysiological recordings (calcium imaging) of neural activity. Such recordings were coupled with differential SER conditioning to determine whether punishment learning induces changes in the neural representation of the learned odorants (Roussel et al., 2010). No differences were found between glomerular responses to the CS+ and the CS− in bees that learned the discrimination, in spite of the fact that in appetitive olfactory PER conditioning, changes in neural activity have been found after differential conditioning (Faber et al., 1999; Rath et al., 2011) (but see Peele et al., 2006).

A possible explanation for this lack of difference between the neural responses to the CS+ and the CS− could be that the aversive olfactory memory traces are located downstream to the antennal lobe, for instance in the mushroom bodies (Gerber et al., 2004). Another possibility relates to the timing of the neural activity recording. In this case, recordings were obtained in parallel to conditioning (i.e. during conditioning trials), taking advantage of the fact that SER conditioning enables simultaneous recording of behavioral output (sting extension) and calcium variation at the neural level. Note that such simultaneity is in principle not possible in PER conditioning because proboscis extension induces muscular activity that interferes with stable calcium-signal recordings in the brain. In the case of SER conditioning, changes in neural activity in response to the CS+ and the CS−, if any, could be only detectable some time after conditioning, upon later memory formation. Further experiments are required in which antennal lobe activity should be measured at different time intervals following conditioning. Similarly, focusing on higher order structures such as the mushroom bodies is crucial.

In appetitive PER conditioning, octopamine mediates the reinforcing properties of sucrose reward in the bee brain (Hammer, 1993; Hammer and Menzel, 1998; Farooqui et al., 2003). Pairing an odor with injections of octopamine in the bee brain leads to olfactory learning in harnessed bees, which exhibit PER to this odor (Hammer and Menzel, 1998). In the fruit fly, where octopamine also mediates sucrose reinforcement (Schwaerzel et al., 2003), dopamine was shown to mediate the aversive properties of the electric shock reinforcement used in olfactory conditioning (Schwaerzel et al., 2003; Claridge-Chang et al., 2009; Aso et al., 2010; Aso et al., 2012). More recently, the distinction between dopaminergic and octopaminergic circuits as separate substrates for aversive- and appetitive-reinforcement signaling has been reconsidered in Drosophila because an interconnection between octopaminergic and dopaminergic pathways was discovered that plays a crucial role in appetitive olfactory conditioning (Burke et al., 2012; Liu et al., 2012). Specifically, a subset of dopaminergic neurons that possesses octopaminergic receptors was found, allowing them to receive signals from octopaminergic neurons signaling the presence of sucrose. These dopaminergic neurons convey the sucrose-reward signal to the mushroom bodies. Their afferences are spatially segregated from those of other subsets of dopaminergic neurons, which convey punishment signals to the mushroom bodies (Burke et al., 2012; Liu et al., 2012).

In order to establish whether dopaminergic signaling is also crucial for aversive US signaling in bees, neuropharmacological experiments were first performed in order to block this signaling and determine whether olfactory SER conditioning was possible (Vergoz et al., 2007a). Separate groups of bees were injected with Ringer solution (control), mianserine or epinastine (octopaminergic blockers), or fluphenazine or flupentixol (dopaminergic blockers) into the brain through the medium ocellus, 30 min before differential conditioning.

Bees injected with Ringer solution learned to discriminate the punished from the non-punished odor (Fig. 5A). One hour later, they remembered the aversive association and extended their sting in response to the previously punished odorant. Octopaminergic antagonists (mianserine or epinastine) did not affect performance at any of the concentrations used in these experiments. Fig. 5B shows that mianserine-injected bees learned to discriminate between the two odorants and responded with the SER only to the odorant paired with the electric shock. Retention tests also showed significant discrimination. Thus, octopaminergic antagonists did not impair aversive olfactory learning in honeybees. Dopaminergic antagonists (fluphenazine and flupentixol), in contrast, has a dramatic effect on aversive olfactory learning. Flupentixol-injected bees did not learn to discriminate between odorants. Consequently, they did not show discrimination in the tests performed 1 h later (Fig. 5C). Fluphenazine produced similar results although it was less effective. These results showed therefore that dopamine signaling, but not octopamine signaling, is necessary for aversive olfactory learning in honeybees (Vergoz et al., 2007a).

These results prompt a precise neuroanatomical characterization of dopaminergic neurons in the honeybee brain. This characterization is necessary because although immunocytochemistry studies using an antiserum against dopamine were performed 25 years ago (Schäfer and Rehder, 1989), the technique used to stain candidate dopaminergic neurons did not allow differentiation between neurons producing dopamine (true dopaminergic neurons) and neurons incorporating dopamine.

In Schäfer and Rehder's study, dopamine-like immunoreactive neurons were identified in most parts of the brain and in the suboesophageal ganglion (Schäfer and Rehder, 1989) (Fig. 4). Only the optic lobes were devoid of staining. Approximately 330 dopamine-immunoreactive cell bodies were found in each brain hemisphere plus the corresponding suboesophageal hemi-ganglion. Most of the stained cell bodies were situated within three clusters: two (C1 and C2) below the α-lobe of the mushroom body, in the inferior medial protocerebrum, and one below the lateral calyx (C3) (Fig. 4). Other stained cell bodies lie dispersed or in small groups around the protocerebral bridge, below the optic tubercles, proximal to the inferior rim of the lobula, and in the lateral and inferior somatal rind of the suboesophageal ganglion. Because of limitations of the staining technique, not all of the dendritic arborizations and axons of these neurons could be visualized, so where and how dopaminergic circuits contact the olfactory pathway remain to be determined (Schäfer and Rehder, 1989). This information is crucial for studying where the association between the odor CS and the electric shock US takes place.

In addition, a dissection of the contribution of the three dopaminergic receptors identified in the honeybee, AmDOP1 (Blenau et al., 1998), AmDOP2 (Humphries et al., 2003) and AmDOP3 (Beggs et al., 2005), to US signaling in aversive learning is necessary. AmDOP1 and AmDOP3 are related to the vertebrate D1-like and D2-like family of dopamine receptors, respectively (Blenau et al., 1998; Beggs et al., 2005), while AmDOP2 appears to be related to invertebrate octopamine receptors and constitutes a distinct ‘invertebrate-type’ dopamine receptor (Humphries et al., 2003). From a functional point of view, it can be referred to as a ‘D1-like receptor’ because it upregulates cAMP. The lack of specific pharmacological blockers of these receptors has until now precluded straightforward analyses of their role in aversive learning. Impairment of aversive learning yields conflicting evidence with respect to this topic: while pharmacological blocking with vertebrate antagonists indicated that AmDOP2 receptors are necessary for aversive learning (Vergoz et al., 2007a), analyses of transcript levels of dopaminergic receptor genes suggested that impairment of aversive learning is associated with an increase of AmDOP2 receptors (Geddes et al., 2013). More experiments are necessary to elucidate whether and how these different receptors contribute to aversive learning.

An interesting twist to the study of aversive learning and dopaminergic signaling is the discovery that 20-hydroxyecdisone (20E), a metabolite of the steroid hormone ecdysone, which intervenes in insect development and reproduction (Riddiford et al., 2000), impairs aversive but not appetitive conditioning in bees (Geddes et al., 2013). This impairment seems to be achieved in part via the dopamine/ecdysone receptor gene AmGPCR19, which is the honeybee ortholog of the dopamine/ecdysone receptor gene 48 (DmDopEcR) identified in Drosophila (Srivastava et al., 2005). Thus, exogenous 20E injection determines both a reduction in AmGPCR19 levels and a decrease in aversive learning performance, therefore indicating that aversive learning in honeybees can be modulated by ecdysteroids (Geddes et al., 2013).

As social insects, honeybees exhibit a division of labor in which different tasks are accomplished by different groups of individuals, usually of different ages (Wilson, 1971). Several models have been proposed to explain why individuals within a social insect colony are differently biased to perform distinct tasks, resulting in task specialization (Beshers and Fewell, 2001). Among these models, the response-threshold model has played an influential role in the explanation of the division of labor in social insects. It posits that differences in sensitivity to external stimuli exist between individuals and that individuals highly sensitive to a given stimulus are prospective candidates for becoming specialized in tasks involving such a stimulus (Page and Erber, 2002). For instance, the well-established difference between nectar and pollen foragers has been explained in terms of their different sensitivity to sucrose (Page and Erber, 2002; Scheiner et al., 2004). These two groups do indeed exhibit differences in sucrose responsiveness, which is assessed by quantifying appetitive PER responses along a series of increasing concentrations of sucrose solution (Pankiw and Page, 1999). The lowest concentration at which the bee starts responding with a PER defines its sucrose responsiveness threshold. Nectar foragers exhibit higher thresholds (i.e. lower responsiveness) than pollen foragers, which exhibit lower thresholds and thus higher responsiveness (Page et al., 1998). Although this difference may appear counterintuitive at first sight, the currently accepted explanation is that nectar foragers are more selective when collecting nectar, and thus only respond to the highest sucrose concentrations, which provide the highest energy gain to the colony. Sucrose responsiveness thresholds vary with multiple factors such as age, caste, sex (Pankiw and Page, 1999), foraging experience, genotype, feeding status (Pankiw and Page, 2001) and season (Scheiner et al., 2003), among others.

The plethora of studies on sucrose responsiveness has led to the general idea that this behavioral trait can explain per se diverse behavioral responses to stimuli as different from sugar as odors or light (Scheiner et al., 2004; Erber et al., 2006). For instance, Page and colleagues (Page et al., 2006) state that, ‘Bees who are sensitive to sucrose are also sensitive to stimuli of other modalities’, so that ‘Sucrose responsiveness can be used as a robust indicator for general differences of processing information in the central nervous system’. This suggestion could, however, be erroneous as the behavioral traits that have been related so far to sucrose responsiveness all have an appetitive framework in common, i.e. they are related to foraging behavior. In a drastically different framework, in which stimuli possess a hedonic value different from sucrose or its related context, would stimulus sensitivities correlate with sucrose sensitivity? In other words, do bees that exhibit high responsiveness to sucrose also display high responsiveness to an aversive stimulus?

To answer this question, Roussel et al. (Roussel et al., 2009) determined whether sucrose responsiveness in forager bees correlates with responsiveness to electric shocks of varying voltage. Like PER for sucrose, SER allows direct quantification of response thresholds to a stimulus that, in this case, is fully independent of a foraging context. PER to a logarithmic series of sucrose solutions of increasing concentration were measured in a first phase, and SER to a series of shocks of increasing voltage were measured in a second phase. In another group of bees, the reversed sequence (first shock, then sucrose) was employed. Neither the responses to the electric shocks nor the responses to the sucrose solutions differed significantly between these two groups, thus showing that the order of stimulation was irrelevant.

Pooled responses are shown in Fig. 6. As expected, bees significantly increased the PER to sucrose solutions of increasing concentration and, similarly, bees significantly increased the SER to electric shocks of increasing voltage. The increase in PER and SER with sucrose concentration and voltage does not, however, answer the question of whether the bees responding more to concentrated sucrose are also those responding more to the highest voltages. To answer this question, both a sucrose responsiveness score and a shock responsiveness score were determined for each bee (Roussel et al., 2009). Scores were quantified as the sum of all responses made along the whole sequence of tested stimulations. For instance, a bee extending its sting at voltages from 0.5 to 8 V, i.e. in response to five out of the six voltages assayed, had a shock responsiveness score of 5 as it responded to five consecutive voltages. This bee also had a sucrose responsiveness score derived from its response to the six concentrations of sucrose solution. Scores may therefore vary from 0 (no response to any stimulus tested in the series) to 6 (responses to all six stimuli of the series) (Fig. 6C).

The results of this analysis are thus represented as a 7×7 matrix in which one axis is defined by sucrose responsiveness scores and the other axis by shock responsiveness scores (Fig. 6C). Colors assigned to each box represent the percentage of bees exhibiting a particular combination of sucrose and shock responsiveness scores. Fig. 6C shows no clear relationship between appetitive and aversive responsiveness, i.e. the correlation analysis performed on the two scores was non-significant. In other words, there is no correlation between responsiveness to two stimuli of opposing hedonic value such as sucrose and electric shock (Roussel et al., 2009).

In studies in which the PER was used, correlated responsiveness was found for stimuli that are related to the appetitive search for food in which bees engage during foraging activity (Scheiner et al., 2004; Humphries et al., 2005; Erber et al., 2006). It seems coherent that responsiveness to odors (which are characteristic of food sources) and to light (which elicits foraging flight), as well as motor activity, are correlated in the same bees (Scheiner et al., 2004; Humphries et al., 2005; Erber et al., 2006). This variety of related sensitivities can be grouped in a ‘foraging behavior syndrome’ (Pankiw, 2005), defined as a set of correlated behaviors reflecting between-individual consistency in behavior across multiple foraging situations (Sih et al., 2004). Yet, this syndrome may just constitute a part of the complex behavioral tuning within a hive.

Several behavioral syndromes may coexist in an insect society. A ‘defensive behavior syndrome’ could be conceived, in which a correlated set of defensive traits could be linked to sensitivity to electric shock. For instance, responsiveness to shock could correlate with defensive responsiveness to alarm pheromone components such as isopentyl acetate (IPA), the main component of the sting pheromone (Boch et al., 1962), and 2-heptanone, an alarm substance released by mandibular glands (Shearer and Boch, 1965). Foraging and defensive syndromes would constitute independent insulated modules coexisting within the same individual and defining its tendency to act as a forager or as a defender (Roussel et al., 2009).

In Pavlovian learning, in which an animal learns that a CS acts as a predictor of the US, sensitivity to the US, which directly determines its salience for the animal, plays a crucial role in learning efficiency and rate (Rescorla and Wagner, 1972). Higher sensitivity to an US results in better learning performance as shown by studies relating sucrose sensitivity and appetitive PER conditioning; bees that are highly sensitive to sucrose show better appetitive learning performance (Scheiner et al., 1999; Scheiner et al., 2001a; Scheiner et al., 2001b; Scheiner et al., 2003; Scheiner et al., 2005). Does shock responsiveness affect olfactory aversive learning in bees in a similar way?

To answer this question, shock responsiveness scores were determined in a group of honeybee foragers (see above), which were then divided into two subgroups according to their scores: bees exhibiting the highest response selectivity and responding only to the highest shock voltages (score 1–3; ‘low responsiveness group’) and bees exhibiting generalized, non-selective responses to 4–6 of the voltages tested including lower ones (‘high responsiveness group’). The next day, bees were trained in a differential conditioning procedure to discriminate an odor paired with a shock (CS+) from an odor not paired with a shock (CS−). Both groups of bees learned to discriminate the CS+ from the CS− and remembered this information 1 h later. Yet, the high responsiveness group showed a higher percentage of conditioned responses to the CS+ than the low responsiveness group. Responses to the CS− did not differ between groups. In the retention tests, bees of the high responsiveness group also responded more to the CS+ than did bees of the low responsiveness group while no differences were found for the CS−.

These results show that the more responsive a bee is to electric shocks, the better it learns to associate an odor with this noxious stimulus. Similarly, in the case of sucrose reinforcement, the more responsive a bee is to sucrose, the better it learns and memorizes CS–US associations in appetitive olfactory and tactile learning protocols (Scheiner et al., 1999; Scheiner et al., 2001a; Scheiner et al., 2001b; Scheiner et al., 2003; Scheiner et al., 2005). Taken together, these results underline the crucial role of US sensitivity for learning and retention performance as underlined by models of classical conditioning, where US salience directly affects learning rate (Rescorla and Wagner, 1972).

We have seen so far that honeybee foragers exhibit a shock responsiveness that does not necessarily correlate with sucrose responsiveness, and that their US sensitivity directly determines their learning success in olfactory SER conditioning. Do these principles apply to other honeybee castes and do castes differ from each other in terms of these variables?

To answer this question, a first study focused on a comparison between guards and foragers in terms of shock responsiveness and aversive learning. Foragers were collected upon arrival at a feeder containing sucrose solution to which they were previously trained, thus ensuring that they were real nectar foragers. Guards were collected at the hive entrance after an attack had been elicited by means of a mechanical disturbance. One day after determining the shock responsiveness scores of these two groups of bees, they were subjected to differential conditioning. Retention tests were again performed 1 h after the last conditioning trial.

Fig. 7A shows that shock responsiveness differed significantly between guards and nectar foragers, the responses of foragers to shocks being generally higher than those of guards, especially for lower voltages. Thus, guards are less sensitive to electric shocks than are nectar foragers. Fig. 7B shows that both guards and nectar foragers learned to discriminate between the CS+ and the CS− and remembered the aversive association 1 h later. Yet, although both groups responded similarly to the CS− during conditioning, at the end of training, nectar foragers responded significantly more to the CS+ than did guards. The same difference was found in the retention tests as nectar foragers remembered the CS+ significantly better than did guards (which reflected their better acquisition) but did not differ in their response to the CS−.

Thus, the more responsive, and presumably more sensitive, foragers are the ones learning and remembering better aversive associations. Although this result appears surprising, it may be adaptive for guards to be less sensitive, and presumably more tolerant, to noxious stimuli. Accordingly, they would assign low values to an aversive reinforcement, thus determining lower acquisition and retention performance. Such a low sensitivity of guards to noxious stimuli may indeed be adaptive for honeybees, as defensive responses are costly for the colony (especially when recruitment takes place), and defensive responses should not be triggered by any kind of aggression, but rather by situations that are potentially dangerous for the colony.

Neural-based explanations could account for the difference found between guards and foragers in shock responsiveness and aversive conditioning. Dopamine levels in the bee brain depend on age (Taylor et al., 1992; Schulz and Robinson, 1999) so that older bees have more dopamine in their brains. Foragers, which are generally older than guards, are more sensitive to shock and thus more likely to learn aversive associations than guards (Roussel et al., 2009). Nurse bees are the youngest adult members of the colony and stay in close contact with the queen. Dopamine levels are even lower in nurses than in guards and foragers (Taylor et al., 1992; Schulz and Robinson, 1999) and, as a consequence, shock sensitivity should be lower and olfactory SER conditioning less successful in these bees. In addition, nurses are exposed to queen mandibular pheromone (QMP) via their close contacts with the queen. QMP is a chemical blend that has priming and acute effects on social control within the colony (Sandoz et al., 2007). Among these effects, QMP induces young workers to feed and groom the queen and primes bees to perform colony-related tasks (Slessor et al., 1988; Keeling et al., 2003).

Olfactory SER conditioning of nurse bees has been studied in relation to the presence of the queen and QMP (Vergoz et al., 2007b). One of the key components of QMP, homovanillyl alcohol (HVA), bears a striking structural resemblance to dopamine. The presence of this compound within the pheromone blend suggested that dopamine function in the brain of recipient young bees might be affected by exposure to QMP (Beggs et al., 2007). Indeed, exposure to QMP, and more precisely to HVA, affected dopamine levels, levels of dopamine receptor gene expression and cellular responses to this amine in young worker bees. These results show that dopamine levels in the bee brain depend not only on age but also on contact with QMP (Beggs et al., 2007).

How does this inhibition of dopaminergic signaling affect aversive olfactory learning in young bees? To answer this question, Vergoz and colleagues (Vergoz et al., 2007b) examined the impact of QMP on associative olfactory learning in young bees (6 days old) exposed to QMP from the time of adult emergence. Bees of the same age maintained under identical conditions but without exposure to QMP were used as controls. These two groups were in turn subdivided into two groups, one trained following appetitive PER conditioning to discriminate an odor reinforced with sucrose from a non-reinforced odor (Fig. 8A,B) and another trained following aversive SER conditioning to discriminate an odor reinforced with shock from a non-reinforced odor (Fig. 8C,D) (Vergoz et al., 2007b). Both exposed and non-exposed bees learned the appetitive discrimination and showed retention 1 h later (Fig. 8A,B). Interestingly, while non-exposed young bees (Fig. 8C) learned the aversive discrimination and remembered it 1 h later, bees of the same age exposed to QMP failed to show aversive learning and retention (Fig. 8D). Thus, QMP suppresses aversive olfactory learning in young bees but leaves their appetitive learning intact (Vergoz et al., 2007b). A possible interpretation of these results is that the inhibition exerted by QMP on aversive learning increases the probability that young nurses remain in close contact with their queen by impeding aversive experiences around her (Vergoz et al., 2007b).

The effect of QMP on the associative learning of young bees resembles that of ecdysteroid hormones like 20E when injected into adult bees (see above). Indeed, 20E impairs aversive but not appetitive learning. Moreover, greater impairment of aversive learning when bees are 2 days old correlates with higher levels of endogenous ecdysone (Hartfelder et al., 2002). Recent results have shown, in addition, that like 20E (see above), QMP impairs aversive learning, inducing a concomitant reduction of the AmGPCR19 receptor (Geddes et al., 2013). The ecdysone/dopamine signaling pathway would therefore be implied in aversive US signaling as well as in social regulation.

Taken together, these results show how aversive olfactory SER conditioning has helped in uncovering unsuspected aspects of social organization and division of labor within the hive. These articulate on specific and variable stimulus sensitivities, which in turn reflect complex regulation of biogenic amine levels and neural signaling, which determine not only distinct aversive learning performance but also different behavioral roles and syndromes (i.e. sets of correlated behaviors across situations) (see Sih et al., 2004) within the hive.

The experiments presented so far used olfactory SER conditioning to answer questions focusing on various topics, from learning and memory to social organization in honeybees. Besides this rich spectrum of research, SER conditioning has also recently been achieved using visual rather than olfactory stimuli as CS (Mota et al., 2011b).

In this protocol, which allows the study of visual learning and memory in intact harnessed bees in the laboratory, two visual stimuli are used as CS+ and CS−. Bees learn to discriminate between CS+ and CS− by using, for instance, chromatic cues (Fig. 9). It should be noted, however, that acquisition levels in visual SER conditioning can be lower than in olfactory SER conditioning. Further improvements are thus necessary in visual SER conditioning to enhance the efficiency of the protocol.

Despite this, the fact that the SER can be visually conditioned opens new doors for accessing the neural correlates of visual learning and memory in honeybees. Indeed, visual learning in bees has been studied for almost 100 years using almost exclusively free-flying bees conditioned to choose visual targets paired with sucrose solution (von Frisch, 1914; Avarguès-Weber et al., 2012). This approach revealed numerous aspects of an organism, the honeybee, which has emerged as a model for the study of visual processing because of its color vision and visual orientation capabilities (Menzel and Backhaus, 1991; Giurfa and Menzel, 1997; Srinivasan, 2010; Avarguès-Weber et al., 2012). Coupling behavioral measures of visual perception and invasive recordings of neural activity at the level of visual centers or pathways in the bee brain has so far been impossible. Important advances have recently been achieved in the neurophysiological study of visual processing in the bee brain thanks to the advent of in vivo optical imaging of visual neuron populations (Mota et al., 2011c; Mota et al., 2013). Visual SER conditioning could provide the basis for the necessary coupling between behavioral and neurophysiological measurements, thus allowing a qualitative improvement of our knowledge of the neural mechanisms underlying visual perception and their intrinsic plasticity.

Here, we have reviewed recent and current developments of aversive conditioning in honeybees based on the SER. This protocol, which has recently been established, gives access to punishment learning in honeybees, which have been a traditional model for studying reward learning. The fact that bees are harnessed but nevertheless exhibit learning and retention of CS–electric shock associations has opened new research avenues to uncover neural principles of associative, aversive learning in a framework that is distinct from appetitive behavioral contexts.

Starting with the demonstration that individual bees learn odor–shock associations, a pluridisciplinary body of research has been developed spanning questions on honeybee social organization, learning and memory, neurobiology and behavior. At the crossroad of these different research avenues is the olfactory SER conditioning protocol, which constitutes, in our opinion, a significant contribution to the study of different aspects of honeybee behavior. We expect through it to overcome the monofaceted view offered for 50 years by the equivalent protocol in the appetitive domain, the olfactory conditioning of PER. The results presented in this review show that the endeavor was successful and offers further promising perspectives.

Special thanks are due to two anonymous referees and to our friend and colleague J. C. Sandoz, who played an essential role in the development of the different aspects of SER conditioning reported in this review. Thanks are also due to H. Tanimoto and A. Borst for much support.