ABSTRACT
Bumblebees were trained in biologically realistic sensorimotor tasks to test how learnt information from more than a single task is organised in memory. Bees (Bombus impatiens and Bombus occidentalis) learned to collect sucrose solution from the arms of small T-mazes. The reward was offered in the right arm of a maze when the entrance was marked blue, and in the left arm when the entrance was yellow. Bees were trained on either one or both of these tasks until after they had reached saturation in terms of speed and accuracy. One group of bees (Bombus impatiens) was evaluated for long-term retention by retesting (a) on the day after training and (b) between 3 and 4 weeks after training. Performance did not decline overnight but, when bees were tested after a delay of several weeks, an increase in error scores and times taken to negotiate mazes occurred. However, bees started out at a better level than during the initial training, indicating that information acquired during initial training had not been entirely erased from memory. A second group of bees (Bombus occidentalis) was tested to see whether it was possible to reverse the association between colour and direction, so that blue entrances now meant that the reward was on the left side, and yellow entrances meant that it was on the right. Bees reached saturation on both tasks as rapidly as they had during the initial training.
However, after a second reversal, bees chose directions randomly for several hundred trials, indicating that the first-learnt colour–direction associations had not been forgotten as a consequence of the first reversal: instead, information from the initial training must have been retained, but actively suppressed, causing difficulties in re-establishing the appropriate sensorimotor associations during the second reversal. Finally, a single Bombus impatiens worker was trained on only a single task, but with multiple reversals. The bee went through an intermediate phase when performance was poor until, after more than seven reversals, it required only three trials to assess that a reversal had taken place. This indicates that bees can learn actively to suppress irrelevant information, but only with an extended training schedule. Taken together, these results suggest that sensorimotor information is rarely lost from long-term memory, either during extended lay-offs or during interfering reversal training sessions. Organizing, correctly retrieving and suppressing information accumulated in memory seems to be more of a challenge than storing new information.
INTRODUCTION
Sensory learning and memory in insects have been studied extensively, so that we know many of the mechanisms, down to the molecular level, that drive the dynamics of the formation and retrieval of memory (Menzel, 1990; Tully, 1996). Motor learning, in contrast, has received comparatively little attention, although it is of tremendous importance in the natural world of bees: many species live in underground cavities and have to navigate to their nests in complete darkness (Kalmus, 1937; L. Chittka, in preparation), they have to build honeycombs, which requires complex, and partially learnt, skills (Oelson and Rademacher, 1979), and they have to handle flowers, which often have immensely intricate structures (Laverty, 1994). In the latter context, bees have to associate sensory inputs (colours, patterns, odours) with learnt motor patterns (Chittka and Thomson, 1996, 1997). In long-distance navigation, landmarks are used to retrieve previously memorized flight patterns from memory (Chittka et al. 1995; Collett and Baron, 1995; Zhang et al. 1996).
In this study, I test for long-term retention and reversal learning in such learnt sensorimotor associations. The objective is not so much to find out whether bees can perform such feats but to use these tests as clues to the organization of memory in bees. In humans, for example, motor memories are much more resistant to being forgotten than are sensory memories (Adams, 1987). One does not forget how to swim or how to type, even during long periods without practice (Schmidt, 1991). It is not known how well motor tasks in insects are retained over long periods. Are learnt sensorimotor and pure motor tasks as resistant to being forgotten as are sensory memories, which are retained throughout the lifetime of a bee after only a few trials (Lindauer, 1963; Menzel, 1968)? Reversal learning can be similarly informative with respect to the organization of memory. We know that bees can reverse their learned preferences in sensory learning (Menzel, 1969, 1990; Meineke, 1978), but what happens to the information stored during the initial training? One possibility is that this information is erased when the opposite information is entrained (interference). Consider the following example in a T-maze experiment: blue is rewarded in the right arm, and yellow in the left arm, during initial training; the opposite is true in the reversal entrainment. In such a situation, what bees may retain in memory is only the information from the second (reversal) training: that yellow means ‘right’ and blue means ‘left’. Alternatively, bees may memorize information from both training procedures, and learn that sometimes yellow means right and sometimes left (and the same for blue), and use additional cues to retrieve the appropriate motor patterns from memory (Menzel, 1969).
Multiple reversal tests can elucidate what cues might be used. Are bees capable of using temporal information from a reversal schedule to solve a multiple reversal task? For example, if one of two options is rewarded on every second day, and the other on the days between, can bees learn to ‘predict’ which option will be correct at the start of each day? Or are a few initial trials necessary after each reversal to assess that a reversal has taken place and that the opposite option is now correct (Mackintosh, 1974)?
I trained bees to forage from computer-monitored T-mazes, the entrances of which bear coloured marks that can be used by the bees to predict the direction in which the reward is located. Learnt associations between colours and motor directions are evaluated following delays of several weeks after training. Bees are also tested in situations where learnt sensorimotor associations are reversed. Serial reversal experiments are conducted to determine how information from more than one conflicting sensorimotor association is laid down in memory and to examine how bees might retrieve the correct association whenever the experimenter reverses the relationship between sensory cues and the motor patterns that lead to a reward.
MATERIALS AND METHODS
A colony of bumblebees (Bombus occidentalis) was raised from a queen caught in spring at the Rocky Mountain Biological Laboratory, Colorado (courtesy of James D. Thomson). A second colony of bumblebees (Bombus impatiens) was obtained from Koppert Biological Systems, Michigan, USA. They were kept in nest boxes connected to a flight arena of 60 cm×40 cm (28 cm in height) by means of a transparent plastic pipe. Manual shutters in the pipe allowed control over which bees entered the arena. A large number of bees were individually marked with number tags (Opalith-Plättchen), and only marked individuals were tested. Pollen was fed directly into the nest. Between experiments, bees could forage freely from a covered Petri dish with holes drilled into the lid, which contained 50% (v/v) sucrose solution and which was located in the centre of the flight arena. The only foraging experience the bees had prior to training was to extract sucrose solution from this openly accessible feeder. Thus, they were naive with respect to the experimental task.
From the flight arena, bees had access to four T-mazes (Fig. 1). The entrances were in one of the walls of the flight arena. They were arranged side by side, 10 cm apart. The entrance hole was square (6 mm×6 mm), and each hole was surrounded by a yellow or blue target (3 cm×3 cm). The entrance tunnel measured 14 mm, and both arms were 17 mm long; all tunnels were the same width as the entrance of the mazes. In the initial training, the reward was offered in the right arm of the maze when the entrance bore a blue mark, and in the left arm when the entrance was yellow.
Sketch of one of the four mazes; top view. FA, flight arena; IRR, infrared receivers; IRT, infrared transmitters; MS, manual shutter to control access to the flower; NR, ‘nectar’ rail, sliding under the arms of the maze, so that the rail forms the floor of the maze in the arms; NW, nectar wells (black, full; white, empty); PB, polyethylene block, which makes up the walls and the central (white) part of the maze floor; P, spring-loaded indexing pawl that engages regularly spaced notches on the nectar rail, allowing fresh nectar wells to be quickly and precisely clicked into position between visits; W, wall of flight arena (which bears the coloured marks at the entrance holes of the flowers). The diagonally hatched area at the top marks the floor of the flight arena; the nectar rail slides under this floor. The arrow marks the direction in which the nectar rail is advanced.
Sketch of one of the four mazes; top view. FA, flight arena; IRR, infrared receivers; IRT, infrared transmitters; MS, manual shutter to control access to the flower; NR, ‘nectar’ rail, sliding under the arms of the maze, so that the rail forms the floor of the maze in the arms; NW, nectar wells (black, full; white, empty); PB, polyethylene block, which makes up the walls and the central (white) part of the maze floor; P, spring-loaded indexing pawl that engages regularly spaced notches on the nectar rail, allowing fresh nectar wells to be quickly and precisely clicked into position between visits; W, wall of flight arena (which bears the coloured marks at the entrance holes of the flowers). The diagonally hatched area at the top marks the floor of the flight arena; the nectar rail slides under this floor. The arrow marks the direction in which the nectar rail is advanced.
The entrances and each arm of the mazes were monitored by an infrared light barrier. These barriers were connected to a computer so that error scores and maze times could be evaluated on-line. Under each maze arm was a sliding Plexiglas rail with wells drilled every 25 mm. On one side of the maze, these wells contained 5 μl of 50% sucrose solution; on the other side, they were empty. Each time a well had been emptied, the rail was advanced by 25 mm so that the next reward was made available. Each Plexiglas rail featured 25 wells. Bees had to visit 20–30 T-mazes before returning to the nest with a full stomach. The insides of the mazes were cleaned with ethanol after every second foraging bout. Throughout all the experiments, the mazes were covered with red acetate. I used several mazes and small rewards because I wished to simulate a ‘foraging world’ similar to that encountered by bees collecting food from natural flowers. Bees have to visit many such flowers to fill their honey stomach, and they have to use sensory stimuli (such as the colours in my experiments, and floral colours in nature) as predictors for the particular motor patterns needed to obtain access to food.
To prevent bees from simply using the position (and not colour) of a maze entrance to identify the correct direction, after every second foraging bout the blue entrance marks were exchanged for yellow ones (and vice versa), and the rails containing the rewards were exchanged accordingly, so that blue-entrance mazes would again contain the reward on the right side and the yellow ones would offer them on the left.
During the tests, bees could choose freely which T-maze to forage from next, with two restrictions: (1) they could not return directly to the maze just visited; and (2) when switching between maze types was enforced, both mazes of the previously visited type were made inaccessible. Access to the mazes was controlled by manual shutters which slid into a 1 mm crevice between the cardboard wall of the flight arena and the entrance tunnel of the mazes. A reward was offered at each visit; even when bees initially turned into the wrong arm of the maze, they were allowed to correct this error and feed in the opposite arm, without leaving the maze.
Bees were trained and tested individually. A single bee was selected from those feeding from the clear Plexiglas feeder by observing the feeder for some time and then picking a bee that arrived at that feeder with a particularly high frequency. No other bees were then allowed into the arena, and training could begin. A detailed description of how bees were enticed to visit the mazes can be found in Chittka and Thomson (1997).
Training and test schedules
Long-term retention – single-task training
Six bumblebees (Bombus impatiens) were trained for a single task, either to visit mazes with yellow entrances, in which the food was located in the left arm, or to visit mazes with blue entrance mark where food was offered on the right. Each bee visited the mazes 400 times on a single day. The number of full trials was actually somewhat lower, because some visits were aborted after the bee had entered the maze entrances but failed to turn into one of the arms of the mazes (Chittka and Thomson, 1997). These abortive visits are not counted as trials here; when they are excluded, the number of trials on day 1 ranged from 360 to 372. All bees were retested on the second day by giving them an additional 200 visits on the same task as on day 1. Four of the six bees were tested for retention between 3 and 4 weeks after their initial training (for exact intervals between training and re-testing, see Table 1). They were allowed 200 visits to the same maze type as during training.
Long-term retention – double-task training
Bees in this group had to learn two tasks, to associate both blue with right turns and yellow with left turns. Subjects were forced to alternate between tasks after each individual trial. Bumblebees (Bombus impatiens) were allowed 200 visits to each maze type. Again, aborted visits are not evaluated here so that the total number of trials is slightly lower than 400 (see Fig. 6). All bees were retested on the next day (200 trials). Three of the bees were given 400 trials on a single day, 3–4 weeks after training. Bees were allowed to forage from Petri dishes in the flight arena during this period, but not from any structures similar to the mazes.
Reversal learning – single-task training
A single worker bumblebee (Bombus impatiens) was trained to collect sucrose solution from the right arm of the mazes, all of which bore a blue entrance mark. 100 trials were allowed on the day of training. During the following 11 days, the location of the reward was changed every day, so that food was located on the left side on the second day, on the right side on the third, and so forth. The bee was allowed 100 visits on each day. The colour of the entrance marks was blue throughout the entire procedure.
Reversal learning – double-task training
Five worker bumblebees (Bombus occidentalis) were trained as described in the penultimate paragraph (200 visits to blue-entrance mazes with rewards on the right, and 200 visits to yellow-entrance mazes with rewards on the left, using a forced alternating schedule). On the day following training, three of these bees were trained to reverse both associations, so that blue entrances now meant rewards on the left and mazes with yellow entrances could be used to predict rewards on the right side. A total of 400 visits were allowed, employing a forced alternate schedule. Two of these bees had to reverse their associations again on the third day, once more using a forced alternate schedule.
Data evaluation and statistics
An error was scored when a bee initially turned into the wrong arm before feeding (Chittka and Thomson, 1997). Error scores were determined as follows. For each bee, the N ordered trials were categorized into N/10 bins of 10 consecutive trials. The errors within each bin were counted. Learning curves were established by plotting the percentage of errors in each bin of 10 consecutive visits as a function of the midpoint of the bins (see Fig. 2).
Error scores for bees trained on only one task (triangles) or on both tasks (circles) and tested after 1 day (A), 2 days (B) or more than 20 days (C). Each point categorizes 10 subsequent visits from each group. Note that the curves differ slightly from those in Chittka and Thomson (1998, their Fig. 2) because aborted visits are excluded here, so that the x-scale is different and the curves end at shorter x-values.
Error scores for bees trained on only one task (triangles) or on both tasks (circles) and tested after 1 day (A), 2 days (B) or more than 20 days (C). Each point categorizes 10 subsequent visits from each group. Note that the curves differ slightly from those in Chittka and Thomson (1998, their Fig. 2) because aborted visits are excluded here, so that the x-scale is different and the curves end at shorter x-values.
RESULTS
Long-term retention
Bees trained on only a single task started out with 25% errors and reached a practically error-free performance after 100 trials (Fig. 2A). Overnight retention was perfect: there was no significant increase in error score in the first 10 trials of day 2 compared with the last 10 trials on day 1 (χ2=2.07, d.f.=1, P=0.154; Fig. 2B). For a more detailed discussion of the results on day 1 and 2, see Chittka and Thomson (1997). After 3–4 weeks without practice, there was an increase in error score (from 0% in the last 10 trials of the initial training to 7.5% in the first 10 trials of the retention test: χ2=4.64, d.f.=1, P=0.03). However, the error score of 7.5% is still significantly lower than that at the start of the initial training (25%; χ2=4.98, d.f.=1, P=0.026), and bees rapidly recovered from their decline in performance: after more than 20 trials, they made practically no errors (Fig. 2C). Thus, the direction in which food was to be found had not been entirely erased from the memory of the bees during a no-practice interval of more than 3 weeks.
Bees trained on two tasks started out with approximately 35% errors, and there was a transient period between 30 and 100 visits when the error score was worse than at the onset of training. After approximately 150 trials, however, performance reached a consistently good level, with error scores mostly below 10%. The error score did not increase significantly on the second day (χ2=0.38, d.f.=1, P=0.54). When these bees were re-tested after 3–4 weeks, they showed a performance remarkably similar to that on day 1 (Fig. 2C): the error score started out at a comparatively low level (20% in the first bin of 10 trials), but then worsened to approximately 40% between 30 and 100 trials, and then again improved to a level of less than 10%. It thus appears that, when the bees have to make associations between colours and motor patterns (which for ‘single-task bees’ is not mandatory: they may only memorize the motor pattern itself, not its association with colour), they may have to re-learn the associations after more than 3 weeks. However, the critical question in proving that the bees have forgotten is whether performance after the delay is truly as poor as during the initial training. The answer is no: first, almost the entire curve after the delay is below that of the training on day 1 during the first 200 trials (20 bins). This difference is significant when the error scores in the 20 bins are compared by means of the sign test (Z=3.21, P=0.001). Second, a significant difference from random is established earlier after the delay (in the ninth bin, i.e. after 80 trials; χ2=5.9, d.f.=1, P=0.015) than during initial training (in the eleventh bin, i.e. after 100 trials; χ2=6.11, d.f.=1, P=0.013). Thus, although a substantial decline in performance did occur over the 3 week delay, the bees had not entirely forgotten the association between colours and motor patterns.
Are similar effects seen when the maze times are evaluated? The raw data for the maze times are shown for a single bee in Fig. 3, and the exponential decay functions for all individuals are presented in Fig. 4. The parameters determined by fitting functions to the data are given in Table 1. For all bees, irrespective of whether they were trained on both tasks or only one, there is no decline in performance from the day of initial training to day 2, but there is a substantial decline at the onset of testing 3 weeks later: all bees start out worse than they had been at the end of the original training procedure on day 1 (compared: A+y0 after 3 weeks with y0 on day 1; Z=2.27, P=0.023; sign test), but better than they had been initially at the onset of training on day 1 (compared: A+y0 after 3 weeks with A+y0 on day 1; Z=2.27, P=0.023; sign test). This means that, while the motor pattern had not been perfectly retained over a period of several weeks, it also had not been entirely forgotten. In addition, the saturation levels y0 were worse in all bees after more than 3 weeks than they were during the initial training (Table 1; Z=2.27, P=0.023, sign test).
Maze times for a single bee (bee 6 in Table 1) trained on both tasks, as a function of the number of correct trials (error-free visits). The continuous curves are exponential decay functions fitted to these data (see Materials and methods). Only the first 100 trials are shown for each day.
Maze times for a single bee (bee 6 in Table 1) trained on both tasks, as a function of the number of correct trials (error-free visits). The continuous curves are exponential decay functions fitted to these data (see Materials and methods). Only the first 100 trials are shown for each day.
Maze times for bees trained on one task (top) and on two tasks (bottom). Joint curves are shown for bees performing both tasks. Exponential decay functions are drawn for each individual bee that was tested for long-term retention. Each symbol type specifies data from one individual. Only the first 100 trials are shown for day 1 and the long-term retention test; only the first 30 trials are shown for day 2.
Maze times for bees trained on one task (top) and on two tasks (bottom). Joint curves are shown for bees performing both tasks. Exponential decay functions are drawn for each individual bee that was tested for long-term retention. Each symbol type specifies data from one individual. Only the first 100 trials are shown for day 1 and the long-term retention test; only the first 30 trials are shown for day 2.
Reversal learning – single-task training
During its initial training, the bee made two errors during the first bin of 10 visits, one error in the second, and then no more errors (Fig. 5). The first reversal brought about a substantial increase in errors, both during early (the first 10) and late (the last 50) trials. During the next couple of reversals, early and late trials showed two divergent trends: while late trials progressively approached an error score of zero, early trials continued to show increasing error scores until, at the fifth reversal, 9 of the 10 first trials started with a turn into the wrong arm of the maze. After that, the error score also improved for the early trials so that, at the ninth to eleventh reversal, the learning curve had the same shape as that for the initial training. At the eighth reversal, the bee made only a single error during the first 10 trials, but two errors during the second 10 trials.
Error scores of a single bee exposed to multiple reversals.(A) Error scores as a function of the midpoint of each consecutive bin of 10 trials, for four representative reversals: the initial training (circles), the first reversal (squares); the fifth reversal (triangles) and the final (eleventh) reversal (identical to the initial training curve).(B) Error scores in the first 10 trials (triangles) and the last 50 trials (circles) as a function of the number of reversal training procedures.
Error scores of a single bee exposed to multiple reversals.(A) Error scores as a function of the midpoint of each consecutive bin of 10 trials, for four representative reversals: the initial training (circles), the first reversal (squares); the fifth reversal (triangles) and the final (eleventh) reversal (identical to the initial training curve).(B) Error scores in the first 10 trials (triangles) and the last 50 trials (circles) as a function of the number of reversal training procedures.
Reversal learning – double-task training
Bombus occidentalis workers trained on two tasks with forced alternate trials showed the same biphasic learning curve as B. impatiens: performance initially worsened from an error score of 20% during the first bin of 10 trials until, between 40 and 70 trials, error scores peaked at approximately 40%. The error rates subsequently declined and reached a steady level of less than 10% after more than 200 trials (Fig. 6A). When associations between colour and direction were reversed on the day following the initial training (Fig. 6B), the error score was initially high (50% in the first bin), but performance rapidly improved and, after 200 trials, a constant level of 0–10% errors was again achieved. The learning curves of the initial training and of the first reversal between 50 and 370 trials are statistically indistinguishable when the error numbers in the 32 bins of 10 trials each are compared by means of the χ2 goodness-of-fit test (χ2=25.7; d.f.=31; P=0.73). Choice of direction becomes significantly different from random in the ninth bin (81–90 trials) in both curves (initial training: χ2=9.9; d.f.=1.1; P=0.002; reversal training (χ2=7.5; d.f.=1; P=0.006). Thus, one might conclude that sensorimotor reversals are learnt with the same speed as they are initially entrained, and that the information from the reversal training has replaced that from the initial training.
Error scores for reversal learning of bees trained on both tasks. Error scores are pooled for the two tasks in the initial training (A) and the first reversal (B) and for all bees, but plotted separately for both tasks and both bees on the second reversal (C,D), since there were strong differences both between bees and between tasks. Bins of 20 are formed for C and D with single bees because of low values of N. Dashed lines show error scores on blue-marked mazes with rewards in the right arm; continuous lines show error scores in yellow-marked mazes with rewards in the left arm.
Error scores for reversal learning of bees trained on both tasks. Error scores are pooled for the two tasks in the initial training (A) and the first reversal (B) and for all bees, but plotted separately for both tasks and both bees on the second reversal (C,D), since there were strong differences both between bees and between tasks. Bins of 20 are formed for C and D with single bees because of low values of N. Dashed lines show error scores on blue-marked mazes with rewards in the right arm; continuous lines show error scores in yellow-marked mazes with rewards in the left arm.
If this were the case, then a second reversal should show the same dynamics in the learning curve as the first. To test for this possibility, two workers were exposed to a second reverse training session following the day of their first reversal (Fig. 6C,D). Surprisingly, error scores were much worse than during the first reversal. A direction choice significantly different from random was established only after more than 200 trials in bee 1 (between 201 and 220 trials – note that bins of 20 are used here; χ2=3.95; d.f.=1.1; P=0.046) and after more than 220 trials in bee 2 (221–240 trials; χ2=7.6; d.f.=1.1; P=0.006). Because performance differed strongly on both tasks (as opposed to the initial training and first reversal), two learning curves are presented for the two different tasks (i.e. one for the mazes with blue entrances and food on the right side, and one for the mazes with yellow entrances and food on the left). Both bees showed a peculiar performance for several hundred trials: bee 1 essentially treated all mazes the same until above 200 trials, when she turned right on almost all trials. Bee 2 switched several times between directions, but made most errors on one task and few in the other in any given phase. Between 141 and 220 trials, she used the same strategy as bee 1, i.e. to turn right in all mazes irrespective of colour. These results indicate that bees had retained information from both the initial training and the first reversal, but apparently in different ways: the first learnt associations had not simply been overwritten, but instead suppressed or ‘labelled’ as incorrect in the memory of the bee.
DISCUSSION
Long-term retention
In humans, many pure motor and sensorimotor skills are extremely resistant to being forgotten, and whatever decreases in performance occur are subject to rapid recovery (Meyers, 1967; Adams, 1987; Schmidt, 1991). When recovery curves are steeper than initial learning curves and performance is initially better after a delay, this means that a skill has not been erased from memory.
This is exactly the effect found in the bumblebees tested here. None of the bees shows any overnight decline in their sensorimotor skills, unlike the imperfect overnight retention found in some other tasks (Craig, 1994; Keasar et al. 1996). After more than 3 weeks, such declines do occur in the present experiments. Nevertheless, as all bees start out at a better level after the delay than they had during initial training, I conclude that the skills had not been fully forgotten. Performance after the delay depended on the number of tasks (bees trained on both tasks were less efficient than those trained on one), but all bees were more efficient at the onset of testing than when they were first trained. Thus, motor skills, like sensory memories (Menzel, 1968, 1990), are very robust to extinction once stored in long-term memory.
It is unclear what caused the levels of decline in performance that were observed. They might be due to a passive decay of memory or to interference from conflicting information acquired during the delay time. However, the influence of interference on information stored in long-term memory is usually limited to similar tasks (Mackintosh, 1974; Chittka and Thomson, 1997). The bees tested here were allowed to feed from Petri dishes with sucrose solution during the delay time, but not from any structures that resembled the T-mazes. This indicates that the motor patterns are partially weakened due to a passive decay of memory.
An interesting effect is that the saturation level in the maze times is worse in bees after the delay than at the end of the initial training period. This might be an age-dependent effect, in that the bees, which are now several weeks older, are slower although they received an equally extensive training period as that of a few weeks previously. The retrieval of information also becomes slower as more information is stored in the brain (Chittka and Thomson, 1997). Although the bees tested here were confined to a flight arena which was poor in cues, and bees could not forage from food sources other than those offered by the experimenter, it cannot be ruled out that they learn numerous motor patterns during regular activities in the nest, which may make them less efficient at retrieving earlier learnt motor programmes used to gain access to a reward. Experiments with bees whose age is known are necessary to decide between these possibilities.
Reversal learning
Reversal learning tasks pose complex problems: options that were previously correct are now incorrect, and previously unrewarded options are now rewarded. In the case of multiple reversals, both situations, and transitions between these, are encountered several times. In the case of flower-visiting insects, such problems are biologically realistic (Menzel, 1990): flowers that are most rewarding in the morning may not be so in the afternoon, while other flowers may show the reverse pattern of nectar production. Efficient foragers should and do adjust to such changes on a daily basis (Heinrich, 1976). Likewise, a given colour in one flower species may be associated with one floral morphology (and the corresponding motor pattern needed to extract the nectar), while another plant species with a similar colour signal may have an entirely different morphology, forcing bees to readjust the association between sensory signals and learnt motor programmes.
There are several conceivable responses that animals may show to reversal learning problems. (a) Reversal learning may be as fast as the initial training, and even several reversals may not alter the shape of the learning curve. This effect, which has been observed in some invertebrates (e.g. Longo, 1964; and references in Menzel, 1968), suggests that information from each reversal training session replaces that from the previous training. It also suggests that these animals are not capable of a higher form of learning that allows them to improve their efficiency at dealing with reversed tasks. (b) Performance might worsen from one reversal to the next so that, after several reversals, both options are chosen randomly. Menzel (1969) found this to be the case in honeybee colour learning. This means that the bee has stored both colours as predictors of reward and has difficulties in assessing that one of them is not rewarding at any one time when the number of extinction trials (trials in which a previously rewarded colour is now encountered without reward) is low. (c) Meineke (1978), in contrast, found that bees can reverse their learnt preferences multiple times when the number of trials is large enough in each training session. In addition, he found effects also observed in many vertebrates (Mackintosh, 1974): initially, large numbers of errors are made at each reversal, until subjects become more and more efficient so that, after many reversals, each new association is acquired faster and with fewer mistakes than even during the first training.
This is similar to what I find in the multiple reversal experiment. The single bee trained with multiple reversals on a daily schedule was initially more inefficient at coping with each new reversal than it had been during its first training, but later improved to its original learning speed. The initial decline in performance is unsurprising: at the first reversal, a motor pattern that was correct for 100 trials is now incorrect. Clearly, the bee is now no longer as unbiased as at the onset of its first training, and the association acquired during that training interferes with that entrained during the reversal training. What causes the later improvement in reversal learning?
One possible strategy to cope with serial reversals is to memorize the schedule in which associations are reversed (provided that, as in the present case, there is such a schedule). Bees might memorize which association was valid on the previous day, and use the opposite information at the onset of testing on the new day. This is not what the bee is doing in the multiple reversal test, since it does make errors at the beginning of each reversal. Rats make similar errors: even with highly predictable schedules, choices are still random at the beginning of each reversal, but rats will eventually need only a single trial to assess that a reversal has taken place (Mackintosh, 1974). The Bombus impatiens worker tested here needed three such trials. It is unclear just what is learnt to produce this behaviour. Some authors have suggested that animals use a concept of ‘the opposite’ to cope with such multiple reversals (Foppa, 1965). Another possibility is that bees simply ‘learn to forget’ the information that was valid during the previous reversal session, but this is unlikely because such active forgetting has not been described previously in learning psychology. Perhaps the most parsimonious explanation is that animals acquire a simple win–stay, lose–shift strategy after an initial random choice at the onset of each reversal (Mackintosh, 1974; Chittka et al. 1997) and then use the outcome of the initial choice as a basis for their behaviour in subsequent trials.
The dual effect in the change of the learning curve shape suggests that the acquisition of this strategy is complex. The number of errors made in the second half of each daily session (the last 50 trials) decreases monotonically from the first to later reversals. This suggests that the bee became more effective at suppressing interference from the information learnt on the previous day during late trials, and did so starting from the second reversal. This suppression may also explain why the initial performance of the bee after each reversal declined during the first five reversals. Using the first initial choices of each daily session to guide the choice of direction for the rest of the day is something that the bee learnt only later.
Interestingly, bees that had to reverse their learnt associations simultaneously on two tasks performed better at the first reversal than did the bee that had to cope with the simple reversal task described above: the bees trained on the double reversal task learnt the first switch in associations as rapidly as the initial associations. A possible explanation is that opposite directions in the double training (and double reversal) task were predictable by opposite colours, whereas colour had no information in the simple reversal task. On the second reversal, however, learning was much slower than during the initial training and the first reversal. This confirms that information from both the initial training and the reversal training is retained, because otherwise there should be no change in performance from one reversal to the next. After the second reversal, the bees had to cope with highly conflicting information: both blue and yellow had predicted rewards both on the left and on the right during previous sessions, and the bees had just learnt to suppress the information from the initial training. It is this suppressed information, however, that became relevant again after the second reversal. Retrieving this information was apparently much more difficult than storing it initially, or even than reversing the learnt preferences for a first time. Gould and Gould (1988) have suggested that such delays in reversal learning may reflect the time required to erase memories before learning new associations. This is unlikely to account for the present findings, since in that case there should be no change in performance from one reversal session to the next. Hence, it is more probable that bees are learning to suppress the information from previous training sessions rather than erasing it.
In conclusion, the present results support the notion that long-term memory capacity in bumblebees is large and holds not only sensory and spatial memories (Lindauer, 1963; Menzel, 1969, 1990) and information about reward probabilities (Greggers and Menzel, 1993) but also motor memories over long periods of time. Organizing and retrieving information in memory appears to be more difficult than storing more information. As more experience is amassed, the problem arises of how to handle memories so that they can be retrieved efficiently and in the appropriate context (Chittka et al. 1995, 1997). It seems that very little information is entirely lost, so that long-term memory accumulates information throughout a lifetime and, consequently, retrieval becomes slower and more prone to errors (Chittka and Thomson, 1997). The serial reversal experiment in the present paper indicates that bees can learn actively to suppress irrelevant information, but achieving this takes a rather extended training schedule and involves an intermediate period of very poor performance. If such problems extend to other animals, this may explain some peculiar phenomena in the neurobiology of learning and memory.
In male canaries, for example, the acquisition of new songs in autumn is associated with loss of neurones and their subsequent regeneration in brain regions involved in song control (Kirn and Nottebohm, 1993). This phenomenon appears to be unique in adult vertebrates, and the reasons are unclear. The authors speculate that ‘the acquisition of new information may require that old information – and the cells that hold it – be discarded’. However, other animals, including insects with much smaller brains, do not suffer from this constraint. Thus, a limited capacity for long-term storage may be less of a reason for neuronal replacement than the increased risk of making errors as memories from many learnt songs accumulate and may be spontaneously recovered while a novel song is performed.
Another peculiar phenomenon is that honeybees (Apis mellifera) and wild-type fruit flies (Drosophila melanogaster) actually learn more slowly than they might: selection experiments show that it is possible to breed faster learners than wild animals in only a few generations (Brandes et al. 1988; Lofdahl et al. 1992; Tully, 1996). If faster learning makes fitter animals, then why have selective pressures not driven these insects to learn faster in natural conditions long ago? One possible explanation is that flies, like bees, encounter increased problems in organizing and retrieving greater amounts of information in long-term memory. Since information is hard to eliminate once stored, limiting the input to long-term memory to information that has shown its salience in large numbers of trials may be adaptive for more efficient retrieval of memories in later life.
Acknowledgement
This work was performed under the auspices of the Department of Ecology and Evolution, State University of New York at Stony Brook. I am grateful to its members for providing the space and intellectual environment for these experiments. For comments on the manuscript, I thank Drs T. Collett, R. Menzel and J. D. Thomson. Financial support came from the DFG.