Using an abstract geometry in virtual reality to explore choice behaviour: visual flicker preferences in honeybees

The study of visual attention in animals requires methods of accurately tracking ongoing choice behaviour, especially in the presence of competing distractors. Attention modulates our perception of the environment by enabling us to focus on specific or related sets of stimuli whilst simultaneously suppressing information from irrelevant stimuli (Bichot and Desimone, 2006; Posner et al., 1980). Shifts in attention are thought to be under the control of two processes: a fast, involuntary mechanism termed ‘bottom-up’ attention that is driven by stimulus salience, and ‘top-down’ attention, which is a stimulus-selective, volitional form of control driven by previous experiences (Itti and Koch, 2000; Treue, 2003). Salience-driven attention appears to be found in most animals including insects such as bees and flies (Sareen et al., 2011; Spaethe et al., 2006; van Swinderen, 2011). Whether insects have volitional control of what they pay attention to is harder to determine. Understanding volitional effects requires methods of tracking the history of previous experiences and choices. Typically, in order to query visual perception in insects, stimuli are presented one at a time or in binary combinations within single experiments that produce one outcome. For example, an experiment may test whether honeybees prefer blue or yellow discs, before and after appetitive conditioning with sugar water (Menzel and Giurfa, 2006). Alternatively, flies may be tested for their preferences toward upright or inverted Ts in a flight arena, before and after aversive conditioning with heat (Wolf and Heisenberg, 1991). While behavioural paradigms can be very different (free flight in the former bee example, and tethered, closed-loop feedback in the latter), the structure of the experiments is similar: the insects have no control in determining what stimuli are being presented to them. Thus, they are constrained to display their choices toward stimuli that the experimenter has deemed relevant (blue and yellow discs, or T-shaped objects) in classical conditioning paradigms. While this Pavlovian-inspired methodology has yielded tremendous insight into the perceptual and cognitive capacities of insects (Menzel and Giurfa, 2006; van Swinderen, 2011), it does not lend itself to efficiently exploring a visual parameter space (e.g. an entire colour spectrum, or a whole range of shapes) or to disambiguating bottom-up salience effects from history-dependent effects. In visual attention experiments, which stimulus is most salient (or conspicuous) clearly depends on both innate preferences and history; however, the relative contribution of these different forms of salience can be difficult to disentangle, even in humans (Treue, 2003).

Virtual reality environments provide an effective way of accurately tracking attention-like processes in small animals such as insects. Traditionally, flight arenas have been used to measure visual choices in flying, tethered Drosophila flies (Brembs, 2008; Heisenberg and Wolf, 1984). In these arenas, flies might indicate their choices by ‘fixating’ on select objects in a closed loop paradigm, where their wing-beat or body torque readouts are used to control the angular position of objects displayed around them. In previous work in Drosophila, we have used such a flight paradigm combined with electrophysiology to show that object fixation and visual salience are associated with neural signatures of attention in the insect's brain (van Swinderen and Greenspan, 2003). In more recent work, we have shown that it is possible to ‘tag’ visual stimuli with distinct flicker frequencies, in order to track attention-like processes in the brains of flying Drosophila flies (van Swinderen, 2012) or walking honeybees (Paulk et al., 2014). One advantage of using such ‘frequency tags’ is that they allow fine-grained analysis of stimulus selection and suppression dynamics in the brain (Norcia et al., 2015; Vialatte et al., 2010), and when these are correlated to behaviour it becomes possible to infer attention-like states in the brain, even when animals are not behaving (van Swinderen, 2007; van Swinderen and Brembs, 2010; van Swinderen et al., 2009). Accordingly, we recently showed that selective attention occurs in the honeybee optic lobes (evidenced by a dominant frequency tag amplitude) even before the bee displays a behavioural choice between two competing objects (Paulk et al., 2014), suggesting that some level of ‘top-down’ control may exist in the insect brain.

Here, we present a new paradigm, based on a simple recursive geometry, to study visual selective attention and choice behaviour in insects. In this paradigm, tethered walking insects are confronted with competing objects displayed on a wrap-around LED arena and reveal their visual choices through their fixation behaviour, as shown previously (Paulk et al., 2014). However, each choice made by the insect engenders a subsequent set of related choices, effectively allowing the animal to ‘walk’ up or down a stimulus gradient. By embedding the successive binary choices within a constrained landscape of inter-linked virtual y-mazes (e.g. a dodecahedron), the insect is thus able to return to previously encountered choice points. Furthermore, recursive exploration of the visual parameter space embedded within this geometry allows stimulus preferences of an insect to be studied within the context of its previous decisions. An attractive choice may become less attractive after multiple exposures, for example.

In a first test of this paradigm, we presented honeybees with visual stimuli whose intensity was modulated at various temporal frequencies, to examine whether honeybees showed a preference for a particular flicker frequency, or a range of frequencies. Whilst it is known that insects can see visual flickers up to 100 Hz (Heisenberg and Wolf, 1984; Srinivasan and Lehrer, 1984), it is not known whether they prefer or avoid some flicker frequencies over others or even whether they can discriminate between various frequencies (but see Srinivasan and Lehrer, 1984). Bees respond reliably to different patterned stimuli (e.g. striped gratings) by modulating their flight speed or orientation for example (Srinivasan et al., 1999) and much is known about the responses of insects to moving gratings (Maisak et al., 2013; Srinivasan et al., 1999). However, this is not the same as static flicker, which rarely occurs in nature. It has long been known that humans respond differently to different flicker frequencies, for example, 7 Hz flicker is often aversive, whereas 10 Hz is relaxing (Walter, 1961). These human reactions can be easily determined because we can ask people about their reactions. Such a dialogue is, however, impossible with an insect and must be replaced by a suitable experimental paradigm that measures and ranks their behavioural choices. In this study, we use a virtual environment and a novel stimulus paradigm that enables efficient determination of a honeybee's visual stimulus preferences (flicker frequency in this case), as well as tracking of these choices through time to investigate their likely dependence on the bee's experience.

Honeybees (Apis mellifera Linnaeus 1758) were captured at a hive in the grounds of The University of Queensland as they exited the hive on foraging flights. The bees were subsequently chilled in 50 ml Falcon tubes under an ice pack until they became unresponsive (typically 3–5 min). While immobile, the dorsal side of the thorax of the bee was scraped gently with a scalpel (Livingstone International) to remove body-hairs and abrade the surface. This allowed a small metal tether to be glued to the thorax of the bee with blue-light cured dental cement (Southern Dental Industries) in an upright position parallel to the back of the bee (Paulk et al., 2014) (Fig. 1A). During this time, the head was also immobilised in the forward position with dental cement. After recovery from the tethering procedure, bees were fed with 1 mol l⁻¹ sucrose solution and left to rest suspended in a box at room-temperature (24°C) for at least 1 h prior to experiments.

Experiments were performed with the tethered bees walking on an air-supported ball surrounded by six 16×16 cm LED panels (Shenzhen Sinorad Medical Electronics Inc.) (Paulk et al., 2014) with a resolution of 32×32 pixels apiece arranged hexagonally (Fig. 1C). Positioning of the bee was achieved with a six-axis micromanipulator platform (Edmund Optics), with a coil-spring placed between the tether and tip of the manipulator to provide some flexibility. Illumination of the setup, which was required for filming, was from three 40 W bulbs positioned above-left, above-middle and above-right of the setup. The combined illumination on the bee from these bulbs and the LED display combined was approximately 260 lx (Mastech). A custom-created software, FicTrac (Moore et al., 2014), running on a Windows 7 (SP 1) platform was used to analyse the motion of the bee on the ball via a camera (Point Grey Laboratories, Vancouver, BC, Canada) positioned in the front of the arena. Bees were presented with one, two or three green vertical bars of angular size approximately 15 deg (h) and 60 deg (v) as previously published (Paulk et al., 2014). In closed-loop, the rotational vector (Δθ) was used to link the movement of the ball 1:1 to the angular position of the stimuli around the arena (i.e. 360 deg rotation of the ball by the bee would drive the bar through a full revolution around the arena). x-y movement vectors were used to record how far the bee had walked in the forward direction. The latency between ball movement and corresponding image movement was around 25 ms. The Python script running the stimuli was synchronised with the refresh rate of the LED panels and ran at 200 cycles s⁻¹, which allowed frequencies up to 100 Hz to be displayed by switching LED pixels on and off, resulting in a square wave pattern (see Results).

All bees were first exposed to three successive closed-loop fixation trials, each involving a 120 s presentation of a single, non-flickering green bar on a dark (unlit) background. Our objective during these trials was to determine which bees were fixating well (i.e. actively walking on the ball and keeping the bar mostly within a 60 deg arc encompassed by the frontal LED panel). In order to verify whether bees were actively attending to the visual stimulus, the bar was randomly moved (‘perturbed’) 60 deg left or right approximately every 15–60 s (Paulk et al., 2014). A return of the bar to the frontal visual field within 3 s was termed a successful perturbation correction. Bees that fixated well during these three single-bar trials were termed ‘cooperative’. This was determined qualitatively by observing whether or not the bee managed to fixate well for the majority of the three trials and whether it was still running well towards the end of the third trial. Cooperative bees then underwent further experimentation using the dodecahedron paradigm.

A conceptual dodecahedron was used to map competing stimuli onto 12 faces. A choice scenario with two presented stimuli is represented by an edge on the dodecahedron with the stimuli represented abstractly as the edge's two adjoining faces (see Fig. 5B). In this study, we investigated green bars with different visual flicker frequencies, each frequency being represented by one face on the dodecahedron. Thus, if the bee is currently on an edge with adjoining faces A and B and face A is selected, it is maintained for the next choice scenario while face B is replaced with a novel competitor, for example face C. On the dodecahedron, this can be abstractly visualised as moving forward to the next connected edge by circling around the chosen face A (so if face A is continuously selected, the different choice scenarios displayed are determined by the edges that border its perimeter). This method of abstractly traversing the dodecahedral geometry allows the bee to ‘choose’ its own path and even eventually revisit old choice scenarios. Pilot trials showed that the median walking speed of bees in this scenario was about 1 cm s⁻¹. For our experiments, about 15 s for the presentation of each choice scenario to the bee was considered desirable. Thus, the forward walking distance that the bee had to cover on the ball before a transition to the next competition scenario was set to be 15 cm. During each choice scenario, the stimulus that evoked the greater measure of fixation strength was the selected stimulus (see following sections for measures of fixation strength in the two-bar and three-bar competition scenarios). Experiments using the dodecahedron paradigm ceased when the bee was qualitatively determined to have ceased activity (for an indication of how many choice transitions each bee accomplished, see Results).

Bees that fixated strongly on a single bar in closed loop were made to undergo experimentation using the dodecahedron paradigm for two competing bars. Using the dodecahedron paradigm, the two adjoining faces of the current edge visited by the bee defined each choice scenario. The investigated visual characteristic was visual flicker frequency superimposed onto a green bar (2, 4, 8, 10, 12.5, 14.2, 16.6, 20, 25, 33.3, 50 or 100 Hz). In total, the bee could explore 30 unique combinations of stimulus pairs. It was expected that the bee would walk towards and fixate on its preferred flicker frequencies. As with the single-bar stimulus, random left or right perturbations of 60 deg occurred every 15 to 60 s to ensure that fixation choices were deliberate and not coincidental. The perturbations also served to bring the non-fixated bar into the visual field of the bee, thus exposing the bee to the other competitor in that choice scenario. To determine the strength of fixation directed to either competing object, the cumulative sum of the value of a sine function [y=Asin(x), where x is the current angular position of the bar in the LED arena, A is an arbitrary scaling value and y represents the strength of fixation directed towards the bar at x] superimposed on each bar's position was summated over time. The bar with the greater cumulative sum after 15 cm forward walking was thus calculated to have evoked stronger fixation behaviour and was hence the selected choice of the bee. To examine choice behaviour over time, only bees that had undergone at least 150 transitions were analysed. These data were divided into equal thirds for each dataset (early, middle and late) and fixation statistics (see below) were performed as previously for these subdivided datasets.

A different group of bees that fixated strongly on a single bar in closed loop were made to undergo experimentation using the dodecahedron paradigm with three competing bars. The three bars were set 120 deg apart. The third bar represents the rejected choice of the previous choice scenario. This rejected choice was the adjoining face of the previous edge that was not chosen. So, after transitioning to the next choice scenario the selected stimulus remained as before, a novel competitor was introduced onto the scene (determined by the new adjoining face of the next edge) and the rejected choice was maintained for a second chance to be chosen. Abstractly, the three faces that share the vertex between the current edge and the previous edge represent the three competing objects of the current choice scenario. Selecting the previously rejected choice enables the bee to revisit the previous edge albeit in a reverse direction. Thus, the bee is also able to navigate ‘backwards’ to a previous stimulus scenario (see Results). Hence, at any given time, every object (or frequency) can either be a previously selected object, a previously rejected object or a novel object. For three competing objects in this scenario, the total number of unique combinations is 60. Perturbations were reduced to a smaller angle of 30 deg because of the increased number of objects in the scene. To measure the strength of fixation evoked by each flicker frequency, a slightly modified sine function compared with the one used in the two-bar competition scenario was used. The zero-point of the sine function was set directly behind the bee and its period doubled such that its greatest value was directly in front of the bee and there was no negative component of the sine function y=Asin[0.5(x+90)]. Again, the bar that accumulated the greatest cumulative sum of the sine function values over 15 cm of forward walking determined which frequency was selected by the bee.

Offline analysis of data was performed primarily in MATLAB 2014a, with the Circular Statistics toolbox (Berens, 2009). Bar position data were converted into polar coordinates, and the angular distribution and magnitude were calculated to generate a median vector. Statistical significance for fixation and walking speed data was calculated by a Kruskal–Wallis test (significance was set at α=0.05). Fixation was deemed to occur when the stimulus bar was within 60 deg of the bee's frontal visual field. Walking speed was calculated in MATLAB from the x-y coordinates derived from FicTrac (Moore et al., 2014). For the single bar perturbation data, a two-tailed t-test (α=0.05) was used to determine statistical significance between successful return proportions for the first and last trials, and a Kruskal–Wallis test was used to compare the significance of the time taken to return the bar to the frontal visual field between the above trials. For the two-bar experiments, data were only analysed if the bee traversed at least 80% of the edges of the topographical dodecahedron before stopping, whereas for the three-bar scenario, this threshold was set at 50%. Stimulus preferences were calculated as proportions; the average proportion of times that each frequency was chosen compared with the others was computed by first counting the number of times each frequency was chosen in each trial (a trial being a succession of choices for one bee confronted with the dodecahedral geometry), converting this to a normalised proportion for the trial and then taking the average of proportions across all trials. The raw count was weighted first by the proportion of edges the bee explored out of the total number of edges (60 for the both the two bar and three bar scenarios). Comparisons between the distribution of proportions from all the bees' trials for each frequency were made using Friedman's test with α=0.05 and these proportions were also compared to expected random values (set at 8.33% for 12 stimuli) with a Wilcoxon signed-rank test. For determining the effect of novelty salience, significance was determined through a two-sample t-test comparing the proportions of choices that were novel or a continuation, respectively. Statistical comparison of the relative number of novelty choices versus continuation choices was performed with a z-test for proportions, and adjusted for multiple comparisons (α=0.05/12=0.004).

Tethered honeybees were placed on an air-supported ball in the middle of a wrap-around LED arena (Fig. 1A,B). The arena was hexagonal in shape and consisted of six square panels of 32×32 LED arrays (Fig. 1C). Bees were able to control the angular position of objects displayed in the arena (e.g. a green vertical bar) by their walking behaviour on the ball, which was monitored by a camera-based closed-loop interface, FicTrac (Moore et al., 2014) (Fig. 1D). The tendency of the bee to turn towards or away from a particular object displayed on the screen resulted in a rotation (Δθ) of the ball about the vertical axis, which was used to proportionally control the angular position of the object (Fig. 1E). In a previous study using a four-sided arena (Paulk et al., 2014), we showed that bees placed in this context adjust their turning behaviour so as to position single objects in their frontal visual field, a behaviour we call ‘fixation’. Here, we confirmed that bees continue to show robust fixation behaviour in our improved hexagonal setup (Fig. 1F).

Fixation behaviour in the arena requires the bees to behave in ways which they have never encountered before: they must ‘understand’ that their walking behaviour is connected to the visual objects displayed around them and that by turning the ball they can effectively place the object in front of them. Indeed, only two thirds of tethered bees were able to fixate at all (N=39/60; see Materials and methods for fixation criteria). For those that did fixate, this often occurred quickly, soon after placing the bee on the ball. We questioned whether fixation behaviour in these bees improved over time, which might be indicative of motor learning. Bees were presented with a single green vertical bar over three successive 2 min trials (see Materials and methods). We found that although fixation behaviour was robust from the start in these bees, it improved significantly by the third trial (Fig. 2A–D). Improved fixation was also associated with increased walking speed (Fig. 2E–H), suggesting increased motivation in these animals as they became accustomed to the somewhat unnatural virtual reality context. It is not entirely clear why bees place the bright green object directly in front of them, but it is possible that by fixating on the bar they are attempting to escape their confinement – this may be a phototactic response.

In order to be certain that bees were actively fixating on the green bar, we introduced random perturbations to the position of the bar in all of our experiments (Paulk et al., 2014 and see Materials and methods). During a perturbation, the bar would move 60 deg to the left or to the right, forcing the bee to undertake corrective behaviour to return the bar to the front (Fig. 3A), if it were attempting to fixate on the bar. Perturbations were programmed to occur at random times during each trial (approximately every 15–60 s). The bar could be in any starting position during a perturbation, but if a perturbation occurred when the bar was in front, this effectively probed the bee's motivation to fixate the bar. To better quantify fixation behaviour, we considered fixation to be corrective if the bee returned the bar to within a 60 deg sector of the frontal visual field (two perturbations and subsequent corrections are shown in Fig. 3B). As we had found earlier that fixation improved over time, we next asked whether the bees' response to perturbations also improved over time. For the same dataset as in Fig. 2, we found that bees could already respond well to perturbations in the initial trials (Fig. 3C). However, in the final trials, their corrective responses appeared to become more accurate (Fig. 3D), with less variance compared with the initial trials (s.d. for final bar position after fixation correction during first trials: 24.7 deg; during last trials: 10.9 deg). Closer comparison of the first and last trials revealed that the proportion of corrective responses was not different between the first and last trials (Fig. 3E). Instead, bees became significantly faster at correcting perturbations by the third trial (Fig. 3F), which is consistent with their improved fixation performance at this later point in time (Fig. 2D). The bees' responsiveness to perturbations can also be measured in terms of reaction time, which is a commonly used metric in human attention experiments. For bees that have become accustomed to this virtual reality environment, reaction time appears to be about 1 s, compared with almost 2 s during initial trials (Fig. 3F).

We have previously shown that bees alternate their fixation between two objects displayed simultaneously on the arena (Paulk et al., 2014). We confirmed this with our improved hexagonal setup: bees presented with two objects 180 deg apart alternately place one or the other object in their frontal visual field (Fig. 4A–D). When presented with three green bars simultaneously (each 120 deg apart), bees distributed their fixations among the three bars (Fig. 4E), such that, on average, fixation was directed equally to each bar (Fig. 4F). This again supports the conclusion that bees view these bright bars as distinct and separate objects rather than as a bound panorama. This is important because, to be able to track choice behaviour over time, it is crucial to be certain that fixation actually represents a choice. In a previous study, we found that when two objects are closer together (e.g. 30–60 deg apart), bees tend to treat both objects as a single object by fixating on a point located between the two objects, rather than alternating between them (Paulk et al., 2014). For the current setup and subsequent study, it appears that objects 120–180 deg apart are still viewed as distinct by the bees.

In a first examination of choice behaviour in this paradigm, we presented bees with competing green bars (180 deg apart) flickering at 12 different frequencies, ranging from 2 to 100 Hz (2, 4, 8, 10, 12.5, 14.2, 16.6, 20, 25, 33, 50, 100 Hz) (Fig. 5A). The choice of frequencies was constrained by the refresh rate of our LED panels (200 Hz, see Materials and methods). A non-redundant pair-wise comparison of all 12 frequencies would require 66 different experiments [V=P(P−1)/2, where P=number of frequencies and V=number of non-redundant paired comparisons]. Clearly, such a thorough pairwise approach is impractical for determining choice preferences: presenting each bee with 66 different experiments is unreasonable and assigning different frequency pairs to different bees is not ideal either because some bees may perform differently to others. Instead, we wondered whether bees could explore a closed virtual environment containing the 12 different flicker frequencies and indicate their preferences, each in their own way, by the history of choices made while traversing this environment. We chose a dodecahedral geometry to represent the set of stimulus frequencies in the choice paradigm, as this platonic solid has 12 identical pentagonal faces (Fig. 5B, left). Each face of the dodecahedron can represent a different visual object, which in our case is a green bar flickering at a particular frequency (Fig. 5B–D). A choice scenario between two competing bars presented 180 deg apart in the visual panorama can be represented abstractly as an edge on the dodecahedron, where the two adjoining faces represent the visual characteristics assigned to the two competing bars (Fig. 5D). We decided to implement the following rule: at each choice point (represented by vertices on the dodecahedron), maintain the object that evoked greater fixation, and replace the less-fixated object with another competitor (Fig. 5E) that is determined by the new adjoining face of the edge directly ahead (Fig. 5E, right). Thus, if a bee continuously preferred a 2 Hz flickering bar, for example, it would follow a path encircling the perimeter of the 2 Hz pentagon face of this abstract dodecahedron (Fig. 5B, right). Thus, there is a directionality associated with dodecahedron navigation that allows the bee to move ‘forwards’ (left or right) through the different choice points and re-visit them (see arrow path example on Fig. 5B). A choice decision is reached after 15 cm of forward walking where the bee's selection is analytically determined to be the object that the bee fixated best on, in other words had the greater measure of fixation strength (see Materials and methods). Only bees that performed well in three successive closed loop, single-object trials (Fig. 2) were tested in the more complex dodecahedron paradigm. Importantly, random perturbations of the stimulus position were embedded in all experiments. This continuously challenged the bee's fixation choice, and allowed competing objects to be brought momentarily into the visual field of the bee.

A robust salience profile for visual flicker was evident for bees (N=13) exploring this virtual environment, with some frequencies being clearly more salient (more fixated upon) than others, and other frequencies clearly avoided (Fig. 6A). Bees regularly avoided lower (<10 Hz) and higher (>33 Hz) flicker frequencies and preferred middle-range frequencies (around 20 Hz). These distinct preferences are also evident by examining the average of fixation polar plots for each frequency when it was in combination with all its possible competitors: strong fixation behaviour was seen on average for 20 Hz amongst all its competitors (Fig. 6B), and likewise aversion (anti-fixation) was seen for 100 Hz (Fig. 6C).

Each object presented to the bees in this environment exists in one of two possible states. It can be either a previously selected object that has remained on the scene or a novel object that has appeared on the scene (Fig. 5E). As such, two different forms of ‘bottom-up’ salience are in direct competition: novelty and innate preference. We questioned whether novelty salience effects influenced choice preferences in this scenario, so we sorted all choices by whether the object was novel or not. As expected, we found that preferred frequencies (e.g. 20 Hz), when chosen, were most often not novel, i.e. they were selected in immediate succession over multiple choice points (Fig. 6D). Indeed, some bees fixated on the same flickering bar for extended amounts of time, ignoring the novel competing frequencies that were continuously being presented (Fig. 7, blue sequences; e.g. dataset 4). However, there was considerable variability among individuals: other bees tended to respond more strongly to novelty in general (Fig. 7, red sequences; e.g. dataset 2). Interestingly, some less-preferred frequencies (2 Hz, 100 Hz) were selected significantly more often when they were novel (Fig. 6D), suggesting that novelty salience can override innate preferences. We found the same trend, as well as a similar frequency salience profile, when we exposed a second smaller set of bees (N=6) to randomly assigned dodecahedron environments (neighbouring ‘faces’ were different for different bees) (Fig. S1), thereby controlling for the fixed topography of the choice environment.

We showed earlier that fixation performance improves over time, when bees are presented with a simple green bar (Fig. 2A–D). Having found innate preferences and novelty effects for different visual flickers (Fig. 6), we wondered whether fixation performance might also change over time when bees were confronted with competing stimuli of variable salience. We examined data from longer-performing bees (N=5, see Fig. 7), which we could divide into three comparably broad ‘epochs’ (early, middle and late; see Materials and methods). We then examined fixation directed toward preferred frequencies and avoided frequencies, based upon our known frequency preference profile (Fig. 6A). We found that anti-fixation of the lower avoided frequencies (2–4 Hz) became less robust with time (Fig. 8A,D). Fixation toward preferred frequencies (20–25 Hz) also became less robust with time (Fig. 8B,D). In contrast, anti-fixation of the higher avoided frequencies (50–100 Hz) remained robust through time (Fig. 8C,D). This result suggests differential habituation effects for these stimuli.

Having seen that fixation preferences could depend on stimulus history (e.g. novelty and habituation effects), we next decided to include ‘history’ as a choice. We did this by providing bees with a third choice: a previously rejected object. By being neither novel nor fixated upon, such an object would not be salient in a ‘bottom-up’ sense, so selection of such an object might reflect other attention-like processes. Knowing that bees can interact with three objects at a time (Fig. 4E), we therefore added a third bar that represents the previously rejected choice (Fig. 9A). This modification to our paradigm also allowed bees to move ‘backwards’ on our abstract dodecahedron: fixating on a previously rejected object effectively ‘reverses’ the bee's direction in this abstract space (Fig. 9A). Diagrammatically, the previously rejected object is the adjoining face of the previous edge (for bees moving ‘forwards’ in the previous choice scenario). In the previous two-bar competition scenario, this object would be replaced with a new competitor from the next edge, but in the three-bar competition scenario it is retained for one more round. Abstractly, the three faces that share the vertex between the old and new edge represent the three competing objects. The same selection rule is maintained as before: selected objects remain while rejected objects are swapped out for a novel object, except we keep the rejected object for one more round, represented as a third bar, which offers the bee the option to reverse direction (see Fig. 9A, right panels). Providing a third bar also gives the bee a second chance to select an object that was rejected in the previous round. More generally, we were interested to see whether similar frequency preferences were evoked when we provided three competing objects that also engendered a different set of navigation rules on the abstract dodecahedron.

With this three-bar scenario, we found that bees displayed a similar salience profile for the 12 competing visual flickers (Fig. 9B), although fewer significant effects were found compared with the previous experiments (Fig. 6A). However, a closer examination of fixation plots for different frequencies revealed similar trends: middle-range frequencies (e.g. 20 Hz) were still preferred compared with lower and higher frequencies (Fig. 9B–D), and avoided frequencies (e.g. 2 Hz and 100 Hz) were selected relatively more often when they were novel (Fig. 9E). When we examined the average fixation strength (by vector length) for each frequency, the similarity with our previous experiment became even more apparent (Fig. 10A,B). Thus, bees arrived at the same answer with regard to their visual flicker preferences, even when the number of simultaneous choices was increased and the navigation rules in their virtual environment were altered.

Knowing that novelty and innate salience are competing factors driving choice behaviour in bees (Fig. 6D, Fig. 7 and Fig. 9E), we next wondered how these factors compared when an object was no longer novel, by retaining previously rejected choices for one more round, as in Fig. 9A. In our three-object scenario, each object exists in one of three possible states: a previously selected object that has remained on the scene, a previously rejected object that has remained on the scene, or a novel object that has appeared on the scene. When we re-examined these data, we found that, on average, previously rejected objects were selected as often as novel objects (Fig. 10C; Fig. S2). In contrast, most choices were directed to previously selected objects, suggesting that innate preferences predominate in this paradigm. This was also evident in our two-choice paradigm (Fig. 10D) and probably explains the robust frequency salience profiles we have uncovered using this new approach.

Tethered virtual reality paradigms provide a valuable approach to studying visual choice behaviour in animals. Importantly, this approach provides excellent experimental control over the visual environment. In free-walking paradigms, it is harder to know what the animal is viewing at any time, and this is also problematic with animals that can shift their gaze. In tethered virtual reality, the experimenter can infer exactly what images impact the animal's retina at different times and, when coupled to a behavioural readout, this allows for a better assessment of the perceptual capacities of the animal. Indeed, few animal paradigms allow for such a thorough control of the visual context as the operant paradigms that have been designed for tethered insects (Brembs and Wiener, 2006). Using these paradigms, researchers have found that flies can learn (Wolf et al., 1998), generalise different contexts (Liu et al., 1999), perceive objects as the same even if they are in a different location (Tang et al., 2004), resolve conflicting cues in an ‘intelligent’ way (Tang and Guo, 2001; Zhang et al., 2007) and pay attention to some objects while ignoring others (Sareen et al., 2011). When combined with electrophysiology, tethered virtual reality paradigms have revealed that selective visual attention is associated with distinct oscillations in the fly brain (van Swinderen and Greenspan, 2003) and that attention-like signals in the honeybee brain precede behavioural choices made by the animal (Paulk et al., 2014). However, all of these insect paradigms have by necessity resorted to simple experimental designs centred on the classic Pavlovian model wherein outcomes are constrained by what stimuli the experimenter has decided are worth using – often a single binary-type experiment that ends with data indicating one preference or another.

Here, we decided to take a different approach to studying visual perception in bees. Instead of subjecting many animals to an exhaustive set of stimulus pairs in order to determine their average preferences, we allowed each animal to navigate its own way through a closed-loop virtual environment where it was exposed to a range of stimulus parameters, thereby allowing their cumulative fixation behaviour to tell us what they preferred. We decided to explore visual flicker preferences in honeybees because little is known about whether bees can discriminate different flickers or whether they prefer some to others. Also, we have used visual flicker in the past as ‘frequency tags’ to track selective attention dynamics in the insect brain (Paulk et al., 2015,, 2014; van Swinderen, 2012) without really knowing whether these tags were appetitive or aversive. Clearly, the ideal frequency-tagged attention experiment should use tags that are of similar valence to the animal, to prevent confounding innate salience effects. In humans, 7–8 Hz flicker has often been used to study visual attention in the brain (Ding et al., 2006; Norcia et al., 2015; Vialatte et al., 2010) and this frequency range is anecdotally agreed upon as being somewhat unpleasant. We found that honeybees fixate preferentially on middle-range flicker (20–25 Hz) and that they avoid lower (<10 Hz) and higher (>33.3 Hz) frequencies. This effect was robust, as we demonstrated this trend by two different choice paradigms. Interestingly, fixation to the preferred frequencies (20–25 Hz) became less robust with time, whereas anti-fixation of some of the avoided frequencies (50–100 Hz) remained robust throughout. Finally, we found that although novelty salience can override innate preferences, the latter seems to dominate the choices made by honeybees in our paradigm. This makes ethological sense: behaviour is largely hardwired, but immediate history provides a level of flexibility to promote exploratory behaviour, which might be beneficial. A conceptually similar result was found by studying the timing of left–right torque behaviour in tethered flies, which was best described by a long-tailed (Levy) distribution (Maye et al., 2007). Such stereotypy, with sporadic forays into new territory, has been suggested to promote effective foraging (Reynolds and Frye, 2007).

The history embedded in our paradigm revealed just how different individuals can be when confronted with the same virtual environment. For example, some bees were much less novelty seeking than others, such as bee 4, as seen in Fig. 7. This bee went through 70+ choice iterations, always fixating on the same flickering object, even though novel choices (and random perturbations) occurred throughout. In contrast, other bees (e.g. bee 2 in Fig. 7) were intensively novelty seeking, turning very regularly to the new objects that flickered around them. Similarly, when bees were given a choice to select previously rejected objects, they displayed individual variability (Fig. S2). For example, bee 5 in Fig. S2 selected historical choices more often than bee 6. Some bees changed behaviour during a trial. For example, bee 12 went through a long stretch favouring continuation choices (marked in blue), but then alternated more between history and novelty (red and green, respectively) towards the end of its trial. Another bee almost never selected the same objects in succession, fixating instead almost exclusively on novel or previously rejected objects (bee 24, Fig. S2). Although it is not clear why bees should behave so differently from one another (all bees were captured as foragers leaving the hive in the morning), our study highlights the value of tracking individual idiosyncrasies to understand variability in choice behaviour. The bees' previous history in their natural environment, their caste, their state of health or their age could all have contributed to the remarkable individual variability we uncovered in our virtual reality environment. These might be better individually addressed in future studies.

Our virtual reality environment can, in principle, be applied to any visual question. While we restricted our study to 2–100 Hz flicker, it is conceivable that a future study might use a similar design to more closely examine flicker discrimination in the 20–30 Hz range specifically, where there appeared to be a major switch in valence for bees (Fig. 6A). Other visual parameters could also be explored using a similar approach: luminosity, contrast, colour, shapes or patterns. Application of an unconditioned stimulus (such as heat) to our paradigm is also a possibility, as has been done for operant conditioning (Wolf et al., 1998), as well as the addition of brain recordings in behaving animals (Paulk et al., 2015,, 2014).

In our first experiments using this novel approach to investigate visual perception in insects, we found that honeybees fixate preferentially on some frequencies (20–25 Hz) and less on others (2–4 Hz and 50–100 Hz). Why might this be? It is difficult to imagine that these preferences are ethologically relevant, as such precise flickers do not often occur in nature. One possibility is that these frequencies co-opt motion detection pathways in the insect's brain (Maisak et al., 2013; Rister et al., 2007). Thus, 20 Hz may reflect optomotor events that are commensurate with normal bee motion, whereas 2 Hz may be unusually slow and >30 Hz unusually fast, hence the preference for a middle frequency range. Indeed, the optomotor response of freely flying honeybees is strongest at frequencies in the vicinity of 25–50 Hz (Srinivasan and Zhang, 1997). Alternatively, frequency preferences may reflect endogenous oscillations that exist in the bee brain that are already being used to perform certain computations relevant to selective attention. In a similar manner, human subjects find 10 Hz flicker relaxing (Walter, 1961) and this ‘alpha-band’ oscillation, previously associated with ‘idling’, has recently been proposed as a visual suppression mechanism in the human brain (Jensen et al., 2012). We have previously shown that 20–30 Hz LFP activity is associated with visual attention in Drosophila flies (van Swinderen and Greenspan, 2003). Perhaps the honeybee preference for 20 Hz flicker (which corresponds to a period of ∼50 ms) reflects a common temporal window for visual processing in the insect brain, which might be centred on this frequency range. Future experiments combining our choice paradigm with electrophysiology (Paulk et al., 2015,, 2014) should resolve how endogenous oscillations might interact with dynamic visual stimuli to guide decision making in behaving insects.

We thank Yanqiong Zhou, Angelique Paulk and Leonie Kirszenblat for help setting up the LED arena.