Innate visual preferences and behavioral flexibility in Drosophila

Animals continuously make decisions to survive in a dynamic environment to, for example, successfully locate an adequate food source, find a way home or avoid something dangerous. Behavioral choices are guided by innate preferences or ‘instinct’, as well as by more flexible cognitive processes such as attention (Smith and Ratcliff, 2009; VanRullen and Thorpe, 2001), learning and memory (Euston et al., 2012; O'Doherty et al., 2017; Odoemene et al., 2018; Tobler et al., 2006). Instinct and experience together determine the valence of stimuli and therefore assign negative or positive associations (Gold and Shadlen, 2007; Lee et al., 2005; Xie and Padoa-Schioppa, 2016). Typically, negative and positive associations to stimuli result in opposing behavioral actions: animals move towards attractive stimuli and away from aversive stimuli. In animal learning experiments, these rudimentary behaviors are usually tested by using a Pavlovian conditioning paradigm, whereby one of two ‘neutral’ stimuli are provided with a valence cue (a punishment or reward) in order to demonstrate increased attraction (or repulsion) towards that stimulus, compared with the other (Balleine and Dickinson, 1998; Dickinson and Balleine, 2002; Rangel et al., 2008; Tully and Quinn, 1985). However, the valence of stimuli is not necessarily hard wired (Janak and Tye, 2015). Inherently attractive objects can become less attractive over time as a result of habituation or pre-exposure, or can become repulsive if associated with punishment. Similarly, inherently repulsive objects might become transiently worth paying attention to (Redondo et al., 2014). Such flexibility seems to be a feature of all animal brains, to allow for adaptive decision making based on experience.

Recent studies suggest that circuits in the central brain of insects, in the central complex (CC), are involved in decision making (Guo et al., 2016; Sun et al., 2017). These insect circuits display some functional similarities to the mammalian basal ganglia (Anderson et al., 2014; Barron et al., 2015; Stephenson-Jones et al., 2011; Strausfeld and Hirth, 2013), especially with regard to the regulation of valence-based decision making (Foti et al., 2011; Gold and Shadlen, 2007; Guitart-Masip et al., 2012; O'Doherty et al., 2017; Rangel et al., 2008; VanRullen and Thorpe, 2001). Also, like the basal ganglia, the insect CC is involved in multisensory integration; it is understood to be involved in sleep (Donlea et al., 2011, 2018; Nitz et al., 2002), learning and memory (Krashes et al., 2009; Liu et al., 2006; Rohwedder et al., 2015; Weir et al., 2014; Zhang et al., 2013), navigation and orientation (Seelig and Jayaraman, 2015), and action selection (Barron and Klein, 2016; Barron et al., 2015; Gerfen and Surmeier, 2011; Gurney et al., 2001). These diverse functions are regulated in the brain for both mammals and insects by monoamines (Gøtzsche and Woldbye, 2016; Ichinose et al., 2015; Kahsai and Winther, 2011; Keene and Waddell, 2007; Waddell, 2010; Weir et al., 2014), gamma-aminobutyric acid (GABA) (Guo et al., 2016; Gurney et al., 2001; Root et al., 2008; Zhang et al., 2013) and neuropeptides (Bannon et al., 2000; Chung et al., 2017; Gøtzsche and Woldbye, 2016; Krashes et al., 2009; Shao et al., 2017). Whether these neuromodulators and neuropeptides play a direct role in controlling valence-based decision making in the insect CC remains unclear.

Drosophila neuropeptide F (dNPF) is the homolog of mammalian neuropeptide Y (NPY) (Garczynski et al., 2002), which is involved in signaling food satiety levels as well as regulation of fear and anxiety (Gøtzsche and Woldbye, 2016; Primeaux et al., 2005; Redrobe et al., 2002). In Drosophila, an increase in dNPF levels in the brain has been associated with increased aggression (Dierick and Greenspan, 2007), arousal (Chung et al., 2017) and reward learning (Krashes et al., 2009; Shao et al., 2017). Further, dNPF modulates olfactory learning by inhibiting dopaminergic neurons that provide positive and negative valence cues to the mushroom body (MB) (Hattori et al., 2017; Krashes et al., 2009; Zhang et al., 2007), a structure that has primarily been associated with olfactory memory (Keene and Waddell, 2007). Interestingly, dNPF-expressing neurons also project to the fan-shaped body (FB) (Kahsai and Winther, 2011; Krashes et al., 2009), a CC neuropil associated with arousal (Donlea et al., 2011; Liu et al., 2012) as well as visual behavior (Liu et al., 2006; Weir et al., 2014). This suggests that dNPF might provide valence cues for visual stimuli, in order to guide visual decision making.

In this study, we investigated visual preferences in Drosophila, to study flexibility in valence-based choice behavior along one visual stimulus parameter: object height. Using a closed-loop virtual reality arena for tethered, walking flies, we found that flies display robust attraction or repulsion behaviors to very specific object heights. We show that these apparently hard-wired visual preferences can be modified by experience and controlled or overwritten by optogenetic activation of dNPF neurons.

Drosophila melanogaster were reared using standard fly media and kept under a 12 h light–dark cycle at 25°C. Canton-S (CS) flies were used as wild-type (control) flies. For optogenetic control of the dNPF circuit, we made use of the Gal4/UAS system (Brand and Perrimon, 1993) to express red-shifted channelrhodopsin ‘Chrimson’, a non-selective ion channel, in dNPF neurons. Exposure to red light results in an activation of these ion channels and therefore an activation of the dNPF-expressing neurons. dNPF-Gal4 flies (kindly provided by Ulrike Heberlein, Janelia Research Campus, Ashburn, VA, USA) were crossed with UAS-CsChrimson::mVenus(attp40) flies (kindly provided by Vivek Jarayaman, Janelia Research Campus) to provide female transgenic flies used in this study. For optogenetic activation of the dNPF-neurons, Gal4;UAS-CsChrimson::mVenus flies were fed with blue-dyed 0.2 mmol l⁻¹ all-trans-retinal (ATR, Sigma-Aldrich, St Louis, MO, USA) food for 2 days before the experiments and kept in darkness until testing (Klapoetke et al., 2014). Non-retinal-fed animals were used as controls.

We used female flies at 3–11 days or 17–40 days post-eclosion. Each fly was immobilized under cold anesthesia (0.5°C) for 60 s and positioned for tethering on a custom-made preparation block. The flies were then glued dorsally to a tungsten rod by means of dental cement (Coltene Whaledent Synergy D6 Flow A3.5/B3) (Fig. 1A), cured with blue light (Radii Plus, Henry Schein Dental). In order to avoid wing movements and to encourage walking of the fly, the fly's wings were tethered to the tungsten rod by folding them against the rod and using dental cement for fixation. Additionally, to stabilize the head, it was fixed by applying dental cement to the neck of the fly (Paulk et al., 2015). After tethering, the animals were provided with water and allowed to rest for about 60 min before testing began.

The virtual reality arena was set up as described in Van De Poll et al. (2015). The hexagon-shaped arena consisted of six 32×32 pixel LED panels (Schenzhen Sinorad Medical Electronics Inc.) (Fig. 1A). In its center, a visually patterned, air-supported Styrofoam ball (40 mg, 15 mm diameter; Spotlight Ltd Pty) was used as a walking medium for the tethered flies (Fig. 1B). For positioning of the flies on the ball, a 6-axis micromanipulator (Edmund Optics) was used. The setup was additionally illuminated by three 40 W bulbs in order to provide adequate lighting for tracking the fly and ball movements by a camera (Point Grey Laboratories) at 60 frames s⁻¹, mounted at the front of the arena. The video was further analyzed by FicTrac (Moore et al., 2014), custom-written tracking software operating in Ubuntu Linux (12.10) running on Windows 7 (SP1). To create a closed-loop environment where the fly could control the position of the stimulus, the movements of the stimuli were linked to the movements of the ball. This was achieved by linking the output (movement of the fly on the ball) of FicTrac with custom-written Python (2.5) scripts (modified after Van de Poll et al., 2015), which in turn generated the visual output with the corresponding stimulus position through VisionEgg software (Straw, 2008). FicTrac extracted the lateral movement (X), the forward movement (Y) and the rotation of the ball (turning ΔΘ) (Fig. 1B) and calculated a fictive path of the fly movements which then resulted in a 1:1 translation between the movement of the ball and the rotation on the stimulus within the 360 deg arena (25 ms delay).

To induce Chrimson activation, three orange–red LED lights (Luxeon Rebel, 617 nm, 700 mA, LXM2-PH01-00700) were mounted around the arena, focusing on the center of the arena. The activation and inactivation of the red LED lights was linked to the position and size of the stimulus (Fig. 6), provided by FicTrac and controlled by a BlinkStick (Agile Innovative Ltd), and an LED controller board, driven by a custom-written Python (2.7) script. For these experiments, position thresholds triggered the activation and inactivation of the LEDs. In Fig. 6, for example, LED activation was induced via BlinkStick when the 26.25 deg bar was positioned by the fly in-between 330 and 30 deg (frontal visual field, FVF).

A lux meter (LX101BS) was used to determine luminosity of the background and the visual stimuli and the mean calculated from four independent measurements in Fig. 4. Colors of the arena were set in the custom-written Python (2.5) script as RGBA values. Colors of the visual stimuli were set as RGB values in the same script.

All behavioral experiments were performed in closed-loop. Flies were positioned on the air-supported ball in the LED arena (Fig. 1A) and allowed to habituate to the new environment for about 2–3 min.

In order to examine general fixation, the flies were exposed for three times 2 min in succession to a solid black (unlit) bar on a green (555 nm) background. The bar was 15 deg (8 pixels) wide and 60 deg (32 pixels) high (Fig. 1B) or 15 deg (8 pixels) wide and 26.25 deg (14 pixels) high (Fig. 2F). If a fly was fixating on the bar, it kept the stimulus within its FVF, which was defined as the width of the frontal panel (32 pixels, 60 deg) (Fig. 1B). Random perturbations were used in order to determine the quality of fixation. The bar was displaced every 10–30 s by 60 deg (32 pixels) to the left or to the right. The threshold for a successful repositioning was 10 s or less.

For the multiple-choice maze, a set of 12 different visual stimuli was presented to the fly (Van De Poll et al., 2015). The stimuli were all solid black (or red) bars on a green (or red/cyan) background and 15 deg wide with different heights (Fig. 2A,B). The center of the bars was linked to the vertical center of the LED panels, so differences in bar height resulted in a symmetrical change. The flies were exposed to two competing stimuli, stimulus 1 and stimulus 2, always 180 deg apart, such that only one could be fixated upon at any point in time (Fig. 2C). A stimulus was regarded as successfully chosen when the fly walked a distance of 7 cm and mostly fixated that stimulus during that time (usually 20–50 s; for further details, see Van De Poll et al., 2015). The non-chosen stimulus was then replaced by a new stimulus (Fig. 2B). Subsequently, the fly had to choose again between the previously fixated/chosen stimulus (continuation) and the new (novelty) stimulus. This allowed us to study choice behavior in a historical context. The experiment was ended after the flies were exposed to at least 80% of the possible choices, which resulted in experiments between 40 and 60 min typically, depending on the walking speed of the flies.

For the pre-exposure experiments in Fig. 5, flies were presented with either a single 60 deg or a 26.25 deg high bar for three consecutive 2 min trials as described for the single-bar fixation experiments. Directly after these three consecutive trials, the flies were presented with the multiple-choice maze as described above. There was a 10 s break between all trials in which the flies walked in the arena without visual stimulation.

For the optogenetic experiment in Fig. 6, we used the same multiple-choice paradigm as described for Fig. 2. In order to activate the red LEDs, the fly had to position the 26.25 deg bar in the FVF (330–30 deg). As soon as the 26.25 deg bar left this area, the LEDs were turned off via a BlinkStick (Agile Innovative Ltd). Analysis of the data was performed as before.

For the learning experiments in Fig. 7, flies were presented with two competing solid black bars (60 deg and 26.25 deg) on a green background. At any time, the bars were 180 deg apart, and flies could fixate on one or the other. Random perturbations ensured active fixation. The experiment was divided into three parts: baseline, training, memory. In the baseline trials, the flies had to choose between the two bars for 2 min in three consecutive trials. In the training trials (3×2 min), red LEDs were activated every time the 26.25 deg bar was in the FVF using a BlinkStick controller board (Agile Innovative Ltd). In the last three trials (2 min each), memory was tested without the activation of the red LEDs.

For the single-bar fixation experiments in Fig. 8, red LEDs were activated when the dark solid bar was at different positions in the arena (right: 60–120 deg; back: 150–210 deg; left: 240–300 deg). As in the last two experiments, LED activation was controlled by a BlinkStick (Agile Innovative Ltd). Each position was tested for three consecutive 2 min trials. Fixation was averaged across these three trials (no significant difference was found between trials, Watson–Williams-test, α=0.05, data not shown). Each time, after one side was tested for fixation, we returned to a 2 min trial of single-bar fixation without optogenetic activation in order to test whether there was a significant learning effect. To test for learning, we extracted 20 s from this trial and analyzed the mean direction of the bar position after dNPF circuit stimulation. We did not observe any significant learning effect for all directions compared with baseline trials (fixation without LEDs activation) (Watson–Williams-test, α=0.05, data not shown), so we averaged the first trial without dNPF circuit activation for all experiments (1st trial LED off).

dNPF-Gal4,UAS-mCD8::GFP flies were collected under CO₂ anesthesia and dissected under cold 1× phosphate-buffered saline (PBS). Samples were then fixed with 4% paraformaldehyde diluted in PBS-T (1× PBS, 0.2× Triton-X 100) for 20 min, followed by 3×20 min washes in PBS-T. They were then blocked with 10% goat serum (Sigma-Aldrich) for 1 h and incubated in primary antibody overnight. We used mouse antibody to nc82 [1:100, Developmental Studies Hybridoma Bank (DSHB)] and rabbit antibody to GFP (1:1000, Invitrogen). After 3×20 min washes with PBS-T, secondary antibody was added and the tube was covered with aluminium foil for overnight incubation. We used AlexaFluor-488 goat anti-rabbit (1:250, Invitrogen) and AlexaFluor-647 goat anti-mouse (1:250, Invitrogen) as secondary antibodies. Following three final washes with PBS-T, samples were transferred to microscope slides and mounted on a drop of Vectashield (Vector Laboratories) for imaging.

Imaging was done using a spinning-disk confocal system (Marianas; 3I, Inc.) consisting of an Axio Observer Z1 (Carl Zeiss) equipped with a CSU-W1 spinning-disk head (Yokogawa Corporation of America), and an ORCA-Flash4.0 v2 sCMOS camera (Hamamatsu Photonics); a 20×0.8 NA PlanApo objective was used and image acquisition was performed using SlideBook 6.0 (3I, Inc.).

FicTrac datasets were imported for offline analysis in MATLAB 2015b as well as in GraphPad Prism 7.0. In order to analyze fixation, bar positions were converted into polar coordinates and their mean vector length calculated using the Circular Statistics Toolbox for MATLAB (Berens, 2009). All data were first tested for normal distribution using the D'Agostino and Pearson test for normality, set at α=0.05. A Brown–Forsythe test was used to test for differences in standard deviations. Groups of parametric data were tested for significant differences using an ordinary one-way ANOVA followed by a Bonferroni multiple comparisons test, set at α=0.05, or a Tukey's multiple comparisons test (α=0.05) to detect specific differences. Groups of non-parametric data were tested for significant differences using a Kruskal–Wallis test followed by a Dunn's multiple comparisons test (α=0.05, corrected). Pairwise data were compared using a t-test (parametric) or Wilcoxon rank-sum test (non-parametric). In some instances, averaged proportioned choices for particular stimuli were compared independently with a chance value of 8.333% using the Wilcoxon rank-sum test α=0.05 and further corrected with the Benjamini–Hochberg method using the fdr_bh() function in Matlab. Significance for the uncorrected values for this analysis is denoted with an asterisk and the corrected values with a double dagger in the figures; the new corrected significance level is listed in the figure legends. For circular data, the distribution of the bar positions and the mean vector length was tested for non-uniformity using the Rayleigh test. The mean direction of the mean vectors was compared using the Watson–Williams test (Berens, 2009; Watson and Williams, 1956).

A female D. melanogaster fly was positioned on an air-supported ball in the center of a hexagonal LED arena (Fig. 1A,B). The fly was presented with a visual stimulus (a dark bar on a lit green background, 15 deg wide and 60 deg high). Walking of the fly resulted in forward, lateral and turning movements of the ball. These movements were translated into corresponding movements of the visual stimulus displayed by the LED arena via a camera-based closed-loop interface (FicTrac; Moore et al., 2014; Fig. 1B). For example, if a fly was moving the ball laterally to the left side by 55 deg, which indicates a turning behavior to the left, the visual stimulus moved to the left by 55 deg as well. The resulting rotation angle (ΔΘ) is the difference between two stimulus positions in the arena, measured from the center of the air-supported ball (Fig. 1B). This setup allowed the fly to keep the visual stimulus in the FVF voluntarily, so we could assess fixation and attention-like parameters. Flies rapidly learned to fixate on the virtual object and increased their fixation significantly over three consecutive 2 min trials (Fig. 1C,D). To ensure that flies actively fixated on the object, we introduced visual perturbations, where the stimulus was randomly moved by 60 deg to the left or to the right every 10–30 s. If the flies were actively attending to the stimulus, they rapidly returned it to their FVF within 10 s (Fig. 1E; see Materials and Methods). We found no significant difference in the proportion of successful returns after the perturbations (mean±s.d.: trial 1: 83.5±20.6, trial 2: 79.9±28.7, trial 3: 86.8±24.4) or the time taken to return the stimulus to the FVF (median: trial 1; 5.8 s, trial 2: 6.1 s, trial 3: 6.0 s) between the three trials (Fig. 1F,G). Further, there was no significant difference in walking speed during the three trials (median: trial 1: 2.7 mm s⁻¹, trial 2: 2.6 mm s⁻¹, trial 3: 2.7 mm s⁻¹) (Fig. 1H).

We next investigated visual decision making in this paradigm. Not all visual stimuli are intrinsically attractive for Drosophila (Maimon et al., 2008; Park and Wasserman, 2018). In order to better understand visual preferences in flies, we implemented a virtual choice maze used previously to study visual preferences in honeybees (Van De Poll et al., 2015). In this previous experiment, bees were able to choose recurrently between 12 visual stimuli, which were green bars flickering at different frequencies. This operant approach revealed a clear preference/aversion profile for specific visual flickers in bees. We implemented a similar recursive approach for Drosophila, to determine visual preferences or aversions among 12 different sized bars, presented in paired competition with one another. The bars were all dark on a lit green background, 15 deg wide and between 3.75 and 60 deg high (Fig. 2A). The flies walked on a fictive path along the edges of a (virtual) dodecahedral structure (Fig. 2B, green arrows). This fictive path represents the whole distance and duration the fly moves while being presented with two competing stimuli. The faces of the geometric structure represented the different visual objects (Fig. 2A,B; bar height in degrees). At any time, the fly was thus presented with two competing objects (faces in Fig. 2B), 180 deg apart (Fig. 2C). Flies fixated on one or the other object, and after walking for 7 cm (fictive path), a decision was arrived at by the program algorithm (see Materials and Methods) depending on which object was most fixated upon. Perturbations occurred throughout, as above (Fig. 1E), to ensure this was an active choice. The more fixated object was retained and the less fixated object was replaced by another object, represented as the next adjacent face on the dodecahedron structure (Fig. 2B). An experiment lasted until at least 80% of all stimuli were seen, or a minimum of 45 min per fly, yielding an average proportioned choice profile (Fig. 2D, see Materials and Methods).

Our closed-loop visual competition experiments revealed that wild-type, female Drosophila selected the large 60 deg bar above chance level (red dashed line at 8.3%, Fig. 2D). Interestingly, flies seemed to select medium-sized bars (37.5–22.5 deg) significantly below chance level, suggesting these are visually repulsive to them. Calculating the mean direction vector for the stimuli revealed that the 60 deg bar was indeed mostly positioned in the FVF (342.1±62.5 deg). In contrast, medium-sized bars were positioned behind the fly: for example, the 26.25 deg bar was positioned in the opposite direction (140.9±70.8 deg) on average (Fig. 2E). The mean directions for the largest (60 deg) bar and the medium (26.25 deg) bar were significantly different from each other (Fig. 2E), and the 60 deg bar was chosen significantly more often than the 26.25 deg bar (Fig. 2D). Having found that the 26.25 deg bar was consistently avoided when presented in competition with other bars, we then asked whether it was aversive even when presented on its own. Indeed, when we presented the 26.25 deg bar on its own (as in Fig. 1 for the 60 deg bar), flies displayed clear anti-fixation behavior (Fig. 2F). This confirms that the 26.25 and 60 deg bars are indeed visually ‘repulsive’ and ‘attractive’, respectively, and that our operant virtual maze design can effectively uncover these innate visual preferences.

We next investigated whether these innate visual preferences persisted through life in older female flies. Older flies (17–40 day) displayed a remarkably similar choice profile to that of the younger (5–10 day) flies (Fig. 3A), again choosing the 60 deg bar significantly more often than the 26.25 deg bar. Interestingly, the smallest bar (3.75 deg) became attractive to older flies, to a similar level to the largest bar. Older flies showed a significant fixation for the 60 deg bar and an anti-fixation for the 26.25 deg bar, with a significant difference in mean vector direction between these objects (Fig. 3B). However, there was no significant difference in mean vector length between the young and old dataset for these two visual objects (Fig. 3C). This indicates that the quality of fixation (and anti-fixation) remains robust with age. Interestingly, old age significantly decreased novelty-seeking behavior in this paradigm (Fig. 3D,E). Every visual choice represents two different historical contingencies: either a continued selection of a preferred object (a ‘continuation’ choice) or a selection of a novel object that has just replaced a previously non-preferred object (a ‘novelty’ choice) (Van De Poll et al., 2015; see Materials and Methods). Younger flies fixated upon the smaller 26.25 deg bar more often when it was novel (Fig. 3D), suggesting that novelty could override repulsion. In contrast, older flies mostly displayed continuation choices (Fig. 3E), and showed significantly less novelty-seeking behavior in general for all objects (Fig. 3F). These experiments show that innate visual preferences remain robust throughout a fly's life, without deterioration in fixation behavior, although more flexibility in younger flies is evident.

A recent study investigating visual attention in Drosophila showed that background luminosity and color affect visual attraction and aversion behavior in flies (Koenig et al., 2016). To test whether this might be the case for the visual objects in our closed-loop paradigm, we ran our virtual maze experiments with different background colors, and we also explored how contrast might affect choice behavior for an attractive stimulus (Fig. 4). Changing the background color from green (RGBA: 0.0, 1.0, 0.0, 1.0) to cyan (RGBA: 0.0, 0.58, 0.58, 1.0) and maintaining the same luminosity (3279 lx) did not alter the choice profile of young wild-type female flies, although more significant effects were noted (Fig. 4A). The 60 deg bar was still chosen significantly more often than the 26.25 deg bar, and the mean direction of each bar position for these objects was still significantly different (Fig. 4A, right box plot). Behavioral processes were also preserved under these different conditions: the 60 deg bar was mostly chosen as a ‘continuation’ event, whereas the 26.25 deg bar was fixated upon mostly if it was novel (Fig. 4B). When we presented competing dark objects on a red background (RGBA: 0.2, 0.0, 0.0, 1.0), the choice profile became flat (Fig. 4C), showing no significant preferences for any object. Drosophila have little perception of red-shifted light because they lack the corresponding photoreceptors (Garbers and Wachtler, 2016), so it is not surprising that they could not perceive the dark objects in this context. In addition to showing an absence of object perception in this context, these experiments indicate that the choice profiles revealed earlier are not an artifact of the maze geometry; only visible objects revealed a significant choice profile. Finally, we also tested choice behavior for different contrasts of the attractive 60 deg bar. We observed a significant difference between the strongest contrasts (7 lx bar, 3279 lx background) compared with the last four visual stimuli with the lowest contrasts (Fig. 4D).

Habituation and pre-exposure can have an effect on valence-based decisions (Rangel et al., 2008). We therefore next investigated whether we could modulate the attractive and repulsive responses towards visual stimuli by pre-exposing the fly to these stimuli prior to running the virtual choice maze experiment. To test this, we pre-exposed flies to either a single 60 deg bar or a 26.25 deg bar for three consecutive 2 min trials (Fig. 5). For the attractive stimulus (60 deg), we found that pre-exposure resulted in a similar choice profile (Fig. 5B, left) to that for non-exposed flies (Fig. 2). The mean direction of both the attractive and aversive bar positions within the LED arena also seemed unchanged: the 60 deg bar was fixated and the 26.25 deg bar was anti-fixated (Fig. 5C, left). In contrast, pre-exposure to the repulsive 26.25 deg bar (Fig. 5A, right) appeared to dampen the average choice profile of flies. The 60 deg bar was not chosen significantly above chance level, although the 26.25 deg bar was still chosen significantly below chance level (Fig. 5B, right, uncorrected data). Accordingly, there was no significant difference between the averaged proportioned choice for the 60 deg bar and the 26.25 deg bar (Fig. 5B, right). The overall positions of both bars within the arena during the experiment remained similar: the 60 deg bar was still significantly within the FVF, the 26.25 deg bar was positioned in the back of the arena, and the mean directions for the attractive and aversive bar were significantly different (Fig. 5C, right). However, the mean vector length of the aversive 26.25 deg stimulus was not significant (Fig. 5C, right, red arrow). By comparing the averaged proportioned choices of the pre-exposed datasets with the averaged proportioned choices of naive flies (Fig. 5D), we observed that pre-exposure to the attractive 60 deg bar resulted in an increase in choice of the 60 deg bar in the subsequent choice maze assay, which was not the case for the flies that were pre-exposed to the 26.25 deg bar (Fig. 5D, left). In contrast, choice of the 26.25 deg bar was unaffected by either pre-exposure condition (Fig. 5D, right). These results suggest that prior experience, even if relatively brief (6 min), can affect valence assignations in the subsequent recursive choice paradigm, particularly for attractive stimuli. However, innate object preferences still appear quite robust.

In the preceding experiments, we attempted to query the fly's motivational state by examining effects of age, stimulus parameters and pre-exposure. We next attempted to modulate visual preferences by directly manipulating Drosophila brain circuits that have been associated with motivational states, such as neurons that express dNPF (Shao et al., 2017). dNPF has been associated with learning in Drosophila (Krashes et al., 2009) by, for example, altering feeding behavior (Chung et al., 2017), social behavior (Wu et al., 2003) and olfactory learning (Chung et al., 2017; Krashes et al., 2009; Shao et al., 2017). It is unclear, however, whether this neuropeptide also plays a role in gating information about the valence of visual stimuli. In the context of our recursive choice maze paradigm, we tested whether the previously established preference profile for object size could be modulated by acute activation of dNPF neurons, by expressing the red-shifted channelrhodopsin variant ‘CsChrimson’ (Klapoetke et al., 2014) in dNPF-expressing neurons (Fig. 6A). To ensure the optogenetic activation of the dNPF circuit was only associated with one object, we transiently activated dNPF-expressing neurons only when flies fixated on the small aversive (26.25 deg) bar, in the context of our recursive choice maze paradigm (Fig. 6B,C). The red light was off (i.e. NPF-expressing neurons were not activated) when the fly fixated on any other visual stimulus. Control animals were not fed retinal, a food supplement required for light-induced activation of the channelrhodopsin (see Materials and Methods).

We found that control flies displayed a similar preference profile to that of wild-type (CS) flies (Fig. 6D, upper panel), despite the red light turning on when the 26.25 deg bar happened to be in the fly's FVF. In contrast, optogenetic activation of dNPF-expressing neurons in retinal-fed flies led to a loss of repulsive behavior towards the 26.25 deg bar, while the larger bar remained attractive (Fig. 6D, lower panel). Activation of dNPF-expressing neurons had no effect on selection of the 60 deg bar, but selection for the 26.25 deg bar was significantly increased (Fig. 6E). This was also reflected by the vector length and orientation, which remained unchanged for the large bar but decreased (as a result of decreased repulsion) for the smaller bar (Fig. 6F,G). Altered behavior toward the smaller bar was not due to altered walking speed, which remained unchanged (Fig. S1A). Increased attraction to the smaller bar was also evident in retinal-fed flies returning this object to their FVF more often, compared with controls, after a perturbation event (Fig. S1B). Thus, negative visual valence can be eliminated by acute dNPF circuit activation, and this manipulation increases the flies' motivation to fixate on an innately repulsive object.

As activation of the dNPF pathway is associated with olfactory learning in Drosophila (Krashes et al., 2009), we next tested whether operant activation of the dNPF circuit in our paradigm could result in visual learning. For this purpose, we devised a closed-loop learning assay (Fig. 7A) where we activated the dNPF neurons specifically during fixation of the smaller (26.25 deg) bar, which was in competition only with a larger (60 deg) bar (see Materials and Methods). We first confirmed our previous result showing that over three consecutive 2 min trials, flies chose the large bar significantly more often than the 26.25 deg bar, for both retinal-fed flies (+ATR) and controls (Fig. 7B, trial 1–3). Training began in trial 4, by turning on the red light only when the smaller bar was in the frontal visual field. Retinal-fed flies (+ATR) lost their aversive behavior towards the small bar by trial 5 (Fig. 7C, right), showing no significant difference in mean choice behavior in trials 5 and 6 and a general non-significant choice behavior for the training trials 4–6 (Fig. 7B). However, as soon as the dNPF circuit was no longer being activated (trial 7), flies returned to their innate preferences (Fig. 7C, right). This suggests that the 26.25 deg bar was rendered transiently attractive by the operant paradigm – and was at least equivalent in valence to the 60 deg bar – but that the valence effect did not persist beyond the training session in our experiments.

To ensure that we had altered valence rather than merely causing indifference to the competing bars, we devised a different operant learning experiment, where we optogenetically manipulated NPF-expressing neurons only when the flies placed an object (the 60 deg bar) in a specified quadrant of the arena (Fig. 8A). Flies were first tested for baseline fixation behavior, which for the larger bar is naturally directed towards the FVF (Fig. 8A,B, first panel from left). To test for operant learning, dNPF-expressing neurons were activated only when the bar was positioned by the fly to its right (between 60 and 120 deg; Fig. 8A, second panel). Accordingly, flies kept the bar significantly more often on their right side (Fig. 8B, second panel). We confirmed this result by activating the dNPF circuit when the bar was positioned behind the fly or to the left, with flies accordingly placing the bar in those respective quadrants in closed loop (Fig. 8A,B, third and fourth panels). This shows that activation of the dNPF circuit is indeed a positive or motivating cue, and not simply abolishing visual behavior. Additionally, after every experiment, we investigated whether flies continued to place the bars in the same locations, by testing the flies again in a 2 min trial without activating dNPF neurons (see Materials and Methods). However, flies immediately returned to keeping the stimulus in their FVF after the operant training (Fig. 8A,B, 1st trial LED off). We conclude that in this particular paradigm, activation of the dNPF circuit provides a transient positive cue that does not have a lasting effect on visual learning.

All animals display strong innate preferences, being attracted to some stimuli and repulsed by others, which influences their ultimate decisions and actions. With odors, this easily relates to chemicals relevant to an animal's survival in a specific environment: the smell of rotten food is repulsive to humans but attractive to a fly. Visual stimuli are more difficult to assign valence, as these tend to be highly context dependent (Brembs and Wiener, 2006; Heisenberg et al., 2001; Gorostiza et al., 2017). Simple visual parameters, such as responses to light intensity (Menzel, 1979; Reichert and Bicker, 1979) and color (Menne and Spatz, 1977; Morante and Desplan, 2008), have been well studied in D. melanogaster. In contrast, fly responses to visual objects with multiple features are less well understood (Paulk et al., 2013). While there has been a considerable amount of research done on visual learning in tethered, closed-loop flight paradigms, these experiments are typically not concerned with uncovering innate preferences but rather focus on feature discrimination of visual objects of a possible equivalent valence such as an upright ‘T’ and an upside down ‘T’ (Heisenberg et al., 2001; Liu et al., 2006; Paulk et al., 2013). Indeed, most visual learning paradigms in animals are unaware of the larger valence landscape wherein experimental stimuli reside – or even whether they are attractive, aversive or neutral – but rather settle on robust responses that produce reliable behavioral readouts. For visual decision making, however, some knowledge about innate valence is important for better understanding of responses to different stimuli (Guitart-Masip et al., 2014).

In this study, we use a closed-loop visual paradigm for walking flies to show that flies find large bars innately attractive and smaller bars repulsive. This confirms previous work done in closed-loop flight (Maimon et al., 2008), indicating surprisingly entrenched valence effects for these simple visual objects. We used a virtual reality maze paradigm for walking flies, previously developed for walking honeybees (Van De Poll et al., 2015), to place these preferences within a larger valence spectrum for object size, particularly bars of different heights but the same width. In this paradigm, flies are able to reveal their visual preferences by iteratively selecting competing objects in a recurrent binary choice design. We found that visual preference profiles are remarkably robust, with larger bars remaining more attractive and smaller bars repulsive even as flies age. Further, changes to the background color did not alter choice behavior when the contrast between stimulus and background was high. Earlier studies have shown that these visual stimulus parameters can affect a fly's attention and therefore learning behavior (Koenig et al., 2016). An aversion towards smaller vertical objects in tethered flying fruit flies has previously been discussed as representing a possible hazard (Maimon et al., 2008; Park and Wasserman, 2018), whereas large vertical bars could represent a natural landmark. We did find, however, that flies fixate on innately repulsive objects if they are perceived as novel, suggesting that an internal switch exists for over-riding these deeply entrenched valence effects. This novelty effect was also observed in honeybees for innately aversive visual flickers (Van De Poll et al., 2015).

An interesting observation was made in older flies compared with younger flies. Whereas older flies displayed similar valence profiles to those of younger flies, and their fixation behavior was just as robust, they were less responsive to visual novelty. This conservative behavior in older flies has some parallels with human behavior: aging in humans affects decision making, resulting in less impulsive and delayed choice behavior, compared with younger participants (Eppinger et al., 2011, 2012). Such superficial similarities in valence-based decision making between older flies and older humans is not surprising considering the possibility of homologous systems being involved in decision making in the two species (Barron and Klein, 2016; Barron et al., 2015; Bogacz and Gurney, 2007; Strausfeld and Hirth, 2013).

One important aspect of valence-based decision making is that it is not necessarily hard wired; it can be influenced by experience (Dayan and Abbott, 2003; Dickinson and Balleine, 2002; Rangel et al., 2008). This was observed to some extent in our experiments. A relatively short exposure to a repulsive or attractive 60 deg bar stimulus altered valence effects in the subsequent ∼1 h in the virtual choice maze. Pre-exposure to the aversive 26.25 deg bar resulted in a more blunted valence profile, where we did not observe a significant difference in choice behavior between the attractive and aversive stimulus, whereas pre-exposure to the attractive 60 deg bar resulted in an increased choice of this bar in the subsequent multiple-choice assay. This can be interpreted as sensitization rather than habituation to a positive stimulus, in this case towards a stimulus with positive valence (Fischbach, 1981). This shows that even a brief visual experience can influence subsequent decision making over a long period of time. Typically, outside of virtual reality environments, it is impossible to ascertain an animal's exact previous experience.

We found that a neuromodulatory circuit in the fly brain might play a key role in governing visual decision making in our paradigm. NPY is a highly conserved neuropeptide (Wahlestedt and Reis, 1993; Feng et al., 2003) that regulates motivational state in animals (Bannon et al., 2000), and the fly homolog dNPF (Garczynski et al., 2002; Shao et al., 2017) seems to play a similar role. In mammals, there is also evidence that NPY plays a role in suppressing anxiety and fear (Fendt et al., 2009; Primeaux et al., 2005; Redrobe et al., 2002; Thorsell, 2000), as well as regulating responsiveness to aversive or stressful stimuli (Bannon et al., 2000; El Bahh et al., 2001). In flies, dNPF has been linked to olfactory learning by regulating dopaminergic input to the mushroom bodies (Krashes et al., 2009), with data suggesting that it may provide a rewarding cue (Rohwedder et al., 2015; Shao et al., 2017). The role of dNPF input to visual centers such as the CC is less clear, although recent studies similarly propose a reinforcing neuromodulatory role (Chung et al., 2017; Krashes et al., 2009; Shao et al., 2017). We found that optogenetic activation of dNPF-expressing neurons could indeed be used as a positive cue, which agrees with a recently published study (Shao et al., 2017): flies could be induced to ‘place’ a visual object at different positions in the arena, by only activating the NPF-expressing neurons when flies kept the object in that specified location. Flies innately fixate on objects in their frontal visual field (Guo et al., 2015; Heisenberg et al., 2001), so their capacity to also fixate to the sides suggests an attention-like effect (Sareen et al., 2011; Sun et al., 2017) which could possibly be linked to a positive cue. Consistent with this result, optogenetically activating the dNPF circuit during the presence of a repulsive object (the smaller bar) made it more attractive. Also, when random perturbations removed the smaller bar from the dNPF activation zone (the FVF), activated flies returned it more often to the FVF (Fig. S1B), suggesting an increased motivation to fixate on this object. However, the transient nature of any changes we were able to impose on innate visual fixation preferences suggests that other systems might need to be recruited to effectively transform these altered preferences into a more persistent memory.

We thank Leonie Kirszenblat, Lucy Heap, Chelsie Rohrsheib, Adam Hines, Kai Feng, Lisa Wittenhagen and Eva Maria Reuter for helpful comments on the manuscript. We thank James Reuben for helping with behavioral experiments. We thank the QBI microscopy facility for help with imaging.