Animals evolve mechanisms to send and receive communication signals through multiple sensory channels during crucial behavioral contexts such as aggression and reproduction. This ensures the transmission of important context-dependent signals that supply either the same (redundant) or different (non-redundant) information to the receiver. Despite the importance of multimodal communication, there are relatively few species in which information on sender signals and receiver responses are known. Further, little is known about where context-dependent unimodal and multimodal information is processed in the brain to produce adaptive behaviors. We used the African cichlid, Astatotilapia burtoni, to investigate how unimodal and multimodal signals are processed within the female brain in a reproductive context. During courtship, dominant males produce low frequency sounds in conjunction with visual displays (quivers) directed towards receptive gravid females. We compared affiliation behaviors and neural activation patterns in gravid females exposed to visual, acoustic and visual–acoustic signals from courting dominant males. Females displayed reduced affiliation in auditory-only conditions, but similar affiliation during visual and visual–acoustic conditions, demonstrating that visual–acoustic signaling from males is non-redundant but vision dominates. Using the neural activation marker cfos, we identified differential activation in specific socially relevant brain nuclei between unimodal and multimodal conditions and distinct neural co-activation networks associated with each sensory context. Combined with our previous work on chemosensory signaling, we propose that A. burtoni represents a valuable vertebrate model for studying context-dependent behavioral and neural decision making associated with non-redundant multimodal communication.

Across taxa, organisms have evolved mechanisms to communicate through multiple sensory channels for mediating social interactions such as reproduction, aggression and parental care. Multimodal communication occurs when one organism produces a signal via more than one sensory channel, which can then be detected by a receiver to either provide the same (redundant) or different (non-redundant) information (Johnstone, 1996; Partan and Marler, 1999, 2005). When presented separately, non-redundant signaling components should have different effects on a receiver, while redundant components should elicit the same or equivalent effects. According to the classification scheme of multimodal communication proposed by Partan and Marler (1999, 2005), recipient responses can include a physical behavioral response, a physiological response or no response at all. To properly classify multimodal signaling as redundant or non-redundant, however, the recipient response to each unimodal component separately as well as the response to the combined signals must be tested. The ability to process signals through multiple senses is crucial in instances where environmental noise may prevent effective transmission of one or more signal types, highlighting the importance of understanding whether signaling is redundant or non-redundant. For example, anthropogenic noise, pollution and climate change can impact both the sending and receiving of acoustic, chemosensory and visual information that is vital to communication in many animals (Halfwerk and Slabbekoorn, 2015; Partan, 2017; Slabbekoorn et al., 2010). This sensory disruption is particularly relevant for non-redundant signaling, in which masking or elimination of information in one sensory channel can have detrimental effects on decisions related to survival and reproductive success (Candolin, 2019; Shannon et al., 2016). After a relevant signal is received, this new information must also be integrated with the receiver's physiology and other environmental factors to produce the appropriate behavioral response. Most studies on multimodal communication largely focus on categorizing signals given by the sender and the resulting behavioral response in the receiver. Little is known, however, about how and where in the brain receivers process this multimodal social information during crucial behavioral interactions.

Although many studies across diverse taxa examine multimodal communication from the perspective of which senses are used and what type of information they provide to a receiver, less is known about how these signals are integrated in the brain and whether there are similar mechanisms of integration across species. Social behaviors are controlled by the brain, and a conserved neural network called the social decision-making network (SDMN) provides a comparative framework to examine how different animals integrate social information to mediate adaptive behaviors (O'Connell and Hofmann, 2012). The SDMN is composed of the social behavior network (SBN: lateral septum, bed nucleus of the stria terminalis/medial amygdala, anterior hypothalamus, ventromedial hypothalamus, preoptic area, periaqueductal gray/central gray) and the mesolimbic reward pathways (lateral septum, basolateral amygdala, bed nucleus of the stria terminalis/medial amygdala, basolateral amygdala, striatum, nucleus accumbens, hippocampus, ventral pallidum, ventral tegmental area) (Goodson and Kingsbury, 2013; Newman, 1999; O'Connell and Hofmann, 2011). Because animals live, interact and make decisions in a multisensory world, it is crucial to examine not only how receivers respond behaviorally to unimodal and multimodal sensory information in social contexts but also which regions of the SDMN are involved in processing these sensory inputs that lead to behavioral decisions. Thus, identification of relevant brain regions also allows for phylogenetic comparisons of nuclei involved in integrating unimodal and multimodal signals across vertebrates.

Fishes represent the most diverse and speciose vertebrate group, with enormous diversity in complex social behaviors. The African cichlid fish, Astatotilapia burtoni, is ideally suited to address questions related to multimodal communication because dominant males direct visual, acoustic, chemosensory, mechanosensory and tactile signals towards females during their courtship and spawning behaviors (Butler and Maruska, 2016b; Field and Maruska, 2017; Maruska et al., 2012). Dominant males are brightly colored, defend a territory, court females and have an upregulated reproductive axis. In addition to the bright coloration depicted by dominant males, visual courtship behaviors (body quiver, waggle, leads into territory), along with chemosensory and acoustic signaling, collectively serve as honest signals of fitness to females (Fernald and Hirata, 1977; Maruska and Fernald, 2012). For example, courtship body quivers are often coincident with the production of low frequency pulsed sounds (Fig. 1), the peak frequency of these sounds is negatively correlated with male body size, and females prefer to affiliate with males producing sounds over those that only provide visual signals (Maruska et al., 2012). Reproductively active females (gravid; full of eggs) are receptive to these mating signals, and also show improved vision, improved hearing and increased affiliation towards males, especially after ovulation (Butler et al., 2019; Maruska et al., 2012). Because multimodal social communication is crucial for their ability to identify mates, reproduce and defend their territories, A. burtoni has become an emerging model in many aspects of behavioral neuroscience and sensory communication.

Fig. 1.

Male Astatotilapia burtoni produce visual–acoustic courtship signals. Brightly colored dominant males (left) generate pulsed low frequency courtship sounds (waveform shown) while curving their bodies, quivering and presenting their anal fin egg-spots towards nearby females (right).

Fig. 1.

Male Astatotilapia burtoni produce visual–acoustic courtship signals. Brightly colored dominant males (left) generate pulsed low frequency courtship sounds (waveform shown) while curving their bodies, quivering and presenting their anal fin egg-spots towards nearby females (right).

The overall goal of this study was to test whether visual–acoustic courtship displays from dominant male A. burtoni provide redundant or non-redundant information to females, and where this sensory information is processed in the brain. Specifically, we quantified affiliation behavior of gravid females when they received visual, acoustic or combined visual–acoustic signals from a courting dominant male and compared this with a non-reproductive context as a control. By performing in situ hybridization for the immediate early gene cfos as a proxy for neural activation, we also investigated where unimodal and multimodal visual–acoustic signals are processed within the female brain. Our results demonstrate that acoustic information from males functions as a non-redundant courtship signal for receptive gravid females and highlights the importance of studying receiver processing in multimodal interactions. Differential cfos expression, classification of females into sensory contexts using cfos expression across brain nuclei, and unique co-activation networks among sensory conditions show that females perceive unimodal and multimodal auditory and visual courtship signals differently in the brain.

Animals

Adult Astatotilapia burtoni (Günther, 1893) were laboratory bred but originally derived from a wild-caught population from Lake Tanganyika, Africa, and kept in community aquaria with conditions similar to their natural environment: pH 7.8–8.0, 28–30°C, 12 h light/dark cycle, and constant aeration. Fish were fed cichlid flakes (AquaDine, Healdsburg, CA, USA) daily and supplemented with brine shrimp twice a week. All experiments were performed in accordance with the recommendations and guidelines provided by the National Institutes of Health Guide for the Care and Use of Laboratory Animals, 2011. The protocol was approved by the Institutional Animal Care and Use Committee (IACUC) at Louisiana State University, Baton Rouge, LA, USA.

Experimental procedure

To understand how female A. burtoni process unimodal and multimodal visual–acoustic courtship signals from males, we used experimental females in the gravid stage of the reproductive cycle. Gravid A. burtoni females have extended abdomens full of eggs, providing a potential visual signal of reproductive readiness to dominant males, and are close to choosing a male for spawning. Female reproductive readiness was confirmed after the behavioral trial by measuring gonadosomatic index [GSI=(gonad mass/body mass)×100]. Only females with GSI≥6.5 were used in the experiment (mean±s.d. GSI 9.23±2.33). Experimental trials were performed in 37.8 l aquaria divided into 2 compartments (community side: 33.5×25×26.5 cm, subject side: 16.5×25×26.5 cm) separated by clear acrylic barriers permanently sealed to the sides and floor of the tank, preventing the transfer of odorants and hydrodynamic stimuli that could be detected by the lateral line system (Fig. 2). While chemosensory, mechanosensory and tactile signals are also used during courtship in this species, our paradigm was designed to test only visual and sound production acoustic signals from males to specifically understand how unimodal and multimodal auditory and visual signals are processed by females. An association zone was marked 15 cm from both sides of the dividing acrylic barrier and used to quantify female affiliation with the dominant male and/or community. Each compartment contained a gravel-covered bottom and a quarter terracotta pot positioned along each side of the barrier to function as a shared territory shelter. The smaller subject female compartment also contained an additional half terracotta pot outside the association zone along the opposing side of the tank to distinguish shelter use for reasons different from affiliation with the male in the shared shelter.

Fig. 2.

Experimental tank setup used to examine female behavior and neural activation patterns in response to male visual–acoustic courtship signals. (A) To examine how unimodal and multimodal courtship signals are processed by female A. burtoni, we used tanks with two compartments that were separated by different types of barriers to allow or block visual and acoustic signals. One compartment (right side of diagram) contained the subject gravid female, and the other compartment (left side of diagram) contained a stimulus dominant male and two recovering females (non-gravid and non-brooding) to form a small community. In control conditions, the left compartment contained only females (no dominant male). A quarter terracotta pot was placed along each side of the barrier, forming a joint territory for interaction between the subject gravid female and dominant courting male. Terracotta pot locations approximately mark the separation between the association zone (dashed vertical lines; marked 15 cm from acrylic barrier) and the remaining tank space. A removable blue opaque acrylic barrier also prevented the transmission of visual signals during the overnight acclimation period. (B) Following the acclimation period, a prostaglandin F2α (PGF)-injected stimulus female was added to the community to stimulate male courtship behaviors, and in all trials except the auditory-only trial, the blue barrier was removed to allow visual signals. Behaviors were recorded for 30 min to quantify dominant male courtship behavior and female affiliation behaviors, followed by fish collection to examine neural activation patterns.

Fig. 2.

Experimental tank setup used to examine female behavior and neural activation patterns in response to male visual–acoustic courtship signals. (A) To examine how unimodal and multimodal courtship signals are processed by female A. burtoni, we used tanks with two compartments that were separated by different types of barriers to allow or block visual and acoustic signals. One compartment (right side of diagram) contained the subject gravid female, and the other compartment (left side of diagram) contained a stimulus dominant male and two recovering females (non-gravid and non-brooding) to form a small community. In control conditions, the left compartment contained only females (no dominant male). A quarter terracotta pot was placed along each side of the barrier, forming a joint territory for interaction between the subject gravid female and dominant courting male. Terracotta pot locations approximately mark the separation between the association zone (dashed vertical lines; marked 15 cm from acrylic barrier) and the remaining tank space. A removable blue opaque acrylic barrier also prevented the transmission of visual signals during the overnight acclimation period. (B) Following the acclimation period, a prostaglandin F2α (PGF)-injected stimulus female was added to the community to stimulate male courtship behaviors, and in all trials except the auditory-only trial, the blue barrier was removed to allow visual signals. Behaviors were recorded for 30 min to quantify dominant male courtship behavior and female affiliation behaviors, followed by fish collection to examine neural activation patterns.

During the overnight acclimation period, a dominant male was added to the community compartment along with three recovering females (neither gravid nor mouthbrooding), and the subject gravid female was placed into the smaller compartment. Stimulus dominant males were selected for experiments based on bright coloration and the performance of typical dominance behaviors for several days prior to use in the trials. A removable blue barrier was also placed alongside the dividing clear barrier, to prevent visual communication between the subject gravid female and neighboring community prior to the experimental trial. The subject female was allowed to acclimate in the tank overnight.

Behavioral trials began the following morning, between 08:00 h and 11:00 h. To stimulate courtship behaviors and motivation from the dominant male across all conditions, a stimulus female was placed into his compartment prior to the trial. This stimulus female was injected with 3 μg g−1 body mass of prostaglandin F2α (PGF) and added to the community at the end of the 10 min pre-stimulus period. PGF levels naturally rise during the time of ovulation in A. burtoni, inducing the onset of female reproductive behaviors, which assists in attaining consistent male courtship behavior across experimental trials (Juntti et al., 2016; Kidd et al., 2013a). The blue acrylic barrier was removed simultaneously as the stimulus female was added to the community, initiating the 30 min stimulus trial period. To test whether male visual–acoustic courtship signals convey redundant or non-redundant information, we used an experimental paradigm to provide females with visual signals only, auditory signals only, or combined visual–auditory signals from the courting dominant males using different types of barriers separating the subject gravid female and the courting male. Visual-only trials consisted of a tank with barriers that created an air–water interface between the two compartments to prevent the transfer of any acoustic signals. In acoustic-only trials, a thin opaque black barrier separated the two compartments, shielding the subject female from visual signals from the courting dominant male while allowing the transmission of acoustic signals (hydrophone recordings across the barrier verified sound transmission). Visual–acoustic trials consisted of a thin transparent barrier that exposed subject females to both visual and acoustic signals from courting males. As a control for the neural activation aspect of the study, subject gravid females were exposed to visual–acoustic signals from the community of recovering females alone, without the presence of a courting dominant male. Thus, the experimental gravid female was exposed to social signals, but no visual or acoustic courtship signals from a male, allowing the controls to serve as a social but non-reproductive context to the subject gravid females. While females are not known to produce intentional sounds, any incidental sounds from feeding, mouth scraping on substances or fish hitting objects was possible. Importantly, the behavioral response of the subject female in this control condition was not necessary to classify male visual–acoustic signaling as redundant or non-redundant (only the three conditions with male signals are required). In contrast, this female-only control allowed us to distinguish neural activation patterns specifically related to male courtship signaling versus activation related to a general social group situation. All trials were video recorded to quantify dominant male courtship behavior and subject female affiliation behaviors during a 10 min pre-stimulus period and a 30 min stimulus period.

Behavioral quantification was performed using behavioral observation research interactive software (BORIS) (Friard and Gamba, 2016). Female affiliation was quantified as time spent in the association zone and time spent at the barrier. Each female was only used once in a single stimulus condition to allow for tissue collection and quantification of neural activation. Therefore, no within-individual behavioral analysis was performed. Because our focus was on female responses to male courtship, we also wanted to determine whether male courtship effort was similar across sensory conditions to ensure that the only difference was that females were receiving different sensory signals in each condition (visual only, acoustic only or combined visual–acoustic). Quantified male courtship displays included body quivers, tail waggles and leads towards the territory shelter. Quantified male behaviors included all behaviors performed by the male, regardless of whether it was directed towards the subject female or community fish. Courtship body quivers occur when the male curves his body in a C-shape and vibrates it while displaying egg spots along the anal fin, and this is the behavior associated with the production of courtship sounds (see Fig. 1). For analyses, we focused only on these quiver behaviors because they are the only behavior that is coincident with sound production.

Tissue preparation

To examine neural activation patterns, subject gravid females were killed 30 min after the end of the stimulus period by gradual cooling in ice-cold cichlid-system water followed by cervical transection. Females were measured for standard length (SL, 39.364±2.936 mm) and body mass (BM, 1.619±0.387 g) and Fulton's body condition factor K was calculated using the formula K=100/(BM/SL3) (26.277±3.815 mm). Brains were exposed, fixed in 4% paraformaldehyde (PFA) in 1× phosphate-buffered saline (1× PBS) overnight at 4°C, washed with 1× PBS and cryoprotected in 30% sucrose in 1× PBS at 4°C for 1–2 days. In preparation for sectioning, brains were embedded in OCT media (TissueTek, Sakura, Torrance, CA, USA), sectioned at 20 μm along the coronal plane and collected onto alternate positively charged slides (Superfrost plus, VWR, Radnor, PA, USA). Slides remained at room temperature for 2 days to allow sections to adhere to them before being stored at −80°C.

cfos in situ hybridization

To investigate where unimodal and multimodal courtship signals are processed within the brain, we used chromogenic in situ hybridization with riboprobes to localize mRNA of the immediate early gene (IEG) cfos as a proxy for neural activation. We chose cfos as a marker because it is more associated with the perception of incoming sensory stimuli as opposed to behavioral output (Teles et al., 2015). Probe specificity, sense controls and protocol design were previously described (Butler and Maruska, 2016a). In short, cfos primers were designed using the A. burtoni-specific cfos sequence to create a probe template through PCR, where digoxigenin (DIG)-labeled nucleotides were incorporated during amplification. After purification, the probe was diluted in hybridization buffer (1:5) and stored at −20°C.

Slides with brain sections were brought to room temperature and a hydrophobic barrier (Immedge, Vector Laboratories) was drawn around the perimeter of the slides and allowed to dry for 30 min at room temperature. Slides were washed with 1× PBS (3×5 min), fixed for 10 min in 4% PFA, washed with 1× PBS (2×5 min), incubated in proteinase K solution (10 min), washed with 1× PBS (10 min), fixed with 4% PFA (20 min), washed with 1× PBS (2×5 min) and incubated in 0.1 mol l−1 triethanolamine-HCl (pH 8.0) with 0.25% acetic anhydride for 10 min. Slides were then washed with 1× PBS again (5 min) and incubated in pre-warmed hybridization buffer for 3 h in a humidified chamber at 60–65°C. Next, the hybridization buffer was replaced with fresh hybridization buffer containing the DIG-labeled cfos probe, covered with hybrislips (Life Technologies), and incubated overnight (12–16 h) at 60–65°C in a sealed, humidified chamber. Hybrislips were carefully removed by immersing slides in prewarmed 2× sodium citrate chloride (SSC):50% formamide solution and slides were sequentially washed in the following solutions at 60–65°C: 2× SSC:50% formamide (2×30 min), 1:1 mixture of 2× SSC:maleate buffer (MABT) (2×15 min), and MABT (2×10 min). Next, slides were washed with MABT (2×10 min) at room temperature and incubated in blocking solution containing MABT with 2% bovine serum albumin to prevent non-specific binding of the anti-DIG antibody. Slides were incubated in alkaline phosphatase (AP)-conjugated anti-DIG Fab fragments (Sigma-Aldrich, St Louis, MO, USA) in blocking solution at a 1:5000 concentration overnight at 4°C in a covered humidified chamber. Slides were washed with MABT (3×30 min) at room temperature, incubated with AP buffer (2×5 min) at room temperature, and reacted with nitro-blue tetrazolium/5-bromo-4-chloro-3′-indolyphosphate (NBT/BCIP) (Sigma-Aldrich) solution in a sealed humidified chamber in the dark at 37°C for 4–5 h. To stop the reaction, slides were washed in 1× PBS (3×5 min), fixed in 4% PFA for 10 min, and washed in 1× PBS (3×5 min) again. Any remaining hydrophobic barrier was removed, slides were coverslipped with aqueous mounting media (Aqua-mount, Fisher Scientific, Waltham, MA, USA), and edges were sealed with clear nail polish after drying overnight.

Quantification of neural activation

To quantify differences in cfos expression across sensory exposures in subject females, we visualized stained slides on a Nikon Eclipse Ni microscope and photographs were taken with a color digital camera (Nikon DSFi2) controlled by Nikon NIS-Elements software. Individuals blind to the experimental condition performed quantifications. A Cresyl Violet-stained reference A. burtoni brain atlas and other relevant neuroanatomical resources were used to identify brain regions of interest (Fernald and Shelton, 1985; Maruska et al., 2017; Munchrath and Hofmann, 2010). We quantified social decision regions of the SDMN and midbrain areas involved in visual and auditory processing. Quantified regions of the SDMN include the lateral nucleus of the dorsal telencephalon, the granular region (Dl-g), the ventral part of the ventral telencephalon (Vv; divided into rostral, Vv-r, and caudal, Vv-c regions), the dorsal part of the ventral telencephalon (Vd), the supracommissural nucleus of the ventral telencephalon (Vs), the central part of the ventral telencephalon (Vc), the anterior part of the periventricular preoptic nucleus (nPPa), the periventricular nucleus of the posterior tuberculum (TPp), and the anterior tuberal nucleus (ATn). We also quantified activation in a midbrain auditory processing region, the torus semicircularis (TS), and a midbrain visual processing region, the torus longitudinalis (TL). For each region, a square grid was overlaid on the section and used to count the number of cfos-positive cells in 4–5 randomly selected boxes. A 30×30 μm grid and 20× magnification was used for the Vv-r; a 50×50 μm grid and 20× magnification for the Vv-c, Vs, nPPa, TPp and ATn; and a 100×100 μm grid and 10× magnification was used for Dl-g. Selected magnification for each region optimized the full view of the region of interest. To determine average cell density (no. cells μm−2) for each region, the total number of positive cells was divided by the cumulative area of the quantified regions. This process was repeated for four consecutive sections as a representative quantification of each region within each animal. Values were then averaged to obtain a mean cfos-expressing cell density for each region within each individual.

Statistical analysis

All statistical analyses were performed in R (3.6.2) and IBM SPSS 27. Differences in male courtship behaviors and female affiliative behaviors across different sensory stimulus conditions were compared using one-way ANOVA. All post hoc analyses were performed with Tukey's tests. A linear mixed model was used to compare the interaction of condition (control, auditory, visual or audio-visual) and brain region on cfos expression in R (package: lme4; Bates et al., 2015). Brain region was used as a within-subject factor, condition as a between-subject factor, body mass as a covariate and animal ID as the random effect. Body mass was used as a covariate in the model because there was slight variation in female body mass across conditions and brain size also varies with fish body mass.

We also used a discriminant function analysis (DFA; in SPSS) on cfos cell densities in all quantified brain regions to examine whether females could be classified into their respective sensory conditions based on neural activation patterns alone. Principal components analysis (PCA; in SPSS) was also used to examine for distinct neural networks among all 11 quantified brain regions and sensory conditions. Group means replaced missing values for each analysis. We used Pearson correlations to examine correlations between male and female time spent in the association zone and to create heat maps of co-activation networks across brain regions for each sensory condition. Clustering analysis was performed using the pvclust package in R. For all analyses, we chose not to correct P-values for multiple tests because these tests can lead to misleading biological interpretations as a result of increases in Type II errors and reduced statistical power (Nakagawa, 2004).

Behavior

A total of 65 subject gravid females were used to conduct behavioral trials. Of these females, 9 had ovulated, with 1 female releasing eggs during the trial. There was no difference in condition factor K, SL or GSI across conditions for experimental females (K: F3,61=2.354, P=0.081; SL: F3,61=2.271, P=0.0893; GSI: F3,61=1.285, P=0.288), but BM of females used in visual-only trials was slightly lower than that of females in other sensory conditions (BM: F3,61=3.039; P=0.036).

Because our goal was to present different male-generated visual–acoustic stimuli to females, we first compared the behavior of dominant males across the three different male sensory conditions (visual, auditory, audio-visual). There was no difference in the total number of quivers performed by dominant males across sensory conditions, verifying that females received similar courtship effort from males in each condition and were only exposed to different types of male-generated courtship sensory information depending on barrier type (F2,45=0.109, P=0.897; Fig. 3).

Fig. 3.

Male courtship effort is similar across experimental conditions. During courtship, males perform body quivers which provide a visual and auditory signal to receptive gravid females. In each condition (auditory, visual, audio-visual), females receive equivalent courtship quivers from males (number of quivers performed by dominant male per 30 min trial), regardless of barrier type. The lower and upper limits of the box represent the 25th and 75th percentile, respectively, and whiskers extend to the smallest and largest values 1.5× the interquartile range. Individual data points are plotted as filled circles, data mean as a cross and median as a line. Different lowercase letters represent significant differences (P<0.05). Sample sizes: N=17 auditory; N=16 visual; N=15 audio-visual.

Fig. 3.

Male courtship effort is similar across experimental conditions. During courtship, males perform body quivers which provide a visual and auditory signal to receptive gravid females. In each condition (auditory, visual, audio-visual), females receive equivalent courtship quivers from males (number of quivers performed by dominant male per 30 min trial), regardless of barrier type. The lower and upper limits of the box represent the 25th and 75th percentile, respectively, and whiskers extend to the smallest and largest values 1.5× the interquartile range. Individual data points are plotted as filled circles, data mean as a cross and median as a line. Different lowercase letters represent significant differences (P<0.05). Sample sizes: N=17 auditory; N=16 visual; N=15 audio-visual.

To examine how females change their behavior at the start of a trial, we compared the percentage change in time spent by the female at the barrier between the pre-stimulus period and the sensory exposure stimulus period (Fig. 4). This analysis showed that it differed across sensory conditions (one-way ANOVA, F3,61=74.19, P<0.001), with the greatest increase in the visual-only trials (pre-stimulus time at the barrier: control 1.71±1.39 min; auditory 0.25±0.59 min; visual 0.61±0.82 min; audio-visual 0.29±0.57 min). While there was little to no change in the female's behavior in the auditory-only condition, females in all other conditions that contained a visual stimulus showed an increase in their time at the barrier once the trial began.

Fig. 4.

Females show an increase in affiliative behavior after barrier removal at the start of the trial when visual stimuli are present. The percentage change in female time spent at the barrier from the 10 min pre-stimulus period to the stimulus period is low in the auditory-only trials, but high in conditions where they receive visual signals, particularly from males (visual and audio-visual) but also from other females (control). See Fig. 3 for box plot descriptions. Sample sizes: N=17 control; N=17 auditory; N=16 visual; N=15 audio-visual).

Fig. 4.

Females show an increase in affiliative behavior after barrier removal at the start of the trial when visual stimuli are present. The percentage change in female time spent at the barrier from the 10 min pre-stimulus period to the stimulus period is low in the auditory-only trials, but high in conditions where they receive visual signals, particularly from males (visual and audio-visual) but also from other females (control). See Fig. 3 for box plot descriptions. Sample sizes: N=17 control; N=17 auditory; N=16 visual; N=15 audio-visual).

During sensory stimulus exposures, gravid females spent less time in the association zone (one-way ANOVA, F3,61=73.79, P<0.001) and at the barrier (one-way ANOVA, F3,61=106, P <0.001) during auditory-only trials compared with audio-visual, visual or control contexts (Fig. 5). These different behavioral responses between the two unimodal reproductive conditions demonstrate that visual and auditory courtship signals from males provide non-redundant information to the gravid female receiver. However, because the female responses in visual-only and audio-visual trials are similar, the visual information dominates in this reproductive context and this can be classified as the ‘dominance’ type of non-redundant communication.

Fig. 5.

Female affiliation differs depending on received sensory signals. Females spend less time in the association zone (A) and at the barrier (B) when auditory-only signals from males are transmitted through the barrier. Females show similar affiliation when receiving visual courtship signals from males (with or without sounds) and from non-reproductive female-only controls. See Fig. 3 for box plot descriptions. Sample sizes: N=17 control; N=17 auditory; N=16 visual; N=15 audio-visual.

Fig. 5.

Female affiliation differs depending on received sensory signals. Females spend less time in the association zone (A) and at the barrier (B) when auditory-only signals from males are transmitted through the barrier. Females show similar affiliation when receiving visual courtship signals from males (with or without sounds) and from non-reproductive female-only controls. See Fig. 3 for box plot descriptions. Sample sizes: N=17 control; N=17 auditory; N=16 visual; N=15 audio-visual.

To further explore the timing of behaviors from both the signaling male and the subject receiver female in different sensory conditions, we used raster plots to depict female affiliation (time in association zone, time at barrier) and male courtship quivers, as well as entries into the shared pot by both the male and female (Fig. 6A). This analysis revealed several important points. First, it showed that gravid females associate with the barrier specifically when dominant males perform quivers on the other side during visual and audio-visual trials. While the time males and females spent in the association zone was positively correlated in all conditions, it was only significant in the audio-visual context (R=0.553, P=0.033) (Fig. 6B). Second, females spent significant time in the association zone and at the barrier throughout the entire trial time in all conditions that contained a visual stimulus, regardless of whether it was a reproductive or non-reproductive (control) context. Similarly, subject females entered the shared pot more often in all conditions that contained a visual stimulus (visual, audio-visual, control) compared with the auditory-only context (one-way ANOVA, F3,61=73.79, P=0.002).

Fig. 6.

Courting males produce courtship quivers when females are in the association zone and at the barrier. (A) Raster plots of representative temporal sequences of each sensory context show that males perform courtship behaviors when females are in the association zone. During visual and audio-visual conditions, males specifically perform body quivers when females are in the association zone and at the barrier. Males and females were also often in the shared pot shelter at the same time. During auditory-only trials, females show fewer affiliative behaviors. Blue and green bars indicate the duration of time spent in the association zone and at the barrier, respectively. Red triangles and yellow diamonds indicate the single events of entries into the shared pot shelter and male body quivers, respectively. (B) Female and male time in the association zone is positively correlated, but only statistically significant in the audio-visual (purple) context (Pearson correlations).

Fig. 6.

Courting males produce courtship quivers when females are in the association zone and at the barrier. (A) Raster plots of representative temporal sequences of each sensory context show that males perform courtship behaviors when females are in the association zone. During visual and audio-visual conditions, males specifically perform body quivers when females are in the association zone and at the barrier. Males and females were also often in the shared pot shelter at the same time. During auditory-only trials, females show fewer affiliative behaviors. Blue and green bars indicate the duration of time spent in the association zone and at the barrier, respectively. Red triangles and yellow diamonds indicate the single events of entries into the shared pot shelter and male body quivers, respectively. (B) Female and male time in the association zone is positively correlated, but only statistically significant in the audio-visual (purple) context (Pearson correlations).

Neural activation patterns

The density of cfos-expressing cells differed among female sensory conditions (control, auditory, visual and audio-visual) and brain regions (condition: F3,24=8.445, P<0.001; region: F10,240=28.270, P<0.001; condition×region: F30,240=2.730, P<0.001). Three regions, the nPPa, ATn and Vv-r, showed differential cfos expression among different sensory contexts. In the nPPa, females exposed to multimodal auditory and visual signals had greater cfos expression compared with females exposed to unimodal auditory signals alone (T186=4.832, P<0.001) and similar expression levels to females from control and visual-only sensory contexts (Fig. 7A). Similarly, in the ATn, females from audio-visual conditions showed greater cfos expression when compared with females from control and auditory-only sensory contexts (audio-visual/auditory: T174=2.982, P=0.017; audio-visual/control: T186=2.731, P=0.035) and similar levels to females exposed to visual signals alone (Fig. 7B). In the Vv-r, females exposed to visual and audio-visual signals displayed the highest levels of cfos expression, followed by controls with intermediate levels, and females exposed to auditory-only signals with the lowest cfos expression (audio-visual/auditory: T174=7.976, P<0.001; audio-visual/control: T186=4.071, P<0.001; control/auditory: T185=3.88, P<0.001; control/visual: T189=7.17 P<0.006; visual/auditory: T189=7.17, P<0.001) (Fig. 7C) (see Table S1 for post hoc values). There were no differences among sensory treatments in the Vv-c, Vd, Vs, Vc, TPp, TS or TL (P>0.05).

Fig. 7.

Females receiving different sensory information show different neural activation patterns in some nuclei of the social decision-making network (SDMN). Females exposed to combined audio-visual courtship signals show greater cfos expression in the nPPa (A) and the ATn (B) when compared with females exposed to auditory signals alone. (C) In the Vv-r, females exposed to visual and audio-visual signals show the highest activation when compared with control and auditory-only conditions, which have intermediate and the lowest activation levels, respectively. Representative cross-sections showing cfos (purple) labeling from females in each of the four conditions are shown on the right. Scale bars: 100 µm. See Fig. 3 for box plot descriptions. Sample sizes: N=7 control; N=8 auditory; N=6 visual; N=7 audio-visual. See Materials and Methods for SDMN region definitions.

Fig. 7.

Females receiving different sensory information show different neural activation patterns in some nuclei of the social decision-making network (SDMN). Females exposed to combined audio-visual courtship signals show greater cfos expression in the nPPa (A) and the ATn (B) when compared with females exposed to auditory signals alone. (C) In the Vv-r, females exposed to visual and audio-visual signals show the highest activation when compared with control and auditory-only conditions, which have intermediate and the lowest activation levels, respectively. Representative cross-sections showing cfos (purple) labeling from females in each of the four conditions are shown on the right. Scale bars: 100 µm. See Fig. 3 for box plot descriptions. Sample sizes: N=7 control; N=8 auditory; N=6 visual; N=7 audio-visual. See Materials and Methods for SDMN region definitions.

To further examine how neural activation differed across sensory contexts, we performed a DFA with cfos densities in all quantified brain regions (Fig. 8A). DFA determines whether animals can be separated into groups based solely on their neural activation patterns across the brain and identifies which variables or brain regions contribute to the proper classification of individuals. The DFA correctly classified 100% of females from auditory-only and visual-only contexts, while one female from each of the control and audio-visual sensory conditions was misclassified into the opposite group. The DFA produced one significant function (Eigenvalue=98.82, χ2=57.536, P=0.005), explaining 70.8% of the variation. Function 1 clearly separates females from audio-visual and auditory conditions and is positively loaded by neural activation in the Dl-g, Vs, nPPa, Vd, TS and Vv-c. Although not reaching significance, function 2 (Eigenvalue=3.012, χ2=18.579, P=0.549) separates females from non-reproductive control conditions from all of the reproductive sensory conditions where males were present, suggesting that courtship signals received from dominant males result in neural activation patterns different from signals received in a non-reproductive context. To examine how neural activation might be associated with female affiliation and courtship behavior from dominant males, we performed Pearson correlations between discriminant function coefficients and behavior for each sensory condition (Fig. 8B) (see Table S2 for Pearson correlation coefficients and P-values). Function 1 did not correlate with any behaviors while function 2 negatively correlated with female time in the association zone (R=−0.76, P=0.047) and female time at the barrier (R=−0.76, P=0.049) only in the audio-visual condition. Female affiliative behaviors did not correlate with neural activation in any individual brain region, suggesting that the observed neural activation patterns determined with cfos are more reflective of incoming sensory information rather than female behavioral output.

Fig. 8.

Females are categorized into their sensory exposure groups based on neural activation patterns alone. (A) Discriminant function analysis (DFA) on cfos-stained cell densities across the 11 analyzed brain regions correctly classified 100% of females from auditory-only and visual-only conditions, and greater than 90% of females from control and audio-visual contexts. Function 1 (primarily loaded by the Vv-c, Vd, Vs, Dl-g, nPPa and TS) separates females in the auditory-only condition from those in the audio-visual condition, showing clear distinction between these two groups, with females from the visual-only and control condition distributed between them along this function. Although not significant, function 2 (primarily loaded by the ATn) separates females from the non-reproductive control condition from all the reproductive sensory conditions. Circles indicate individual females and stars represent group centroids for each condition. (B) Pearson correlation coefficients of DFA functions with behaviors. Function 1 does not correlate with any male or female behaviors in any condition. Function 2 negatively correlates with female time in the association zone (AZ) and female time spent at the barrier only in the audio-visual condition. Asterisks indicate significant correlations (P<0.05).

Fig. 8.

Females are categorized into their sensory exposure groups based on neural activation patterns alone. (A) Discriminant function analysis (DFA) on cfos-stained cell densities across the 11 analyzed brain regions correctly classified 100% of females from auditory-only and visual-only conditions, and greater than 90% of females from control and audio-visual contexts. Function 1 (primarily loaded by the Vv-c, Vd, Vs, Dl-g, nPPa and TS) separates females in the auditory-only condition from those in the audio-visual condition, showing clear distinction between these two groups, with females from the visual-only and control condition distributed between them along this function. Although not significant, function 2 (primarily loaded by the ATn) separates females from the non-reproductive control condition from all the reproductive sensory conditions. Circles indicate individual females and stars represent group centroids for each condition. (B) Pearson correlation coefficients of DFA functions with behaviors. Function 1 does not correlate with any male or female behaviors in any condition. Function 2 negatively correlates with female time in the association zone (AZ) and female time spent at the barrier only in the audio-visual condition. Asterisks indicate significant correlations (P<0.05).

Because SDMN brain regions are interconnected, examining co-activation networks may better reflect complex sensory processing in behavioral contexts than single brain nuclei comparisons. To examine the connectivity of each brain region in different sensory contexts, we generated heat maps using Pearson correlations of cfos cell densities (Fig. 9). Overall, we found that patterns of activation across brain regions differed for each sensory condition. For example, the control, visual and audio-visual context had two significant clusters while the auditory-only condition had three and there was little overlap among significantly correlated regions and clusters across different sensory conditions. This emphasizes the complexity of perceiving unimodal and multimodal courtship signals in A. burtoni females. Tables S3–S6 show Pearson correlation coefficients and P-values for control, auditory, visual and audio-visual conditions, respectively.

Fig. 9.

Females exposed to unimodal or multimodal courtship signals have different co-activation patterns in socially relevant brain regions. Pearson correlation coefficients were used to create heat maps (r indicated by color scale) across brain regions for females from control (A), auditory-only (B), visual-only (C) and audio-visual (D) conditions. The axes of each condition are organized to highlight significant clusters identified by hierarchical clustering analysis. Each heatmap reveals a unique functional network of co-activation, which is absent in other sensory conditions. Outlined boxes represent significant clusters and asterisks indicate significant correlations (P<0.05). See Materials and Methods for SDMN and midbrain visual processing region definitions.

Fig. 9.

Females exposed to unimodal or multimodal courtship signals have different co-activation patterns in socially relevant brain regions. Pearson correlation coefficients were used to create heat maps (r indicated by color scale) across brain regions for females from control (A), auditory-only (B), visual-only (C) and audio-visual (D) conditions. The axes of each condition are organized to highlight significant clusters identified by hierarchical clustering analysis. Each heatmap reveals a unique functional network of co-activation, which is absent in other sensory conditions. Outlined boxes represent significant clusters and asterisks indicate significant correlations (P<0.05). See Materials and Methods for SDMN and midbrain visual processing region definitions.

To identify neural activation networks across all brain regions analyzed for females from all conditions, we performed a PCA which identified three distinct networks (Kaiser–Meyer–Olkin measure of sampling adequacy=0.613, χ2=128.97, P<0.001) (Fig. 10). The first includes nPPa, ATn, Vd, Dl-g and Vc (representing 48.26% of variance) and may represent regions that process multimodal audio-visual information because most of these regions have high cfos expression in audio-visual conditions. The second includes Vv-c, Vv-r, Vs and TPp (representing 20.28% of variance) and likely represents regions that process unimodal acoustic or visual signals, because these regions have high activation in visual or audio-visual conditions and low activation in auditory-only conditions. The third network includes TS and TL (representing 9.41% of variance) and represents regions characterized by high activation in control and visual or audio-visual conditions so may be involved in processing sensory information in any social context.

Fig. 10.

Principal component analysis (PCA) reveals three distinct co-activation neural networks. The PCA used data for cfos cell densities across all brain nuclei and conditions to estimate regions that correlate. The first network, composed of the nPPa, ATn, Vd, Dl-g and Vc, represents 48.26% of the variation, the second network includes the Vv-c, TPp, Vv-r and Vs and accounts for 20.28% of the variation and the final network contains the TL and TS, accounting for 9.41% of the variance. Component plots in varimax rotated space for components 1 and 2 are shown. See Materials and Methods for SDMN and midbrain visual processing region definitions.

Fig. 10.

Principal component analysis (PCA) reveals three distinct co-activation neural networks. The PCA used data for cfos cell densities across all brain nuclei and conditions to estimate regions that correlate. The first network, composed of the nPPa, ATn, Vd, Dl-g and Vc, represents 48.26% of the variation, the second network includes the Vv-c, TPp, Vv-r and Vs and accounts for 20.28% of the variation and the final network contains the TL and TS, accounting for 9.41% of the variance. Component plots in varimax rotated space for components 1 and 2 are shown. See Materials and Methods for SDMN and midbrain visual processing region definitions.

We examined the behavior of A. burtoni females to test whether auditory and visual courtship signals from dominant males provide redundant or non-redundant signals to females. Females showed different affiliative behaviors in each unimodal sensory condition, demonstrating that auditory and visual signals from dominant males provide non-redundant information to receptive gravid females. Because female affiliation behaviors were similar between the unimodal visual condition and the multimodal audio-visual condition, the visual signals dominate over the auditory signals and signaling can be classified as the ‘dominance’ type of non-redundant communication. We also identified differential neural activation patterns in the brain using in situ hybridization for cfos within the SDMN and midbrain sensory processing regions. These distinct co-activation networks across socially relevant brain regions in females exposed to different sensory conditions highlight the complex neural computations utilized by females to make mating decisions based on different sensory signals received from courting males.

Behavior

Dominant A. burtoni males use non-redundant multimodal audio-visual signaling to court receptive gravid females, with the visual component dominating the auditory. Males produce elaborate courtship displays consisting of intensifications of body coloration (yellow/blue body coloration, black vertical eye bar and orange spots along the anal, caudal and dorsal fins) and increased body quivers performed in association with low frequency sounds. While most studies focus on the production of multimodal signals, our study highlights how signals are interpreted by the receiver. Females showed increased affiliative behaviors when they received visual sensory information, demonstrating the importance of the visual component(s) of the courtship display. As females approach ovulation, the sensitivity of their visual and auditory systems improves (Butler et al., 2019; Maruska et al., 2012) and they are behaviorally more receptive to male courtship (Kidd et al., 2013b). Females have greater auditory sensitivity in the spectral range of male courtship sounds and show a preference for sound-producing males, indicating that the acoustic component plays a role in female mate choice (Maruska et al., 2012). Females from auditory-only conditions showed no response to male courtship sounds alone, demonstrating that auditory signals alone are insufficient to motivate females for reproduction. According to the multimodal communication classification scheme proposed by Partan and Marler (2005), recipient responses from individual components of a multimodal signal can be a physical behavioral response, internal physiological response or no response at all. In A. burtoni, the visual and auditory components elicit distinct responses in receptive females, either increased affiliation or no response, respectively. Thus, courting males may enhance their primary visual courtship displays by providing some additional information to females via sound production (e.g. size, fitness) that is not available from the visual display alone. Females may show reduced or no affiliation in the auditory-only context if vision is necessary for directional responses to auditory signals, especially as the auditory component is always accompanied by the visual component. Females may require a visual signal to identify the location of auditory signals or to perceive it as salient since auditory signals do not occur without visual displays. During audio-visual contexts only, male and female affiliation time was positively correlated, suggesting that both the visual and acoustic component are valuable for courtship in A. burtoni and when presented in combination provide new information to females. The visual component functions as the dominant signal and may provide information on location while sound may supply additional information (e.g. male size, quality) to the female when accompanied by the visual component to increase correlated behavior between males and females. Similar examples exist in crayfish courtship, where males show no response to female chemical signals but positively respond to multimodal chemical and visual stimuli (Acquistapace et al., 2002).

Several fish species use audio-visual signaling to convey information in courtship and reproductive contexts. However, in species with elaborate visual display components (coloration and movement), and as for our results in A. burtoni, sound production alone often fails to elicit female positive responses unless a visual signal is also present. Thus, multimodal visual–acoustic signaling likely provides non-redundant information in many fish species. For example, the painted goby produces visual courtship displays in conjunction with low frequency drum sounds and females in this species use the sound as an assessment of male quality (Amorim et al., 2013). Many Lake Malawi cichlids also use visual–acoustic signaling as part of their courtship repertoires and related sympatric species with similar coloration patterns have different sound characteristics, suggesting that acoustic signals may be important for both species identity and signaler phenotypic traits (Amorim et al., 2008; Smith and Staaden, 2009). In the damselfish Dascyllus albisella, male mating success is positively correlated with their courtship rate (combined visual–acoustic signaling) and females choose mates based on this condition-dependent trait (Oliver and Lobel, 2013). Similarly, in A. burtoni, peak sound frequency and the percentage of visual quivers accompanied by sounds is related to male body size, so females may receive honest information on male qualities such as size, dominance status, health and fitness that can be used in mating decisions.

Non-redundant multimodal signaling is also advantageous in situations where environmental conditions may mask one or more sensory channels, to ensure some transmission of information in reproductive contexts. Wild A. burtoni live in shallow shore pools and river systems of Lake Tanganyika, which are subject to disturbances (e.g. runoff, increased turbidity from large animal traffic) that could disrupt the transmittance of visual signals between males and females. Evolving alternative sensory pathways such as sound production to convey reproductive readiness and fitness qualities to receptive females would be advantageous in such environments to ensure species persistence. Several vertebrate taxa implement multimodal shifting from one sensory channel to another to ensure signal detection. For example, in several species of frogs, males initially produce mating call sounds to receptive females but shift to visual signals, such as foot flagging, when their environment becomes too noisy from streams or waterfalls (Hodl and Amezquita, 2001). Similarly, zebrafish (Danio rerio) shift perception from vision to olfaction after exposure to dim light for 6 weeks, displaying the ability to adapt perception to the sensory modality that best suits their environmental conditions (Suriyampola et al., 2020). Thus, non-redundant communication allows for behavioral signaling plasticity under changing environmental or social conditions to ensure transfer of information.

Females from the control conditions were exposed to non-reproductive signals from community females and displayed affiliative behaviors at levels similar to those for male-generated visual and audio-visual conditions. Astatotilapia burtoni are social animals, highly dependent on interactions, and therefore they typically increase affiliative behaviors in the presence of any other fish, regardless of sex. It is important to note that the behavioral response of subject females in this female-only non-reproductive control condition was not needed to determine the non-redundant classification of male audio-visual signaling in the reproductive context. Rather, this control was included to better distinguish neural activation patterns specifically related to the reception of male courtship signals compared with those resulting from any general social context. Exploring neural activation patterns in females from each sensory condition will assist in uncovering how reproductive versus non-reproductive visual signals are processed in A. burtoni females.

Context-dependent neural activation

While several studies focus on how courtship signals are produced by signalers, few focus on how unimodal and multimodal information is processed in the brain by receivers. Here, we used in situ hybridization for the immediate early gene cfos as a marker of neural activation in nuclei of the SDMN along with two midbrain sensory regions, the TL and TS, to understand how unimodal and multimodal visual and acoustic male courtship signals are processed in A. burtoni females. The SDMN, which regulates social behavior across vertebrates, is composed of the social behavior network and the mesolimbic reward system, and A. burtoni females displayed differential cfos activation in three nuclei of the SDMN, the Vv-r, nPPa and ATn. Across these regions, there was greater cfos expression in audio-visual contexts when compared with auditory alone, showing that auditory courtship information is processed differently by the female brain when visual information is absent. This pattern is consistent with the behavioral results where acoustic signals alone from males failed to elicit a positive female affiliation response. Thus, based on both behavior and neural activation patterns, auditory courtship signals alone are insufficient to provide the required context that leads to successful reproduction.

In the Vv-r, females in the visual and audio-visual context showed greater neural activation when compared with those in the control and auditory-only conditions. Previous work in A. burtoni showed that gravid females had greater cfos expression in the Vv-r when exposed to sexually relevant visual, auditory, mechanosensory (lateral line) and chemosensory stimuli from dominant males (Field and Maruska, 2017). By solely focusing here on visual and auditory signals, we further demonstrate the importance of the Vv-r in specifically processing visual courtship signals from dominant males. The Vv-r, homologous in part to the mammalian lateral septum, is a shared region in both the social behavior network and mesolimbic reward systems (O'Connell and Hofmann, 2011). In mammals, the lateral septum is heavily involved in integrating internal (age, sex, reproductive/social status, motivational state) and external information to produce appropriate behavioral responses (Menon et al., 2022). In rodents, sex steroid receptor expression in females further implicates the role of the lateral septum in modulating reproductive behaviors (Tobet and Baum, 1982). Female A. burtoni in the control context also displayed higher activation in the Vv-r when compared with females from auditory-only conditions. In mammals, the lateral septum plays a role in processing sensory information from general social interactions, although these pathways are not well understood (Menon et al., 2022). Together, our results show that the Vv-r may be involved in contextualizing sensory information by processing general visual social information (from control conditions) while being more heavily involved in processing visual information during reproductive contexts.

Females exposed to multimodal audio-visual courtship signals had greater activation in the ATn than those receiving unimodal auditory courtship signals and control females from the non-reproductive context. The discriminant function analysis revealed that function 2, primarily loaded by the ATn, was significantly correlated with female affiliative behaviors only in females from the audio-visual condition, further showing its importance in processing multimodal courtship signals. The ATn is homologous in part to the mammalian ventromedial hypothalamus (VMH), which contains progesterone receptors and is known to control female reproduction and sexual receptivity (Yang et al., 2013). The functional role of the ATn in fishes remains largely unknown but our results suggest it may play an important role in processing visual and acoustic multimodal courtship signals. In the midshipman fish, females displayed greater cfos expression in the ATn when exposed to conspecific male courtship calls compared with heterospecific calls or ambient noise (Mohr et al., 2018). Conversely, goldfish exposed to non-reproductive unimodal, bimodal and multimodal visual, acoustic and hydrodynamic stimuli showed activation in the ATn, supporting the idea that the ATn functions as a integrative center while contradicting the notion that it plays a bigger role in specifically processing reproductive sensory information (Kirsch et al., 2002). Although more studies are needed to elucidate the role of the ATn in social behavior and processing of sensory information in fishes, our results suggest that the ATn may work to integrate and discriminate between auditory and visual signals in reproductive and non-reproductive contexts.

Female A. burtoni exposed to multimodal audio-visual courtship signals had greater neural activation in the nPPa when compared with females from the auditory-only context, but showed similar activation in all conditions with a visual stimulus. This result suggests that the nPPa processes socially relevant visual information and may function as an integrative center to process sexually relevant visual and acoustic courtship signals in A. burtoni females. Further, the nPPa had greater cfos expression in A. burtoni females exposed to multimodal visual, chemosensory, acoustic and mechanosensory courtship signals from males compared with no-fish controls and similar sensory information from other females, showing it functions to integrate various types of sexually relevant sensory information (Field and Maruska, 2017). The nPPa is a subregion of the preoptic area that functions in modulating social behaviors across vertebrates including reproduction, parental care and aggression (Forlano and Bass, 2011). Neurons in the nPPa are heterogeneous and this region contains different neuron populations that express various types of neuromodulators, including gonadotropin-releasing hormone, arginine vasotocin, galanin and receptors for steroid hormones that collectively play a role in regulating behavioral output and neuroendocrine functions (Butler et al., 2021, 2020; Maruska and Tricas, 2011). Using co-activation networks, we show that cfos expression in the nPPa is heavily correlated with expression in other nuclei in the brain that is dependent on sensory condition. Because the nPPa expresses various neuromodulators that can impact the activity of different circuits and suites of neurons, this region likely functions as an integrative center to process different types of sensory information. Thus, it may help evaluate the salience of communication signals based on an individual's internal physiological state.

To assess differences in neural activation among different regions in the brain that process sensory information, we quantified cfos expression in two midbrain nuclei outside the SDMN, the TL and TS. The TL is strongly associated with the optic tectum, a primary visual center in non-mammalian vertebrates, and plays a large role in visual processing and directing visual attention (Folgueira et al., 2020). Conversely, the TS is an important auditory and lateral line processing region, but also receives visual and somatosensory inputs in fishes (Schellart, 1983; Schellart et al., 1987). The TS is also thought to help distinguish between conspecific and heterospecific acoustic calls in vertebrates. For example, female tungara frogs exposed to male-specific mating calls displayed greater neural activation in the TS than those exposed to heterospecific calls or ambient noise (Mangiamele and Burmeister, 2011). Our PCA demonstrated that the TS and TL form a neural network, showing that these regions may work together to process sensory information and relay it to behavioral decision circuits. Although we did not find differences in cfos expression across sensory conditions for the TL or TS, we revealed some differences in co-activation between the TL and TS and regions of the SDMN. For example, females exposed to audio-visual courtship stimuli had a significant positive correlation between activation of the TL and Dl-g, while activation of the TS and Vv-r was negatively correlated in females from the auditory-only context. The Dl-g is homologous in part to the mammalian hippocampus and is involved in spatial cognition, which is heavily dependent on processing spatial visual information (Broglio et al., 2003). Taken together, this suggests that the TL may be more involved in conditions where visual information is processed while the TS is more involved in auditory contexts when females only receive acoustic sensory information. However, in visual-only conditions, the TL and TS formed a significant cluster with the ATn, TPp and Vv-c that was not present in any other context, showing that more complex interactions occur between the TL and TS and regions of the SDMN to modulate behavioral output in different social contexts.

Conclusions

We show that dominant A. burtoni males use multimodal visual–acoustic courtship signals to provide non-redundant signals to receptive females. Females showed lower affiliation when exposed to auditory signals alone, demonstrating that the visual component(s) of male courtship signals dominates over the auditory signal and is necessary for positive female affiliative behaviors. However, several lines of evidence indicate that the sound production provided by males during the multimodal visual–acoustic display is important and used by females for mating decisions. (1) Peak sound frequency and the percentage of quivers with associated sounds is related to male traits (e.g. body size, quiver rate) providing potential honest signals for female choice. Thus, while the visual signal component is necessary for a directed response, sounds may provide information on male characteristics. (2) Female hearing improves (increased sensitivity) as she approaches ovulation, specifically in the frequency range of male courtship sounds, possibly allowing her to better discriminate acoustic information that conveys male traits. (3) Playback experiments show that females prefer to affiliate with males that have sounds coming from their territories compared with males with only visual signals (Maruska et al., 2012). We now add to this information by demonstrating that females process visual–acoustic unimodal and multimodal courtship stimuli differently in the brain, showing context-dependent activation in several regions of the SDMN and unique co-activation networks. This research expands our knowledge of how sensory information is processed by the receiver, especially in a species whose reproductive success is tightly linked to its ability to process information delivered via different sensory modalities.

We thank members of the Maruska lab, particularly Julie Butler, Cy Bryson, Adaoma Ngari and Austin Nguyen for experimental assistance, and Rose Wayne for artwork in Fig. 1.

Author contributions

Conceptualization: T.K., B.T., K.P.M.; Methodology: T.K., B.T., K.P.M.; Validation: T.K., E.J.R., B.T., K.P.M.; Formal analysis: T.K., E.J.R., K.P.M.; Investigation: T.K., E.J.R., B.T., K.P.M.; Resources: K.P.M.; Data curation: T.K., E.J.R., B.T., K.P.M.; Writing - original draft: T.K.; Writing - review & editing: T.K., E.J.R., K.P.M.; Visualization: T.K., E.J.R., K.P.M.; Supervision: T.K., K.P.M.; Project administration: K.P.M.; Funding acquisition: K.P.M.

Funding

Funding was provided by the National Science Foundation (IOS-1456004 and IOS-1456558 to K.P.M.). T.K. was supported by a Louisiana Board of Regents Fellowship and a National Science Foundation Graduate Research Fellowship (GR-00001414).

Acquistapace
,
P.
,
Aquiloni
,
L.
,
Hazlett
,
B. A.
and
Gherardi
,
F.
(
2002
).
Multimodal communication in crayfish: Sex recognition during mate search by male Austropotamobius pallipes
.
Can. J. Zool.
80
,
2041
-
2045
.
Amorim
,
M. C. P.
,
Simões
,
J. M.
,
Fonseca
,
P. J.
and
Turner
,
G. F.
(
2008
).
Species differences in courtship acoustic signals among five Lake Malawi cichlid species (Pseudotropheus spp
.
J. Fish Biol.
72
,
1355
-
1368
.
Amorim
,
M. C.
,
Pedroso
,
S. S.
,
Bolgan
,
M.
,
Jordao
,
J. M.
,
Caiano
,
M.
and
Fonseca
,
P. J.
(
2013
).
Painted gobies sing their quality out loud: acoustic rather than visual signals advertise male quality and contribute to mating success
.
Funct. Ecol.
27
,
289
-
298
.
Bates
,
D.
,
Mächler
,
M.
,
Bolker
,
B.
and
Walker
,
S.
(
2015
).
Fitting linear mixed-effects models using lme4
.
J. Stat. Softw
67
,
1
-
48
.
Broglio
,
C.
,
Rodríguez
,
F.
and
Salas
,
C.
(
2003
).
Spatial cognition and its neural basis in teleost fishes
.
Fish Fish.
4
,
247
-
255
.
Butler
,
J. M.
and
Maruska
,
K. P.
(
2016a
).
The Mechanosensory lateral line system mediates activation of socially-relevant brain regions during territorial interactions
.
Front. Behav. Neurosci.
10
,
93
.
Butler
,
J. M.
and
Maruska
,
K. P.
(
2016b
).
Mechanosensory signaling as a potential mode of communication during social interactions in fishes
.
J. Exp. Biol.
219
,
2781
-
2789
.
Butler
,
J. M.
,
Whitlow
,
S. M.
,
Rogers
,
L. S.
,
Putland
,
R. L.
,
Mensinger
,
A. F.
and
Maruska
,
K. P.
(
2019
).
Reproductive state-dependent plasticity in the visual system of an African cichlid fish
.
Horm. Behav.
114
,
104539
.
Butler
,
J. M.
,
Herath
,
E. M.
,
Rimal
,
A.
,
Whitlow
,
S. M.
and
Maruska
,
K. P.
(
2020
).
Galanin neuron activation in feeding, parental care, and infanticide in a mouthbrooding African cichlid fish
.
Horm. Behav.
126
,
104870
.
Butler
,
J. M.
,
Anselmo
,
C. M.
and
Maruska
,
K. P.
(
2021
).
Female reproductive state is associated with changes in distinct arginine vasotocin cell types in the preoptic area of Astatotilapia burtoni
.
J. Comp. Neurol.
529
,
987
-
1003
.
Candolin
,
U.
(
2019
).
Mate choice in a changing world
.
Biol. Rev. Camb. Philos. Soc.
94
,
1246
-
1260
.
Fernald
,
R. D.
and
Hirata
,
N. R.
(
1977
).
Field study of Haplochromis burtoni: quantitative behavioural observations
.
Anim. Behav.
25
,
964
-
975
.
Fernald
,
R. D.
and
Shelton
,
L. C.
(
1985
).
The organization of the diencephalon and the pretectum in the cichlid fish, Haplochromis burtoni
.
J. Comp. Neurol.
238
,
202
-
217
.
Field
,
K. E.
and
Maruska
,
K. P.
(
2017
).
Context-dependent chemosensory signaling, aggression and neural activation patterns in gravid female African cichlid fish
.
J. Exp. Biol.
220
,
4689
-
4702
.
Folgueira
,
M.
,
Riva-Mendoza
,
S.
,
Ferreño-Galman
,
N.
,
Castro
,
A.
,
Bianco
,
I. H.
,
Anadon
,
R.
and
Yanez
,
J.
(
2020
).
Anatomy and connectivity of the torus longitudinalis of the adult Zebrafish
.
Front. Neural Circuits
14
,
8
.
Forlano
,
P. M.
and
Bass
,
A. H.
(
2011
).
Neural and hormonal mechanisms of reproductive-related arousal in fishes
.
Horm. Behav.
59
,
616
-
629
.
Friard
,
O.
and
Gamba
,
M.
(
2016
).
BORIS: a free, versatile open-source event-logging software for video/audio coding and live observations
.
Method. Ecol. Evol.
7
,
1325
-
1330
.
Goodson
,
J. L.
and
Kingsbury
,
M. A.
(
2013
).
What's in a name? Considerations of homologies and nomenclature for vertebrate social behavior networks
.
Horm. Behav.
64
,
103
-
112
.
Günther
,
A. C. L. G.
(
1893
).
Descriptions of the reptiles and fishes collected by Mr E. Coode-Hore on Lake Tanganyika
.
Proc. Zool. Soc. Lond.
1893
,
628
-
632
.
Halfwerk
,
W.
and
Slabbekoorn
,
H.
(
2015
).
Pollution going multimodal: the complex impact of the human-altered sensory environment on animal perception and performance
.
Biol. Lett.
11
,
20141051
.
Hodl
,
W.
and
Amezquita
,
A
. (
2001
).
Visual Signalling in Anuran Amphibians. Anuran Communication Washington
(ed.
M. J.
Ryan
), pp.
121
-
141
.
Smithsonian Institution Press
.
Johnstone
,
R. A.
(
1996
).
Multiple displays in animal communication: “backup signals” and “multiple messages
.
Philos. Trans. R. Soc. Lond. B
351
,
329
-
338
.
Juntti
,
S. A.
,
Hilliard
,
A. T.
,
Kent
,
K. R.
,
Kumar
,
A.
,
Nguyen
,
A.
,
Jimenez
,
M. A.
,
Loveland
,
J. L.
,
Mourrain
,
P.
and
Fernald
,
R. D.
(
2016
).
A neural basis for control of cichlid female reproductive behavior by prostaglandin F2alpha
.
Curr. Biol.
26
,
943
-
949
.
Kidd
,
M. R.
,
Dijkstra
,
P. D.
,
Alcott
,
C.
,
Lavee
,
D.
,
Ma
,
J.
,
O'Connell
,
L. A.
and
Hofmann
,
H. A.
(
2013a
).
Prostaglandin F2α facilitates female mating behavior based on male performance
.
Behav. Ecol. Sociobiol.
67
,
1307
-
1315
.
Kidd
,
M. R.
,
O'Connell
,
L. A.
,
Kidd
,
C. E.
,
Chen
,
C. W.
,
Fontenot
,
M. R.
,
Williams
,
S. J.
and
Hofmann
,
H. A.
(
2013b
).
Female preference for males depends on reproductive physiology in the African cichlid fish Astatotilapia burtoni
.
Gen. Comp. Endocrinol.
180
,
56
-
63
.
Kirsch
,
J. A.
,
Hofmann
,
M. H.
,
Mogdans
,
J.
and
Bleckmann
,
H.
(
2002
).
Response properties of diencephalic neurons to visual, acoustic and hydrodynamic stimulation in the goldfish, Carassius auratus
.
Zoology
105
,
61
-
70
.
Mangiamele
,
L. A.
and
Burmeister
,
S. S.
(
2011
).
Auditory selectivity for acoustic features that confer species recognition in the tungara frog
.
J. Exp. Biol.
214
,
2911
-
2918
.
Maruska
,
K. P.
and
Fernald
,
R. D.
(
2012
).
Contextual chemosensory urine signaling in an African cichlid fish
.
J. Exp. Biol.
215
,
68
-
74
.
Maruska
,
K. P.
and
Tricas
,
T. C.
(
2011
).
Gonadotropin-releasing hormone (GnRH) modulates auditory processing in the fish brain
.
Horm. Behav.
59
,
451
-
464
.
Maruska
,
K. P.
,
Ung
,
U. S.
and
Fernald
,
R. D.
(
2012
).
The African cichlid fish Astatotilapia burtoni uses acoustic communication for reproduction: sound production, hearing, and behavioral significance
.
PLoS One
7
,
e37612
.
Maruska
,
K. P.
,
Butler
,
J. M.
,
Field
,
K. E.
and
Porter
,
D. T.
(
2017
).
Localization of glutamatergic, GABAergic, and cholinergic neurons in the brain of the African cichlid fish, Astatotilapia burtoni
.
J. Comp. Neurol.
525
,
610
-
638
.
Menon
,
R.
,
Suss
,
T.
,
Oliveira
,
V. E. M.
,
Neumann
,
I. D.
and
Bludau
,
A.
(
2022
).
Neurobiology of the lateral septum: regulation of social behavior
.
Trends Neurosci.
45
,
27
-
40
.
Mohr
,
R. A.
,
Chang
,
Y.
,
Bhandiwad
,
A. A.
,
Forlano
,
P. M.
and
Sisneros
,
J. A.
(
2018
).
Brain activation patterns in response to conspecific and heterospecific social acoustic signals in female plainfin midshipman fish, porichthys notatus
.
Brain Behav. Evol.
91
,
31
-
44
.
Munchrath
,
L. A.
and
Hofmann
,
H. A.
(
2010
).
Distribution of sex steroid hormone receptors in the brain of an African cichlid fish, Astatotilapia burtoni
.
J. Comp. Neurol.
518
,
3302
-
3326
.
Nakagawa
,
S.
(
2004
).
A farewell to Bonferroni: the problems of low statistical power and publication bias
.
Behav. Ecol.
15
,
1044
-
1045
.
Newman
,
S. W.
(
1999
).
The medial extended amygdala in male reproductive behavior. A node in the mammalian social behavior network
.
Ann. N. Y. Acad. Sci.
877
,
242
-
257
.
O'Connell
,
L. A.
and
Hofmann
,
H. A.
(
2011
).
The vertebrate mesolimbic reward system and social behavior network: a comparative synthesis
.
J. Comp. Neurol.
519
,
3599
-
3639
.
O'Connell
,
L. A.
and
Hofmann
,
H. A.
(
2012
).
Evolution of a vertebrate social decision-making network
.
Science
336
,
1154
-
1157
.
Oliver
,
S. J.
and
Lobel
,
P. S.
(
2013
).
Direct mate choice for simultaneous acoustic and visual courtship displays in the damselfish, Dascyllus albisella (Pomacentridae)
.
Environ. Biol. Fishes
96
,
447
-
457
.
Partan
,
S. R.
(
2017
).
Multimodal shifts in noise: switching channels to communicate through rapid environmental change
.
Anim. Behav.
124
,
325
-
337
.
Partan
,
S.
and
Marler
,
P.
(
1999
).
Communication goes multimodal
.
Science
283
,
1272
-
1273
.
Partan
,
S. R.
and
Marler
,
P.
(
2005
).
Issues in the classification of multimodal communication signals
.
Am. Nat.
166
,
231
-
245
.
Schellart
,
N. A.
(
1983
).
Acousticolateral and visual processing and their interaction in the torus semicircularis of the trout, Salmo gairdneri
.
Neurosci. Lett.
42
,
39
-
44
.
Schellart
,
N. A. M.
,
Kamermans
,
M.
and
Nederstigt
,
L. J. A.
(
1987
).
An electrophysiological study of the topographical organization of the multisensory torus semicircularis of the rainbow trout
.
Comp. Biochem. Physiol. A Physiol.
88
,
461
-
469
.
Shannon
,
G.
,
McKenna
,
M. F.
,
Angeloni
,
L. M.
,
Crooks
,
K. R.
,
Fristrup
,
K. M.
,
Brown
,
E.
,
Warner
,
K. A.
,
Nelson
,
M. D.
,
White
,
C.
,
Briggs
,
J.
et al. 
(
2016
).
A synthesis of two decades of research documenting the effects of noise on wildlife
.
Biol. Rev. Camb. Philos. Soc.
91
,
982
-
1005
.
Slabbekoorn
,
H.
,
Bouton
,
N.
,
van Opzeeland
,
I.
,
Coers
,
A.
,
ten Cate
,
C.
,
Popper
,
A. N.
(
2010
).
A noisy spring: the impact of globally rising underwater sound levels on fish
.
Trends Ecol. Evol.
25
,
419
-
427
.
Smith
,
A. R.
and
Staaden
,
M. J.
(
2009
).
The association of visual and acoustic courtship behaviors in African cichlid fishes
.
Mar. Freshw. Behav. Physiol.
42
,
211
-
216
.
Suriyampola
,
P. S.
,
Lopez
,
M.
,
Ellsworth
,
B. E.
and
Martins
,
E. P.
(
2020
).
Reversibility of multimodal shift: zebrafish shift to olfactory cues when the visual environment changes
.
Integr. Comp. Biol.
60
,
33
-
42
.
Teles
,
M.
,
Almeida
,
O.
,
Lopes
,
J.
and
Oliveira
,
R.
(
2015
).
Social interactions elicit rapid shifts in functional connectivity in the social decision-making network of zebrafish
.
Proc. Biol. Sci.
282
,
20151099
.
Tobet
,
S. A.
and
Baum
,
M. J.
(
1982
).
Implantation of dihydrotestosterone propionate into the lateral septum inhibits sexual receptivity in estrogen-primed, ovariectomized rats
.
Neuroendocrinology
34
,
333
-
338
.
Yang
,
C. F.
,
Chiang
,
M. C.
,
Gray
,
D. C.
,
Prabhakaran
,
M.
,
Alvarado
,
M.
,
Juntti
,
S. A.
,
Unger
,
E. K.
,
Wells
,
J. A.
and
Shah
,
N. M.
(
2013
).
Sexually dimorphic neurons in the ventromedial hypothalamus govern mating in both sexes and aggression in males
.
Cell
153
,
896
-
909
.

Competing interests

The authors declare no competing or financial interests.

Supplementary information