Behavior and neural activation patterns of non-redundant visual and acoustic signaling during courtship in an African cichlid fish

Across taxa, organisms have evolved mechanisms to communicate through multiple sensory channels for mediating social interactions such as reproduction, aggression and parental care. Multimodal communication occurs when one organism produces a signal via more than one sensory channel, which can then be detected by a receiver to either provide the same (redundant) or different (non-redundant) information (Johnstone, 1996; Partan and Marler, 1999, 2005). When presented separately, non-redundant signaling components should have different effects on a receiver, while redundant components should elicit the same or equivalent effects. According to the classification scheme of multimodal communication proposed by Partan and Marler (1999, 2005), recipient responses can include a physical behavioral response, a physiological response or no response at all. To properly classify multimodal signaling as redundant or non-redundant, however, the recipient response to each unimodal component separately as well as the response to the combined signals must be tested. The ability to process signals through multiple senses is crucial in instances where environmental noise may prevent effective transmission of one or more signal types, highlighting the importance of understanding whether signaling is redundant or non-redundant. For example, anthropogenic noise, pollution and climate change can impact both the sending and receiving of acoustic, chemosensory and visual information that is vital to communication in many animals (Halfwerk and Slabbekoorn, 2015; Partan, 2017; Slabbekoorn et al., 2010). This sensory disruption is particularly relevant for non-redundant signaling, in which masking or elimination of information in one sensory channel can have detrimental effects on decisions related to survival and reproductive success (Candolin, 2019; Shannon et al., 2016). After a relevant signal is received, this new information must also be integrated with the receiver's physiology and other environmental factors to produce the appropriate behavioral response. Most studies on multimodal communication largely focus on categorizing signals given by the sender and the resulting behavioral response in the receiver. Little is known, however, about how and where in the brain receivers process this multimodal social information during crucial behavioral interactions.

Although many studies across diverse taxa examine multimodal communication from the perspective of which senses are used and what type of information they provide to a receiver, less is known about how these signals are integrated in the brain and whether there are similar mechanisms of integration across species. Social behaviors are controlled by the brain, and a conserved neural network called the social decision-making network (SDMN) provides a comparative framework to examine how different animals integrate social information to mediate adaptive behaviors (O'Connell and Hofmann, 2012). The SDMN is composed of the social behavior network (SBN: lateral septum, bed nucleus of the stria terminalis/medial amygdala, anterior hypothalamus, ventromedial hypothalamus, preoptic area, periaqueductal gray/central gray) and the mesolimbic reward pathways (lateral septum, basolateral amygdala, bed nucleus of the stria terminalis/medial amygdala, basolateral amygdala, striatum, nucleus accumbens, hippocampus, ventral pallidum, ventral tegmental area) (Goodson and Kingsbury, 2013; Newman, 1999; O'Connell and Hofmann, 2011). Because animals live, interact and make decisions in a multisensory world, it is crucial to examine not only how receivers respond behaviorally to unimodal and multimodal sensory information in social contexts but also which regions of the SDMN are involved in processing these sensory inputs that lead to behavioral decisions. Thus, identification of relevant brain regions also allows for phylogenetic comparisons of nuclei involved in integrating unimodal and multimodal signals across vertebrates.

Fishes represent the most diverse and speciose vertebrate group, with enormous diversity in complex social behaviors. The African cichlid fish, Astatotilapia burtoni, is ideally suited to address questions related to multimodal communication because dominant males direct visual, acoustic, chemosensory, mechanosensory and tactile signals towards females during their courtship and spawning behaviors (Butler and Maruska, 2016b; Field and Maruska, 2017; Maruska et al., 2012). Dominant males are brightly colored, defend a territory, court females and have an upregulated reproductive axis. In addition to the bright coloration depicted by dominant males, visual courtship behaviors (body quiver, waggle, leads into territory), along with chemosensory and acoustic signaling, collectively serve as honest signals of fitness to females (Fernald and Hirata, 1977; Maruska and Fernald, 2012). For example, courtship body quivers are often coincident with the production of low frequency pulsed sounds (Fig. 1), the peak frequency of these sounds is negatively correlated with male body size, and females prefer to affiliate with males producing sounds over those that only provide visual signals (Maruska et al., 2012). Reproductively active females (gravid; full of eggs) are receptive to these mating signals, and also show improved vision, improved hearing and increased affiliation towards males, especially after ovulation (Butler et al., 2019; Maruska et al., 2012). Because multimodal social communication is crucial for their ability to identify mates, reproduce and defend their territories, A. burtoni has become an emerging model in many aspects of behavioral neuroscience and sensory communication.

The overall goal of this study was to test whether visual–acoustic courtship displays from dominant male A. burtoni provide redundant or non-redundant information to females, and where this sensory information is processed in the brain. Specifically, we quantified affiliation behavior of gravid females when they received visual, acoustic or combined visual–acoustic signals from a courting dominant male and compared this with a non-reproductive context as a control. By performing in situ hybridization for the immediate early gene cfos as a proxy for neural activation, we also investigated where unimodal and multimodal visual–acoustic signals are processed within the female brain. Our results demonstrate that acoustic information from males functions as a non-redundant courtship signal for receptive gravid females and highlights the importance of studying receiver processing in multimodal interactions. Differential cfos expression, classification of females into sensory contexts using cfos expression across brain nuclei, and unique co-activation networks among sensory conditions show that females perceive unimodal and multimodal auditory and visual courtship signals differently in the brain.

Adult Astatotilapia burtoni (Günther, 1893) were laboratory bred but originally derived from a wild-caught population from Lake Tanganyika, Africa, and kept in community aquaria with conditions similar to their natural environment: pH 7.8–8.0, 28–30°C, 12 h light/dark cycle, and constant aeration. Fish were fed cichlid flakes (AquaDine, Healdsburg, CA, USA) daily and supplemented with brine shrimp twice a week. All experiments were performed in accordance with the recommendations and guidelines provided by the National Institutes of Health Guide for the Care and Use of Laboratory Animals, 2011. The protocol was approved by the Institutional Animal Care and Use Committee (IACUC) at Louisiana State University, Baton Rouge, LA, USA.

To understand how female A. burtoni process unimodal and multimodal visual–acoustic courtship signals from males, we used experimental females in the gravid stage of the reproductive cycle. Gravid A. burtoni females have extended abdomens full of eggs, providing a potential visual signal of reproductive readiness to dominant males, and are close to choosing a male for spawning. Female reproductive readiness was confirmed after the behavioral trial by measuring gonadosomatic index [GSI=(gonad mass/body mass)×100]. Only females with GSI≥6.5 were used in the experiment (mean±s.d. GSI 9.23±2.33). Experimental trials were performed in 37.8 l aquaria divided into 2 compartments (community side: 33.5×25×26.5 cm, subject side: 16.5×25×26.5 cm) separated by clear acrylic barriers permanently sealed to the sides and floor of the tank, preventing the transfer of odorants and hydrodynamic stimuli that could be detected by the lateral line system (Fig. 2). While chemosensory, mechanosensory and tactile signals are also used during courtship in this species, our paradigm was designed to test only visual and sound production acoustic signals from males to specifically understand how unimodal and multimodal auditory and visual signals are processed by females. An association zone was marked 15 cm from both sides of the dividing acrylic barrier and used to quantify female affiliation with the dominant male and/or community. Each compartment contained a gravel-covered bottom and a quarter terracotta pot positioned along each side of the barrier to function as a shared territory shelter. The smaller subject female compartment also contained an additional half terracotta pot outside the association zone along the opposing side of the tank to distinguish shelter use for reasons different from affiliation with the male in the shared shelter.

During the overnight acclimation period, a dominant male was added to the community compartment along with three recovering females (neither gravid nor mouthbrooding), and the subject gravid female was placed into the smaller compartment. Stimulus dominant males were selected for experiments based on bright coloration and the performance of typical dominance behaviors for several days prior to use in the trials. A removable blue barrier was also placed alongside the dividing clear barrier, to prevent visual communication between the subject gravid female and neighboring community prior to the experimental trial. The subject female was allowed to acclimate in the tank overnight.

Behavioral trials began the following morning, between 08:00 h and 11:00 h. To stimulate courtship behaviors and motivation from the dominant male across all conditions, a stimulus female was placed into his compartment prior to the trial. This stimulus female was injected with 3 μg g⁻¹ body mass of prostaglandin F2α (PGF_2α) and added to the community at the end of the 10 min pre-stimulus period. PGF_2α levels naturally rise during the time of ovulation in A. burtoni, inducing the onset of female reproductive behaviors, which assists in attaining consistent male courtship behavior across experimental trials (Juntti et al., 2016; Kidd et al., 2013a). The blue acrylic barrier was removed simultaneously as the stimulus female was added to the community, initiating the 30 min stimulus trial period. To test whether male visual–acoustic courtship signals convey redundant or non-redundant information, we used an experimental paradigm to provide females with visual signals only, auditory signals only, or combined visual–auditory signals from the courting dominant males using different types of barriers separating the subject gravid female and the courting male. Visual-only trials consisted of a tank with barriers that created an air–water interface between the two compartments to prevent the transfer of any acoustic signals. In acoustic-only trials, a thin opaque black barrier separated the two compartments, shielding the subject female from visual signals from the courting dominant male while allowing the transmission of acoustic signals (hydrophone recordings across the barrier verified sound transmission). Visual–acoustic trials consisted of a thin transparent barrier that exposed subject females to both visual and acoustic signals from courting males. As a control for the neural activation aspect of the study, subject gravid females were exposed to visual–acoustic signals from the community of recovering females alone, without the presence of a courting dominant male. Thus, the experimental gravid female was exposed to social signals, but no visual or acoustic courtship signals from a male, allowing the controls to serve as a social but non-reproductive context to the subject gravid females. While females are not known to produce intentional sounds, any incidental sounds from feeding, mouth scraping on substances or fish hitting objects was possible. Importantly, the behavioral response of the subject female in this control condition was not necessary to classify male visual–acoustic signaling as redundant or non-redundant (only the three conditions with male signals are required). In contrast, this female-only control allowed us to distinguish neural activation patterns specifically related to male courtship signaling versus activation related to a general social group situation. All trials were video recorded to quantify dominant male courtship behavior and subject female affiliation behaviors during a 10 min pre-stimulus period and a 30 min stimulus period.

Behavioral quantification was performed using behavioral observation research interactive software (BORIS) (Friard and Gamba, 2016). Female affiliation was quantified as time spent in the association zone and time spent at the barrier. Each female was only used once in a single stimulus condition to allow for tissue collection and quantification of neural activation. Therefore, no within-individual behavioral analysis was performed. Because our focus was on female responses to male courtship, we also wanted to determine whether male courtship effort was similar across sensory conditions to ensure that the only difference was that females were receiving different sensory signals in each condition (visual only, acoustic only or combined visual–acoustic). Quantified male courtship displays included body quivers, tail waggles and leads towards the territory shelter. Quantified male behaviors included all behaviors performed by the male, regardless of whether it was directed towards the subject female or community fish. Courtship body quivers occur when the male curves his body in a C-shape and vibrates it while displaying egg spots along the anal fin, and this is the behavior associated with the production of courtship sounds (see Fig. 1). For analyses, we focused only on these quiver behaviors because they are the only behavior that is coincident with sound production.

To examine neural activation patterns, subject gravid females were killed 30 min after the end of the stimulus period by gradual cooling in ice-cold cichlid-system water followed by cervical transection. Females were measured for standard length (SL, 39.364±2.936 mm) and body mass (BM, 1.619±0.387 g) and Fulton's body condition factor K was calculated using the formula K=100/(BM/SL³) (26.277±3.815 mm). Brains were exposed, fixed in 4% paraformaldehyde (PFA) in 1× phosphate-buffered saline (1× PBS) overnight at 4°C, washed with 1× PBS and cryoprotected in 30% sucrose in 1× PBS at 4°C for 1–2 days. In preparation for sectioning, brains were embedded in OCT media (TissueTek, Sakura, Torrance, CA, USA), sectioned at 20 μm along the coronal plane and collected onto alternate positively charged slides (Superfrost plus, VWR, Radnor, PA, USA). Slides remained at room temperature for 2 days to allow sections to adhere to them before being stored at −80°C.

To investigate where unimodal and multimodal courtship signals are processed within the brain, we used chromogenic in situ hybridization with riboprobes to localize mRNA of the immediate early gene (IEG) cfos as a proxy for neural activation. We chose cfos as a marker because it is more associated with the perception of incoming sensory stimuli as opposed to behavioral output (Teles et al., 2015). Probe specificity, sense controls and protocol design were previously described (Butler and Maruska, 2016a). In short, cfos primers were designed using the A. burtoni-specific cfos sequence to create a probe template through PCR, where digoxigenin (DIG)-labeled nucleotides were incorporated during amplification. After purification, the probe was diluted in hybridization buffer (1:5) and stored at −20°C.

Slides with brain sections were brought to room temperature and a hydrophobic barrier (Immedge, Vector Laboratories) was drawn around the perimeter of the slides and allowed to dry for 30 min at room temperature. Slides were washed with 1× PBS (3×5 min), fixed for 10 min in 4% PFA, washed with 1× PBS (2×5 min), incubated in proteinase K solution (10 min), washed with 1× PBS (10 min), fixed with 4% PFA (20 min), washed with 1× PBS (2×5 min) and incubated in 0.1 mol l⁻¹ triethanolamine-HCl (pH 8.0) with 0.25% acetic anhydride for 10 min. Slides were then washed with 1× PBS again (5 min) and incubated in pre-warmed hybridization buffer for 3 h in a humidified chamber at 60–65°C. Next, the hybridization buffer was replaced with fresh hybridization buffer containing the DIG-labeled cfos probe, covered with hybrislips (Life Technologies), and incubated overnight (12–16 h) at 60–65°C in a sealed, humidified chamber. Hybrislips were carefully removed by immersing slides in prewarmed 2× sodium citrate chloride (SSC):50% formamide solution and slides were sequentially washed in the following solutions at 60–65°C: 2× SSC:50% formamide (2×30 min), 1:1 mixture of 2× SSC:maleate buffer (MABT) (2×15 min), and MABT (2×10 min). Next, slides were washed with MABT (2×10 min) at room temperature and incubated in blocking solution containing MABT with 2% bovine serum albumin to prevent non-specific binding of the anti-DIG antibody. Slides were incubated in alkaline phosphatase (AP)-conjugated anti-DIG Fab fragments (Sigma-Aldrich, St Louis, MO, USA) in blocking solution at a 1:5000 concentration overnight at 4°C in a covered humidified chamber. Slides were washed with MABT (3×30 min) at room temperature, incubated with AP buffer (2×5 min) at room temperature, and reacted with nitro-blue tetrazolium/5-bromo-4-chloro-3′-indolyphosphate (NBT/BCIP) (Sigma-Aldrich) solution in a sealed humidified chamber in the dark at 37°C for 4–5 h. To stop the reaction, slides were washed in 1× PBS (3×5 min), fixed in 4% PFA for 10 min, and washed in 1× PBS (3×5 min) again. Any remaining hydrophobic barrier was removed, slides were coverslipped with aqueous mounting media (Aqua-mount, Fisher Scientific, Waltham, MA, USA), and edges were sealed with clear nail polish after drying overnight.

To quantify differences in cfos expression across sensory exposures in subject females, we visualized stained slides on a Nikon Eclipse Ni microscope and photographs were taken with a color digital camera (Nikon DSFi2) controlled by Nikon NIS-Elements software. Individuals blind to the experimental condition performed quantifications. A Cresyl Violet-stained reference A. burtoni brain atlas and other relevant neuroanatomical resources were used to identify brain regions of interest (Fernald and Shelton, 1985; Maruska et al., 2017; Munchrath and Hofmann, 2010). We quantified social decision regions of the SDMN and midbrain areas involved in visual and auditory processing. Quantified regions of the SDMN include the lateral nucleus of the dorsal telencephalon, the granular region (Dl-g), the ventral part of the ventral telencephalon (Vv; divided into rostral, Vv-r, and caudal, Vv-c regions), the dorsal part of the ventral telencephalon (Vd), the supracommissural nucleus of the ventral telencephalon (Vs), the central part of the ventral telencephalon (Vc), the anterior part of the periventricular preoptic nucleus (nPPa), the periventricular nucleus of the posterior tuberculum (TPp), and the anterior tuberal nucleus (ATn). We also quantified activation in a midbrain auditory processing region, the torus semicircularis (TS), and a midbrain visual processing region, the torus longitudinalis (TL). For each region, a square grid was overlaid on the section and used to count the number of cfos-positive cells in 4–5 randomly selected boxes. A 30×30 μm grid and 20× magnification was used for the Vv-r; a 50×50 μm grid and 20× magnification for the Vv-c, Vs, nPPa, TPp and ATn; and a 100×100 μm grid and 10× magnification was used for Dl-g. Selected magnification for each region optimized the full view of the region of interest. To determine average cell density (no. cells μm⁻²) for each region, the total number of positive cells was divided by the cumulative area of the quantified regions. This process was repeated for four consecutive sections as a representative quantification of each region within each animal. Values were then averaged to obtain a mean cfos-expressing cell density for each region within each individual.

All statistical analyses were performed in R (3.6.2) and IBM SPSS 27. Differences in male courtship behaviors and female affiliative behaviors across different sensory stimulus conditions were compared using one-way ANOVA. All post hoc analyses were performed with Tukey's tests. A linear mixed model was used to compare the interaction of condition (control, auditory, visual or audio-visual) and brain region on cfos expression in R (package: lme4; Bates et al., 2015). Brain region was used as a within-subject factor, condition as a between-subject factor, body mass as a covariate and animal ID as the random effect. Body mass was used as a covariate in the model because there was slight variation in female body mass across conditions and brain size also varies with fish body mass.

We also used a discriminant function analysis (DFA; in SPSS) on cfos cell densities in all quantified brain regions to examine whether females could be classified into their respective sensory conditions based on neural activation patterns alone. Principal components analysis (PCA; in SPSS) was also used to examine for distinct neural networks among all 11 quantified brain regions and sensory conditions. Group means replaced missing values for each analysis. We used Pearson correlations to examine correlations between male and female time spent in the association zone and to create heat maps of co-activation networks across brain regions for each sensory condition. Clustering analysis was performed using the pvclust package in R. For all analyses, we chose not to correct P-values for multiple tests because these tests can lead to misleading biological interpretations as a result of increases in Type II errors and reduced statistical power (Nakagawa, 2004).

A total of 65 subject gravid females were used to conduct behavioral trials. Of these females, 9 had ovulated, with 1 female releasing eggs during the trial. There was no difference in condition factor K, SL or GSI across conditions for experimental females (K: F_3,61=2.354, P=0.081; SL: F_3,61=2.271, P=0.0893; GSI: F_3,61=1.285, P=0.288), but BM of females used in visual-only trials was slightly lower than that of females in other sensory conditions (BM: F_3,61=3.039; P=0.036).

Because our goal was to present different male-generated visual–acoustic stimuli to females, we first compared the behavior of dominant males across the three different male sensory conditions (visual, auditory, audio-visual). There was no difference in the total number of quivers performed by dominant males across sensory conditions, verifying that females received similar courtship effort from males in each condition and were only exposed to different types of male-generated courtship sensory information depending on barrier type (F_2,45=0.109, P=0.897; Fig. 3).

To examine how females change their behavior at the start of a trial, we compared the percentage change in time spent by the female at the barrier between the pre-stimulus period and the sensory exposure stimulus period (Fig. 4). This analysis showed that it differed across sensory conditions (one-way ANOVA, F_3,61=74.19, P<0.001), with the greatest increase in the visual-only trials (pre-stimulus time at the barrier: control 1.71±1.39 min; auditory 0.25±0.59 min; visual 0.61±0.82 min; audio-visual 0.29±0.57 min). While there was little to no change in the female's behavior in the auditory-only condition, females in all other conditions that contained a visual stimulus showed an increase in their time at the barrier once the trial began.

During sensory stimulus exposures, gravid females spent less time in the association zone (one-way ANOVA, F_3,61=73.79, P<0.001) and at the barrier (one-way ANOVA, F_3,61=106, P <0.001) during auditory-only trials compared with audio-visual, visual or control contexts (Fig. 5). These different behavioral responses between the two unimodal reproductive conditions demonstrate that visual and auditory courtship signals from males provide non-redundant information to the gravid female receiver. However, because the female responses in visual-only and audio-visual trials are similar, the visual information dominates in this reproductive context and this can be classified as the ‘dominance’ type of non-redundant communication.

To further explore the timing of behaviors from both the signaling male and the subject receiver female in different sensory conditions, we used raster plots to depict female affiliation (time in association zone, time at barrier) and male courtship quivers, as well as entries into the shared pot by both the male and female (Fig. 6A). This analysis revealed several important points. First, it showed that gravid females associate with the barrier specifically when dominant males perform quivers on the other side during visual and audio-visual trials. While the time males and females spent in the association zone was positively correlated in all conditions, it was only significant in the audio-visual context (R=0.553, P=0.033) (Fig. 6B). Second, females spent significant time in the association zone and at the barrier throughout the entire trial time in all conditions that contained a visual stimulus, regardless of whether it was a reproductive or non-reproductive (control) context. Similarly, subject females entered the shared pot more often in all conditions that contained a visual stimulus (visual, audio-visual, control) compared with the auditory-only context (one-way ANOVA, F_3,61=73.79, P=0.002).

The density of cfos-expressing cells differed among female sensory conditions (control, auditory, visual and audio-visual) and brain regions (condition: F_3,24=8.445, P<0.001; region: F_10,240=28.270, P<0.001; condition×region: F_30,240=2.730, P<0.001). Three regions, the nPPa, ATn and Vv-r, showed differential cfos expression among different sensory contexts. In the nPPa, females exposed to multimodal auditory and visual signals had greater cfos expression compared with females exposed to unimodal auditory signals alone (T₁₈₆=4.832, P<0.001) and similar expression levels to females from control and visual-only sensory contexts (Fig. 7A). Similarly, in the ATn, females from audio-visual conditions showed greater cfos expression when compared with females from control and auditory-only sensory contexts (audio-visual/auditory: T₁₇₄=2.982, P=0.017; audio-visual/control: T₁₈₆=2.731, P=0.035) and similar levels to females exposed to visual signals alone (Fig. 7B). In the Vv-r, females exposed to visual and audio-visual signals displayed the highest levels of cfos expression, followed by controls with intermediate levels, and females exposed to auditory-only signals with the lowest cfos expression (audio-visual/auditory: T₁₇₄=7.976, P<0.001; audio-visual/control: T₁₈₆=4.071, P<0.001; control/auditory: T₁₈₅=3.88, P<0.001; control/visual: T₁₈₉=7.17 P<0.006; visual/auditory: T₁₈₉=7.17, P<0.001) (Fig. 7C) (see Table S1 for post hoc values). There were no differences among sensory treatments in the Vv-c, Vd, Vs, Vc, TPp, TS or TL (P>0.05).

To further examine how neural activation differed across sensory contexts, we performed a DFA with cfos densities in all quantified brain regions (Fig. 8A). DFA determines whether animals can be separated into groups based solely on their neural activation patterns across the brain and identifies which variables or brain regions contribute to the proper classification of individuals. The DFA correctly classified 100% of females from auditory-only and visual-only contexts, while one female from each of the control and audio-visual sensory conditions was misclassified into the opposite group. The DFA produced one significant function (Eigenvalue=98.82, χ²=57.536, P=0.005), explaining 70.8% of the variation. Function 1 clearly separates females from audio-visual and auditory conditions and is positively loaded by neural activation in the Dl-g, Vs, nPPa, Vd, TS and Vv-c. Although not reaching significance, function 2 (Eigenvalue=3.012, χ²=18.579, P=0.549) separates females from non-reproductive control conditions from all of the reproductive sensory conditions where males were present, suggesting that courtship signals received from dominant males result in neural activation patterns different from signals received in a non-reproductive context. To examine how neural activation might be associated with female affiliation and courtship behavior from dominant males, we performed Pearson correlations between discriminant function coefficients and behavior for each sensory condition (Fig. 8B) (see Table S2 for Pearson correlation coefficients and P-values). Function 1 did not correlate with any behaviors while function 2 negatively correlated with female time in the association zone (R=−0.76, P=0.047) and female time at the barrier (R=−0.76, P=0.049) only in the audio-visual condition. Female affiliative behaviors did not correlate with neural activation in any individual brain region, suggesting that the observed neural activation patterns determined with cfos are more reflective of incoming sensory information rather than female behavioral output.

Because SDMN brain regions are interconnected, examining co-activation networks may better reflect complex sensory processing in behavioral contexts than single brain nuclei comparisons. To examine the connectivity of each brain region in different sensory contexts, we generated heat maps using Pearson correlations of cfos cell densities (Fig. 9). Overall, we found that patterns of activation across brain regions differed for each sensory condition. For example, the control, visual and audio-visual context had two significant clusters while the auditory-only condition had three and there was little overlap among significantly correlated regions and clusters across different sensory conditions. This emphasizes the complexity of perceiving unimodal and multimodal courtship signals in A. burtoni females. Tables S3–S6 show Pearson correlation coefficients and P-values for control, auditory, visual and audio-visual conditions, respectively.

To identify neural activation networks across all brain regions analyzed for females from all conditions, we performed a PCA which identified three distinct networks (Kaiser–Meyer–Olkin measure of sampling adequacy=0.613, χ²=128.97, P<0.001) (Fig. 10). The first includes nPPa, ATn, Vd, Dl-g and Vc (representing 48.26% of variance) and may represent regions that process multimodal audio-visual information because most of these regions have high cfos expression in audio-visual conditions. The second includes Vv-c, Vv-r, Vs and TPp (representing 20.28% of variance) and likely represents regions that process unimodal acoustic or visual signals, because these regions have high activation in visual or audio-visual conditions and low activation in auditory-only conditions. The third network includes TS and TL (representing 9.41% of variance) and represents regions characterized by high activation in control and visual or audio-visual conditions so may be involved in processing sensory information in any social context.

We examined the behavior of A. burtoni females to test whether auditory and visual courtship signals from dominant males provide redundant or non-redundant signals to females. Females showed different affiliative behaviors in each unimodal sensory condition, demonstrating that auditory and visual signals from dominant males provide non-redundant information to receptive gravid females. Because female affiliation behaviors were similar between the unimodal visual condition and the multimodal audio-visual condition, the visual signals dominate over the auditory signals and signaling can be classified as the ‘dominance’ type of non-redundant communication. We also identified differential neural activation patterns in the brain using in situ hybridization for cfos within the SDMN and midbrain sensory processing regions. These distinct co-activation networks across socially relevant brain regions in females exposed to different sensory conditions highlight the complex neural computations utilized by females to make mating decisions based on different sensory signals received from courting males.

Dominant A. burtoni males use non-redundant multimodal audio-visual signaling to court receptive gravid females, with the visual component dominating the auditory. Males produce elaborate courtship displays consisting of intensifications of body coloration (yellow/blue body coloration, black vertical eye bar and orange spots along the anal, caudal and dorsal fins) and increased body quivers performed in association with low frequency sounds. While most studies focus on the production of multimodal signals, our study highlights how signals are interpreted by the receiver. Females showed increased affiliative behaviors when they received visual sensory information, demonstrating the importance of the visual component(s) of the courtship display. As females approach ovulation, the sensitivity of their visual and auditory systems improves (Butler et al., 2019; Maruska et al., 2012) and they are behaviorally more receptive to male courtship (Kidd et al., 2013b). Females have greater auditory sensitivity in the spectral range of male courtship sounds and show a preference for sound-producing males, indicating that the acoustic component plays a role in female mate choice (Maruska et al., 2012). Females from auditory-only conditions showed no response to male courtship sounds alone, demonstrating that auditory signals alone are insufficient to motivate females for reproduction. According to the multimodal communication classification scheme proposed by Partan and Marler (2005), recipient responses from individual components of a multimodal signal can be a physical behavioral response, internal physiological response or no response at all. In A. burtoni, the visual and auditory components elicit distinct responses in receptive females, either increased affiliation or no response, respectively. Thus, courting males may enhance their primary visual courtship displays by providing some additional information to females via sound production (e.g. size, fitness) that is not available from the visual display alone. Females may show reduced or no affiliation in the auditory-only context if vision is necessary for directional responses to auditory signals, especially as the auditory component is always accompanied by the visual component. Females may require a visual signal to identify the location of auditory signals or to perceive it as salient since auditory signals do not occur without visual displays. During audio-visual contexts only, male and female affiliation time was positively correlated, suggesting that both the visual and acoustic component are valuable for courtship in A. burtoni and when presented in combination provide new information to females. The visual component functions as the dominant signal and may provide information on location while sound may supply additional information (e.g. male size, quality) to the female when accompanied by the visual component to increase correlated behavior between males and females. Similar examples exist in crayfish courtship, where males show no response to female chemical signals but positively respond to multimodal chemical and visual stimuli (Acquistapace et al., 2002).

Several fish species use audio-visual signaling to convey information in courtship and reproductive contexts. However, in species with elaborate visual display components (coloration and movement), and as for our results in A. burtoni, sound production alone often fails to elicit female positive responses unless a visual signal is also present. Thus, multimodal visual–acoustic signaling likely provides non-redundant information in many fish species. For example, the painted goby produces visual courtship displays in conjunction with low frequency drum sounds and females in this species use the sound as an assessment of male quality (Amorim et al., 2013). Many Lake Malawi cichlids also use visual–acoustic signaling as part of their courtship repertoires and related sympatric species with similar coloration patterns have different sound characteristics, suggesting that acoustic signals may be important for both species identity and signaler phenotypic traits (Amorim et al., 2008; Smith and Staaden, 2009). In the damselfish Dascyllus albisella, male mating success is positively correlated with their courtship rate (combined visual–acoustic signaling) and females choose mates based on this condition-dependent trait (Oliver and Lobel, 2013). Similarly, in A. burtoni, peak sound frequency and the percentage of visual quivers accompanied by sounds is related to male body size, so females may receive honest information on male qualities such as size, dominance status, health and fitness that can be used in mating decisions.

Non-redundant multimodal signaling is also advantageous in situations where environmental conditions may mask one or more sensory channels, to ensure some transmission of information in reproductive contexts. Wild A. burtoni live in shallow shore pools and river systems of Lake Tanganyika, which are subject to disturbances (e.g. runoff, increased turbidity from large animal traffic) that could disrupt the transmittance of visual signals between males and females. Evolving alternative sensory pathways such as sound production to convey reproductive readiness and fitness qualities to receptive females would be advantageous in such environments to ensure species persistence. Several vertebrate taxa implement multimodal shifting from one sensory channel to another to ensure signal detection. For example, in several species of frogs, males initially produce mating call sounds to receptive females but shift to visual signals, such as foot flagging, when their environment becomes too noisy from streams or waterfalls (Hodl and Amezquita, 2001). Similarly, zebrafish (Danio rerio) shift perception from vision to olfaction after exposure to dim light for 6 weeks, displaying the ability to adapt perception to the sensory modality that best suits their environmental conditions (Suriyampola et al., 2020). Thus, non-redundant communication allows for behavioral signaling plasticity under changing environmental or social conditions to ensure transfer of information.

Females from the control conditions were exposed to non-reproductive signals from community females and displayed affiliative behaviors at levels similar to those for male-generated visual and audio-visual conditions. Astatotilapia burtoni are social animals, highly dependent on interactions, and therefore they typically increase affiliative behaviors in the presence of any other fish, regardless of sex. It is important to note that the behavioral response of subject females in this female-only non-reproductive control condition was not needed to determine the non-redundant classification of male audio-visual signaling in the reproductive context. Rather, this control was included to better distinguish neural activation patterns specifically related to the reception of male courtship signals compared with those resulting from any general social context. Exploring neural activation patterns in females from each sensory condition will assist in uncovering how reproductive versus non-reproductive visual signals are processed in A. burtoni females.

While several studies focus on how courtship signals are produced by signalers, few focus on how unimodal and multimodal information is processed in the brain by receivers. Here, we used in situ hybridization for the immediate early gene cfos as a marker of neural activation in nuclei of the SDMN along with two midbrain sensory regions, the TL and TS, to understand how unimodal and multimodal visual and acoustic male courtship signals are processed in A. burtoni females. The SDMN, which regulates social behavior across vertebrates, is composed of the social behavior network and the mesolimbic reward system, and A. burtoni females displayed differential cfos activation in three nuclei of the SDMN, the Vv-r, nPPa and ATn. Across these regions, there was greater cfos expression in audio-visual contexts when compared with auditory alone, showing that auditory courtship information is processed differently by the female brain when visual information is absent. This pattern is consistent with the behavioral results where acoustic signals alone from males failed to elicit a positive female affiliation response. Thus, based on both behavior and neural activation patterns, auditory courtship signals alone are insufficient to provide the required context that leads to successful reproduction.

In the Vv-r, females in the visual and audio-visual context showed greater neural activation when compared with those in the control and auditory-only conditions. Previous work in A. burtoni showed that gravid females had greater cfos expression in the Vv-r when exposed to sexually relevant visual, auditory, mechanosensory (lateral line) and chemosensory stimuli from dominant males (Field and Maruska, 2017). By solely focusing here on visual and auditory signals, we further demonstrate the importance of the Vv-r in specifically processing visual courtship signals from dominant males. The Vv-r, homologous in part to the mammalian lateral septum, is a shared region in both the social behavior network and mesolimbic reward systems (O'Connell and Hofmann, 2011). In mammals, the lateral septum is heavily involved in integrating internal (age, sex, reproductive/social status, motivational state) and external information to produce appropriate behavioral responses (Menon et al., 2022). In rodents, sex steroid receptor expression in females further implicates the role of the lateral septum in modulating reproductive behaviors (Tobet and Baum, 1982). Female A. burtoni in the control context also displayed higher activation in the Vv-r when compared with females from auditory-only conditions. In mammals, the lateral septum plays a role in processing sensory information from general social interactions, although these pathways are not well understood (Menon et al., 2022). Together, our results show that the Vv-r may be involved in contextualizing sensory information by processing general visual social information (from control conditions) while being more heavily involved in processing visual information during reproductive contexts.

Females exposed to multimodal audio-visual courtship signals had greater activation in the ATn than those receiving unimodal auditory courtship signals and control females from the non-reproductive context. The discriminant function analysis revealed that function 2, primarily loaded by the ATn, was significantly correlated with female affiliative behaviors only in females from the audio-visual condition, further showing its importance in processing multimodal courtship signals. The ATn is homologous in part to the mammalian ventromedial hypothalamus (VMH), which contains progesterone receptors and is known to control female reproduction and sexual receptivity (Yang et al., 2013). The functional role of the ATn in fishes remains largely unknown but our results suggest it may play an important role in processing visual and acoustic multimodal courtship signals. In the midshipman fish, females displayed greater cfos expression in the ATn when exposed to conspecific male courtship calls compared with heterospecific calls or ambient noise (Mohr et al., 2018). Conversely, goldfish exposed to non-reproductive unimodal, bimodal and multimodal visual, acoustic and hydrodynamic stimuli showed activation in the ATn, supporting the idea that the ATn functions as a integrative center while contradicting the notion that it plays a bigger role in specifically processing reproductive sensory information (Kirsch et al., 2002). Although more studies are needed to elucidate the role of the ATn in social behavior and processing of sensory information in fishes, our results suggest that the ATn may work to integrate and discriminate between auditory and visual signals in reproductive and non-reproductive contexts.

Female A. burtoni exposed to multimodal audio-visual courtship signals had greater neural activation in the nPPa when compared with females from the auditory-only context, but showed similar activation in all conditions with a visual stimulus. This result suggests that the nPPa processes socially relevant visual information and may function as an integrative center to process sexually relevant visual and acoustic courtship signals in A. burtoni females. Further, the nPPa had greater cfos expression in A. burtoni females exposed to multimodal visual, chemosensory, acoustic and mechanosensory courtship signals from males compared with no-fish controls and similar sensory information from other females, showing it functions to integrate various types of sexually relevant sensory information (Field and Maruska, 2017). The nPPa is a subregion of the preoptic area that functions in modulating social behaviors across vertebrates including reproduction, parental care and aggression (Forlano and Bass, 2011). Neurons in the nPPa are heterogeneous and this region contains different neuron populations that express various types of neuromodulators, including gonadotropin-releasing hormone, arginine vasotocin, galanin and receptors for steroid hormones that collectively play a role in regulating behavioral output and neuroendocrine functions (Butler et al., 2021, 2020; Maruska and Tricas, 2011). Using co-activation networks, we show that cfos expression in the nPPa is heavily correlated with expression in other nuclei in the brain that is dependent on sensory condition. Because the nPPa expresses various neuromodulators that can impact the activity of different circuits and suites of neurons, this region likely functions as an integrative center to process different types of sensory information. Thus, it may help evaluate the salience of communication signals based on an individual's internal physiological state.

To assess differences in neural activation among different regions in the brain that process sensory information, we quantified cfos expression in two midbrain nuclei outside the SDMN, the TL and TS. The TL is strongly associated with the optic tectum, a primary visual center in non-mammalian vertebrates, and plays a large role in visual processing and directing visual attention (Folgueira et al., 2020). Conversely, the TS is an important auditory and lateral line processing region, but also receives visual and somatosensory inputs in fishes (Schellart, 1983; Schellart et al., 1987). The TS is also thought to help distinguish between conspecific and heterospecific acoustic calls in vertebrates. For example, female tungara frogs exposed to male-specific mating calls displayed greater neural activation in the TS than those exposed to heterospecific calls or ambient noise (Mangiamele and Burmeister, 2011). Our PCA demonstrated that the TS and TL form a neural network, showing that these regions may work together to process sensory information and relay it to behavioral decision circuits. Although we did not find differences in cfos expression across sensory conditions for the TL or TS, we revealed some differences in co-activation between the TL and TS and regions of the SDMN. For example, females exposed to audio-visual courtship stimuli had a significant positive correlation between activation of the TL and Dl-g, while activation of the TS and Vv-r was negatively correlated in females from the auditory-only context. The Dl-g is homologous in part to the mammalian hippocampus and is involved in spatial cognition, which is heavily dependent on processing spatial visual information (Broglio et al., 2003). Taken together, this suggests that the TL may be more involved in conditions where visual information is processed while the TS is more involved in auditory contexts when females only receive acoustic sensory information. However, in visual-only conditions, the TL and TS formed a significant cluster with the ATn, TPp and Vv-c that was not present in any other context, showing that more complex interactions occur between the TL and TS and regions of the SDMN to modulate behavioral output in different social contexts.

We show that dominant A. burtoni males use multimodal visual–acoustic courtship signals to provide non-redundant signals to receptive females. Females showed lower affiliation when exposed to auditory signals alone, demonstrating that the visual component(s) of male courtship signals dominates over the auditory signal and is necessary for positive female affiliative behaviors. However, several lines of evidence indicate that the sound production provided by males during the multimodal visual–acoustic display is important and used by females for mating decisions. (1) Peak sound frequency and the percentage of quivers with associated sounds is related to male traits (e.g. body size, quiver rate) providing potential honest signals for female choice. Thus, while the visual signal component is necessary for a directed response, sounds may provide information on male characteristics. (2) Female hearing improves (increased sensitivity) as she approaches ovulation, specifically in the frequency range of male courtship sounds, possibly allowing her to better discriminate acoustic information that conveys male traits. (3) Playback experiments show that females prefer to affiliate with males that have sounds coming from their territories compared with males with only visual signals (Maruska et al., 2012). We now add to this information by demonstrating that females process visual–acoustic unimodal and multimodal courtship stimuli differently in the brain, showing context-dependent activation in several regions of the SDMN and unique co-activation networks. This research expands our knowledge of how sensory information is processed by the receiver, especially in a species whose reproductive success is tightly linked to its ability to process information delivered via different sensory modalities.

We thank members of the Maruska lab, particularly Julie Butler, Cy Bryson, Adaoma Ngari and Austin Nguyen for experimental assistance, and Rose Wayne for artwork in Fig. 1.