Multisensory enhancement of electromotor responses to a single moving object

Multisensory integration is essential to the formation of a unified sensory percept (Ghazanfar and Schroeder,2006), and its influence extends across a wide range of taxa. Humans use the optimal integration of visual and haptic information to determine the orientation of an object(Helbig and Ernst, 2007). Drosophila hone visually guided compensations in flight control with mechanosensory input from their halteres(Sherman and Dickinson, 2004). Barn owls integrate auditory and visual information to produce bimodal head saccades with the shorter latency of auditory saccades and the greater accuracy of visual saccades (Whitchurch and Takahashi, 2006). The basic rules governing multisensory integration are fairly well established: (1) facilitation occurs when cross-modal stimuli are spatiotemporally aligned to a receptive field(Meredith et al., 1987; Meredith and Stein, 1996); (2)the magnitude of multisensory enhancement is inversely related to the effectiveness of its unisensory components(Stanford et al., 2005) and(3) the perceptual weight of each sensory modality during a multisensory experience is inversely related to the variance in the stimulus(Ernst and Banks, 2002).

Weakly electric fish possess three types of sensory organs structured in arrays with overlapping receptive fields. They are the tuberous electrosense(Tub), ampullary electrosense (Amp) and mechanosensory lateral line (LL). Each sensory modality simultaneously encodes unique and interrelated information about objects in the near-field environment(Nelson et al., 2002). Nonetheless, the importance of multisensory integration across electrosensory and mechanosensory arrays during moving object recognition is not yet established. Behavioral (Ciali et al.,1997; Moller,2002; Moller et al.,1982; von der Emde and Bleckmann, 1998) and neurophysiological(Bastian, 1982; Bleckmann and Zelick, 1993; Prechtl et al., 1998) evidence indicates that weakly electric fish are multisensory integrators.

Weakly electric fish are most notably recognized for their high-frequency,active electric sense performed by the tuberous sense organs. Active electrolocation results from the detection of localized distortions in the self-generated electric field that are caused by nearby objects with an electrical impedance different from the surrounding water(Gomez et al., 2004; Heiligenberg, 1973). The tuberous sense is capable of determining the distance(von der Emde et al., 1998),3-D shape (von der Emde and Schwarz,2000) and electrical properties of objects(Aguilera and Caputi, 2003; Caputi et al., 2003). Weakly electric fish also possess a low-frequency, passive electric sense performed by the ampullary sense organs. Passive electrolocation results from the detection of a transdermal potential caused by the presence of objects that possess a bioelectric field (Wilkens et al., 2002). The ampullary sense is capable of determining prey location (von der Emde and Bleckmann,1998) and proximity to a shelter(Rojas and Moller, 2002). Weakly electric fish also have the mechanosensory lateral line system. The lateral line is sensitive to water movement, which can be caused by a nearby moving object (Mogdans and Bleckmann,1998). The lateral line system is capable of discriminating an object's directional motion, speed, size(Vogel and Bleckmann, 2000)and location (Coombs et al.,2001).

The so-called `novelty response' (NR) in pulse-type weakly electric fishes is a stimulus-induced transient increase in electric organ discharge (EOD)rate (Aguilera and Caputi,2003; Barrio et al.,1991; Hall et al.,1995) with psychophysical properties akin to orienting responses in mammals (Post and von der Emde,1999). The NR is a tractable behavior for measuring perception,since it is reducible into scalar values and persists under curarization.

The NR may temporarily augment vigilance because it increases the sampling rate of the tuberous electrosensory system. The energetic cost of the electric fish brain is exceptionally high (Nilsson,1996), and, in hypoxic environments, pulse-type electric fishes will lower the rate and/or amplitude of their EOD(Crampton, 1998). Consequently, the regulation of the NR may ultimately impact the survival of electric fishes. Therefore, the NR probably indicates the perceived value and saliency of a stimulus. In fact, the probability and amplitude of the NR are directly related to electric image contrast(Caputi et al., 2003) and light intensity (Post and von der Emde,1999). NR habituation is inversely related to interstimulus interval (Barrio et al., 1991; Caputi et al., 2003; Post and von der Emde,1999).

The NR of the mormyrid electric fish Brienomyrus(Brevimyrus) niger contains two components: `acceleration'and `scallop'. Scallops and accelerations are fairly similar except that scallops contain a short sequence (4–6 intervals) of EODs with a peak frequency much higher than accelerations(Carlson, 2002; Serrier and Moller, 1989). We utilized the NR to evaluate the role of multisensory integration in moving object recognition. By manipulating the fish's EOD and surrounding environment we were able to selectively stimulate three sensory modalities with a single moving object. Principally, we discovered that multisensory NRs have a significantly larger duration and magnitude than the linear sum of their component unisensory NRs. Additionally, multisensory NRs have a significantly greater amplitude, probability and rate of scallop production than their component unisensory NRs.

The mormyrid pulse-type weakly electric fish Brienomyrus(Brevimyrus) niger (Günther 1866) was used for this study. B. niger were acquired from commercial suppliers and housed in 150 liter aquaria connected to a centralized water filtration system. They were fed with live, black worms daily and maintained under a 12 h:12 h light:dark cycle. Fish were tested between 4 and 9 hours after the lights turned on during the day. Water conductivity and temperature were maintained at 20–80 μscm^–1 and 25–27°C. Fish ranging from 6 to 9 cm in total length were used (N=21). Fish were first anesthetized with MS-222 (1:5000) and then immobilized with a 3 μl intra-muscular injection of gallamine triethiodide (3 mg ml ^–1; Sigma Chemical Co., St Louis, MO, USA) prior to introduction into the experimental tank. These procedures are in accordance with the guidelines established by the National Institutes of Health and were approved by the University of Virginia Animal Care and Use Committee.

The experimental tank was a 50 cm length × 50 cm wide aquarium with a height of 15 cm. Water temperature was maintained at 26°C with a conductivity ranging from 70 to 110μscm^–1. The immobilized fish was supported by a 12 cm-long, narrow platform with a foam strip on top that cradled its shape. Aerated water was passed over its gills through a fitted glass tube placed inside its mouth. In intact B. niger, EODs occur 2.5–4.5 ms after the first negative peak of a triphasic volley of the electromotor neuron (EMN). EMN activity was monitored by placing the caudal peduncle inside a piece of nylon tubing containing a pair of silver wire electrodes. The EMN signal was amplified 10,000 times, low-pass (1.6 kHz)filtered and sent to a Schmitt trigger that drove an event timer to record the timing of its activity with 1 μs resolution (model ET1, Tucker-Davis Technologies, Gainesville, FL, USA). Time stamps were collected with a Matlab program for off-line analysis.

The experiment took place in a light-shielded environment. Luminance level in the room during stimulus object presentation was 4×10^–8 Wcm^–2 (UDT instruments,Baltimore, MD, USA), and the experimenter's ability to even roughly visualize the experimental setup was impossible without continuous dark adaptation for approximately 30 min. To prevent the fish from dark adaptation, a large fluorescent ceiling light was turned on and, from a short distance, a 40 W lamp was directed at it between periods of object stimulation.

The stimulator consisted of a vertically oriented center axis with an L-shaped plastic arm connected to the base of the axis(Fig. 1). The vertical portion of the L-shaped arm (9 cm length, 6.5 mm diameter) served as the stimulus object. The top of the stimulus object was above the fish's dorsal surface. The horizontal portion of the L-shaped arm was approximately 6 cm below the fish's ventral body surface and extended 12 cm out from the center axis. The water surface of the tank was kept more than 1 cm above the stimulus object to minimize surface waves. A DC motor (Oriental Motor Corp., AXU series, Tokyo,Japan) rotated the center axis on ball bearings via a pulley system. Object speed was calculated from the frequency of a square pulse produced by the DC motor's speed control unit. Starting at an angle of 180 deg. from the fish's lateral body surface, each 360 deg. rotation of the stimulus object was initiated by a software-triggered square pulse and was terminated by the interruption of an infrared hardware switch.

The injection of gallamine triethiodide immobilized the fish and abolished its EOD, making the fish unable to electrolocate. Recovery time from the drug was several hours longer than the duration of the experiment. A digitally stored natural B. niger EOD was delivered through a pair of electrodes, one from inside the fish's stomach and the other from behind the tail to serve as a substitute. EMN activity triggered the EOD substitute at a delay of 3–4.5 ms from the first negative peak of the EMN volley. Turning off the EOD substitute allowed for the selective elimination of stimuli for the tuberous electrosensory system. Before beginning any experimental trials, the immobilized fish was allowed to acclimate to the experimental environment for 30–45 min while the EOD substitute was active. Measured through the agar wall (see below), the amplitude of the EOD substitute was set to approximately 45–60 m V cm^–1 at 1.6 cm away from the fish's abdomen to closely resemble its natural electric field strength. The EOD substitute was turned on or off immediately following the session of stimulus rotations.

The local distortion of the fish's electric field caused by the plastic stimulus object was measured with the 3 mm-wide fork electrode positioned 5 mm away from the fish's lateral body wall and 1.3 cm from the stimulus object. The replacement electric field was generated by a function generator and consisted of a biphasic pulse that approximated the duration and amplitude of B. niger's EOD. The mean reduction in peak-to-peak electric field amplitude was 1.6±0.7% (N=4) at a stimulus object distance of 2.1 cm from the lateral body surface.

Two insulated silver wires with bare tips were glued to opposing sides of the stimulus object so that the tips were directly facing the fish's lateral body surface at the object's minimum lateral distance (MLD). By flipping a switch on a battery, direct current was passed into the wires to create a DC electric field. A potentiometer connected to the battery allowed the voltage supplied to the wires to be adjusted. Selective delivery of the ampullary stimulus was controlled by the mechanical switch. Different strengths of the DC field were delivered, ranging from 200 to 600 μVcm^–1 at a distance of 3 cm from the fork electrode.

A 6 mm-thick acrylic frame with a rectangular opening, approximately 10 cm long and 5 cm wide was placed between the fish and the moving object(Fig. 1). The frame was firmly clamped in place. A hard agar gel (5% agarose, Sigma Chemical Co., A0576, 8 mm thick, within 10% conductivity of water) was fitted tightly inside and clamped to the frame opening to securely block hydrodynamic stimulation of the lateral line. Electrical current could readily pass through the agar wall but water movement was obstructed. The agar gel could be removed from the acrylic frame,without displacing the fish, when lateral line stimulation was desired. The object's MLD from the fish was between 2.4 and 2.8 cm. The object traveled at a speed of 4.5–5.0 cm s ^–1.

Each fish was presented with a specific stimulus only once, for a maximum of seven different stimulus types including the control(Table 1). For each stimulus type presented, 10 identical 360 deg. rotations of the stimulus object were delivered with a 10 s pause between them. Ten rotations of a single stimulus type will hereafter be referred to as a `session'. The within-session stimulus object MLD interval was approximately 26 s. There was a 10 min pause between each session to reduce the effect of habituation. The different stimulus type sessions were randomly ordered within two blocks. In the first block of stimuli, the agar wall was present, and in the second block, the agar wall was removed for the addition of lateral line stimulation. The sessions without lateral line stimulation occurred first because although the agar wall was relatively easy to remove without displacing the fish, fitting it back inside the plastic frame was difficult and could interfere with experimental protocol. At the beginning of every experiment, a control test was conducted in which the agar wall was present and both of the electric fields were off. If the fish did not respond with probability (see Data analysis) greater than 30% during the control, stimulus isolation was deemed successful, and the experiment was included in the analysis.

Time stamps of fictive EOD activity were converted into a series of delta functions. These were then convolved with a Gaussian function, with a width of one standard deviation set to 68.75 ms, to generate a spike density function(SDF; Fig. 2C)(Carlson and Hopkins, 2004; Szucs, 1998). This width of the Gaussian function was chosen because it provided an accurate representation of NRs (relative to instantaneous frequency, Fig. 2A) against the background variation in EOD intervals. The result was a continuous, low-pass filtered(23.5 Hz) function representing the EOD rate in units of pulses s^–1. The main purpose of using the SDF was to create a continuous function of EOD rate to allow the arithmetic summation of unisensory responses. The mean SDF and its time derivative [spike density derivative (SDD); Fig. 2D] were calculated across each session of 10 rotations. The mean SDF was evaluated instead of single SDFs because it was often problematic to accurately determine the start time and end time of a NR during a single rotation if the fish had a tendency to scallop or displayed frequent spontaneous accelerations. The mean SDF and SDD will be hereafter referred to as `SDF' and`SDD', unless specified otherwise.

Several NR parameters were quantified from the SDF and SDD: start time, end time, baseline, duration, area, amplitude and probability (hybrid SDF and instantaneous EOD rate). They are defined as follows. The maximum SDD was determined within a 2 s window centered on the time of the object's MLD. The time of the last negative-to-positive zero-crossing before the time of this maximum SDD was defined as the start time(Fig. 2C,D). Prior to the start time, each fish exhibited a variable discharge rate. In some fish, noise related to the start of the stimulus object's movement caused an increase in the variation of their EOD behavior that could persist to the time of the MLD. Since the NR has been defined as a change in EOD rate(Caputi et al., 2003; Caputi et al., 1998; Hall et al., 1995; Post and von der Emde, 1999),the period of time immediately before the start time was chosen to determine baseline, in order to improve the accuracy of its measurement. The baseline of each session was defined as the mean of the SDF over the final second leading up to the start time. The end time was defined as the time of the first negative-to-positive zero-crossing after the maximum SDD with a corresponding SDF below a threshold level. The within-session threshold level was baseline plus two standard deviations during the baseline period and was used to avoid registering local minima as end time while the SDF remained high(Fig. 2C,D). In cases where a local minimum did not happen before the SDF returned to baseline, the time at which the SDF intersected the baseline was chosen as the end time. The duration of the NR was calculated by subtracting start time from end time. The area of the NR was calculated as the integral of the baseline-subtracted SDF from start time to end time. To compare multisensory and unisensory SDFs,baseline was subtracted from the SDFs. The unisensory SDFs were then arithmetically summed to compare with multisensory SDFs. In the special case of Fig. 5, the duration and area of the individual NRs were calculated using the aforementioned methods applied to mean NRs.

To determine the probability of the NR during each session, each SDF from a single stimulus rotation was examined. Within each session, probability was defined as the fraction of stimulus rotations in which the individual SDF exceeded the threshold value at the time of NR maximum. The within-session threshold was set at baseline + 2α, where α is the mean of the standard deviations across the 10 individual SDFs during the baseline period. Amplitude was calculated by subtracting baseline from the maximum value of the NR. The number of scallops near the object's MLD was calculated using the unfiltered EOD intervals. Scallops were counted as any string of EODs with one or more consecutive intervals less than 30 ms and located between 1 s before or 2 s after the time of the object's MLD. To determine if differences in within-session response variation could explain multisensory enhancement, we compared the standard deviations and coefficients of variation of the SDFs(between start time and end time of the mean NR) among the stimulus types. To estimate the within-session temporal alignment of individual NRs, we calculated the standard deviation of the time of SDF maximum within the bounds of the mean NR. The sample mean of the standard deviation will be expressed as the symbol σ.

Within-subject (fish) differences among stimulus types were tested using a repeated–measures analysis of variance (ANOVA) with a Bonferroni correction for individual comparisons. Non-parametric tests for scalloping among the non-tuberous stimuli were performed with a sign test. Tests for covariance between scallop rate and NR magnitude were performed by calculating Pearson's correlation coefficient. A Shapiro–Wilk test was used for determining the normality of data.

When the object stimulus was near the MLD, B. niger transiently increased its EOD rate in a pattern typical of an acceleration(Carlson and Hopkins, 2004),thereby resembling previous descriptions of the so-called `novelty response'. However, B. niger would also scallop(Serrier and Moller, 1989)quite often. Therefore, the definition of novelty response in the current study includes both accelerations and scallops.

Area and duration were most strongly affected by multimodal stimulation. Fig. 3 compares baseline-subtracted SDFs for different combinations of multisensory and unisensory stimulation. Each row of graphs is from a different fish. In each example, the duration and area of the multisensory NRs are larger than their component unisensory NRs. Overall, the mean area and duration of multisensory NRs were 3.17 and 1.72 times larger than unisensory NRs, respectively. Multisensory NRs from each stimulus type had a significantly greater area than their component unisensory NRs (P<0.01, N=13–21; Fig. 4A). A significant difference existed for NR duration as well, except when comparing Tub+LL to LL(P>0.05, N=16; Fig. 4C). To help address response variation, Fig. 5 compares the within-session distribution of NR duration and area among three stimulus types from a representative fish. In this example, there is a clear distinction between the distributions of multisensory and unisensory responses. For this session, 6 responses to the Tub+Amp stimulation had a duration greater than 2 s (Fig. 5A). By contrast, only three responses to the Tub and zero responses to the Amp unisensory stimuli were of equivalent duration. Similarly, four of the responses to Tub+Amp stimulation had an area of more than seven pulses, while only one response to Tub and zero responses to Amp unisensory stimulation had an equivalent area(Fig. 5B).

Fig. 3B,D,F compare the actual multisensory NR to the calculated sum of its component unisensory NRs. In each example, the actual multisensory NRs have a larger duration and area than the calculated sums of their component unisensory responses. On average,the actual multisensory NRs have a 1.59 and 1.62 times larger area and duration, respectively, than their summed component unisensory NRs. The summary data comparing the area and duration of actual multisensory NRs to the area and duration of their calculated summed NRs are plotted in bar graphs according to stimulus type (Fig. 4B,D). The actual multisensory NRs had a significantly greater duration and area than the calculated summed NRs (P<0.05, N=13-21; Fig. 4B,D). However, the difference in area between the actual Tub+LL response and calculated Tub+LL response was not significant (P>0.08, N=16; Fig. 4B). Interestingly, one fish failed to demonstrate an NR to any of the unimodal stimuli but responded to each multimodal stimulus(Fig. 6). In this example, the supralinear multisensory enhancement of duration and area was most evident.

Although the time of NR maximum was stable across stimulus type(P>0.4, N=13–21), NR amplitude varied significantly. The unimodal Amp stimuli had the weakest effect(1.65±0.90 pulses s^–1, N=21) on amplitude,while the trimodal Tub+Amp+LL stimuli had the strongest effect(6.75±3.28 pulses s^–1, N=13). The largest unisensory amplitude (3.32±1.92 pulses s^–1, N=16) was elicited by the LL stimuli. The multisensory NRs had significantly greater amplitude than their component unisensory NRs(Fig. 7; P<0.05, N=13–21). The difference between the actual multisensory amplitudes and the amplitudes of the summed unisensory responses was not as great. The amplitude of the actual Tub+Amp NR (5.11±2.11 pulses s^–1) was significantly greater than the amplitude of the calculated Tub+Amp NR (3.89±2.64 pulses s^–1, P<0.05, N=21). However, the amplitudes of the actual Tub+LL (5.87±2.62 pulses s^–1) and Tub+Amp+LL(6.72±3.30 pulses s^–1) NRs were not significantly greater than the amplitudes of their summed component unisensory responses(4.89±2.96 pulses s^–1, P>0.05, N=16 and 5.59±3.67 pulses s^–1, P>0.05, N=13, respectively).

Fig. 8A shows how the probability of the NR was calculated from a stimulus session. Mean probability for each stimulus type is outlined in a bar graph(Fig. 8B). The mean probabilities were relatively low for the unimodal Tub (40±27%, N=21) and unimodal Amp (47±26%, N=21) stimuli. However, the stimulus object elicited a relatively high probability LL NR(73±28%, N=16). In fact, the probability from unimodal LL stimuli was significantly greater than the probability from unimodal Tub or unimodal Amp stimuli (P<0.02, N=16). There were no significant differences in NR probability between stimulus types containing LL stimuli (P>0.7, N=16). However, NR probability from bimodal Tub+Amp stimulation was significantly greater than NR probability elicited by its component unimodal stimuli (P<0.01, N=21). NR probability from most of the stimulus types was normally distributed (P>0.1), except for unimodal LL, which was leptokurtotic (P<0.05). A NR probability of 100% did not occur often. None (N=21) of the fish showed 100% probability during unimodal Tub stimulation, while only 4.8% (N=21) of fish showed 100%probability to unimodal Amp stimulation. Bimodal Tub+Amp stimulation yielded an increase, with 100% probability occurring 19% (N=21) of the time. The three stimulus types containing a LL component had the best chance of 100%probability (LL, 31%, N=16; T+LL, 25%, N=16; T+A+LL, 31%, N=13). To evaluate the contribution of NR probability to NR magnitude, the area, duration and amplitude of each mean NR were recalculated using only the within-session responses that crossed the probability threshold. Interestingly, the removal of these below-threshold responses had no significant affect on NR duration across all stimulus types(P>0.05). However, it caused a significant increase in NR amplitude for all stimulus types except unisensory LL (P<0.05). Similarly, only the LL and Tub+Amp+LL responses did not display a significant increase in NR area.

The scallop of B. niger is a stereotypical EOD burst pattern that is characterized by a high instantaneous frequency with a fast rise and fall(Carlson, 2002; Serrier and Moller, 1989). Across all of the sessions of stimulus presentation without the EOD substitute, the probability of having at least one scallop near (1 s before or 2 s after) the object's MLD was only 13.5%. During stimulus presentations with the EOD substitute, the probability increased to 84.5%. The distribution of scallop production during unimodal Amp and unimodal LL stimulation was highly skewed relative to stimulus types containing tuberous input(Fig. 9A). Hence, without the EOD substitute, scallop production was not normally distributed(P<0.001). Sign tests show that the rate of scallop production from unimodal Tub stimulation is significantly greater than unimodal Amp(P<0.001, N=21) and LL (P<0.01, N=16) stimulation. However, there was no significant difference between Amp and LL unimodal stimuli (P>0.3, N=16). Interestingly, the number of scallops near the object's MLD for each multimodal stimulus was significantly greater than the number of scallops during its component unimodal Tub stimulation(Fig. 9B, P<0.05, N=13–21). Fig. 9C,D shows an example of an increase in scallop production near the object's MLD due to the addition of LL stimulation. Due to the short duration (4–6 intervals), yet high frequency, of a scallop's signature burst, the rate of their occurrence was only significantly correlated to NR amplitude. Table 2 shows the correlation coefficient between scallop rate and NR amplitude, duration and area across each stimulus type that contains the EOD substitute.

Within-session standard deviations of the SDFs between the start time and end time of the mean NR were significantly different when comparing stimulus types with EOD replacement to stimulus types without EOD replacement(P<0.05). Importantly, the standard deviations are approximately equal among all stimulus types with EOD replacement (P>0.05). The same is true when comparing stimulus types without EOD replacement (Amp and LL). The average standard deviation in EOD rate for stimulus types with EOD replacement is 3.19±1.36 pulses s^–1. The average standard deviation in EOD rate for stimulus types without EOD replacement is 1.77±0.91 pulses s^–1. Interestingly, the CVs among all stimulus types were approximately equal (P>0.05). Therefore, the significant difference among standard deviations is likely to be caused by the significantly larger scallop rate with EOD replacement compared with trials without EOD replacement. Since the CV is normalized, the effect of the jump in NR amplitude caused by scalloping is reduced. The average CV in EOD rate for stimulus types with EOD replacement is 0.47±0.18. The average CV in EOD rate for stimulus types without EOD replacement is 0.47±0.24. Therefore, differences in within-session variation cannot explain differences in response magnitude among stimulus types.

Overall, the within-session temporal alignment (σ) of SDF maxima was found to be approximately equal between multisensory responses and their component unisensory responses (P>0.05). Temporal alignment to Tub+Amp stimulation (σ=0.60, N=21) was not different from that obtained with either Tub (σ=0.47, N=21) or Amp (σ=0.38, N=21) stimulation (P>0.05). Unimodal LL stimulation,however, elicited relatively tight temporal alignment (σ=0.30, N=16). This, coupled with the prevalence of scalloping during EOD replacement, led to significant differences when responses to Tub+Amp+LL(σ=0.60, N=13) or Tub+LL (σ=0.51, N=16) were compared with responses to LL stimulation (P<0.05). Scallops spread the temporal alignment of SDF maxima because the occurrence of their signature burst is not limited to a specific phase of the NR.

We were able to determine a rudimentary relationship between stimulus strength and NR magnitude before arriving at the stimulus parameters used in the main body of evidence of the present study. During a set of experiments where the MLD was 1.8–2.0 cm and the DC voltage was greater than 600μVcm^–1, much larger and more consistent unisensory responses were elicited that integrated linearly or sublinearly with the other sensory modality (Fig. 10A–D). To allow for the possibility of elucidating supralinear multisensory enhancement, the stimuli were weakened. Eventually,we discovered that a stimulus object distance of 2.4–2.7 cm often elicited relatively weak and/or inconsistent unisensory tuberous NRs. After this distance was determined, we discovered that unimodal Amp stimuli of approximately 200–400 μVcm^–1 generally elicited weak and/or inconsistent NRs. The vast majority of the statistically analyzed data in the current paper is within these parameters. The relationship between stimulus strength and response strength was directly related during unimodal Amp stimulation (Fig. 10E). However, during LL stimulation, the relationship seemed less plastic and susceptible to response suppression (Fig. 10F).

The aim of the current study was to evaluate the role of multisensory integration in moving object recognition. We discovered that multisensory stimulation can result in a supralinear enhancement to moving object recognition. Specifically, the duration and area of multisensory NRs were significantly greater than the NR duration and area calculated from the arithmetic sum of their component unisensory responses(Fig. 4B,D). This finding is unique because it is based on a comparison between the direct sum of two continuous, scalar unisensory behavioral responses and their actual multisensory equivalent (Fig. 3B,D,F).

NR duration and area provide logical indications to the perceived value or saliency of a stimulus. Intuitively, they represent the length and the sum of increased energy expenditure in the currency of EODs. If the function of the NR, like other orienting responses, is to augment sensory acquisition(Spinks et al., 1985), then the fish would logically expend greater energy augmenting sensory information that has a greater perceived value. In the natural environment, common moving objects such as fish or crustaceans, are likely to stimulate more sensory modalities than stationary inanimate objects. For example, a stationary fish would seldom, if ever, encounter a nearby moving object that stimulates the Amp electrosense and not the LL or Tub electrosense. Therefore, supralinear multisensory enhancement may be driven by the ecological significance or qualitative relevance of multimodal stimuli. However, it is not clear whether a nonlinear relationship between stimulus strength and NR duration can explain supralinear enhancement. Post and von der Emde showed what appears to be the beginning of a sigmoidal response curve in the visual sense but a decaying curve in the auditory sense as stimulus strength increases(Post and von der Emde, 1999). LL stimulation stronger (closer object distance) than that applied in the current paper often caused the fish to suppress its pulsing behavior; however,unisensory ampullary responses remained directly related to DC voltage(Fig. 10F). Therefore, the relationship between stimulus strength and NR magnitude is likely to be different between mechanosensory and electrosensory modalities. Nonetheless,it is clear from the main results of the present study that adding electrosensory stimulation to LL stimulation (i.e. Tub+Amp+LL), which effectively increases overall `stimulus strength', does not mimic LL response suppression, but causes response enhancement. However, a multimodal stimulus containing an NR suppressing LL component and a 100% effective bimodal Tub+Amp component still causes response suppression(Fig. 10E). Perhaps a hierarchy exists, whereby the relative weight of each sensory modality dictates the dynamics of the multisensory response curve. Nonetheless, as the strength of the unisensory stimuli cross a threshold level, additional stimulus modalities become less influential to NR magnitude(Fig. 10A–D).

The augmentation of perception from the integration of two sub-threshold unimodal stimuli (Ramos-Estebanez et al.,2007) is virtually unexplored in electric fish. However, in the current study, some fish gave a weak or undetectable NR to one or all of the unimodal stimuli but responded fairly well to their multimodal combination(Fig. 6 and Fig. 3C). Perhaps these data offer additional support for the multimodal stimulus object having a greater ecological significance or some form of qualitative uniqueness. It is also possible that these particular fish required a combination of stimulus modalities to satisfy a higher than normal NR threshold. Nonetheless, a physiological counterpart to this form of gating was discovered in the optic tectum of the rattlesnake, whereby certain neurons were virtually unresponsive to unimodal stimuli but responded to the bimodal visual–infrared combination (Newman and Hartline,1981). For a predator like the rattlesnake that specializes in endothermic prey, the ecological significance of such a neuron is evident.

In weakly electric fish, multisensory neurons have been identified in the tectum mesencephali (Bastian,1982; Bleckmann and Zelick,1993; Heiligenberg and Rose,1987) as well as in the cerebral pallium(Prechtl et al., 1998). While the NR is ultimately regulated by output from the EOD command nucleus(Carlson, 2002), the relative importance of either the tectum or pallium to NR magnitude is unknown. Future research in our lab will focus on exploring whether multisensory integration in mid-brain neurons can, at least partially, explain the supralinear enhancement to the NR discussed in the present study. The cerebral pallium may also be involved because the ablation of two association cortices precludes the development of multisensory enhancement of behavioral(Jiang et al., 2007) and neuronal responses (Jiang et al.,2006) in the cat. Therefore, it is similarly possible that multisensory enhancement of the NR is derived from a sensory process formed from the interaction between the mid-brain and cerebral pallium.

Multisensory integration had a significant, positive effect on the amplitude of the NR (Fig. 7). The amplitude of each multisensory NR was significantly greater than the amplitude of its component unisensory NRs. However, only the amplitude of the Tub+Amp NR was significantly greater than the amplitude of its arithmetically summed unisensory components. Therefore, the level of LL stimulation applied in the present study was probably too close to saturation of NR amplitude for additional stimuli to elicit a supralinear increase. Under curare, the EOD rates during a typical NR acceleration normally had a ceiling around 10 pulses s^–1. EOD bursts above this rate were typically in the form of the high-frequency (30–80 Hz) burst component found within a scallop. However, due to the brevity (4–6 intervals) of this burst component, it contributed little to the overall amplitude of the SDF(Fig. 2A). Therefore, the lack of supralinear enhancement in NR amplitude may result from a short behavioral ceiling on acceleration-like changes, since all stimuli greater than or equal to causing this ceiling effect would elicit approximately equal NR amplitudes.

The rate of the NR's occurrence near the MLD of the stimulus object was dependent upon stimulus type. Probability to unimodal Tub and unimodal Amp stimuli was relatively low while being relatively high for unimodal LL stimuli(Fig. 8B). Bimodal Tub+Amp stimulation yielded a significantly higher probability than its component unimodal stimuli. However, NR probability was approximately equal between all stimuli containing the LL modality. Therefore, unimodal LL stimulation had a strong effect on probability and could not be augmented by the addition of electrosensory input. It is interesting that the same strength of LL influence did not extend to NR magnitude. Perhaps the centers of LL sensation in the brain are tightly linked to the NR decision process but, to a lesser extent,to the determination of NR magnitude (Fig. 8B,D). Therefore, the relationship between LL stimulus strength and NR probability might be fairly steep.

NR duration, area and amplitude were recalculated after removing the within-session trials below the probability threshold. No significant change occurred in NR duration. Therefore, it is likely that the multisensory enhancement of NR duration is not simply caused by an increase in response probability. Not surprisingly, a significant inflation of NR amplitude and area occurred in all but the most consistent responses. Therefore,multisensory enhancement of NR area can be partially explained by an increase in NR probability.

Although an individual's tendency to scallop may be relevant to social interactions (Moller et al.,1989; Serrier and Moller,1989), the current study provides evidence that scalloping is involved in electrolocation. With the EOD substitute, scallop production regularly occurred when the object was near its MLD. Scallop production between 1 s before and 2 s after the object's MLD was significantly greater during each multimodal stimulus than its component unimodal tuberous stimulus(Fig. 9B). However, scallop production was very rare without the EOD substitute and had a highly skewed distribution among subjects (Fig. 9A). In fact, during unimodal LL stimulation, only one of the 16 fish scalloped during the aforementioned time window. Therefore, the rate of scallop production significantly increased during multimodal stimulation and relied heavily on tuberous stimulation. Even though the rate of scalloping significantly varied with stimulus type, it had no correlation to NR duration. Not surprisingly, scallop rate was significantly correlated to NR amplitude(Table 2). The scallop's signature burst may function as a complementary method of electrosensory scanning that provides a higher temporal resolution than accelerations but with minimal cost due to its brevity.

Our `single object' experimental paradigm provides evidence that the spatial and temporal congruence of multiple stimuli is sufficient to induce multisensory enhancement in electric fish. However, are the spatial determinants to multisensory enhancement for electro-mechanically and audio-visually elicited orienting responses(Stein et al., 1988; Whitchurch and Takahashi,2006) similar? The Tub, Amp, and LL (canal) modalities are interesting because they are all confined to the near field. By contrast,vision and audition both work over a much broader spatial range. Therefore,electrosensory and lateral line systems are less likely than the visual and auditory systems to integrate spatially incongruent stimuli segregated by two unrelated, yet temporally coincident events. Additionally, a small, point source visual stimulus, such as a flying insect, produces a relatively diffuse auditory stimulus. Yet, a similarly sized Daphnia would produce electrosensory and mechanosensory stimuli that attenuate at equal rates(Coombs et al., 2002; Nelson et al., 2002). This natural distinction may have a functional significance to the spatial determinants of multisensory enhancement. Perhaps multisensory neurons in electric fish have much narrower receptive fields. Yet, in the mid-brain of electric fishes, nothing is known about the receptive field of multisensory neurons. Moreover, very little is known about the relationship between the spatial registration and convergence of electrosensory and mechanosensory sensory space in neurons of the pallium of electric fish(Prechtl et al., 1998). Therefore, it is not clear how combining `spatially incongruent'electrosensory and mechanosensory stimuli would affect sensory integration and perception. The NR could possibly serve as a probe in determining the spatial boundaries of multisensory enhancement by studying the relationship between NR magnitude and the physical disparity between sensory stimuli.

The current study provides evidence that B. niger predictably increases its NR magnitude according to the number of sensory modalities present in a single moving object. Since the NR has energetic costs and theoretically improves electrolocation, it is logical to speculate that a multimodal moving object is perceived as more valuable than a moving object comprised of one of its unimodal components. Since this increase in stimulus value, measured by NR magnitude, was supralinear, multimodal object stimuli may cause a qualitative change in perception not elicited by unimodal object stimuli, which likely lack the same ecological significance.

LIST OF ABBREVIATIONS

 
• Amp

  ampullary electrosense

 
• EMN

  electromotor neuron

 
• EOD

  electric organ discharge

 
• LL

  mechanosensory lateral line

 
• MLD

  minimum lateral distance

 
• NR

  novelty response

 
• SDD

  spike density derivative

 
• SDF

  spike density function

 
• Tub

  tuberous electrosense

The authors are grateful to Dr Bruce Carlson for making helpful suggestions throughout the preparation and completion of this project. Financial Support was provided by grants from the National Science Foundation. S.R.P. would like to thank his graduate committee for their continued support.