Viewpoint-independent face recognition via extrapolation in paper wasps Free

Visual recognition of three-dimensional signals is challenging because visual signals appear different when seen from different viewpoints or distances, or under different conditions (Echeverri et al., 2021; Riesenhuber and Poggio, 2000; Tarr and Cheng, 2003; Tarr et al., 1998). Differences in viewing angle have a particularly strong impact on the appearance of three-dimensional signals (Tarr and Pinker, 1989; Hancock et al., 2000; Peissig and Goode, 2012). For example, the front and profile view of a human face have very different characteristics. In fact, the front views of two different people are more similar than the front and side views of the same person (Hill et al., 1997). Nevertheless, humans are adept at identifying individuals even when the faces are presented from different viewing angles (Burke et al., 2007; Young and Burton, 2018). The way animals recognize three-dimensional objects despite variation in viewing angles and conditions has been a focus of research in psychology, computer science and neuroscience (Riesenhuber and Poggio, 2000; Zhao et al., 2003). However, less is known about three-dimensional recognition in non-model systems and how different viewing angles influence receiver perception and recognition behavior.

Animals could use multiple different mechanisms to identify three-dimensional objects. Animals with viewpoint-dependent recognition always view stimuli from a particular perspective. Viewpoint-dependent recognition has limited flexibility, though it is likely less cognitively challenging than other recognition mechanisms (Hill et al., 1997; Tarr et al., 1998). Animals could also learn multiple views of each object such that known objects can be identified from multiple viewpoints. Learning multiple views requires experience with particular objects from multiple views as well as substantial memory investment to allow recall of many different views. Animals could also generalize based on previous experience, such that they can identify objects from multiple angles regardless of experience with a particular object at a particular angle. Image interpolation allows animals to identify novel views that fall within known views, while extrapolation allows identification of novel views that fall outside known views (Biederman and Gerhardstein, 1993). Vertebrates use both interpolation and extrapolation to generalize over different viewpoints of the same individual (Bülthoff and Edelman, 1992; Logothetis et al., 1994; Spetch and Friedman, 2003). To our knowledge, only one previous study tested how insects recognize rotated objects, finding that honeybees are relatively poor at identifying novel views of complex objects. After sufficient experience, honeybees can learn to recognize novel viewpoints via image interpolation (Dyer et al., 2005).

Polistes paper wasps provide a good model system to study how viewing angle influences visual recognition because they have three-dimensional visual signals of individual identity (Fig. 1). Wasps learn the unique faces of other wasps and use individual recognition during interactions on and off nests to manage dominance interactions and division of work, reproduction and food (Sheehan and Tibbetts, 2008; Tibbetts, 2002; Tibbetts et al., 2020b). Thus far, no research has explored how viewpoint influences wasp face recognition. Instead, previous work has focused on either social interactions between live wasps, which allows viewing from multiple perspectives, or training, where wasps are trained and tested on pictures of conspecifics from a single, front perspective.

Here, we tested how wasps apply the concept of sameness and difference to images of wasps taken from different viewpoints. Previous work has shown that P. fuscatus can learn a general concept of sameness or difference and apply the concept to new samples and types of stimuli (Weise et al., 2022). During training, wasps are exposed to two stimuli concurrently that are either identical (AA) or non-identical (BC). Then, wasps must select a novel stimulus set representing the same relationship (same or different) between stimuli as the trained stimulus pair. For example, an animal trained to select a pair of different stimuli would be asked to choose between (DD) and (EF) with (EF) being the correct choice. We trained wasps using pairs of colors, then tested their capacity to apply the same/different concept to pairs of face pictures taken from different perspectives (Fig. 1, 0/0, 0/30 or 0/60 deg rotation). For example, wasps were asked to choose between two sets of pictures: (1) two pictures of the same wasp taken at different angles (wasp 1 at 0 deg and 30 deg rotation) and (2) two pictures of two different wasps taken at different angles (wasp 2 at 0 deg rotation and wasp 3 at 30 deg rotation). If wasps are capable of viewpoint-independent recognition, we predict wasps would apply the concept of sameness and difference to different viewpoints of the same wasp. Therefore, the wasp will choose the pair of face pictures with the correct relationship. However, if recognition is viewpoint dependent, we predict wasps would not be able to apply the concept of sameness and difference to novel viewpoints of wasp faces. Therefore, the wasp will choose randomly.

Polistes fuscatus (Fabricius 1793) queens were initially trained to approach either ‘same’ or ‘different’ color stimuli (Fig. 1) using methods similar to Weise et al. (2022). During training, wasps were placed in a 4 cm wide×4 cm long×0.7 cm high wood and Plexiglas box with color stimuli glued onto the inside walls. Wasps were alternately exposed to ‘same’ stimuli, where the box had identical color squares on the walls, and ‘different’ stimuli, where each side of the box had two different color squares on the walls. For half the wasps, the different stimuli were associated with a mild shock (CS−) and the same stimuli were associated with safety (CS+). In the other half the trials, the same stimuli were associated with a mild shock (CS−) and the different stimuli were associated with safety (CS+). Electric shock was provided by an electrified pad made of anti-static conductive foam electrified by two copper wires connected to a Variac transformer. The pad provided a continuous 0.4 V AC current. In trials with no shock, the Variac transformer was turned off so the foam was not electrified. Between each 2 min bout, the wasp was given a 1 min break in a holding container. This sequence of one CS+ and one CS– bout was repeated five times per wasp, so wasps experienced five CS+ and five CS− bouts in total. Wasps were trained using five pairs of same stimuli and five pairs of different stimuli, a unique set of stimuli for each bout of training. After training, the wasp was given a 45 min break in a holding container with access to sugar and water.

Wasps were tested with pairs of P. fuscatus face images (Fig. 1; Fig. S1). Face images were photographs of wasps from Michigan, USA. Pictures were taken head-on (0 deg rotation), rotated 30 deg and rotated 60 deg. Performance was assessed by measuring whether the wasp approached the correct or incorrect image pair over 10 trials. Testing occurred in a 3 cm wide×10 cm long×0.7 cm high rectangular box. One end of the rectangle displayed a pair of face images from the same wasp, while the other end of the rectangle displayed a pair of face images from two different wasps. Pairs of face images were either entirely unrotated (0/0 deg) or had one face unrotated and one face rotated (0/30 deg, 0/60 deg). The center of the rectangle had two removable, clear partitions that confined the wasp. At the beginning of each trial, the wasp was placed in the center of the rectangle between the clear partitions, the two partitions were removed simultaneously, and the wasp was free to walk through the rectangle. A choice was scored when the wasp's head and thorax entered one of the 2.5 cm zones closest to each stimulus, at each end of the rectangle. Wasps typically chose rapidly, within 2 s. After a wasp made a choice, it was removed from the testing arena and given a 1 min break in a holding container. The placement of the stimuli (right or left side) was determined randomly and changed between trials. This ensured that wasps did not associate a particular direction with correct choices.

The data were analyzed in SPSS v.29. Learning was measured as the total number of correct choices (out of 10). We used a general linear model to compare choice accuracy. The dependent variable was the number of correct choices (out of 10). The independent variables were degree of rotation (categorical: 0/0, 0/30, 0/60 deg), whether same or different was associated with safety (categorical: same safe, different safe), and their interaction. The interaction was not statistically significant, so it was removed from the final model. We also compared the number of correct choices with the 50:50 random expectation using binomial tests. The binomial test provides an exact test of whether the number of correct versus incorrect choices differs from a 50:50 random expectation. Binomial tests provide P-values with no test statistics. A total of 46 wasps were trained (n=7 0/0 deg different, n=6 0/0 deg same, n=9 0/30 deg different, n=8 0/30 deg same, n=8 0/60 deg different, n=8 0/60 deg same).

This study illustrates that P. fuscatus wasps apply the concept of sameness and difference to novel views of conspecifics via extrapolation. Wasps identify a pair of pictures of the same wasp as the ‘same’, even if the pictures are taken from different viewpoints (e.g. a pair consisting of one face picture at 0 deg rotation and one face picture at 60 deg rotation). Accuracy was not influenced by whether wasps compared a pair of unrotated faces or a pair with one unrotated face and one face rotated 30 or 60 deg (Fig. 2; F_2,42=1.4, P=0.29). Accuracy was also not influenced by whether wasps were trained to approach pairs of the same face or pairs of two different faces (F_1,42=0.78, P=0.41). In all cases, wasps chose the pair with the correct relationship more often than expected by chance. When the data are split into six groups (0/0 deg same, 0/0 deg different, 0/30 deg same, 0/30 deg different, 0/60 deg same, 0/60 deg different), each group made more correct choices than the 50:50 random expectation (all P<0.001). Our data suggest that wasps use image extrapolation rather than interpolation because wasps had never previously encountered the individuals in the pictures and had no opportunity to view the faces from different viewpoints prior to testing. Nevertheless, wasps quickly identified novel views of the same wasp.

To our knowledge, this study provides the first evidence of image extrapolation in a non-vertebrate. Viewpoint-independent recognition may be based on three different mechanisms. Receivers could learn multiple views of each object, use interpolation to identify novel views that fall within stored views, or use extrapolation to identify novel views that fall outside stored views (Biederman and Gerhardstein, 1993). Vertebrates use both interpolation and extrapolation to generalize over different viewpoints of the same individual (Bülthoff and Edelman, 1992; Logothetis et al., 1994; Spetch and Friedman, 2003). One previous study tested how insects recognize rotated three-dimensional objects, finding that honeybees are relatively poor at identifying novel views of complex objects. After sufficient experience, honeybees can learn to recognize novel viewpoints by interpolating between or ‘averaging’ views they have experienced (Dyer et al., 2005). For example, bees trained to 0 and 60 deg rotated human faces are able to identify 30 deg rotated human faces. However, bees trained with only 0 deg rotated human faces or bees trained with only 60 deg rotated faces are unable to identify 30 deg rotated faces. In contrast to bees, wasps were able to identify novel 30 and 60 deg rotated wasp faces without any previous experience, consistent with image extrapolation.

Polistes fuscatus wasps may excel at identifying novel views of conspecific faces because individual face recognition is an important aspect of their social lives. Wasps rely on individual face recognition to manage social relationships both on and off nests (Tibbetts, 2002; Tibbetts et al., 2020). Previous work has shown that wasps have evolved specialized cognitive mechanisms for identifying conspecific faces that are not used to identify non-face objects (Sheehan and Tibbetts, 2008) or heterospecific faces (Tibbetts et al., 2021). Effective social recognition requires wasps to rapidly and accurately identify conspecifics from multiple views. Perhaps the ability to rapidly identify novel viewpoints of conspecific faces is another cognitive adaptation for individual face recognition. Future comparative work that assesses viewpoint-independent recognition of diverse stimuli in related species will provide insight into whether extrapolation is a widespread ability found in many insect species and contexts or a specialized ability restricted to conspecific faces in P. fuscatus.

Previous work has shown that experience enhances recognition, so it is notable that P. fuscatus can recognize novel viewpoints of unfamiliar wasp faces. Previous work in humans and non-human primates has shown that experience viewing faces from multiple perspectives enhances face recognition. Familiarity enables the formation of robust individual face representations that apply across different perspectives and viewing conditions (Hancock et al., 2000). As a result, humans and chimpanzees exhibit more accurate recognition of novel viewpoints for familiar faces than for unfamiliar faces (Parr et al., 2011). In this study, P. fuscatus were tested using pictures of unfamiliar individuals they had never previously encountered. Nevertheless, P. fuscatus rapidly and accurately identified whether pairs of face images were from the same or different wasp. Future work that compares performance on both familiar and unfamiliar conspecific faces will be useful to assess whether familiarity also improves performance in paper wasps.

This study is different from other work on viewpoint-independent recognition because we used same/different concept learning to assess whether wasps perceived different viewpoints of the same wasp as the same. Concept learning provides a useful method to assess how animals form and use concepts of conspecifics. Our results suggest that P. fuscatus likely form a concept of individual conspecifics that incorporates multiple viewpoints of the same individual. Previous work has shown that receivers often form individual concepts that incorporate multiple aspects of conspecific phenotype. In humans, this concept incorporates traits such as an individual's face, gait and voice (Latinus and Belin, 2011; Yovel and O'Toole, 2016). Horses and lions form cross-modal representation of known individuals based on auditory, acoustic and visual information (Gilfillan et al., 2016; Proops et al., 2009). These complex and multifaceted individual representations likely facilitate accurate recognition across diverse situations. There is much potential for recognition research using concept learning as a window into the information that receivers incorporate into their ‘gestalt’ perception of conspecific phenotypes.

Overall, P. fuscatus can apply a concept of sameness and difference to both rotated and unrotated conspecific faces. Evidence of image extrapolation in P. fuscatus suggests that extrapolation may be a more taxonomically widespread mechanism for viewpoint-independent recognition than previously thought. Extrapolation is thought to be a relatively sophisticated behavior that requires cognitive flexibility, so the capacity for extrapolation in P. fuscatus is noteworthy given their relatively small brain size (wasps <1 million neurons; pigeons 310 million neurons, macaque 6 billion neurons) (Menzel and Giurfa, 2001).