Basic and complex emotion recognition in children with autism: cross-cultural findings

Autism spectrum conditions (ASC) are neurodevelopmental conditions characterized by social communication and interaction difficulties, circumscribed interests, and a preference for sameness and repetition. Individuals with ASC experience difficulties processing and interpreting socio-emotional cues [1–3]. These emotion recognition (ER) difficulties are recognized as part of the social communication challenge, characteristic of ASC [4] and predict adaptive socialization skills [5]. The recognition of others’ emotions and mental states relies on the integration of emotional cues from various channels. These include facial expressions, vocal intonation, and body language, as well as contextual information [6].

Due to its centrality in emotional expression, the majority of ER research has focused on facial expression. In typical development (TD), the skills necessary to recognize and discriminate between emotional expressions are present from infancy [7, 8]. Throughout development, facial ER improves, with children gradually becoming “emotion experts” [9]. In contrast, children and adults with ASC show reduced attention to faces and facial expressions [10–13]. Altered attention, specifically to the eye region of the face [14, 15], has a significant effect on facial affect recognition in individuals with ASC [16]. In addition, children with ASC process faces in a piecemeal fashion rather than holistically [17–19] (see [20], for a different perspective). Consequently, children with ASC show reduced “face expertise” [11].

Some studies of facial ER have reported deficits in participants with ASC [21–24]. Other studies, however, have found facial emotion recognition skills in children with ASC to be equal to those in TD peers [25–28]. Recent review of facial expression in ASC had proposed that these differences in documented ER are dependent on participant demographics, task selection, and stimulus type [29]. Despite the extensive research on emotion recognition through facial expressions in ASC, most studies have either used still, rather than dynamic, stimuli [29, 30], allowed participants unlimited exposure to stimuli [31–33], or else limited their research to a narrow set of basic emotions [34, 35].

Another important channel of emotional information is prosody, which includes vocal intonation, stress, and rhythm [36]. Prosody plays an important role in linguistic functions as well as in emotional expression and comprehension [37, 38]. Whereas considerable research has documented the deficits children with ASC have in language production and comprehension, relatively few studies have examined the abilities of these children to recognize emotions through prosody [38, 39]. Studies on ER from prosody in individuals with ASC have yielded mixed results. Some studies have reported that adults or children with ASC experienced difficulties in using emotional prosody to identify the emotions of others [22, 36, 38, 40–43]. In contrast, several studies have reported that the emotional prosody perception in individuals with ASC is similar to that of TD controls [37, 44–47]. These mixed results could be due to methodological reasons [45], such as the use of cross-modal (e.g., face-voice) paradigms [48, 49] or the use of speech stimuli in which ER could rely on linguistic, rather than prosodic cues [3, 50].

A third channel of relevant emotional cues is body language, which includes gestures and postural changes. TD infants as young as 4 to 6 months show distinct attention to human biological motion [51]. At 2 years of age, children already rely on body language cues to predict the behavior of people they are watching [52], understand the goals of actions they observe [53], and anticipate people’s intentions [54]. At 3 years of age, children can verbally interpret biological motion, an ability that peaks by about 5 years of age [55]. Finally, children as young as 4 years successfully use body language to recognize social interactions, an ability that peaks by the age of 7 to 8 years [56].

In ASC, studies demonstrate difficulties discriminating biological from non-biological motion [57–59] and no preference for biological motion over object motion [60, 61]. Studies of emotional body language comprehension in ASC are scarce. The existing ones reported difficulties recognizing emotional, but not non-emotional motion from point-light displays in individuals with ASC, compared to TD controls [62, 63]. Children with ASC can distinguish social from non-social scenes, solely based on whole-body movements presented in point-light and stick-figure displays [56], but fail to interpret the nature of the actions presented, especially in the case of social scenes [56, 64–66]. Hence, the body motion recognition deficit in ASC appears to be restricted to percepts requiring mentalization and emotional interpretation [67–69].

Most studies that examined the ability of people with ASC to understand body language have either used stimuli such as point-light displays [67–69] or stick figures [56] or limited their investigation to a narrow set of basic emotions [24, 70]. It is therefore important to investigate whether this documented deficit exists in full body video stimuli, demonstrating a wider range of emotions, including complex ones.

In addition to the unimodal emotion-processing deficits described above, individuals with ASC show problems integrating information across multiple modalities [71]. Many of the atypical perceptual experiences reported by people with ASC may stem from an inability to efficiently filter, process, and integrate information from different modalities, when presented simultaneously [72].

Most studies that examine multimodal processing focus on the integration of auditory and visual social stimuli linked to communication, such as speech and its corresponding lip movements [73, 74]. The results of most of these studies indicate that the ability to integrate audiovisual social stimuli is impaired in individuals with ASC [75–78]. On the other hand, there is little research on the integration of information from facial expressions and body language or from multiple modalities in context. Multimodal integration ER studies reveal that presenting emotional cues in different channels does not necessarily help, and may even hamper the ability of individuals with ASC to recognize emotions and mental states [79, 80]. Previous studies of multimodal ER in ASC have shown deficits in adults [81] and in children [82]. However, the stimuli presented in these studies included verbal content, in addition to the visual and auditory emotional cues. Since individuals with ASC rely on verbal content as a compensatory mechanism [25], an examination of multimodal ER in children with ASC, in which stimuli have no verbal content to rely on, is desirable. In the absence of a linguistic component, such stimuli could also be used for cross-cultural comparison of ER in ASC.

As described above, studies showing no ER differences between TD individuals and individuals with ASC focus mostly, if not solely, on basic emotions [24, 34, 35, 70]. The six emotions referred to as “basic” (happiness, sadness, fear, anger, surprise, and disgust) are suggested to be cross-culturally expressed and recognized [83] and are to some extent neurologically distinct [84]. Unlike basic emotions, complex emotions are considered more context and culture dependent [85, 86]. They involve attributing a cognitive state as well as an emotion and may be belief- rather than situation-based [87]. The examination of basic ER in children with ASC yielded mixed results, with some studies reporting basic ER deficit in this group [21, 49, 88], whereas others reporting no difficulties in recognition of the basic emotions in children with ASC [25, 26, 28]. In contrast, studies investigating recognition of complex emotions and other mental states by children with ASC, compared to TD children, have shown more conclusive deficits [3, 89]. However, no study so far has conducted a direct comparison of basic and complex ER in children with ASC and their TD peers.

In terms of cross-cultural ER differences, meta-analyses have documented evidence for an in-group advantage, in that emotional expressions are more accurately recognized by individuals within the same culture, versus other cultures [90, 91]. However, individuals with ASC have been found to have poorer understanding of emotional cues (as shown above), as well as lower sensitivity to social cues [10] and lower social conformity [92]. Since individuals with ASC are less socio-emotionally sensitive within their own cultures, it is possible that cross-cultural differences in ER, if found, would be smaller in the ASC groups than in the TD groups.

The current study aimed to compare the recognition of the six basic and 12 complex emotions by children with ASC and TD controls. We examined ER unimodally through faces, voices, and body language, as well as multimodally through integrative scenarios with no verbal content and tested basic and complex ER cross-culturally in three different countries: Israel, Britain, and Sweden. We assessed differences between 5 to 9-year-old children with autism in the average IQ range and TD controls, comparable for age, sex, and IQ. We predicted that (a) the ASC group would have lower scores on the different ER tasks, compared to controls; (b) basic emotions would be recognized more easily than complex emotions; (c) the group differences would be greater for complex ER than for basic ER; (d) cultural differences between the three sites would be greater for complex ER than for basic ER; and (e) cultural differences on ER would be greater in the TD groups than in the ASC groups.

The current study tested if there are differences in the recognition of basic and complex emotions between children with ASC and typically developing children across three countries: Israel, Britain, and Sweden. Emotion recognition (ER) was tested using dynamic facial expressions, body language, vocal expressions, and integrative scenarios. Children with ASC showed ER deficits in all three modalities and their integration in context. These deficits were found cross-culturally. Participants with ASC showed ER deficits in both basic and complex emotions. Cross-cultural agreement was found for all major findings, with minor deviations, reported below. Our findings highlight the multimodal nature of ER deficits in ASC, which exists for basic and complex emotions and is relatively stable cross-culturally.

As predicted, the ASC group had more difficulties in ER from facial expressions, body language, vocal expressions, and integrative scenarios, compared to the TD participants, even after controlling for age and verbal IQ. The significant group differences on task scores replicates previous findings of difficulties among individuals with ASC on emotion and mental state recognition from visual, auditory, and contextual stimuli [22, 24, 40, 43, 56, 66, 76–78]. These findings provide further support for ER and mentalizing deficits in children with ASC, which are evident cross-culturally.

As predicted, our findings show that basic emotions were recognized more easily than complex emotions on all modalities, regardless of child’s diagnosis. However, contrary to our hypothesis, group differences for complex ER were not greater than those found on basic ER. With the exception of an interaction effect between complexity and group in the voice task, our findings showed ER deficits in 5–9-year olds with ASC exist over and above complexity level. These findings match some reports on basic ER deficits in high-functioning children with ASC [23, 24, 35, 41–43, 70] and reports on complex ER deficits in this group [3, 40, 82]. The deficit we describe in the ASC group in basic ER, which was not found in some other studies (e.g., [28, 46]), may result from the age group tested. Whereas in older age, children with ASC may develop compensatory mechanisms allowing them to recognize basic emotions [25], our examination reveals a comprehensive ER deficit in 5–9-year-old children with ASC.

Another finding of developmental importance was the age by complexity interaction found in the current study. This interaction, found over and above group, revealed moderate correlations between age and ER in complex emotions, across all modalities, suggesting 5–9-year olds are still developing their ability to recognize complex emotions, comprising mental, as well as situational, aspects. Weaker correlations were found for age with basic ER in the voice, body, and integrative tasks, but not on the face task, suggesting basic ER skills have matured for at least some of the children in the tested age group. Interestingly, our findings show that the facial ER skills of basic and complex emotions alike continue to develop in this age group. In view of the centrality of facial expressions in ER, these findings provide an interesting example of the continuing development of facial expertise [9].

It is important to note that all of our visual stimuli used video clips, rather than still pictures. It is possible that processing of dynamic stimuli, which are more ecological in nature, may be more challenging to children with ASC, in comparison to still images. Specific deficits in processing of dynamic stimuli [107] may explain why our paradigm, which employed only dynamic stimuli in the visual channel, had demonstrated ER deficits in basic, as well as in complex, emotions in the visual tasks, but only complex ER deficits in the auditory task.

A striking finding in our study was the lack of major cultural differences. Our hypotheses that complex ER would show greater cultural variability than basic ER and that TD children would show greater cultural variability than children with ASC were not supported. These findings stand in contrast with previous arguments about greater cultural diversity in complex ER [85, 86]. It is possible that such cultural differences would be more salient in older age groups, when the acquisition of culture-specific nuances has been completed.

However, some cultural differences were found on the face and body language tasks. For facial ER, children with ASC in Sweden had performed more poorly than TD controls only on complex emotions and not on basic ER. This was in contrast to the British and Israeli ASC groups, which showed facial ER deficits both for basic and complex emotions. In addition, a significant interaction was found between country and complexity in the body language task. The analysis revealed that regardless of diagnosis, participants in Sweden had more difficulties recognizing complex, compared to basic, emotions from body language. This difference was not found in Israel and Britain. It should be noted that the body task stimuli were filmed in Britain, comprising actors of various ethnicities. Whereas the basic, more automatic and cross-cultural emotions were easily recognized in Sweden, as they were in Britain and in Israel, the more subtle, complex manifestations of emotion in body language may have been more challenging for children in Sweden. An examination of body language emotions, performed by Scandinavian (in comparison to non-Scandinavian) actors should be conducted in order to support (or refute) this potential findings of an in-group advantage [108].

A few limitations of the study should be noted: The current study examined cultural differences in ER as part of a cross-cultural examination of differences between individuals with and without ASC. Whereas the sample size is sufficiently large for the main question of the study, a larger sample of TD individuals within each country may be needed in order to appropriately examine ER cultural differences in the general population. The examination of ER cultural differences (and ASC-TD differences) in the various modalities should also be extended to additional, non-western cultures, such as African, or Eastern cultures [90], as these may reveal greater cross-cultural differences than the ones reported here.

In the current study, we were unable to examine ER gender differences, due to the relatively small number of females in the ASC groups. Gender differences in ER were found among adult participants with ASC in some studies [35, 40] but not in others [109] A replication of the current study with larger samples could examine the existence of gender differences in children with ASC as well.

Another limitation lies in some methodological variability across the sites and more specifically to the administration of assessments at home vs. the lab. These differences had resulted from participants’ difficulties to attend the lab in Israel and in Britain (due to distance and to parents’ needs). Hence, the clinical teams traveled to children’s homes. Despite the attempts to maintain a standardized testing environment (e.g., standardized protocol, individual assessment in a quiet room), it is possible that the different location for assessments had affected the results of this study. However, since the ASC and TD groups within each country were tested under similar conditions, it is unlikely that the different testing conditions have affected the group differences. Nonetheless, we recommend further studies to endorse more standardized structure if possible.

Our findings demonstrate a supra-modal emotion recognition deficit in children with ASC, cross-culturally, for basic and complex emotion alike. Although the ER skills of children with ASC improve with age, like their TD peers, this overall ER deficit persists and calls for interventions [110, 111] which might narrow the developmental ER gap between children with ASC and their TD peers at the earliest possible stage.