Atypical emotional anticipation in high-functioning autism
© Palumbo et al. 2015
Received: 17 March 2015
Accepted: 27 July 2015
Published: 15 August 2015
Understanding and anticipating others’ mental or emotional states relies on the processing of social cues, such as dynamic facial expressions. Individuals with high-functioning autism (HFA) may process these cues differently from individuals with typical development (TD) and purportedly use a ‘mechanistic’ rather than a ‘mentalistic’ approach, involving rule- and contingency-based interpretations of the stimuli. The study primarily aimed at examining whether the judgments of facial expressions made by individuals with TD and HFA would be similarly affected by the immediately preceding dynamic perceptual history of that face. A second aim was to explore possible differences in the mechanisms underpinning the perceptual judgments in the two groups.
Twenty-two adults with HFA and with TD, matched for age, gender and IQ, were tested in three experiments in which dynamic, ‘ecologically valid’ offsets of happy and angry facial expressions were presented. Participants evaluated the expression depicted in the last frame of the video clip by using a 5-point scale ranging from slightly angry via neutral to slightly happy. Specific experimental manipulations prior to the final facial expression of the video clip allowed examining contributions of bottom-up mechanisms (sequential contrast/context effects and representational momentum) and a top-down mechanism (emotional anticipation) to distortions in the perception of the final expression.
In experiment 1, the two groups showed a very similar perceptual bias for the final expression of joy-to-neutral and anger-to-neutral videos (overshoot bias). In experiment 2, a change in the actor’s identity during the clip removed the bias in the TD group, but not in the HFA group. In experiment 3, neutral-to-joy/anger-to-neutral sequences generated an undershoot bias (opposite to the overshoot) in the TD group, whereas no bias was observed in the HFA group.
We argue that in TD individuals the perceptual judgments of other’s facial expressions were underpinned by an automatic emotional anticipation mechanism. In contrast, HFA individuals were primarily influenced by visual features, most notably the contrast between the start and end expressions, or pattern extrapolation. We critically discuss the proposition that automatic emotional anticipation may be induced by motor simulation of the perceived dynamic facial expressions and discuss its implications for autism.
KeywordsHigh-functioning autism (HFA) Dynamic facial expressions Perceptual distortions Prediction Anticipation Embodied simulation Theory of mind
The ability to interpret social cues conveyed by an agent as reflecting the agent’s mental or emotional state, allowing the observer to anticipate the agent’s behaviour, is essential for successful social interactions. In contrast to individuals with typical development (TD), individuals with autism spectrum disorders (ASD) are thought to have difficulties with this skill, or at least with its spontaneous employment [1, 2]. ASD is a pervasive developmental condition characterized by impaired social development and stereotypical, repetitive behaviours, which are often accompanied by obsessive interests and a lack of empathy [3–5]. High-functioning autism (HFA) is a mild form of autism with normal intelligence quotient (IQ) distribution, but with a delayed development of language skills and difficulties in social and emotional domains .
The extent to which atypical social and emotional behaviour in individuals with HFA reflects an impaired ability for implicit (automatic, non-volitional) rather than explicit (inferential, volitional, involving awareness) understanding is still unclear [7, 8]. Traditionally, the emphasis has been on explicit deficits in social understanding, such as the failure to attribute epistemic mental states (such as beliefs and desires) to others [9–11] or on failures in executive functioning . However, more recently, impairments in implicit social processing increasingly gained attention, with suggestions that they may form a primary deficit [1, 2]. A failure in implicit or automatic understanding may be (partly) compensated by explicit reasoning on the basis of contextual information [2, 13].
A primary source of information on the basis of which implicit mental and emotional attributions can be formed is action sequences, either consisting of limb/body articulations or of changing facial expressions [14–16]. In humans, the perception of such simple actions, and indeed even single frames taken from the action sequence, has been shown to activate (pre)motor areas that are also involved in the execution of these actions, both with respect to limb actions  and facial expressions [18–21]. The matching of observation and execution of actions at the level of individual cells has been referred to as the mirror neuron mechanism (MNM) [22, 23]. The MNM may enable an automatic motor simulation of the observed action, which resembles the neuronal activity when the observer carries out the observed action him/herself. This may provide the observer with an appreciation of the other’s action ‘from within’, i.e. from a first person perspective rather than from a third person perspective . Importantly, the motor simulation might additionally provide the observer with an automatic anticipation of what the other’s most likely next action or goal is. In other words, it may provide a view into the immediate future, on the basis of the immediate past. This idea is closely linked to the notion that action representations are neurologically arranged in action chains [15, 16, 25], where the chain specifies a goal. The automatic unfolding of the chain activity enables the automatic anticipation of the other’s next action or state of mind .
It is still much debated whether individuals with HFA have difficulties with action anticipation and whether this may be underpinned by impaired motor simulation. Difficulties in anticipating others’ actions in HFA have recently been suggested [27, 28]. Impaired motor simulation in ASD has been reported  but has also been contested .
Using perceptual distortions to study emotional anticipation
The anticipation of actions consisting of dynamic facial expressions has recently been studied through a new task, which, in TD individuals, creates robust distortions in the perception of other’s facial expressions that are presumably due to anticipatory processes [31, 32]. This task involved the perception of short video clips of ‘ecologically valid’ dynamic facial expressions , in which intense happy or angry facial expressions morphed rapidly into neutral expressions (happy and angry offsets). The perceptual histories biased the evaluation of the neutral expressions, such that following a happy offset the neutral expression was evaluated as slightly angry, and following an angry offset as slightly happy. This was called an ‘overshoot’ response bias, involving a crossing of the category boundary. A series of manipulations were applied in order to clarify the nature of this bias . For example, a change of facial identity just before the final neutral expression removed the overshoot bias . This ruled out a major contribution of relatively low-level visual processes such as (sequential) contrast effects [34–36], adaptation [37, 38] and representational momentum (RM) [33, 39]. Sequential contrast/context effects were ruled out because the identity change did not alter the degree of contrast in emotion between the first and last expressions. Thus, if contrast was the driving force behind the overshoot bias, then the bias should not have been removed by the identity-change manipulation. Aftereffects induced by adaption to the initial facial expression (presented for 300 ms) were ruled out as they typically generalize across different identities. An explanation in terms of RM, which is the phenomenon that an observer’s memory for the final position of a moving target is displaced further along the observed trajectory, was also deemed unlikely. RM, when applied to dynamic facial expressions, could in principle operate on two distinct levels: (1) The level of dynamic geometrical facial features (e.g. the U shape of the mouth in a smile turning flat and moving into a reversed U shape). The changes in the facial expressions when going from one identity to the other are smooth enough not to disturb or reset RM. Indeed, when the identity transition was made really smooth by using morphing procedures , the TD group still did not report an overshoot bias in the identity-change condition; (2) The level of an underlying valence dimension (a positive to negative dimension in the joy-to-neutral condition, and a negative to positive dimension in the anger-to-neutral condition). However, the valence does not depend on the identity but on the expression, and therefore an identity change should not diminish RM.
Thus, extrapolations on both levels are unlikely explanations for the overshoot bias as neither should be affected by an identity change.
On the basis of these findings, the authors suggested that ‘emotional anticipation’, which is the involuntary and automatic anticipation of the other’s emotional state of mind on the basis of the immediate perceptual history, better explained the results. Emotional anticipation can be seen as a low-level mind-reading mechanism, enabling the tacit and intuitive understanding of another’s emotional state of mind . Thus, in the joy-to-neutral videos, the observer implicitly anticipated the agent’s emotional state of mind to move into a ‘negative’ state, and in the anger-to-neutral videos to move into a ‘positive’ state. The idea is that these implicit ‘mentalizing’ activities in turn influenced perception .
The current study
The current study explores the issue of whether emotion anticipation is impaired in HFA by utilizing the above paradigm . The reasoning was that if individuals with HFA are impaired in the involuntary and automatic anticipation of the direction into which another’s emotional state develops, on the basis of the immediate dynamic perceptual history, then they would not report the visual distortion found in TD individuals. A number of experimental manipulations were introduced to examine whether the mechanisms underpinning perceptual report by the HFA group might be any different from those in the TD group. Individuals with HFA have been reported to use alternative or compensatory strategies, which may rely on physical or dynamical characteristics of the stimulus [27, 41]. In principle, any aberrant perceptual report in the HFA individuals might be due to either an over-reliance on contrast effects or RM, or to impaired emotional anticipation. The experimental manipulations were specifically designed to disentangle these mechanisms.
Short video clips of dynamic facial expressions, which morphed from 100 % joy or 100 % anger to either a neutral expression, a 10 % joy or a 10 % anger expression , were presented to the HFA and TD groups to assess any differences in their perceptual report of the facial expression displayed in the final frame. The primary hypothesis was that the individuals with HFA do not engage in ‘emotional anticipation’, which should result in the absence of an overshoot response bias. Alternatively, on the basis of findings suggesting that individuals with HFA can anticipate someone else’s actions by using alternative, compensatory strategies , it might be that they do show an overshoot bias but achieved it via a different route.
Demographic information for the HFA group and the TD group
5 F; 11 M
7 F; 11 M
5 F; 12 M
6 F; 11 M
5 F; 10 M
6 F; 13 M
The following description applies to all 22 HFA individuals. They all had previously received a diagnosis of HFA or Asperger syndrome from a clinical psychologist or psychiatrist based on DSM-IV-TR  or ICD-10  criteria. All were recruited through disability services from universities in the north-east of England (UK). Evidence of diagnostic history for ASD was acquired, and in order to confirm ASD symptomology, the ADOS (Autism Diagnostic Observation Schedule, module 4) was completed with a trained examiner (HGB). The ADOS is a semi-structured, standardized assessment of communication, social interaction and imagination, designed for use with children and adults suspected of having ASD . A score of 7–9 indicates autism spectrum, and a score of 10 or more indicates autism. The 22 individuals included in the study all met the ADOS criteria for HFA or Asperger syndrome (of the initial group, three individuals were excluded for having ADOS scores below 7). They also all completed the Autism Spectrum Quotient questionnaire (AQ) , which is a 50-statement, self-administered questionnaire, designed to measure the degree to which an adult with normal intelligence possesses autistic-like traits.
From the 17 students with HFA participating in experiment 1, one was removed following the application of exclusion criteria to the data (see ‘Data reduction’ below for details). The remaining 16 students (5 females, 11 males; mean age = 20.3 years, SD = 3.2) had a mean total ADOS score of 8.0 (SD = 0.9) and a mean AQ score of 32.1 (SD = 7.4). Their mean total IQ score was 117.1 (SD = 10.9; Table 1), assessed using the Wechsler Adult Intelligence Scale, WAIS-III .
All TD participants in the three experiments were undergraduate psychology students from Hull University. None of them took part in more than one experiment. All were asked if they had previously obtained a head injury or had received a diagnosis of ASD or of another mental health or developmental disorder. No participants disclosed this. Twenty TD individuals took part in experiment 1; applying the exclusion criteria to the data (see ‘Data reduction’ below) removed two individuals. The remaining 18 participants (7 females, 11 males; mean = 20.1 years, SD = 4.0) had a mean AQ score of 15.3 (SD = 5.2) and a mean total IQ score of 113.4 (SD = 8.3). The TD group did not differ from the HFA group in terms of age (t(32) = .554, p = .583), gender ratio (X 2(1, 33) = .216, p = .642) or IQ (t(32) = −1.14, p = .264). As expected, AQ scores were significantly higher in the HFA group (t(32) = −7.74, p < .0001).
An important feature of the study was that the HFA group could be matched very closely with the control TD group, as both groups consisted of university students with fairly similar daily routines, resulting in a good approximation of the influence of the factor ‘HFA’. All HFA and TD participants had normal or corrected-to-normal vision and provided written consent prior to the experiment. Participants received course credits or a fee for taking part. The study was approved by the Ethics Committee of the Department of Psychology of Hull University.
The stimuli and experimental procedure were the same as in . Pictures of eight actors displaying facial expressions of joy and anger were selected from the Pictures of Facial Affect (four males: EM, JJ, PE, WF; four females: C, MO, PF, SW) . All faces were shown from frontal view with their eye gaze directed straight ahead. The photographs were in grayscale. The hair had been blackened so as to merge with the black background. The pictures were digitally adjusted to match in contrast and brightness. The eyes of all actors were positioned at approximately the same screen location. Faces measured about 13 × 20 cm when displayed on the screen, subtending approximately 8° vertically.
The experiment started with a calibration condition, in which the eight stimulus faces showed static neutral expressions (i.e. neutral expressions according to the ratings from ). Sixteen calibration trials were presented (eight actors, two repetitions each; random order) in which the neutral face was shown for 600 ms (preceded by a 500-ms fixation cross). Participants were prompted to rate the ‘neutral’ expressions using a 5-point scale, ranging from slightly angry (1) via neutral (3) to slightly happy (5), by pressing one of five labelled keys on a button box (SR-Box, Psychology Software Tools, Inc., USA).
Following the instructions displayed on the screen, eight practice trials were presented, in order to make sure that participants did understand the instructions and also to make them familiar with the task. The practice was followed by 64 experimental trials. Each trial started with a fixation cross (500 ms), directly followed by a video clip. As soon as the video ended, a blank screen appeared with a prompt to provide a response. The participant’s task was to evaluate the last expression using the same 5-point scale as used for the calibration trials. The 64 experimental trials were presented randomized, half of them starting with a happy expression (called joy-to-‘neutral’ videos) and half with an angry expression (called anger-to-‘neutral’ videos). Of both types of videos, 50 % of trials ended with the neutral expression, 25 % of trials ended with 10 % joy and 25 % of trials with 10 % anger expressions. The duration of the entire experimental procedure was 20 min.
Trials in which Reaction Times (RTs) were below 250 ms or above 3000 ms were considered outliers and were removed (HFA, 3.8 %; TD, 2.9 %). Participants were excluded if more than 25 % of their RT values fell outside the above range (HFA, n = 0; TD, n = 1) and when they pressed the same key for more than 90 % of trials (HFA, n = 0; TD, n = 0). A ±2.5 SD rule was applied to the mean difference of the ratings per participant, i.e. rating in the anger-to-neutral condition minus rating in the joy-to-neutral condition (HFA, n = 1; TD, n = 1).
For both groups, the mean calibration scores obtained for each of the eight stimulus faces were used to adjust the scores in the experimental trials; a calibration factor [equal to 3.00 minus the calibration score] was added to the experimental scores. The calibration procedure allowed performing one-sample t tests with test value 3.00 on the scores in the two perceptual histories. All statistical analyses were performed on the calibrated scores.
The endpoint by group interaction was non-significant (F(2, 64) = .94, p = .40, η p 2 = .029), nor was the perceptual history by group interaction (F(1, 32) = 3.3, p = .078, η p 2 = .095). The endpoint by perceptual history interaction was significant (F(2, 64) = 4.1, p = .021, η p 2 = .11), reflecting that across both groups the difference between the judgments in the two perceptual histories at the 10 % joy endpoint was larger than at the neutral (t(33) = 2.7, p = .030) and 10 % anger (t(33) = 2.6, p = .013) endpoints. The three-way interaction was non-significant (F(2, 64) = .063, p = .94, η p 2 = .002).
The judgments made at the neutral video endpoint were examined in more detail, as this endpoint, due to being neither positive nor negative, is most sensitive to a misattribution of emotional content involving a crossing of the category boundary. In the HFA group, the mean ratings in the joy-to-neutral videos (M = 2.73, SD = 0.22) and anger-to-neutral videos (M = 3.21, SD = 0.28) differed significantly from each other (t(15) = −6.94, p < .0001), and each of them differed significantly from 3.00 (joy video, p < .0001; anger video, p = .010). The TD group showed a very similar pattern; the mean ratings in the joy-to-neutral videos (M = 2.71, SD = 0.27) and anger-to-neutral videos (M = 3.36, SD = 0.31) differed significantly from each other (t(17) = −7.04, p < .0001), and each of them differed significantly from 3.00 (joy video, p < .001; anger video, p < .0001).
Both HFA and TD participants showed a robust overshoot response bias, with the last expression of the joy-to-neutral videos evaluated as slightly angry and the identical last expression of the anger-to-neutral videos evaluated as slightly happy. The main conclusion is that the evaluations of both groups were influenced by the perceptual histories, and to roughly the same extent (with a tendency for the TD group to be influenced more than the HFA group). The results seem to be in line with a study by  who reported that a group of individuals with developmental pervasive disorders overestimated the intensity of facial expressions in neutral-to-emotional expression video clips similar to a TD group, although to a lesser extent.
If the HFA group had shown no overshoot bias, then that would have suggested that they do not engage in an ‘emotional anticipation’ process. The presence of the overshoot bias leaves open two possibilities: (1) the HFA group engages in involuntary emotional anticipation similar to the TD group, and (2) they rely on another mechanism, which compensates for the absence of automatic emotional anticipation. Experiments 2 and 3 were conducted to clarify this issue. Specifically, they examined whether individuals with HFA may have processed the dynamic facial expressions predominantly in terms of physical characteristics, without keeping track of the changes in the agent’s emotional state of mind that cause the changes in facial expression.
Experiment 2 was conducted to test whether low-level visual mechanisms, underpinning contrast and RM effects, may explain the overshoot effect found in experiment 1 in the HFA group. Hereto, the identity of the actor was changed in the last frame of the clips, similar to the manipulation performed in . Thus, participants needed to judge the last expression of the videos depicting a new identity for which no immediate perceptual history was available. The rational was that if the overshoot bias in HFA individuals resulted from sequential contrast effects or from RM operating on an underlying positive–negative valence dimension, rather than from emotional anticipation, then a change in identity should have no effect. A change in the agent’s identity does not affect the contrast between the first and last frames nor does it interfere with the underlying positive–negative valence dimension. On the other hand, if the bias in the HFA group did result from emotional anticipation, then they should, similar to the TD group in , not show an overshoot response bias. After all, one cannot anticipate the emotional state of someone on the basis of a perceptual history pertaining to a different identity.
Twenty individuals with HFA took part in experiment 2, of which three were removed following the application of data exclusion criteria (see ‘Data reduction’ below). The remaining 17 individuals (5 females, 12 males; mean age = 22.1 years, SD = 7.2) had a mean total ADOS score of 8.1 (SD = 0.9) and a mean AQ score of 29.0 (SD = 8.2). Their mean total IQ score was 115.9 (SD = 8.1; WAIS-III; Table 1).
Twenty-one TD individuals took part. Applying the data exclusion criteria removed four participants. The remaining 17 participants (6 females, 11 males; mean age = 22.1 years, SD = 5.5) had a mean AQ score of 15.5 (SD = 6.6) and a mean total IQ score of 114.5 (SD = 7.7). The TD group did not differ significantly from the HFA group in terms of age (t(32) = .675, p = .10), gender ratio (X 2(1, 34) = .134, p = .714) or IQ (t(32) = −.54, p = .591). AQ scores were significantly higher in the HFA group (t(32) = 5.28, p < .0001).
All HFA and TD participants had normal or corrected-to-normal vision and provided written consent prior to the experiment. Participants received course credits or a fee for taking part. The study was approved by the Ethics Committee of the Department of Psychology of Hull University.
The procedure was the same as for experiment 1. Following the calibration phase and the practice trials, 128 experimental trials (8 actors × 2 perceptual histories × 2 conditions × 4 repetitions) were presented. The same-identity and identity-change conditions were presented in separate blocks (each 64 trials).
Exclusion criteria were the same as for experiment 1. Trials in which RTs were below 250 ms or above 3000 ms were removed (HFA, 2.3 %; TD, 5.7 %). Participants were excluded if more than 25 % of their RT values fell outside the above range (HFA, n = 0; TD, n = 2) and when they pressed the same key on more than 90 % of trials (HFA, n = 1; TD, n = 0). The ±2.5 SD rule was applied to the mean difference of the ratings in the ‘joy’ and ‘anger’ conditions per participant (HFA, n = 2; TD, n = 2).
A 2 × 2 ANOVA showed a significant main effect of perceptual history (F(1, 32) = 70.4, p < .0001, η p 2 = .68). The main effect of group was non-significant (F(1, 32) = 3.23, p = .082, η p 2 = .092), and the perceptual history by group interaction was non-significant (F(1, 32) = .25, p = .62, η p 2 = .008). Thus, the results of the TD and HFA groups on the same-identity task replicated those of experiment 1, both showing a robust overshoot response bias.
The 2 × 2 ANOVA showed significant main effects of both perceptual history (F(1, 32) = 11.17, p = .002, η p 2 = .259) and group (F(1, 32) = 10.02, p = .003, η p 2 = .239). Crucially, the perceptual history by group interaction was significant (F(1, 32) = 8.70, p = .006, η p 2 = .214). In the TD group, the ratings for the neutral expressions did not differ between the two perceptual histories (t(16) = .437, p = .67). Thus, in TD participants, a change of identity of the actor effectively removed the overshoot bias. In the AS group, however, the overshoot bias persisted (t(16) = 3.52, p = .003). While in the same-identity condition the two groups performed almost identically, in the identity-change condition the groups differed significantly, with the HFA group, but not the TD group, showing an overshoot response bias. This was also clearly apparent when the results are shown for each actor separately (Fig. 5b).
Could impaired identity recognition skills in the HFA group explain their overshoot bias in the identity-change condition?
When the video clips contained no change in identity, both HFA and TD individuals produced a robust overshoot response bias, confirming the results of experiment 1. However, following the identity-change manipulation, the overshoot bias persisted in the HFA group but was absent in the TD group. Separate analysis of same gender and different gender trials in the identity-change condition suggested that this most probably could not be explained by impaired identity recognition in the HFA group.
Whereas the results of the same-identity condition might suggest that HFA individuals show emotional anticipation, similar to TD individuals, the results of the identity-change condition show that this is probably not the case. Emotional anticipation would not produce an overshoot bias in response to a new agent B the observer knows nothing about (i.e. no perceptual history), other than that agent B depicts a neutral expression. These findings open up the possibility that the HFA group relied on an alternative mechanism or strategy to produce the bias in the same-identity condition and that the same mechanism/strategy was applied in the identity-change condition, hence their bias in the latter condition.
One possibility is that the HFA group might have been influenced by the contrast in facial expressions between the 100 % emotional expression shown at the start of the clip and the neutral expression shown at the end of the clip (both presented for 300 ms). RM [39, 50], involving an extrapolation of the dynamic geometries in the face or an extrapolation on an underlying positive–negative valence dimension, forms in principle another possible explanation. The underlying positive–negative valence dimension and to a large extent also the extrapolation of the dynamic geometries of the face remain unaffected by an identity change.
The absence of an overshoot bias in the TD group in the identity-change condition confirmed the findings reported in . It suggests that the robust overshoot in the same-identity condition was not produced by sequential contrast effects or RM, because if TD participants relied on these low-level, bottom-up processes, then they should also have been affected by them in the identity-change condition. This finding favours a top-down emotional anticipation explanation for the TD group.
In experiment 3, the video sequence was modified such that it started from a neutral expression and morphed via a maximally happy or angry expression back to the same neutral expression (loop condition). The loop task was designed to shed light on the merits of RM [33, 39] and contrast effects [34–36] as an explanation for the overshoot response bias in the HFA group. The rational was that if the response bias found in the previous experiments in HFA individuals was induced by the contrast between the to-be-evaluated last neutral expression (presented for 300 ms) and the ‘anchor’ stimulus (the first happy or angry expression, also presented for 300 ms), then in the loop condition, one would expect this last neutral expression to be evaluated as neutral and not to obtain an overshoot bias. Alternatively, RM would predict to find an overshoot response bias in the loop condition, i.e. an extrapolation of the second half of the sequence that went from maximal joy/anger to neutral. In the study by Palumbo and Jellema , the TD group produced an ‘undershoot’ effect in the loop condition, consisting of a bias in the direction of the immediately preceding emotion (i.e. a bias towards happy in the neutral-to-happy-to neutral condition and a bias towards angry in the neutral-to-angry-to-neutral condition).
Eighteen individuals with HFA took part, of which three were removed following the application of data exclusion criteria (see ‘Data reduction’ below). The remaining 15 individuals (5 females, 10 males; mean age = 22.3 years, SD = 7.1) had a mean total ADOS score of 8.2 (SD = 1.6), a mean AQ score of 33.7 (SD = 5.7) and a mean total IQ score of 118.9 (SD = 8.9; WAIS-III; Table 1).
Twenty-two TD individuals took part. Applying the data exclusion criteria removed three participants. The remaining 19 individuals (6 females, 13 males; mean age = 20.6 years, SD = 2.52) had a mean AQ score of 14.7 (SD = 6.1) and a mean total IQ score of 115.1 (SD = 8.1).
The TD group did not differ from the HFA group in terms of age (t(32) = 1.00, p = .323), gender ratio (X 2(1, 34) = .012, p = .914) or IQ (t(32) = 1.49, p = .147). AQ scores were significantly higher in the HFA group (t(32) = 9.29, p < .0001).
All HFA and TD participants had normal or corrected-to-normal vision and provided written consent prior to the experiment. Participants received course credits or a fee for taking part. The study was approved by the Ethics Committee of the Department of Psychology of Hull University.
Following the calibration and the practice trials as in the previous experiments, participants were presented with 96 randomized trials: 8 actors × 2 perceptual histories × 2 conditions × 3 repetitions.
Trials in which RTs were below 250 ms or above 3000 ms were removed (HFA, 9.2 %; TD, 7.6 %). Participants were excluded if more than 25 % of their RT values fell outside this range (HFA, n = 0; TD, n = 2) and for pressing the same key more than 90 % of trials (HFA, n = 1; TD, n = 0). A ±2.5 SD rule was applied to the mean difference of the ratings in the ‘joy’ and ‘anger’ conditions per participant (HFA, n = 2; TD, n = 1).
In the control condition (no loop), the HFA and TD groups showed a very similar overshoot response bias, confirming the results of experiments 1 and 2. In the loop task, both the TD and HFA groups showed the reversed pattern, i.e. a tendency towards an undershoot response bias: the last frame of the neutral-to-joy-to-neutral sequence was evaluated as ‘slightly happy’ and the last frame of the neutral-to-anger-to-neutral sequence as slightly angry. This confirmed previous findings in .
In neither group, an overshoot bias was found in the loop task, suggesting that RM was not the underpinning mechanism for either group. The sequential contrast hypothesis predicted that the last frame would be evaluated as neutral, as the first and last frames were identical (both neutral). The tendency towards an undershoot response in both groups renders a major contribution of sequential contrast unlikely. However, the explorative analysis suggested that the undershoot bias in the HFA group was rather weak; its reliability will have to be determined by further testing. An absence of a bias in the loop condition in the HFA group might suggest that their responses were determined by sequential contrast effects, which would open up the possibility that their overshoot biases in the control condition were also induced by sequential contrast effects, rather than by emotional anticipation.
A possible explanation for the undershoot response in the TD group is that the neutral-joy-neutral sequence conveyed a positive social signal directed towards the observer (a brief smile, as used e.g. when colleagues acknowledge each other while passing in the corridor). Similarly, the neutral-anger-neutral sequence could convey a negative social signal. This induced in the observer the impression that the agent either liked or disliked them, and, as we tend to like those who like us and dislike those who dislike us , the agent was evaluated accordingly. It is important to note that susceptibility to the ‘social signal’ does not necessarily imply that the observer is capable of ‘emotional anticipation’.
The current study examined whether individuals with HFA show similar or dissimilar biases to TD individuals in their evaluations of dynamic facial expressions, and whether the mechanisms they rely on in making these evaluations might be any different from those of TD individuals. Palumbo and Jellema  suggested that the dynamic perceptual history preceding a neutral target face gives rise to an involuntary anticipation of how the emotional state of the agent would change next, called ‘emotional anticipation’, which in turn biases the perception of that neutral face . These processes can be conceptualized as low-level emotion reading [8, 52], rather than as resulting from low-level visual processes (bottom-up).
In experiment 1, both HFA and TD participants showed a robust overshoot response bias with the last neutral expression of the joy-to-neutral videos evaluated as slightly angry and the identical last neutral expression of the anger-to-neutral videos as slightly happy. Contrary to the initial prediction, the extent of overshoot bias in HFA individuals did not differ from that in TD individuals. This finding suggested that the individuals with HFA were not impaired in emotional anticipation or, alternatively, that they were impaired in emotional anticipation but relied on other (compensatory) mechanisms. Experiment 2 showed that in the TD group, a change in the identity of the agent during the video clips removed the overshoot bias, confirming the findings in Palumbo and Jellema . These results are compatible with top-down emotional anticipation, involving the integration of emotion and identity information available in the immediate perceptual history. Remarkably, the HFA group continued to show an overshoot bias, similar to their bias in experiment 1, which could not be ascribed to a failure to detect the change in identity. It is possible that the individuals with HFA failed to implicitly integrate the information about the identity with the information about the emotional expression. Experiment 3 further tested whether RM and contrast effects might have been responsible for the generation of the response bias in the HFA group by using video sequences that started and ended with the same neutral expression (loop condition). The responses of the HFA and TD participants did not differ in this condition, though subsequent explorative analyses suggested that the undershoot bias in the HFA group may be weaker than in the TD group, which would be compatible with the notion that in the HFA group, sequential contrast effects played a role. The ‘undershoot’ bias found in the TD group might reflect that the ‘ecologically valid’ brief smiles and frowns were seen as positive and negative signals directed at them, which made them evaluate the agent in a reciprocal manner.
Emotional anticipation and alternative mechanisms
The notion that anticipation (or expectation) forms an integral part of perception and acts to shape perception is widely supported . Especially when the anticipations are implicit or tacit, they can drive the perception of the current stimulus into the shape or direction the stimulus is expected to adopt in the immediate future and thereby distort the perception of the current stimulus [54, 55]. The emotional expressivity of a face, especially when it is embedded in a dynamic sequence, may automatically lead to an anticipation of what the agent’s emotional state most likely will be in the immediate future, possibly via a low-level mind-reading process underpinned by motor simulation involving mirror neuron mechanisms (MNM) , which then in turn affects perception. There is accumulating evidence that the MNM in the premotor cortex becomes activated not only when observing other’s bodily actions but also when observing other’s emotional facial expressions [18–21]. It has been suggested that such a motor simulation might underlie the tacit/intuitive understanding of others’ facial expressions , which in turn might shape the perception of those expressions. In this view, MNM activity is part of a low-level mind-reading mechanism [8, 52], not involving any cognitive inferential processes or deliberate reasoning, which enables emotional anticipation.
What mechanism could have driven the response bias in the HFA group in the identity-change condition? It is possible that in the HFA individuals a motor simulation of the observed dynamic expressions took place, which caused their bias in perceptual report, similar to what may have happened in the TD individuals. However, this motor simulation may not have been guided properly by identity information and therefore was applied indiscriminately, giving rise to a bias in perceptual report also in the identity-change condition. There are conflicting reports as to whether motor simulation is impaired in HFA . Alternatively, it is possible that atypical means were employed as compensation for a failure in automatic emotional anticipation. It is well documented that individuals with HFA may adopt systemizing mechanisms (e.g. based on statistical regularities or input-operation-output relations), which typically govern the motions of non-agentive objects, to understand and anticipate others’ behaviour [56–59]. One such atypical approach may be a reliance on the low-level visual characteristic of the sequential contrast between the first and last frames, which were both well perceived as they were presented for 300 ms. Sequential contrast effects would predict an overshoot response in the identity-change task, as these effects are immune to identity. In principle, reliance on RM could form another atypical means resulting in perceptual biases. Although the presence of an overshoot response in the identity-change task is compatible with the notion of RM, the absence of an overshoot bias in the loop task seems to exclude this possibility.
It cannot be excluded though that the evaluations in the HFA group in the loop task were influenced by an altogether different mechanism, namely sensitivity to the pattern present in the video sequence. HFA individuals might be tuned to detecting specific sequences of changes in the stimuli and expect the pattern to repeat itself, in line with the notion that individuals with HFA are preferentially tuned to detect statistical regularities and contingencies . Such a ‘pattern detection and extrapolation’ hypothesis would predict the loop to repeat itself, resulting in an undershoot bias. Subsequent experiments are required to examine the merits of the ‘sequential contrast’ and ‘pattern extrapolation’ hypotheses further in the HFA group.
Could the atypical behaviour of the HFA group have been caused by one or more (visual) difficulties characteristic for this neurodevelopmental disorder? The finding that the TD and HFA groups performed very similarly in experiment 1, and in the control conditions of experiments 2 and 3, with equally sized overshoot response biases, speaks against a contribution of such factors to the judgements. Possible impairments in visual attention and visual working memory , in visual perception of coherent motion  and in global processing  are therefore all unlikely contributors. Recognition of emotional facial expressions may be impaired in HFA, but recognition of prototypical happy and angry expressions is intact . This is supported by the finding of very similar scores in the TD and HFA groups in the calibration phase at the start of the experiments, which also excludes the possibility that the HFA individuals might have been impaired in distinguishing between faint facial expressions of joy and anger .
Future studies could, in addition to investigate the role played by sensitivity to the pattern present in the video sequence in bringing about the perceptual distortions in the HFA group, also further examine the role of motor simulation in bringing about the perceptual distortions. This could be done, for example, by disrupting activity in (premotor) mirror mechanisms using transcranial magnetic stimulation. Furthermore, it would be useful to examine whether individuals with ASD having more severe symptoms than those with HFA might be even more prone to apply alternative strategies to try and make sense of other people’s emotional states from social cues.
TD individuals are equipped with an automatic and mandatory mechanism, which allows them to implicitly anticipate how the emotional state of others may change on the basis of the immediate perceptual history. The current findings suggested that individuals with HFA may not rely on such an emotional anticipation mechanism, but may use alternative mechanisms.
Emotional anticipation may be underpinned by motor simulation. However, whether or not motor simulation is compromised in ASD is still debated [30, 64]. A compromised implicit and automatic anticipation mechanism would undoubtedly contribute to problems in navigating the social world, which characterize HFA.
Autism Diagnostic Observation Schedule
Autism Spectrum Quotient
autism spectrum disorder
Diagnostic and Statistical Manual of Mental Disorders
International Statistical Classification of Diseases and Related Health Problems
mirror neuron mechanism
theory of mind
LP is supported by the Economic and Social Research Council (ESRC, Ref. ES/K000187/1). HGB and TJ are supported by the Arts and Humanities Research Council (AHRC, Ref. AH/E511147/1) and the European Science Foundation (ESF). We are grateful to the volunteers with HFA for their participation in the research. We thank David Perrett for providing the morphed facial expressions.
- Jellema T, Lorteije J, van Rijn S, van’t Wout M, de Haan E, van Engeland H, et al. Involuntary interpretation of social cues is compromised in autism spectrum disorders. Autism Res. 2009;2:192–204.View ArticlePubMedGoogle Scholar
- Senju A, Southgate V, White S, Frith U. Mindblind eyes: an absence of spontaneous theory of mind in Asperger syndrome. Science. 2009;325:883–5.View ArticlePubMedGoogle Scholar
- Rutter M. Diagnosis and definition of childhood autism. J Autism Child Schiz. 1978;8:139–61.View ArticleGoogle Scholar
- World Health Organization. ICD-10: International Statistical Classification of Diseases and Related Health Problems (10th Rev. ed.). New York: World Health Organization; 2008.
- American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (5th ed.). Washington, DC: Author; 2013.
- McPartland J, Klin A. Asperger syndrome. Adolesc Med Clin N Am. 2006;17:771–8.Google Scholar
- Gallese V. Intentional attunement: a neurophysiological perspective on social cognition and its disruption in autism. Brain Res. 2006;1079:15–24.View ArticlePubMedGoogle Scholar
- Goldman AI. Simulating minds: the philosophy, psychology and neuroscience of mindreading. Oxford: Oxford University Press; 2006.View ArticleGoogle Scholar
- Frith U. Asperger and his syndrome. In: Frith U, editor. Autism and Asperger syndrome. Cambridge: Cambridge University Press; 1991. p. 1–36.View ArticleGoogle Scholar
- Baron-Cohen S. Mindblindness: an essay on autism and theory of mind. Cambridge, MA: MIT Press; 1995.Google Scholar
- Baron-Cohen S, Campbell R, Karmiloff-Smith A, Grant J, Walker J. Are children with autism blind to the mentalistic significance of the eyes. Brit J Dev Psychol. 1995;13:379–98.View ArticleGoogle Scholar
- Russell J. Autism as an executive disorder. Oxford: Oxford University Press; 1998.View ArticleGoogle Scholar
- Kuzmanovic B, Schilbach L, Lehnhardt FG, Bente G, Vogeley K. A matter of words: impact of verbal and nonverbal information on impression formation in high-functioning autism. Res Autism Spect Dis. 2011;5:604–13.View ArticleGoogle Scholar
- Blakemore SJ, Decety J. From the perception of action to the understanding of intention. Nat Rev Neurosci. 2001;2:561–7.View ArticlePubMedGoogle Scholar
- Jellema T, Perrett DI. Perceptual history influences neural responses to face and body postures. J Cognitive Neurosci. 2003;15:961–71.View ArticleGoogle Scholar
- Jellema T, Perrett DI. Cells in monkey STS responsive to articulated body motions and consequent static posture: a case of implied motion? Neuropsychologia. 2003;41:1728–37.View ArticlePubMedGoogle Scholar
- Kilner JM, Neal A, Weiskopf N, Friston KJ, Frith CD. Evidence of mirror neurons in human inferior frontal gyrus. J Neurosci. 2009;29:10153–9.PubMed CentralView ArticlePubMedGoogle Scholar
- Bastiaansen JA, Thioux M, Keysers C. Evidence for mirror systems in emotions. Philos T Roy Soc Lon B Biol Sci. 2009;364(1528):2391–404.View ArticleGoogle Scholar
- Carr L, Iacoboni M, Dubeau MC, Mazziotta JC, Lenzi GL. Neural mechanisms of empathy in humans: a relay from neural systems for imitation to limbic areas. Proc Natl Acad Sci U S A. 2003;100:5497–502.PubMed CentralView ArticlePubMedGoogle Scholar
- Leslie KR, Johnson-Frey SH, Grafton ST. Functional imaging of face and hand imitation: towards a motor theory of empathy. Neuroimage. 2004;21:601–7.View ArticlePubMedGoogle Scholar
- Moore A, Gorodnitskya I, Pineda J. EEG mu component responses to viewing emotional faces. Behav Brain Rese. 2012;226:309–16.View ArticleGoogle Scholar
- Gallese V, Fadiga L, Fogassi L, Rizzolatti G. Action recognition in the premotor cortex. Brain. 1996;119:593–609.View ArticlePubMedGoogle Scholar
- Rizzolatti G, Craighero L. The mirror-neuron system. Annu Rev Neurosci. 2004;27:169–92.View ArticlePubMedGoogle Scholar
- Rizzolatti G, Fabbri-Destro M, Cattaneo L. Mirror neurons: from discovery to autism. Exp Brain Res. 2009;200:223–37.View ArticlePubMedGoogle Scholar
- Fogassi L, Ferrari PF, Gesierich B, Rozzi S, Chersi F, Rizzolatti G. Parietal lobe: from action organization to intention understanding. Science. 2005;308:662–7.View ArticlePubMedGoogle Scholar
- Cattaneo L, Fabbri-Destro M, Boria S, Pieraccini C, Monti A, Cossu G, et al. Impairment of actions chains in autism and its possible role in intention understanding. P Natl Acad Sci-Biol. 2007;104:17825–30.View ArticleGoogle Scholar
- Hudson M, Burnett HG, Jellema T. Anticipating action intentions in autism spectrum disorder. J Autism Dev Disord. 2012;42:1684–93.View ArticlePubMedGoogle Scholar
- Zalla T, Labruyere N, Clement A, Georgieff N. Predicting ensuing actions in children and adolescents with autism spectrum disorders. Exp Brain Res. 2010;201:809–19.View ArticlePubMedGoogle Scholar
- Dapretto M, Davies MS, Pfeifer JH, Scott AA, Sigman M, Bookheimer SY, et al. Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorders. Nat Neurosci. 2006;9:28–30.PubMed CentralView ArticlePubMedGoogle Scholar
- Hamilton AFD. Goals, intentions and mental states: challenges for theories of autism. J Child Psychol Psyc. 2009;50:881–92.View ArticleGoogle Scholar
- Jellema T, Pecchinenda A, Palumbo L, Tan EG. Biases in the perception and affective valence of neutral facial expressions induced by the immediate perceptual history. Vis Cogn. 2011;19:616–34.View ArticleGoogle Scholar
- Palumbo L, Jellema T. Beyond face value: does involuntary emotional anticipation shape the perception of dynamic facial expressions? Plos One. 2013;8, e56003.PubMed CentralView ArticlePubMedGoogle Scholar
- Yoshikawa S, Sato W. Dynamic expressions of emotion induce representational momentum. Cogn Affect Behav Ne. 2008;8:25–31.View ArticleGoogle Scholar
- Suzuki S, Cavanagh P. A shape-contrast effect for briefly presented stimuli. J Exp Psychol Human. 1998;24:1315–41.View ArticleGoogle Scholar
- Tanaka-Matsumi J, Attivissimo D, Nelson S, D’urso T. Context effects on the judgment of basic emotions in the face. Motiv Emotion. 1995;19:139–55.View ArticleGoogle Scholar
- Thayer S. The effect of expression sequence and expressor identity on judgments of intensity of facial expression. J Nonverbal Behav. 1980;5:71–9.View ArticleGoogle Scholar
- Fox CJ, Barton JJ. What is adapted in face adaptation? The neural representations of expression in the human visual system. Brain Res. 2007;1127:80–9.View ArticlePubMedGoogle Scholar
- Hsu SM, Young AW. Adaptation effects in facial expression recognition. Vis Cogn. 2004;11:871–99.View ArticleGoogle Scholar
- Freyd JJ, Finke RA. Representational momentum. J Exp Psychol Learn. 1984;10:126–32.View ArticleGoogle Scholar
- Teufel C, Fletcher PC, Davis G. Seeing other minds: attributed mental states influence perception. Trends Cogn Sci. 2010;14:376–82.View ArticlePubMedGoogle Scholar
- Ashwin C, Hietanen JK, Baron-Cohen S. Atypical integration of social cues for orienting to gaze direction in adults with autism. Mol Autism. 2015;6:5.PubMed CentralView ArticlePubMedGoogle Scholar
- American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (4th edition-TR ed.). Washington, DC: Author; 2000.
- Lord C, Rutter M, DiLavore PC, Risi S. Autism Diagnostic Observation Schedule. Western Psychological Services: Los Angeles, CA; 1999.Google Scholar
- Baron-Cohen S, Wheelwright S, Hill J, Raste Y, Plumb I. The ‘Reading the Mind in the Eyes’ Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. J Child Psychol Psyc. 2001;42:241–52.View ArticleGoogle Scholar
- Wechsler D. WAIS-III administration and scoring manual. San Antonio, TX: The Psychological Corporation; 1997.Google Scholar
- Ekman P, Friesen WV. Pictures of facial affect. Palo Alto, CA: Consulting Psychologists Press; 1976.Google Scholar
- Perrett DI, May KA, Yoshikawa S. Facial shape and judgements of female attractiveness. Nature. 1994;368:239–42.View ArticlePubMedGoogle Scholar
- Uono S, Sato W, Toichi M. Brief report: representational momentum for dynamic facial expressions in pervasive developmental disorder. J Autism Dev Disord. 2010;40:371–7.View ArticlePubMedGoogle Scholar
- Behrmann M, Thomas C, Humphreys K. Seeing it differently: visual processing in autism. Trends Cogn Sci. 2006;10:258–64.View ArticlePubMedGoogle Scholar
- Freyd JJ. Dynamic mental representations. Psychol Rev. 1987;94:427–38.View ArticlePubMedGoogle Scholar
- Jones BC, DeBruine LM, Little AC, Conway CA, Feinburg DR. Integrating gaze direction and expression in preferences for attractive faces. Psychol Sci. 2006;17:588–91.View ArticlePubMedGoogle Scholar
- Goldman AI, Sripada CS. Simulationist models of face-based emotion recognition. Cognition. 2005;94:193–213.View ArticlePubMedGoogle Scholar
- Clark A. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behav Brain Sci. 2013;36:181–204. doi:10.1017/S0140525X12000477.View ArticlePubMedGoogle Scholar
- Hudson M, Liu CH, Jellema T. Anticipating intentional actions: the effect of eye gaze direction on the judgement of head rotation. Cognition. 2009;112:423–34.View ArticlePubMedGoogle Scholar
- Hudson M, Jellema T. Resolving ambiguous behavioral intentions by means of involuntary prioritization of gaze processing. Emotion. 2011;11:681–6.View ArticlePubMedGoogle Scholar
- Baron-Cohen S. The extreme male brain theory of autism. Trends Cogn Sci. 2002;6:248–54.View ArticlePubMedGoogle Scholar
- Jellema T, Perrett DI. Coding of visible and hidden actions. In: Prinz W, Hommel B, editors. Common mechanisms in perception and action, attention and performance. London: Academic; 2002. p. 356–80.Google Scholar
- Ristic J, Mottron L, Friesen CK, Iarocci G, Burack JA, Kingstone A. Eyes are special but not for everyone: the case of autism. Cogn Brain Res. 2005;24:715–8.View ArticleGoogle Scholar
- Jellema T, Perrett DI. Neural pathways of social cognition. In: Dunbar RIM, Barrett L, editors. The Oxford handbook of evolutionary psychology. Oxford: Oxford University Press; 2012.Google Scholar
- Kaiser M, Shiffrar M. The visual perception of motion by observers with autism spectrum disorder: a review and synthesis. Psychon B Rev. 2009;16:761–77.View ArticleGoogle Scholar
- Happe´ F, Frith U. The weak coherence account: detail-focused cognitive style in autism spectrum disorders. J Autism Dev Disord. 2006;36:5–25.
- Uljarevic M, Hamilton A. Recognition of emotions in autism: a formal meta-analysis. J Autism Dev Disord. 2012. doi:10.1007/s10803-012-1695-5.Google Scholar
- Law Smith MJ, Montagne B, Perrett MG, Gallagher L. Detecting subtle facial emotion recognition deficits in high-functioning autism using dynamic stimuli of varying intensities. Neuropsychologia. 2010;48:2777–81.View ArticlePubMedGoogle Scholar
- Southgate V, Gergely G, Csibra G. Does the mirror neuron system and its impairment explain human imitation and autism? In: Pineda JA, editor. The role of mirroring processes in social cognition. New York: Humana Press; 2010.Google Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.