The EU-AIMS Longitudinal European Autism Project (LEAP): design and methodologies to identify and validate stratification biomarkers for autism spectrum disorders

Overall design of the Longitudinal European Autism Project

The project was designed by academic and industry partners and in consultation with the European Medicines Agency (EMA) to increase the chances that stratification biomarkers identified in this study may be qualified to support regulatory decisions for future clinical trials [18]. An overview of our study protocol is given in Table 1 and Additional file 1. Our protocol and standard operation procedures (SOPs) are accessible on https://www.eu-aims.eu/fileadmin/websites/eu-aims/media/EU-AIMS_LEAP/EU-AIMS-LEAP_SOP_StudyProtocol.zip. The study was approved by national and local ethics review boards at each study site and is carried out to Good Clinical Practice (ICH GCP) standards. An overview of the recruitment and study procedures is given in Fig. 1

Participant selection criteria

In the ASD group, inclusion criteria were an existing clinical diagnosis of ASD according to DSM-IV/ICD-10 or DSM-5 criteria. All psychiatric comorbidities (except for psychosis or bipolar disorder) are allowed as up to 70% of people with ASD have one or more co-occurring psychiatric conditions [19]. Similarly, we include participants on stable medication because 30–50% [20] children and adults with ASD in Europe and up to 70% in the USA are prescribed at least one medication for features, such as aggression, anxiety, hyperactivity, or sleep problems [21]. The intellectual disability (ID) group (defined by IQ between 50 and 74) comprises individuals with both idiopathic and syndromic forms of mild intellectual impairments.

Clinical characterisation measures in ASD

In the ASD group, diagnosis is confirmed using the combined information of the Autism Diagnostic Interview-Revised (ADI-R [22]) and the Autism Diagnostic Observation Schedule 2 (ADOS-2). Cut-offs on the ADI-R/ADOS-2 are not used as exclusion criteria [23]. To assess dimensional symptom severity across ASD core domains, we used several parent-report instruments with relative emphasis on social-communication [24, 25], repetitive and restricted behaviours [26, 27], sensory processing anomalies [28, 29], and overall autism symptom severity [30, 31] (see Additional file 1). All scales had been validated for the targeted age ranges. In-house translations/back-translations were carried out for some scales. In adolescents and adults with average IQ, companion self-report versions are also included.

We assess a range of psychiatric disorders using the Development and Well-being Assessment (DAWBA, [32]). The most common comorbidities (attention-deficit/hyperactivity disorder (ADHD), depression, anxiety, sleep anomalies) are also assessed at the symptom level using parent and self-report questionnaires. Parental interviews on adaptive behaviour [33] and parent and/ or self-report questionnaires on quality of life [34] provide additional outcome measures.

Biomarker methodologies

Core measures were selected to be suitable across all age and ability levels to determine whether biomarkers are present only at distinct ages or change with age. We chose measures on which individuals with ASD were previously reported to differ from controls on average and that test several of the most influential neurocognitive (see Table 1) and neurobiological hypotheses of ASD (e.g. differences in brain connectivity [35], excitatory-inhibitory balance [36]). Measures that provide comparable read-outs in animal models and humans were prioritised so that findings can be translated to drug discovery. Some of these measures were taken from the high-risk infant sibling study EUROSIBS (www.eurosibs.eu) allowing us to establish whether some cognitive or neurobiological markers identified in this study also confer risk for developing ASD.

Theory of mind (ToM) [37] (also called mentalising or mindreading) refers to the ability to represent mental states, such as beliefs, desires, and intentions to predict and explain (others’ and own) behaviour. ToM deficits have been widely regarded as core social-cognitive deficits in ASD and have been hypothesised to underlie, or contribute to, a range of social-communicative impairments. However, severity of impairments has been shown to depend on age and ability level. A useful distinction has been made between explicit (i.e. verbally or cognitively mediated) [38] and implicit or spontaneous ToM [39, 40]. Many high-functioning adolescents and adults with ASD acquire some degree of explicit ToM, whereas abnormalities may primarily consist of persistent deficits in the (typically developmentally earlier emerging) implicit theory of mind usage. Therefore, we compare for each participant abnormalities in explicit vs. implicit/spontaneous ToM at the behavioural and (using fMRI) neurofunctional levels because atypical brain processes can persist despite apparently ‘intact’ behaviour.

Emotion recognition refers to the ability to infer other people’s emotions from their facial expressions and is therefore critical for many aspects of social-communication. We assess the ability to recognise a range of simple and complex facial expressions in behavioural tests [41] and using eye-tracking [42] and examine neural responses during the processing of facial expressions using fMRI [43].

Another influential theory has proposed that diminished social motivation may represent a primary deficit in ASD [44] and is often operationalized as diminished spontaneous attention to social (vs. non-social) information when observing (naturalistic) images or social situations, using eye-tracking. Social motivation deficits may in turn be rooted in diminished social reward sensitivity [45], i.e. the reward value of faces and other social information.

Executive functions (EF) is an umbrella term for a set of cognitive processes that rely on prefrontal regions and that include attentional control, inhibitory control, working memory, cognitive flexibility, reasoning, problem solving, and planning. Originally, EF deficits, perhaps notably impairments in cognitive flexibility, were hypothesised to underlie repetitive and restricted behaviours [46, 47]. Whereas evidence for the role of EF in RRBIs is mixed, EF deficits may contribute to both social and non-social ASD symptoms, possibly, by interacting—developmentally or online—with other cognitive systems. Intact EF skills in some individuals may serve as a compensatory mechanism [48] that scaffolds adaptive behaviour [49]. We assess spatial working memory [50] and probabilistic reversal learning [51] using computerised tests and inhibitory control while fMRI blood-oxygen-level dependent (BOLD) responses are recorded.

Weak central coherence (WCC) [52] describes a local, detail-focused information processing style, paired with difficulties in global processing, processing information for meaning and in integrating information in context. WCC is thought to pervade different areas of perception and cognition. This account explicitly aims to address islets of talent (in absolute or relative terms) and spared skills. We include the un/segmented block design task as an index of WCC.

Systemizing [53] describes a cognitive style characterised by the motivation to predict lawful events (using if-then rules) and observations of input-operation-output relationships and includes good attention to detail. Systemizing is thought to represent a continuum where, on average, males exceed females, and individuals with ASD (both males and females) are shifted to the extreme end of aptitude. It also aims to address relative and absolute strengths in ASD. We assess systemizing using age-appropriage versions of the Systemizing Quotient [54].

Top-down processing refers to the fundamental cognitive principle that we use our past experiences and prior knowledge to make sense of the present and predict the future. Top-down processing anomalies have been linked to superior perceptual skills and a more ‘accurate’ or veridical memory in ASD [55].

Predictive coding [56, 57] assumes that the brain constantly matches incoming (external) stimuli against a set of (internal) expectations of what will likely happen. Abnormalities in predictive coding may potentially implicated in several facets of repetitive behaviours, sensory processing anomalies, talents as well as social cognitive abilities [58]. Top-down processing is assessed using eye-tracking change detection and event memory tasks; predictability using by studying mismatch negativity derived from an auditory oddball task (using EEG).

Together, this aims to create a comprehensive profile of each participant’s strengths and weaknesses across cognitive domains. Such a cross-domain profile (or composite markers) may potentially be better in predicting symptom severity or functional outcome than the severity of deficits/differences in one single domain. For core domains (e.g. theory of mind), we use convergent methodologies (e.g. behavioural testing, eye-tracking, and fMRI) in the same participant, to identify atypical processes and compensatory mechanisms.

Markers of brain structure, function, and connectivity

We use magnetic resonance imaging (MRI) and electroencephalography (EEG) to study differences in brain structure [65], function, and connectivity [35, 66]. These methods are critical to delineating ASD subgroups based on systems-level abnormalities and provide the basis for identifying the mechanisms through which (future) treatments may produce improvements in functioning.

Structural connectivity reflects physical connections between neurons. Its strength depends on the number and efficacy of synapses and in turn affects functional connectivity. We derive indices of structural connectivity both from structural MRI scans and diffusion tensor imaging (DTI). For example, intrinsic grey matter connectivity can be estimated by examining differences in local and global wiring costs [70] and differences in short and long-range white matter tracts using tractography analysis of specific pathways [71].

The paradigms have been adapted such that each task can be acquired within 5–10 min, as it is challenging for young children and some individuals with ASD (and especially ID) to remain still in the scanner for longer periods of time.

Good test-retest reliability of the fMRI battery was demonstrated in typically developing adults [75, 76]. A pilot study confirmed the feasibility of the tasks for use in children and individuals with ASD. We will use region-of-interest analyses of known areas comprising a particular network (Table 2) as well as exploratory whole-brain analyses in order to investigate potential abnormalities in both activation and functional connectivity within and across tasks.

Stratification by participant characterisation criteria

For each measure, we will first test for overall case-control differences and then stratify the sample by age, IQ, sex, and the presence of comorbidities. To investigate age effects in, for example, brain anatomy, resting-state connectivity, or cognitive skills, we first create ‘cross-sectional developmental trajectories’ or ‘growth charts’ for the typically developing (TD) group that test for linear and nonlinear (e.g. quadratic) developmental patterns and determine the typical variability at a particular age [98]. Then, we use confidence intervals around the TD trajectory to assess for each individual with ASD whether, and by how far, he or she falls outside the range of performance expected for their age group. This will help determine whether the abnormality is only detected at a certain developmental stage, or in a subgroup of individuals with ASD across ages. We will also compare several trajectories simultaneously using mixed design linear regression models [98] to establish whether in the ASD group, performance develops with delay or is uneven across domains or component processes. Cross-sectional age-related patterns will be compared to longitudinal (within person) trajectories once follow-up data are available.

Sex differences have previously been reported both in typical development and ASD groups at multiple levels, including serum biomarkers, brain structure and function, several aspects of cognition, and clinical symptom presentation [16]. Likewise, IQ or psychiatric comorbidities may significantly impact on brain and cognitive profile. Hence, we will test diagnosis-by-sex models to identify potentially sex-specific biomarkers and explore whether neurocognitive or neurobiological abnormalities vary with IQ or the presence of psychiatric comorbidities, using both dimensional and categorical approaches. We will also consider potentially mediating (e.g. sleep problems) or moderating factors, such as handedness (lateralisation), medication, and the narrow vs. broader ASD spectrum (ADI-R/ADOS-2 cut-offs).

Progress has also been made in developing machine learning techniques for neuroimaging data in order to make clinically relevant predictions [99, 100]. Previous proof-of-concept data show that multivariate pattern classification approaches using structural MRI data discriminated individuals with ASD from healthy controls and non-autistic neurodevelopmental disorders with 90% accuracy [101]. We will apply multivariate approaches to see whether they can discriminate a priori defined subtypes (sex, comorbidities).

To test the potential value of each candidate marker as a ‘surrogate end-point’, we will use correlation and regression analyses to establish whether it relates to or predicts symptom severity (overall, or in a particular domain) or level of adaptive behaviour [102]. For each measure, we will report p values adjusted for the number of these core analyses, as well as nominally significant p values (to enable comparison with previous studies), effect sizes, and descriptive information on frequency and severity of deficits. For quantitative stratification markers to be of clinical utility, it will be essential to delineate reference values and cut-offs to aid the interpretation of individual scores.

Stratification by unsupervised, data-driven approaches

The second approach uses data-driven multivariate analysis techniques to identify subgroups based on the pattern of the data itself.

For example, cluster analyses are a widely used set of techniques to divide data into prototypical groups based on only the data points and their relationships to one another. Input variables could be multiple cognitive [103], eye-tracking indices [104], EEG values, or a combination of values from different data types. The optimal number of (meaningful) clusters can be determined based on height differences in a cluster tree, while cluster robustness will be evaluated using cross-validation techniques, such as bootstrapping.

Normative modelling approaches have also recently been extended to model biological variation across the entire study sample or a typical population [105]. Gaussian process regression is used to predict a set of biological responses (e.g. structural indoor connectivity indices) from a set of clinically relevant covariates (e.g. quantitative cognitive or symptom scores), while estimating predictive confidence for every prediction. This approach enables identification of individuals who are outliers within this distribution and to quantify the degree of deviation in relation to specific symptom domains.

Functional data analysis (FDA) takes advantage of trial level data, using curves or trajectories as observational units (for example, eye-tracking gaze paths over time, individual ERP waveforms over the course of the experiment), rather than signals averaged across trials. One recent EEG study reported increased variability of task-related activity in ASD [106]. By combining this approach with a robust multi-level clustering method, recent findings showed distinct learning patterns in particular ASD subgroups [107].

We will also apply recently developed unsupervised techniques to identify meaningful subtypes from the structural and functional neuroimaging data [100]. Further extensions of these methods enable combining different data types (e.g. connectivity indices derived from DTI and resting-state EEG and fMRI) [108], which may further increase specificity/sensitivity of classifiers. Mandatory for these approaches is splitting the data into training and test data sets to avoid ‘overfitting’ and to establish how well the classifiers can predict to which subgroup a new individual belongs.

Molecular biomarkers are potentially of particularly high value to predict treatment response. We will use novel network-based stratification approaches similar to those that have recently been validated in cancer research to identify tumour subtypes that are predictive of patient survival or response to therapy [109]. This method integrates genomic information from each individual with functional gene networks (e.g. protein-protein interaction), leveraging prior knowledge to stratify patients in subgroups with specific molecular profiles. We then aim to map those molecular subgroups to biological pathways, structural and functional biomarkers, and clinical symptom profiles.

Longitudinal follow-up

To test the value of candidate stratification markers in predicting symptom progression, we will initially track the relationship between changes in the neurobiological/ cognitive measure and clinical or behaviour indices at 12–24 months follow-up. In addition, we are seeking additional funding for a third assessment wave to construct for each individual developmental trajectories at multiple levels. This will enable us to ascertain whether subgroups whose (social-communicative, RRBI) symptoms improve, remain the same, or worsen over development [110] differ in terms of their neurobiological/cognitive profile at a given time or the rate of changes (e.g. arrested, uneven across component processes) across particular developmental stages.

Twin data

Twin data are analysed by applying a statistical framework of multiply adjusted (conditional) linear regressions based on generalised estimations equations (GEE) and allowing both categorical and dimensional ASD outcomes. In addition to the GEE model, an additive genetics, common environment, and unique environmental (ACE) model will be computed to determine heritability estimates. For all analyses, probability estimates for different twin groups will be included, based on the population-based twin cohorts, which allows generalizability of the results.

Biomarker validation

We will adopt biomarker validation criteria and steps similar to those employed in other biomedical fields, such as oncology, where biomarkers are ‘fit for purpose’, i.e. used in clinical practice [111]. Key criteria against which candidate stratification biomarkers will be validated are performance characteristics (accuracy and reliability) of the measure, reliability in relating to a particular clinical endpoint/clinical symptoms, and its prognostic and/or predictive value. For stratification markers of a priori defined subgroups (e.g. sex, comorbidity), accuracy (i.e. sensitivity and specificity, positive and negative predictive value) can be established using receiver operating characteristic (ROC) curves. For subgroups derived from data-driven, unsupervised approaches, external validation is essential as these groups do not necessarily differ in terms of their clinical profile. They may be validated by demonstrating their biological plausibility (i.e. that they have different genetic causes or molecular mechanisms) or functional value (that they differ in terms of their developmental trajectory or respond differentially to a given treatment). The latter cannot be tested in observational studies. Instead, the marker will need to be included in the design of treatment studies or clinical trials in order to compare responders and non-responders in terms of their biomarker characteristics [112]. To ascertain reproducibility, replication in independent samples is essential. For this purpose, we are sharing our protocols and SOPs with other interested international research groups with whom we also have formal data-sharing agreements. They currently include the Australian Cooperative Research Centres (CRC), the French Fondation FondaMental, the Chinese Key 973 program, the Foundation for the National Institutes of Health (FNIH) Autism Biomarker Consortium, and the Province of Ontario Neurodevelopmental Disorders (POND) network. To investigate whether any of these stratification markers are specific to ASD we have aligned several measures with parallel European networks focused on ADHD, obsessive-compulsive disorder, and conduct disorder (MATRICS, TACTICS, NeuroIMAGE [113]).

Demographic information of the baseline cohort and assessment rates

Recruitment and assessment of the baseline case-control cohort was carried out between January 2014 (first subject first visit, FSFV) and August 2016, except for schedule D, where recruitment and assessment are ongoing. Follow-up assessments began in September 2015, and all assessments (including schedule D baseline visits) are scheduled to be completed by August 2017. As per protocol, the twins at KI (schedule E) are only seen at one time point. Also, adults at UCBM are only seen at one time point. At the time of writing (08 April 2017), 448 participants have completed their follow-up assessments. Across study sites, retention rates range between 80.3 and 96.2%. (KCL 96.2%, RUNMC 84.7%, UMCU 86.8, UCAM 87.8%, CIMH 80.3%).

‘Lessons learnt’ and future directions

The scale and level of integrated phenotypic and genomic characterisation of the LEAP cohort is unprecedented in ASD research. Within a 5-year period, we have been able to comprehensively assess a large cohort of children, adolescents, and young adults from 6 to 30 years at two time points. As the final sample slightly exceeds the original recruitment target, the study (accounting for missing data and data loss after QC procedures) has the power to detect subgroups with medium effect sizes.

However, we also wish to acknowledge limitations of the study, and ‘share lessons’ learnt regarding the study design, recruitment, and data acquisition.

We deliberately used few participant exclusion criteria, and LEAP is one of the few biomarker—and more specifically neuroimaging—studies that includes people with ASD and mild intellectual disability. However, we did exclude individuals with moderate, severe, and profound ID. This decision was largely made because at the start of the study, it was deemed difficult to use some of the same measures/tests in individuals with severe ID, including MRI scanning (without sedation). However, we are currently carrying out a companion single-site study (EU-AIMS Synaptic Gene Project, SynaG) that focuses on individuals with rare monogenic forms of ASD (e.g. SHANK3), most of whom have severe-to-profound ID.