Multimodal AI to Unravel the Neurodevelopmental Cascade: From Early Movements to Advanced Communicative and Social Skills

This thesis explores the application of multimodal Artificial Intelligence (AI) to advance early diagnosis and understanding of Neurodevelopmental Disorders (NDD) within the framework of the neurodevelopmental cascade theory. NDD present significant diagnostic and therapeutic challenges due to their heterogeneity and overlapping symptoms, requiring new scalable and precise methodologies. By integrating AI across multiple domains and examining how motor, communicative, attentional, and socio-emotional behaviors dynamically interact throughout development, this research identifies early biomarkers, uncovers novel patterns, and addresses traditional assessment limitations. A marker-less analysis of newborns’ spontaneous movements, using Deep Learning (DL), identified kinematic patterns of delays in foot motor development in 10-day-old infants, predicting adverse outcomes with 85% accuracy. Furthermore, longitudinal analyses of motor behaviors during social engagement and objectreaching tasks revealed cascading effects of early hand movements at six months, highlighting how the emergence of reaching serves as a crucial precursor by enabling active exploration. This early motor foundation plays a pivotal role in shaping later communicative and social development. A microanalytic study of gestures, gaze, and language coordination in naturalistic parent-child interactions characterized socio-communicative behaviors, highlighting reduced complexity in neurodivergent toddlers. Building upon this, an automatic coding system based on a transformer architecture was developed to identify deictic gestures from videos with high performance, demonstrating the feasibility of scalable gesture recognition. Gaze behaviors, modeled through a novel eye-tracking paradigm and Markov chains, revealed divergences in attentional dynamics in preschool-aged children with NDD, such as increased gaze aversion and repetitive nonsocial focus, reflecting sensory coping strategies and their downstream effects on social cognition. Additionally, the analysis of the expression of vitality forms linked motor behaviors with socio-communicative skills, emphasizing how emotional expressions differ in social contexts, further reinforcing the cascading nature of developmental processes. Finally, a hierarchical AI model combining Machine Learning (ML) and DL was developed to support automatic speech analysis in school-aged children, identifying dysarthria with 90% accuracy and stratifying its severity with 80% accuracy. This final stage of the research exemplifies how initial sensory-motor and attentional variations contribute to later expressive and linguistic differences, reinforcing the longitudinal approach of this study. These findings demonstrate the dynamic potential of AI to integrate multimodal dimensions and unravel the complex interplay underlying neurodiverse trajectories. By adopting a developmental perspective, this study underscores how disruptions in foundational skills propagate through interconnected domains, influencing later functional outcomes. This approach paves the way for improved diagnostic accuracy, enabling earlier and more personalized interventions that align with individual neurodevelopmental profiles.