Parkinson’s disease (PD) and mild cognitive impairment (MCI) affect both motor and cognitive linguistic abilities. This work proposes a multimodal framework for detecting PD and MCI from naturalistic retelling tasks by combining acoustic and linguistic features. Speech features capture prosodic, articulatory, and phonemic properties linked to hypokinetic dysarthria, while language features model lexical, syntactic, and semantic complexity, including a novel motility-based representation that quantifies the use of action-related verbs. Each modality is independently evaluated and combined following early and late fusion strategies based on support vector machines. The results confirm speech features as good biomarkers to model PD and MCI, and show the motility-based language features improve specificity, particularly in distinguishing cognitive decline in Parkinson’s patients, i.e., PD patients with MCI vs. patients without MCI. Fusion strategies further improve classification performance, confirming the complementarity of speech and language. These findings support the use of retelling-based multimodal analyses as promising tools for early and non-invasive screening of neurodegenerative conditions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Verb Motility Dynamics Reveals Cognitive Impairment in Parkinson’s Disease: A Speech-Language Fusion Approach

  • Jhon Fredy Mercado-Agudelo,
  • Daniel Escobar-Grisales,
  • Cristian David Ríos-Urrego,
  • Adolfo M. García,
  • Yamile Bocanegra,
  • Leonardo Moreno,
  • Elmar Nöth,
  • Juan Rafael Orozco-Arroyave

摘要

Parkinson’s disease (PD) and mild cognitive impairment (MCI) affect both motor and cognitive linguistic abilities. This work proposes a multimodal framework for detecting PD and MCI from naturalistic retelling tasks by combining acoustic and linguistic features. Speech features capture prosodic, articulatory, and phonemic properties linked to hypokinetic dysarthria, while language features model lexical, syntactic, and semantic complexity, including a novel motility-based representation that quantifies the use of action-related verbs. Each modality is independently evaluated and combined following early and late fusion strategies based on support vector machines. The results confirm speech features as good biomarkers to model PD and MCI, and show the motility-based language features improve specificity, particularly in distinguishing cognitive decline in Parkinson’s patients, i.e., PD patients with MCI vs. patients without MCI. Fusion strategies further improve classification performance, confirming the complementarity of speech and language. These findings support the use of retelling-based multimodal analyses as promising tools for early and non-invasive screening of neurodegenerative conditions.