1

Multimodal Dialog Based Remote Patient Monitoring of Motor Function in Parkinson’s Disease and Other Movement Disorder

The need for remote monitoring to support Parkinson’s Disease (PD) patients, caregivers, and healthcare professionals in their collaborative efforts for better care has never been greater. We have earlier presented Tina, a virtual agent that is …

On the Robust Automatic Computation of Speaking and Articulation Duration in ALS Patients Versus Healthy Controls

Speaking and articulation duration have been shown to be important biomarkers of disorder severity in ALS and other motor speech disorders. While signal processing algorithms afford significant advantages over manual annotation in that they allow for …

Multimodal Dialog Based Speech and Facial Biomarkers Capture Differential Disease Progression Rates for ALS Remote Patient Monitoring

Continuous monitoring of ALS symptoms is crucial to improve quality of life because both survival and progression rate can vary substantially between individuals. Speech and facial biomarkers can serve as useful proxies for disease progression and …

Voice Activity Detection Considerations in a Dialog Agent for Dysarthric Speakers

Conversational dialog technology is increasingly being recognized as auseful means of automating remote patient monitoring and diagnostics for dysarthric speakers at scale. However, the characteristics of dysarthric speech introduce multiple …

Unsupervised Online Grounding for Social Robots (Extended Abstract)

Multimodal Conversational Technology for Remote Assessment of Symptom Severity in People with Schizophrenia

The causes of schizophrenia are complex and largely unknown. Current treatments focus on the management of positive or psychotic symptoms. However, negative symptoms like lack of facial and verbal expression, monotone speech, social withdrawal and …

Investigating the Interplay Between Affective, Phonatory and Motoric Subsystems in Autism Spectrum Disorder Using a Multimodal Dialogue Agent

We explore the utility of an on-demand multimodal conversational platform in extracting speech and facial metrics in children with Autism Spectrum Disorder (ASD). We investigate the extent to which these metrics correlate with objective clinical …

Investigating the Utility of Multimodal Conversational Technology and Audiovisual Analytic Measures for the Assessment and Monitoring of Amyotrophic Lateral Sclerosis at Scale

We propose a cloud-based multimodal dialog platform for the remote assessment and monitoring of Amyotrophic Lateral Sclerosis (ALS) at scale. This paper presents our vision, technology setup, and an initial investigation of the efficacy of the …

Emotion Intensity and Gender Detection via Speech and Facial Expressions

Human emotion detection has received increasing attention over the last decades for a variety of applications and systems. However, detecting the intensity of the expressed emotion has not been investigated as much as detecting the type of the …

Toward Remote Patient Monitoring of Speech, Video, Cognitive and Respiratory Biomarkers Using Multimodal Dialog Technology

We demonstrate a multimodal conversational platform for remote patient diagnosis and monitoring. The platform engages patients in an interactive dialog session and automatically computes metrics relevant to speechacoustics and articulation, oro-motor …