This work presents a real-time, edge-deployable system for multimodal stress detection with natural language feedback, prioritizing explainability, privacy, and emotional intelligence. Lightweight machine learning models, including logistic regression and gradient boosting, independently process physiological signals (heart rate variability, skin conductance, skin temperature), behavioral patterns (keystroke dynamics, mouse movements), and facial expressions. Each modality produces an interpretable stress score normalized to the range [−1, +1], allowing users to identify the most influential signals rather than receiving a single opaque output. A local large-language model (Mistral-7B-Instruct) translates these scores into concise two-sentence feedback: the first summarizes stress on a four-level scale from calm to high stress, while the second suggests a coping strategy such as grounding or breathing exercises. The integration of a local LLM enables context-aware, natural feedback generation while safeguarding user privacy. As all inference runs locally, the system avoids cloud dependency and ensures privacy, making it suitable for resource-constrained or sensitive environments. The preliminary evaluations with synthetic and real sensor inputs demonstrate that the system is modular, interpretable, and effective in providing actionable feedback, paving the way for human-centered and explainable stress-aware technologies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Novel Explainable Multimodal Edge Framework for Stress Detection

  • Ojasvi,
  • Kashish,
  • Ravneet Kaur,
  • Rajendra Kumar Roul,
  • Shalini Batra

摘要

This work presents a real-time, edge-deployable system for multimodal stress detection with natural language feedback, prioritizing explainability, privacy, and emotional intelligence. Lightweight machine learning models, including logistic regression and gradient boosting, independently process physiological signals (heart rate variability, skin conductance, skin temperature), behavioral patterns (keystroke dynamics, mouse movements), and facial expressions. Each modality produces an interpretable stress score normalized to the range [−1, +1], allowing users to identify the most influential signals rather than receiving a single opaque output. A local large-language model (Mistral-7B-Instruct) translates these scores into concise two-sentence feedback: the first summarizes stress on a four-level scale from calm to high stress, while the second suggests a coping strategy such as grounding or breathing exercises. The integration of a local LLM enables context-aware, natural feedback generation while safeguarding user privacy. As all inference runs locally, the system avoids cloud dependency and ensures privacy, making it suitable for resource-constrained or sensitive environments. The preliminary evaluations with synthetic and real sensor inputs demonstrate that the system is modular, interpretable, and effective in providing actionable feedback, paving the way for human-centered and explainable stress-aware technologies.