Stress and accompanying physiological responses can occur when everyday emotional, mental and physical challenges exceed one’s ability to cope. Long-Term exposure to stressful situations can have negative health consequences, such as increased risk of cardiovascular diseases and immune system disorder. It is also shown to adversely affect productivity, wellbeing, and self-confidence, which can lead to social and economic inequality. Hence, timely stress recognition can contribute to better strategies for its management and prevention in the future. Stress can be detected from multimodal physiological signals (e.g., skin conductance and heart rate) using well-trained models. However, these models need to be adapted to a new target domain and personalized for each test subject. In this paper, we propose a deep reconstruction classification network and multitask learning (MTL) for domain adaptation and personalization of stress recognition models. The domain adaptation is achieved via a hybrid model consisting of temporal convolutional and recurrent layers that perform shared feature extraction through supervised source label predictions and unsupervised target data reconstruction. Furthermore, MTL based neural network approach with hard parameter sharing of mutual representation and task-specific layers is utilized to acquire personalized models. The proposed methods are tested on multimodal physiological time-series data collected during driving tasks, in both real-world and driving simulator settings.

Model Adaptation

Unsupervised Domain Adaptation. We formulate model adaptation as a cross-domain and cross-user transfer learning problem. Here, a model trained on a dataset collected in a specific setting (simulator) or source domain has to be adapted to perform the same task in a different situation (real-world) or target domain. There are several challenges for learning an optimal model in this case, such as unavailability of ground-truth for the target domain, expensive process of acquiring a large number of labels and dynamic shift in data distribution. Therefore, target data cannot be directly used for fine-tuning an existing model in a supervised manner. In this work, we utilize deep reconstruction and classification network for unsupervised domain adaptation to learn from source and target datasets jointly. This learning setting resembles MTL in the sense that learning an auxiliary task can help improve performance for the actual task through a shared representation. For more information see section II.B of [1].


Personalization (Subjects as Tasks)

Multi-task Learning for Personalization. A subject-independent global model for stress detection may perform poorly due to large interpersonal variations in physiological responses, e.g. due to age, gender, sleep, and diet. In order to take these disparities into account, we personalize a model by applying deep MTL with the subjects-as-tasks approach. MTL involves finding a unified model for solving more than one task with a shared representation. In this work, we develop two model architectures for the MTL setting, one based on the temporal convolutional neural network for end-to-end representation learning and second a feed-forward neural network trained with manually extracted features from heart rate and skin conductance.



Paper [1] Paper [2]


[1] Aaqib Saeed, Tanir Ozcelebi, Johan Lukkien, Jan B.F. van Erp, and Stojan Trajanovski, "Model Adaptation and Personalization for Physiological Stress Detection." in IEEE International Conference on Data Science and Advanced Analytics (DSAA), 2018.

[2] Aaqib Saeed and Stojan Trajanovski, "Personalized Driver Stress Detection with Multi-task Neural Networks using Physiological Signals." in Machine Learning for Health Workshop at 31st Conference on Neural Information Processing Systems, 2017.


    title={Model Adaptation and Personalization for Physiological Stress Detection},
    author={Saeed, Aaqib and Ozcelebi, Tanir and Lukkien, Johan and van Erp, Jan and Trajanovski, Stojan},
    booktitle={2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA)},
    title={Personalized driver stress detection with multi-task neural networks using physiological signals},
    author={Saeed, Aaqib and Trajanovski, Stojan},
    journal={arXiv preprint arXiv:1711.06116},


  • Rich Caruana. "Multitask learning." Learning to learn. Springer US, 1998. 95-133.
  • A. L. Dougall and A. Baum, “Stress, health, and illness,” Handbook of health psychology, pp. 321–337, 2001.
  • Muhammad Ghifary, et al. "Deep reconstruction-classification networks for unsupervised domain adaptation." ECCV. Springer, Cham, 2016.
  • J. Healey and R. W. Picard, "Driver stress data", Retrieved June 26th from MIT Affective Computing Group: http://affect.media.mit.edu, 2002.
  • Aaqib Saeed, et al. "Deep physiological arousal detection in a driving simulator using wearable sensors." ICDMW. IEEE, 2017.
  • S. Taamneh, et al. "A multimodal dataset for various forms of distracted driving", Scientific data, vol. 4, p. 170110, 2017.