Multimodal Sentiment Analysis and Improved Transformer-Based Emotional Music Generation Method for Elderly People in Nursing Homes

Abstract

Aiming at the predicaments of loneliness, depression, and cognitive decline commonly faced by elderly people in nursing homes-where existing manual emotional support struggles to achieve accurate responses due to limited nursing resources. This study conducts research on multimodal sentiment analysis (integrating facial images, EEG signals, and EOG signals) and improved Transformer-based emotional music generation for this group. The research holds significant theoretical supplementary value and practical application significance: it not only provides a new paradigm of multimodal fusion for the field of elderly affective computing but also offers a precise emotional and cognitive intervention solution for the elderly in nursing homes, alleviating their negative emotions and reducing the burden on caregivers. The core innovations of the study are reflected in two aspects: First, a multimodal sentiment analysis model is constructed with optimized design in the feature extraction stage. An improved PFLD model incorporating a Feature Module is adopted to enhance the accuracy of facial landmark detection through multi-scale feature fusion; the Linear Dynamic System (LDS) is used to smooth EEG signal features for reducing noise interference; continuous wavelet transform and peak detection are combined to extract key features (e.g., blinking, fixation) from EOG signals; and a 1D Convolutional Neural Network (1DCCNN) is designed to adapt to the temporal characteristics of physiological signals. Finally, multimodal features are fused via a convolutional encoder and input into a Recurrent Neural Network (RNN) for sentiment classification, breaking the limitations of single-modal recognition. Second, the Discrete-Emo Transformer emotional music generation model is proposed: the Flash attention mechanism is introduced to alleviate memory bottlenecks and improve inference speed; the segment recurrence mechanism of Transformer-XL is integrated to enhance the model’s ability to learn long-range dependencies in long music sequences; and the REMI-EMO representation method is used to simultaneously model note features and emotional information, enabling high-quality symbolic music generation under emotional conditions. Experimental results show that the multimodal sentiment analysis model achieves an accuracy of 97.5%, outperforming mainstream models such as FDMER and MISA. The Discrete-Emo Transformer model reaches an emotional accuracy of 78.4% and performs optimally in the Perplexity (PPL) index (1.71), generating music that is more in line with target emotions and human creative characteristics. This study provides key technical support for "technology-empowered elderly care" and effectively improves the accuracy and efficiency of emotional intervention in nursing homes.