Authors: Muhammad Umair, Nasir Rashid, Umar Shahbaz Khan, Amir Hamza, Ayesha Zeb, Tahir Habib Nawaz, Ali R. Ansari
Understanding human emotions has long been a challenge, but the evolving field of human-computer interaction offers new opportunities to advance our efforts. Emotions provide additional information about the environment and significantly influence decision-making. They also support us in healthcare and educational systems by personalizing our experiences based on emotional states. Considering the significance of emotions, their classification is essential. This paper presents a novel multimodal emotional effect classification approach (NegSl-AIS), designed to classify emotional effects efficiently. The research aims to synthesize information from multiple modalities (electroencephalograph, body temperature, galvanic skin response, electrocardiogram and respiration amplitude) to achieve a holistic understanding of emotional effects. The proposed model NegSl-AIS is inspired by the biological immune system and employs a negative selection-based artificial immune system to classify emotional effects. The recommended solution performs hybrid feature fusion and generates class-specific detectors based on a predefined threshold () to distinguish between self and non-self samples. NegSl-AIS has achieved a classification accuracy of 96.48 % for arousal and 98.63 % for valence on the MAHNOB-HCI dataset. In addition to these parameters, the proposed model attained an overall accuracy of 94 %, with a Cohen’s Kappa of 0.919 and a Matthews Correlation Coefficient (MCC) of 0.920, indicating strong agreement and reliable performance across classes. Obtained results demonstrate commendable classification accuracy, showing competitive performance compared with existing methods. The proposed model integrates multimodal data and aims to enhance the classification accuracy of emotional effects. As technology continues to evolve, emotional effect classification is expected to be integrated into various real-time applications.
