The correlation between visual perception and the evoked emotional response has been investigated extensively in the area of cognitive psychology using the method of electroencephalography (EEG). Activities in the visual cortex have been found to be correlated with psychophysical performance, visual attention, and subjective perceptual experience. While EEG could record the cerebral activities accurately, the subject's visual attention was left out of consideration.
The schematic diagram of the system
In the study, we proposed and developed a multimodal detecting and analytic system for visual perception and emotional response using a combination of EEG, eye tracking, and heart rate. We maximized the portability of the system while ensuring the accuracy and reliability of the data. The effectiveness of the system was tested by an experiment based on negative, positive, and neutral images selected from the ThuPIS image system. 10 participants took part in the user study and their emotions were analyzed in valence and arousal dimensions as they were observing the images.
The results indicated positive correlations between valence and power of EEG signals over the occipital lobe at low frequencies and over the left and right temporal lobes at high frequencies. The saccade magnitude and fixation duration of eye movements also function as indicators of emotion. We designed a compact module of the system for psychological diagnosis and treatment. Our system may contribute to improving the precision of visual-based psychological treatment by providing real-time emotion analysis.
The correlations between the emotional dimensions and the images from ThuPIS
The design for the psychological diagnosis and treatment module
The design for the psychological diagnosis and treatment module (front panel)
Kunpeng Huang, "Multimodal detecting and analytic system for visual perception and emotional response," 2017 IEEE Integrated STEM Education Conference (ISEC), 2017, pp. 112-112, DIO: 10.1109/ISECon.2017.7910224.