Log in

Relevant bibliographies by topics / Multimodal perception of emotion / Journal articles

To see the other types of publications on this topic, follow the link: Multimodal perception of emotion.

Journal articles on the topic 'Multimodal perception of emotion'

Author: Grafiati

Published: 9 March 2023

Last updated: 19 July 2025

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Multimodal perception of emotion.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

1

Barabanschikov, Vladimir A., and Ekaterina V. Suvorova. "Expression and perception of multimodal emotional states." National Psychological Journal 51, no. 3 (2023): 106–27. http://dx.doi.org/10.11621/npj.2023.0311.

Full text

Abstract:

Background. The urge for deeper knowledge on the nature of emotion expression and perception in ecologically and socially valid conditions is of grate importance. The scope is to develop experimental procedures recording not only the demonstration of emotions, but also actual human emotional experience. Objective. The objective is to reveal the patterns of expression and identification of multimodal dynamic emotional states, based on the stimuli, created by professional actors. Sample. The experiments involved 96 (48 women and 48 men) specialists in various fields of practice as well as underg

APA, Harvard, Vancouver, ISO, and other styles

2

Chen, Zhongbo. "Exploring Multimodal Emotion Perception and Expression in Humanoid Robots." Applied and Computational Engineering 174, no. 1 (2025): 86–91. https://doi.org/10.54254/2755-2721/2025.po24880.

Full text

Abstract:

Traditional single-modal emotion recognition is limited by environmental sensitivity, which means that under different environmental conditions, the effect of emotion recognition may be affected, resulting in a decrease in recognition accuracy. Multimodal emotion recognition improves the accuracy of emotion recognition through the complementarity of features between different modalities. In view of the key role of emotion in human cognition and behavior, this paper explains the adaptation laws of emotional robots in medical care, education and growth scenarios by systematically analyzing unimo

APA, Harvard, Vancouver, ISO, and other styles

3

de Boer, Minke J., Deniz Başkent, and Frans W. Cornelissen. "Eyes on Emotion: Dynamic Gaze Allocation During Emotion Perception From Speech-Like Stimuli." Multisensory Research 34, no. 1 (2020): 17–47. http://dx.doi.org/10.1163/22134808-bja10029.

Full text

Abstract:

Abstract The majority of emotional expressions used in daily communication are multimodal and dynamic in nature. Consequently, one would expect that human observers utilize specific perceptual strategies to process emotions and to handle the multimodal and dynamic nature of emotions. However, our present knowledge on these strategies is scarce, primarily because most studies on emotion perception have not fully covered this variation, and instead used static and/or unimodal stimuli with few emotion categories. To resolve this knowledge gap, the present study examined how dynamic emotional audi

APA, Harvard, Vancouver, ISO, and other styles

4

Gao, Xiyuan, Shekhar Nayak, and Matt Coler. "Enhancing sarcasm detection through multimodal data integration: A proposal for augmenting audio with text and emoticon." Journal of the Acoustical Society of America 155, no. 3_Supplement (2024): A264. http://dx.doi.org/10.1121/10.0027441.

Full text

Abstract:

Sarcasm detection presents unique challenges in speech technology, particularly for individuals with disorders that affect pitch perception or those lacking contextual auditory cues. While previous research [1, 2] has established the significance of pitch variation in sarcasm detection, these studies have primarily focused on singular modalities, often overlooking the potential synergies of integrating multimodal data. We propose an approach that synergizes auditory, textual, and emoticon data to enhance sarcasm detection. This involves augmenting sarcastic audio data with corresponding text u

APA, Harvard, Vancouver, ISO, and other styles

5

Vallverdú, Jordi, Gabriele Trovato, and Lorenzo Jamone. "Allocentric Emotional Affordances in HRI: The Multimodal Binding." Multimodal Technologies and Interaction 2, no. 4 (2018): 78. http://dx.doi.org/10.3390/mti2040078.

Full text

Abstract:

The concept of affordance perception is one of the distinctive traits of human cognition; and its application to robots can dramatically improve the quality of human-robot interaction (HRI). In this paper we explore and discuss the idea of “emotional affordances” by proposing a viable model for implementation into HRI; which considers allocentric and multimodal perception. We consider “2-ways” affordances: perceived object triggering an emotion; and perceived human emotion expression triggering an action. In order to make the implementation generic; the proposed model includes a library that c

APA, Harvard, Vancouver, ISO, and other styles

6

Shackman, Jessica E., and Seth D. Pollak. "Experiential Influences on Multimodal Perception of Emotion." Child Development 76, no. 5 (2005): 1116–26. http://dx.doi.org/10.1111/j.1467-8624.2005.00901.x.

Full text

APA, Harvard, Vancouver, ISO, and other styles

7

Portnova, Galina V., and Daria A. Stebakova. "The multimodal emotion perception in codependent individuals." Neuroscience Research Notes 6, no. 1 (2023): 210. http://dx.doi.org/10.31117/neuroscirn.v6i1.210.

Full text

Abstract:

The emotional disturbances of individuals with codependency are often ignored. This study aimed to investigate the emotional perception of codependent individuals in four modalities – visual, auditory, tactile and olfactory. An EEG study was performed and presented pleasant and unpleasant stimuli selected by a panel of experts for each modality. Participants (fifteen codependent individuals and fifteen healthy volunteers) were instructed to assess the emotional impact and pleasantness of stimuli. The method of EEG spaces was used to visualize how close perceived stimuli were according to EEG d

APA, Harvard, Vancouver, ISO, and other styles

8

Barabanschikov, V. A., and E. V. Suvorova. "Gender Differences in the Recognition of Emotional States." Психологическая наука и образование 26, no. 6 (2021): 107–16. http://dx.doi.org/10.17759/pse.2021260608.

Full text

Abstract:

As a rule, gender differences in the perception of human emotional states are studied on the basis of static pictures of face, gestures or poses. The dynamics and multiplicity of the emotion expression remain in the «blind zone». This work is aimed at finding relationships in the perception of the procedural characteristics of the emotion expression. The influence of gender and age on the identification of human emotional states is experimentally investigated in ecologically and socially valid situations. The experiments were based on the Russian-language version of the Geneva Emotion Recognit

APA, Harvard, Vancouver, ISO, and other styles

9

Marino, David, Max Henry, Pascal E. Fortin, Rachit Bhayana, and Jeremy Cooperstock. "I See What You're Hearing: Facilitating The Effect of Environment on Perceived Emotion While Teleconferencing." Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023): 1–15. http://dx.doi.org/10.1145/3579495.

Full text

Abstract:

Our perception of emotion is highly contextual. Changes in the environment can affect our narrative framing, and thus augment our emotional perception of interlocutors. User environments are typically heavily suppressed due to the technical limitations of commercial videoconferencing platforms. As a result, there is often a lack of contextual awareness while participating in a video call, and this affects how we perceive the emotions of conversants. We present a videoconferencing module that visualizes the user's aural environment to enhance awareness between interlocutors. The system visualiz

APA, Harvard, Vancouver, ISO, and other styles

10

Bi, Xin, and Tian Zhang. "Analysis of the fusion of multimodal sentiment perception and physiological signals in Chinese-English cross-cultural communication: Transformer approach incorporating self-attention enhancement." PeerJ Computer Science 11 (May 23, 2025): e2890. https://doi.org/10.7717/peerj-cs.2890.

Full text

Abstract:

With the acceleration of globalization, cross-cultural communication has become a crucial issue in various fields. Emotion, as an essential component of communication, plays a key role in improving understanding and interaction efficiency across different cultures. However, accurately recognizing emotions across cultural backgrounds remains a major challenge in affective computing, particularly due to limitations in multimodal feature fusion and temporal dependency modeling in traditional approaches. To address this, we propose the TAF-ATRM framework, which integrates Transformer and multi-hea

APA, Harvard, Vancouver, ISO, and other styles

11

Yamauchi, Takashi, Jinsil Seo, and Annie Sungkajun. "Interactive Plants: Multisensory Visual-Tactile Interaction Enhances Emotional Experience." Mathematics 6, no. 11 (2018): 225. http://dx.doi.org/10.3390/math6110225.

Full text

Abstract:

Using a multisensory interface system, we examined how people’s emotional experiences change as their tactile sense (touching a plant) was augmented with visual sense (“seeing” their touch). Our system (the Interactive Plant system) senses the electrical capacitance of the human body and visualizes users’ tactile information on a flat screen (when the touch is gentle, the program draws small and thin roots around the pot; when the touch is more harsh or abrupt, big and thick roots are displayed). We contrasted this multimodal combination (touch + vision) with a unimodal interface (touch only o

APA, Harvard, Vancouver, ISO, and other styles

12

Lavan, Nadine, and Carolyn McGettigan. "Increased Discriminability of Authenticity from Multimodal Laughter is Driven by Auditory Information." Quarterly Journal of Experimental Psychology 70, no. 10 (2017): 2159–68. http://dx.doi.org/10.1080/17470218.2016.1226370.

Full text

Abstract:

We present an investigation of the perception of authenticity in audiovisual laughter, in which we contrast spontaneous and volitional samples and examine the contributions of unimodal affective information to multimodal percepts. In a pilot study, we demonstrate that listeners perceive spontaneous laughs as more authentic than volitional ones, both in unimodal (audio-only, visual-only) and multimodal contexts (audiovisual). In the main experiment, we show that the discriminability of volitional and spontaneous laughter is enhanced for multimodal laughter. Analyses of relationships between aff

APA, Harvard, Vancouver, ISO, and other styles

13

Portnova, Galina, Aleksandra Maslennikova, Natalya Zakharova, and Olga Martynova. "The Deficit of Multimodal Perception of Congruent and Non-Congruent Fearful Expressions in Patients with Schizophrenia: The ERP Study." Brain Sciences 11, no. 1 (2021): 96. http://dx.doi.org/10.3390/brainsci11010096.

Full text

Abstract:

Emotional dysfunction, including flat affect and emotional perception deficits, is a specific symptom of schizophrenia disorder. We used a modified multimodal odd-ball paradigm with fearful facial expressions accompanied by congruent and non-congruent emotional vocalizations (sounds of women screaming and laughing) to investigate the impairment of emotional perception and reactions to other people’s emotions in schizophrenia. We compared subjective ratings of emotional state and event-related potentials (EPPs) in response to congruent and non-congruent stimuli in patients with schizophrenia an

APA, Harvard, Vancouver, ISO, and other styles

14

Vani Vivekanand, Chettiyar. "Performance Analysis of Emotion Classification Using Multimodal Fusion Technique." Journal of Computational Science and Intelligent Technologies 2, no. 1 (2021): 14–20. http://dx.doi.org/10.53409/mnaa/jcsit/2103.

Full text

Abstract:

As the central processing unit of the human body, the human brain is in charge of several activities, including cognition, perception, emotion, attention, action, and memory. Emotions have a significant impact on human well-being in their life. Methodologies for accessing emotions of human could be essential for good user-machine interactions. Comprehending BCI (Brain-Computer Interface) strategies for identifying emotions can also help people connect with the world more naturally. Many approaches for identifying human emotions have been developed using signals of EEG for classifying happy, ne

APA, Harvard, Vancouver, ISO, and other styles

15

Jolibois, Simon Christophe, Akinori Ito, and Takashi Nose. "The Development of an Emotional Embodied Conversational Agent and the Evaluation of the Effect of Response Delay on User Impression." Applied Sciences 15, no. 8 (2025): 4256. https://doi.org/10.3390/app15084256.

Full text

Abstract:

Embodied conversational agents (ECAs) are autonomous interaction interfaces designed to communicate with humans. This study investigates the impact of response delays and emotional facial expressions of ECAs on user perception and engagement. The motivation for this study stems from the growing integration of ECAs in various sectors, where their ability to mimic human-like interactions significantly enhances user experience. To this end, we developed an ECA with multimodal emotion recognition, both with voice and facial feature recognition and emotional facial expressions of the agent avatar.

APA, Harvard, Vancouver, ISO, and other styles

16

Mittal, Trisha, Aniket Bera, and Dinesh Manocha. "Multimodal and Context-Aware Emotion Perception Model With Multiplicative Fusion." IEEE MultiMedia 28, no. 2 (2021): 67–75. http://dx.doi.org/10.1109/mmul.2021.3068387.

Full text

APA, Harvard, Vancouver, ISO, and other styles

17

Barabanschikov, V. A., and E. V. Suvorova. "Vivid Face Perception as a Constructive Component of Multimodal Affective States." Experimental Psychology (Russia) 17, no. 4 (2024): 4–27. https://doi.org/10.17759/exppsy.2024170401.

Full text

Abstract:

<p>The features of expression and perception of vivid facial expressions in the system of multimodal affective states of a person are studied. The study is based on the Russian-language version of the Geneva Emotion Recognition Test (GERT) and consists of two series. The first serie of experiment was devoted to demonstration of short audio&mdash;video clips of 14 affective states expressed by specially trained actors, the second &mdash;to the same videos without sound accompaniment (intonation of voice in pseudolinguistic utterances). The subjects &mdash; 72 women in each ser

APA, Harvard, Vancouver, ISO, and other styles

18

Wang, Yonggu, Kailin Pan, Yifan Shao, Jiarong Ma, and Xiaojuan Li. "Applying a Convolutional Vision Transformer for Emotion Recognition in Children with Autism: Fusion of Facial Expressions and Speech Features." Applied Sciences 15, no. 6 (2025): 3083. https://doi.org/10.3390/app15063083.

Full text

Abstract:

With advances in digital technology, including deep learning and big data analytics, new methods have been developed for autism diagnosis and intervention. Emotion recognition and the detection of autism in children are prominent subjects in autism research. Typically using single-modal data to analyze the emotional states of children with autism, previous research has found that the accuracy of recognition algorithms must be improved. Our study creates datasets on the facial and speech emotions of children with autism in their natural states. A convolutional vision transformer-based emotion r

APA, Harvard, Vancouver, ISO, and other styles

19

Kdouri, Lahoucine, Youssef Hmamouche, Amal El Fallah Seghrouchni, and Thierry Chaminade. "Predicting Activity in Brain Areas Associated with Emotion Processing Using Multimodal Behavioral Signals." Multimodal Technologies and Interaction 9, no. 4 (2025): 31. https://doi.org/10.3390/mti9040031.

Full text

Abstract:

Artificial agents are expected to increasingly interact with humans and to demonstrate multimodal adaptive emotional responses. Such social integration requires both perception and production mechanisms, thus enabling a more realistic approach to emotional alignment than existing systems. Indeed, existing emotion recognition methods rely on behavioral signals, predominantly facial expressions, as well as non-invasive brain recordings, such as Electroencephalograms (EEGs) and functional Magnetic Resonance Imaging (fMRI), to identify humans’ emotions, but accurate labeling remains a challenge. T

APA, Harvard, Vancouver, ISO, and other styles

20

Barabanschikov, V. A., E. V. Suvorova, and A. V. Malionok. "Perception of the Prosodic Formative of Multimodal Affective States." Experimental Psychology (Russia) 17, no. 3 (2024): 30–51. http://dx.doi.org/10.17759/exppsy.2024170303.

Full text

Abstract:

<p>The features of the expression and perception of vocal expressions as one of the forming multimodal affective states of a person are studied. The experiment, designed on the basis of the Russian-language version of the Geneva Emotion Recognition Test (GERT), involved two groups of women aged 18-45, 72 women each. One group was shown audio-video clips of 14 affective states, lasting 3-5 seconds, played by 10 professional actors according to the Stanislavsky system. The other group was presented with audio clips extracted from the same clips through headphones. It was demanded to recogn

APA, Harvard, Vancouver, ISO, and other styles

21

Montembeault, Maxime, Estefania Brando, Kim Charest, et al. "Multimodal emotion perception in young and elderly patients with multiple sclerosis." Multiple Sclerosis and Related Disorders 58 (February 2022): 103478. http://dx.doi.org/10.1016/j.msard.2021.103478.

Full text

APA, Harvard, Vancouver, ISO, and other styles

22

Bai, Jie. "Optimized Piano Music Education Model Based on Multimodal Information Fusion for Emotion Recognition in Multimedia Video Networks." Mobile Information Systems 2022 (August 24, 2022): 1–12. http://dx.doi.org/10.1155/2022/1882739.

Full text

Abstract:

Emotion is the important information that people transmit in the process of communication, and the change of emotional state affects people’s perception and decision-making, which introduces the emotional dimension into human-computer interaction. The modes of emotional expression include facial expressions, speech, posture, physiological signals, text, and so on. Emotion recognition is essentially a multimodal fusion problem. This paper investigates the different teaching modes of the teachers and students of our school, designs the load capacity through the K-means algorithm, builds a multim

APA, Harvard, Vancouver, ISO, and other styles

23

Chen, Junwei. "Discussing Solutions to the Data Imbalance Problem in Emotion Recognition." Applied and Computational Engineering 174, no. 1 (2025): 23–31. https://doi.org/10.54254/2755-2721/2025.po24697.

Full text

Abstract:

Emotion recognition technology has been widely used in human-computer interaction, medical health and other fields. However, in practical applications, emotion datasets often have class imbalance problems, which lead to the model being seriously biased towards the majority class, significantly reducing the recognition accuracy and reliability of minority emotion classes. This paper focuses on comparing and analyzing methods such as ESC-GAN generative data augmentation technology, DER-GCN dialogue and event relationship perception graph model, and MultiEMO multimodal fusion framework to solve t

APA, Harvard, Vancouver, ISO, and other styles

24

Zhang, Jingjing, and Wei Chen. "A Decade of Music Emotion Computing: A Bibliometric Analysis of Trends, Interdisciplinary Collaboration, and Applications." Education for Information 41, no. 3 (2025): 227–55. https://doi.org/10.1177/01678329251323441.

Full text

Abstract:

This study systematically reviews the evolution of Music Emotion Computing (MEC) over the past decade, focusing on its two core branches: Music Emotion Recognition (MER) and Music Sentiment Analysis (MSA). Through a comprehensive bibliometric analysis, the research aims to uncover emerging trends, interdisciplinary and cross-regional collaboration patterns, and key application areas within this field. Using data collected from the Web of Science Core Collection (WoSCC), we conducted a comprehensive bibliometric analysis to map global academic output, highlighting influential studies, leading a

APA, Harvard, Vancouver, ISO, and other styles

25

Gauraangi Praakash. "Multimodal Emotion Recognition: A Tri-modal Approach Using Speech, Text, and Visual Cues for Enhanced Interaction Analysis." Journal of Information Systems Engineering and Management 10, no. 39s (2025): 654–63. https://doi.org/10.52783/jisem.v10i39s.7269.

Full text

Abstract:

In an age dominated by the rapid development of human-computer interaction, knowledge of the user emotions has become a critical building block in creating engagement as well as responsiveness. This paper presents a tri-modal system towards real-time emotion perception through the merging of textual, visual, and audio information. Our method utilizes strong deep learning networks in each of the three modalities: DistilBERT to perform sentiment analysis on text data (on the SST-2 dataset), ViT (Vision Transformer, vit-base-patch16-224-in21k) to detect emotions from faces, and a task-specific Co

APA, Harvard, Vancouver, ISO, and other styles

26

Li, Jialu. "Integrating Multimodal Data for Deep Learning-Based Facial Emotion Recognition." Highlights in Science, Engineering and Technology 124 (February 18, 2025): 362–67. https://doi.org/10.54097/gpy08650.

Full text

Abstract:

With the rapid development of neural networks, emotion recognition has become a research area of great concern. It has important applications not only in marketing and human-computer interaction but also holds significant importance for improving emotional computing and user experience. This paper studies various methods for emotion recognition in images and videos, utilizing convolutional neural networks (CNN), multi-layer perceptron (MLP), and fusion models. The Facial Expression Recognition 2013 (FER2013) image dataset and the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVD

APA, Harvard, Vancouver, ISO, and other styles

27

Hancock, Megan R., and Tessa Bent. "Multimodal emotion perception: Influences of autism spectrum disorder and autism-like traits." Journal of the Acoustical Society of America 148, no. 4 (2020): 2765. http://dx.doi.org/10.1121/1.5147698.

Full text

APA, Harvard, Vancouver, ISO, and other styles

28

Bänziger, Tanja, Marcello Mortillaro, and Klaus R. Scherer. "Introducing the Geneva Multimodal expression corpus for experimental research on emotion perception." Emotion 12, no. 5 (2012): 1161–79. http://dx.doi.org/10.1037/a0025827.

Full text

APA, Harvard, Vancouver, ISO, and other styles

29

Udeh, Chinonso Paschal, Luefeng Chen, Sheng Du, Min Li, and Min Wu. "Multimodal Facial Emotion Recognition Using Improved Convolution Neural Networks Model." Journal of Advanced Computational Intelligence and Intelligent Informatics 27, no. 4 (2023): 710–19. http://dx.doi.org/10.20965/jaciii.2023.p0710.

Full text

Abstract:

In the quest for human-robot interaction (HRI), leading to the development of emotion recognition, learning, and analysis capabilities, robotics plays a significant role in human perception, attention, decision-making, and social communication. However, the accurate recognition of emotions in HRI remains a challenge. This is due to the coexistence of multiple sources of information in utilizing multimodal facial expressions and head poses as multiple convolutional neural networks (CNN) and deep learning are combined. This research analyzes and improves the robustness of emotion recognition, an

APA, Harvard, Vancouver, ISO, and other styles

30

Sable, Prof R. Y., Aqsa Sayyed, Baliraje Kalyane, Kosheen Sadhu, and Prathamesh Ghatole. "Enhancing Music Mood Recognition with LLMs and Audio Signal Processing: A Multimodal Approach." International Journal for Research in Applied Science and Engineering Technology 12, no. 7 (2024): 628–42. http://dx.doi.org/10.22214/ijraset.2024.63590.

Full text

Abstract:

Abstract: Music Mood Recognition aims to allow computers to understand the emotions behind music the way humans do, in order to facilitate better perception of media by computers to aid in enhanced services like music recommendations, therapeutic interventions, and Human Computer Interaction. In this paper, we propose a novel approach to improving Music Mood Recognition using a multi-modal model that uses lyrical and audio features of a song. Lyrical features are analysed using stateof-the-art open-source Large Language Models like Microsoft Phi-3 to classify lyrics from one of the four possib

APA, Harvard, Vancouver, ISO, and other styles

31

Horii, Takato, Yukie Nagai, and Minoru Asada. "Modeling Development of Multimodal Emotion Perception Guided by Tactile Dominance and Perceptual Improvement." IEEE Transactions on Cognitive and Developmental Systems 10, no. 3 (2018): 762–75. http://dx.doi.org/10.1109/tcds.2018.2809434.

Full text

APA, Harvard, Vancouver, ISO, and other styles

32

Yamamoto, Hisako W., and Akihiro Tanaka. "The Development of multimodal emotion perception from the combination of bodies and voices." Proceedings of the Annual Convention of the Japanese Psychological Association 86 (2022): 1EV—062—PO—1EV—062—PO. http://dx.doi.org/10.4992/pacjpa.86.0_1ev-062-po.

Full text

APA, Harvard, Vancouver, ISO, and other styles

33

de Gelder, Beatrice, and Jean Vroomen. "Rejoinder - Bimodal emotion perception: integration across separate modalities, cross-modal perceptual grouping or perception of multimodal events?" Cognition & Emotion 14, no. 3 (2000): 321–24. http://dx.doi.org/10.1080/026999300378842.

Full text

APA, Harvard, Vancouver, ISO, and other styles

34

Luna-Jiménez, Cristina, Ricardo Kleinlein, David Griol, Zoraida Callejas, Juan M. Montero, and Fernando Fernández-Martínez. "A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS Dataset." Applied Sciences 12, no. 1 (2021): 327. http://dx.doi.org/10.3390/app12010327.

Full text

Abstract:

Emotion recognition is attracting the attention of the research community due to its multiple applications in different fields, such as medicine or autonomous driving. In this paper, we proposed an automatic emotion recognizer system that consisted of a speech emotion recognizer (SER) and a facial emotion recognizer (FER). For the SER, we evaluated a pre-trained xlsr-Wav2Vec2.0 transformer using two transfer-learning techniques: embedding extraction and fine-tuning. The best accuracy results were achieved when we fine-tuned the whole model by appending a multilayer perceptron on top of it, con

APA, Harvard, Vancouver, ISO, and other styles

35

Barabanschikov, V. A., and O. A. Korolkova. "Perception of “Live” Facial Expressions." Experimental Psychology (Russia) 13, no. 3 (2020): 55–73. http://dx.doi.org/10.17759/exppsy.2020130305.

Full text

Abstract:

The article provides a review of experimental studies of interpersonal perception on the material of static and dynamic facial expressions as a unique source of information about the person’s inner world. The focus is on the patterns of perception of a moving face, included in the processes of communication and joint activities (an alternative to the most commonly studied perception of static images of a person outside of a behavioral context). The review includes four interrelated topics: face statics and dynamics in the recognition of emotional expressions; specificity of perception of movin

APA, Harvard, Vancouver, ISO, and other styles

36

Kim, Bora, and Bhuja Chung. "Affective Perception Characteristics in School-aged Children with High-functioning Autism Spectrum Disorder according to Prosody and Semantic Consistency." Communication Sciences & Disorders 29, no. 4 (2024): 776–86. https://doi.org/10.12963/csd.240075.

Full text

Abstract:

Objectives: The perception of emotions through prosody is crucial to social communication. However, children with Autism Spectrum Disorder (ASD) often struggle to recognize others’ emotions based on various non-verbal cues. This study aims to investigate whether school-aged children with high-functioning ASD rely more on semantic cues or prosodic cues when perceiving others’ emotions. Additionally, the study examines how emotion recognition differs depending on the consistency of emotional cues. Methods: Seven school-aged children with high-functioning ASD and seven typically developing school

APA, Harvard, Vancouver, ISO, and other styles

37

Michael, Stefanus, and Amalia Zahra. "Multimodal speech emotion recognition optimization using genetic algorithm." Bulletin of Electrical Engineering and Informatics 13, no. 5 (2024): 3309–16. http://dx.doi.org/10.11591/eei.v13i5.7409.

Full text

Abstract:

Speech emotion recognition (SER) is a technology that can detect emotions in speech. Various methods have been used in developing SER, such as convolutional neural networks (CNNs), long short-term memory (LSTM), and multilayer perceptron. However, sometimes in addition to model selection, other techniques are still needed to improve SER performance, namely optimization methods. This paper compares manual hyperparameter tuning using grid search (GS) and hyperparameter tuning using genetic algorithm (GA) on the LSTM model to prove the performance increase in the multimodal SER model after optimi

APA, Harvard, Vancouver, ISO, and other styles

38

Miao, Haotian, Yifei Zhang, Daling Wang, and Shi Feng. "Multi-Output Learning Based on Multimodal GCN and Co-Attention for Image Aesthetics and Emotion Analysis." Mathematics 9, no. 12 (2021): 1437. http://dx.doi.org/10.3390/math9121437.

Full text

Abstract:

With the development of social networks and intelligent terminals, it is becoming more convenient to share and acquire images. The massive growth of the number of social images makes people have higher demands for automatic image processing, especially in the aesthetic and emotional perspective. Both aesthetics assessment and emotion recognition require a higher ability for the computer to simulate high-level visual perception understanding, which belongs to the field of image processing and pattern recognition. However, existing methods often ignore the prior knowledge of images and intrinsic

APA, Harvard, Vancouver, ISO, and other styles

39

Ryumin, Dmitry, Elena Ryumina, and Denis Ivanko. "EMOLIPS: Towards Reliable Emotional Speech Lip-Reading." Mathematics 11, no. 23 (2023): 4787. http://dx.doi.org/10.3390/math11234787.

Full text

Abstract:

In this article, we present a novel approach for emotional speech lip-reading (EMOLIPS). This two-level approach to emotional speech to text recognition based on visual data processing is motivated by human perception and the recent developments in multimodal deep learning. The proposed approach uses visual speech data to determine the type of speech emotion. The speech data are then processed using one of the emotional lip-reading models trained from scratch. This essentially resolves the multi-emotional lip-reading issue associated with most real-life scenarios. We implemented these models a

APA, Harvard, Vancouver, ISO, and other styles

40

Babaoğlu, Gizem, Başak Yazgan, Pınar Erturk, et al. "Vocal emotion recognition by native Turkish children with normal hearing and with hearing aids." Journal of the Acoustical Society of America 151, no. 4 (2022): A278. http://dx.doi.org/10.1121/10.0011335.

Full text

Abstract:

Development of vocal emotion recognition in children with normal hearing takes many years before reaching adult-like levels. In children with hearing loss, decreased audibility and potential loss of sensitivity to relevant acoustic cues may additionally affect vocal emotion perception. Hearing aids (HAs) are traditionally optimized for speech understanding, and it is not clear how children with HAs are performing in perceiving vocal emotions. In this study, we investigated vocal emotion recognition in native Turkish normal hearing children (NHC, age range: 5–18 years), normal hearing adults (N

APA, Harvard, Vancouver, ISO, and other styles

41

Berkane, Mohamed, Kenza Belhouchette, and Hacene Belhadef. "Emotion Recognition Approach Using Multilayer Perceptron Network and Motion Estimation." International Journal of Synthetic Emotions 10, no. 1 (2019): 38–53. http://dx.doi.org/10.4018/ijse.2019010102.

Full text

Abstract:

Man-machine interaction is an interdisciplinary field of research that provides natural and multimodal ways of interaction between humans and computers. For this purpose, the computer must understand the emotional state of the person with whom it interacts. This article proposes a novel method for detecting and classify the basic emotions like sadness, joy, anger, fear, disgust, surprise, and interest that was introduced in previous works. As with all emotion recognition systems, the approach follows the basic steps, such as: facial detection and facial feature extraction. In these steps, the

APA, Harvard, Vancouver, ISO, and other styles

42

Dessai, Amita, and Hassanali Virani. "Multimodal and Multidomain Feature Fusion for Emotion Classification Based on Electrocardiogram and Galvanic Skin Response Signals." Sci 6, no. 1 (2024): 10. http://dx.doi.org/10.3390/sci6010010.

Full text

Abstract:

Emotion classification using physiological signals is a promising approach that is likely to become the most prevalent method. Bio-signals such as those derived from Electrocardiograms (ECGs) and the Galvanic Skin Response (GSR) are more reliable than facial and voice recognition signals because they are not influenced by the participant’s subjective perception. However, the precision of emotion classification with ECG and GSR signals is not satisfactory, and new methods need to be developed to improve it. In addition, the fusion of the time and frequency features of ECG and GSR signals should

APA, Harvard, Vancouver, ISO, and other styles

43

Resende Faria, Diego, Abraham Itzhak Weinberg, and Pedro Paulo Ayrosa. "Multimodal Affective Communication Analysis: Fusing Speech Emotion and Text Sentiment Using Machine Learning." Applied Sciences 14, no. 15 (2024): 6631. http://dx.doi.org/10.3390/app14156631.

Full text

Abstract:

Affective communication, encompassing verbal and non-verbal cues, is crucial for understanding human interactions. This study introduces a novel framework for enhancing emotional understanding by fusing speech emotion recognition (SER) and sentiment analysis (SA). We leverage diverse features and both classical and deep learning models, including Gaussian naive Bayes (GNB), support vector machines (SVMs), random forests (RFs), multilayer perceptron (MLP), and a 1D convolutional neural network (1D-CNN), to accurately discern and categorize emotions in speech. We further extract text sentiment f

APA, Harvard, Vancouver, ISO, and other styles

44

Zhang, Lihong, Chaolong Liu, and Nan Jia. "Uni2Mul: A Conformer-Based Multimodal Emotion Classification Model by Considering Unimodal Expression Differences with Multi-Task Learning." Applied Sciences 13, no. 17 (2023): 9910. http://dx.doi.org/10.3390/app13179910.

Full text

Abstract:

Multimodal emotion classification (MEC) has been extensively studied in human–computer interaction, healthcare, and other domains. Previous MEC research has utilized identical multimodal annotations (IMAs) to train unimodal models, hindering the learning of effective unimodal representations due to differences between unimodal expressions and multimodal perceptions. Additionally, most MEC fusion techniques fail to consider the unimodal–multimodal inconsistencies. This study addresses two important issues in MEC: learning satisfactory unimodal representations of emotion and accounting for unimo

APA, Harvard, Vancouver, ISO, and other styles

45

Kholkhal, Mourad, Mohammed Sofiane Bendelhoum, Nabil Dib, and Mohammed Ridha Youbi. "Decoding the brain: EEG applications in sensory processing, emotion recognition, and pain assessment." STUDIES IN ENGINEERING AND EXACT SCIENCES 5, no. 2 (2024): e9688. http://dx.doi.org/10.54021/seesv5n2-388.

Full text

Abstract:

This work explores the diverse applications of electroencephalography (EEG) across sensory processing, emotion recognition, and pain assessment. It analyzes recent studies employing EEG to investigate auditory perception, visual processing, emotional states, and pain experiences, highlighting common methodological approaches such as preprocessing techniques, feature extraction methods, and classification algorithms. Key findings reveal EEG’s potential in decoding complex brain states, from identifying neural correlates of music perception to detecting pain signals, and discuss the advancement

APA, Harvard, Vancouver, ISO, and other styles

46

Kalam, Mr Md Abdul. "A STUDY ON A JOURNEY THROUGH STORIES IN SEQUENTIAL FRAMES." International Scientific Journal of Engineering and Management 04, no. 05 (2025): 1–7. https://doi.org/10.55041/isjem03444.

Full text

Abstract:

ABSTRACT: This project is aimed at educating machines to narrate visual stories through creating descriptive and coherent text from image sequences. Based on analyzing objects, actions, emotions, and scene changes, the system constructs narrative text reflecting the story in the images. It intends to simulate human storytelling by extracting temporal and contextual relationships among images. Applications are automatic photo album description, digital storytelling, and assistive technology for the visually impaired. By closing the gap between visual perception and language generation, this app

APA, Harvard, Vancouver, ISO, and other styles

47

Yang, Yi, Hao Feng, Yiming Cheng, and Zhu Han. "Emotion-Aware Scene Adaptation: A Bandwidth-Efficient Approach for Generating Animated Shorts." Sensors 24, no. 5 (2024): 1660. http://dx.doi.org/10.3390/s24051660.

Full text

Abstract:

Semantic communication technology in the 6G wireless system focuses on semantic extraction in communication, that is, only the inherent meaning of the intention in the information. Existing technologies still have challenges in extracting emotional perception in the information, high compression rates, and privacy leakage due to knowledge sharing in communication. Large-scale generative-model technology could rapidly generate multimodal information according to user requirements. This paper proposes an approach that leverages large-scale generative models to create animated short films that ar

APA, Harvard, Vancouver, ISO, and other styles

48

Fiorini, Laura, Grazia D'Onofrio, Alessandra Sorrentino, et al. "The Role of Coherent Robot Behavior and Embodiment in Emotion Perception and Recognition During Human-Robot Interaction: Experimental Study." JMIR Human Factors 11 (January 26, 2024): e45494. http://dx.doi.org/10.2196/45494.

Full text

Abstract:

Background Social robots are becoming increasingly important as companions in our daily lives. Consequently, humans expect to interact with them using the same mental models applied to human-human interactions, including the use of cospeech gestures. Research efforts have been devoted to understanding users’ needs and developing robot’s behavioral models that can perceive the user state and properly plan a reaction. Despite the efforts made, some challenges regarding the effect of robot embodiment and behavior in the perception of emotions remain open. Objective The aim of this study is dual.

APA, Harvard, Vancouver, ISO, and other styles

49

Prajapati, Vrinda, Rajlakshmi Guha, and Aurobinda Routray. "Multimodal prediction of trait emotional intelligence–Through affective changes measured using non-contact based physiological measures." PLOS ONE 16, no. 7 (2021): e0254335. http://dx.doi.org/10.1371/journal.pone.0254335.

Full text

Abstract:

Inability to efficiently deal with emotionally laden situations, often leads to poor interpersonal interactions. This adversely affects the individual’s psychological functioning. A higher trait emotional intelligence (EI) is not only associated with psychological wellbeing, educational attainment, and job-related success, but also with willingness to seek professional and non-professional help for personal-emotional problems, depression and suicidal ideation. Thus, it is important to identify low (EI) individuals who are more prone to mental health problems than their high EI counterparts, an

APA, Harvard, Vancouver, ISO, and other styles

50

Piwek, Lukasz, Karin Petrini, and Frank E. Pollick. "Auditory signal dominates visual in the perception of emotional social interactions." Seeing and Perceiving 25 (2012): 112. http://dx.doi.org/10.1163/187847612x647450.

Full text

Abstract:

Multimodal perception of emotions has been typically examined using displays of a solitary character (e.g., the face–voice and/or body–sound of one actor). We extend investigation to more complex, dyadic point-light displays combined with speech. A motion and voice capture system was used to record twenty actors interacting in couples with happy, angry and neutral emotional expressions. The obtained stimuli were validated in a pilot study and used in the present study to investigate multimodal perception of emotional social interactions. Participants were required to categorize happy and angry

APA, Harvard, Vancouver, ISO, and other styles

We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!