To see the other types of publications on this topic, follow the link: Facial expression analysis.
Dissertations / Theses on the topic 'Facial expression analysis'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 dissertations / theses for your research on the topic 'Facial expression analysis.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse dissertations / theses on a wide variety of disciplines and organise your bibliography correctly.
1
Baltrušaitis, Tadas. "Automatic facial expression analysis." Thesis, University of Cambridge, 2014. https://www.repository.cam.ac.uk/handle/1810/245253.
Full textAbstract:
Humans spend a large amount of their time interacting with computers of one type or another. However, computers are emotionally blind and indifferent to the affective states of their users. Human-computer interaction which does not consider emotions, ignores a whole channel of available information. Faces contain a large portion of our emotionally expressive behaviour. We use facial expressions to display our emotional states and to manage our interactions. Furthermore, we express and read emotions in faces effortlessly. However, automatic understanding of facial expressions is a very difficult task computationally, especially in the presence of highly variable pose, expression and illumination. My work furthers the field of automatic facial expression tracking by tackling these issues, bringing emotionally aware computing closer to reality. Firstly, I present an in-depth analysis of the Constrained Local Model (CLM) for facial expression and head pose tracking. I propose a number of extensions that make location of facial features more accurate. Secondly, I introduce a 3D Constrained Local Model (CLM-Z) which takes full advantage of depth information available from various range scanners. CLM-Z is robust to changes in illumination and shows better facial tracking performance. Thirdly, I present the Constrained Local Neural Field (CLNF), a novel instance of CLM that deals with the issues of facial tracking in complex scenes. It achieves this through the use of a novel landmark detector and a novel CLM fitting algorithm. CLNF outperforms state-of-the-art models for facial tracking in presence of difficult illumination and varying pose. Lastly, I demonstrate how tracked facial expressions can be used for emotion inference from videos. I also show how the tools developed for facial tracking can be applied to emotion inference in music.
2
Li, Jingting. "Facial Micro-Expression Analysis." Thesis, CentraleSupélec, 2019. http://www.theses.fr/2019CSUP0007.
Full textAbstract:
Les micro-expressions (MEs) sont porteuses d'informations non verbales spécifiques. Cependant, en raison de leur nature locale et brève, il est difficile de les détecter. Dans cette thèse, nous proposons une méthode de détection par reconnaissance d'un motif local et temporel de mouvement du visage. Ce motif a une forme spécifique (S-pattern) lorsque la ME apparait. Ainsi, à l'aide de SVM, nous distinguons les MEs des autres mouvements faciaux. Nous proposons également une fusion spatiale et temporelle afin d'améliorer la distinction entre les MEs (locaux) et les mouvements de la tête (globaux). Cependant, l'apprentissage des S-patterns est limité par le petit nombre de bases de données de ME et par le faible volume d'échantillons de ME. Les modèles de Hammerstein (HM) est une bonne approximation des mouvements musculaires. En approximant chaque S-pattern par un HM, nous pouvons filtrer les S-patterns réels et générer de nouveaux S-patterns similaires. Ainsi, nous effectuons une augmentation et une fiabilisation des S-patterns pour l'apprentissage et améliorons ainsi la capacité de différencier les MEs d'autres mouvements. Lors du premier challenge de détection de MEs, nous avons participé à la création d’une nouvelle méthode d'évaluation des résultats. Cela a aussi été l’occasion d’appliquer notre méthode à longues vidéos. Nous avons fourni le résultat de base au challenge.Les expérimentions sont effectuées sur CASME I, CASME II, SAMM et CAS(ME)2. Les résultats montrent que notre méthode proposée surpasse la méthode la plus populaire en termes de F1-score. L'ajout du processus de fusion et de l'augmentation des données améliore encore les performances de détectionThe Micro-expressions (MEs) are very important nonverbal communication clues. However, due to their local and short nature, spotting them is challenging. In this thesis, we address this problem by using a dedicated local and temporal pattern (LTP) of facial movement. This pattern has a specific shape (S-pattern) when ME are displayed. Thus, by using a classical classification algorithm (SVM), MEs are distinguished from other facial movements. We also propose a global final fusion analysis on the whole face to improve the distinction between ME (local) and head (global) movements. However, the learning of S-patterns is limited by the small number of ME databases and the low volume of ME samples. Hammerstein models (HMs) are known to be a good approximation of muscle movements. By approximating each S-pattern with a HM, we can both filter outliers and generate new similar S-patterns. By this way, we perform a data augmentation for S-pattern training dataset and improve the ability to differentiate MEs from other facial movements. In the first ME spotting challenge of MEGC2019, we took part in the building of the new result evaluation method. In addition, we applied our method to spotting ME in long videos and provided the baseline result for the challenge. The spotting results, performed on CASME I and CASME II, SAMM and CAS(ME)2, show that our proposed LTP outperforms the most popular spotting method in terms of F1-score. Adding the fusion process and data augmentation improve even more the spotting performance
3
Carter, Jeffrey R. "Facial expression analysis in schizophrenia." Thesis, National Library of Canada = Bibliothèque nationale du Canada, 2000. http://www.collectionscanada.ca/obj/s4/f2/dsk2/ftp03/NQ58398.pdf.
Full text
4
Munasinghe, Kankanamge Sarasi Madushika. "Facial analysis models for face and facial expression recognition." Thesis, Queensland University of Technology, 2018. https://eprints.qut.edu.au/118197/1/Sarasi%20Madushika_Munasinghe%20Kankanamge_Thesis.pdf.
Full textAbstract:
This thesis examines the research and development of new approaches for face and facial expression recognition within the fields of computer vision and biometrics. Expression variation is a challenging issue in current face recognition systems and current approaches are not capable of recognizing facial variations effectively within human-computer interfaces, security and access control applications. This thesis presents new contributions for performing face and expression recognition simultaneously; face recognition in the wild; and facial expression recognition in challenging environments. The research findings include the development of new factor analysis and deep learning approaches which can better handle different facial variations.
5
Shang, Lifeng, and 尚利峰. "Facial expression analysis with graphical models." Thesis, The University of Hong Kong (Pokfulam, Hong Kong), 2012. http://hub.hku.hk/bib/B47849484.
Full textAbstract:
Facial expression recognition has become an active research topic in recent
years due to its applications in human computer interfaces and data-driven animation. In this thesis, we focus on the problem of how to e?ectively use domain,
temporal and categorical information of facial expressions to help computer understand human emotions. Over the past decades, many techniques (such as
neural networks, Gaussian processes, support vector machines, etc.) have been
applied to facial expression analysis. Recently graphical models have emerged as
a general framework for applying probabilistic models. They provide a natural
framework for describing the generative process of facial expressions. However,
these models often su?er from too many latent variables or too complex model
structures, which makes learning and inference di±cult. In this thesis, we will
try to analyze the deformation of facial expression by introducing some recently
developed graphical models (e.g. latent topic model) or improving the recognition
ability of some already widely used models (e.g. HMM).
In this thesis, we develop three di?erent graphical models with di?erent representational assumptions: categories being represented by prototypes, sets of
exemplars and topics in between. Our ¯rst model incorporates exemplar-based
representation into graphical models. To further improve computational e±-
ciency of the proposed model, we build it in a local linear subspace constructed
by principal component analysis. The second model is an extension of the recently
developed topic model by introducing temporal and categorical information into
Latent Dirichlet Allocation model. In our discriminative temporal topic model
(DTTM), temporal information is integrated by placing an asymmetric Dirichlet
prior over document-topic distributions. The discriminative ability is improved by
a supervised term weighting scheme. We describe the resulting DTTM in detail
and show how it can be applied to facial expression recognition. Our third model
is a nonparametric discriminative variation of HMM. HMM can be viewed as a
prototype model, and transition parameters act as the prototype for one category.
To increase the discrimination ability of HMM at both class level and state level,
we introduce linear interpolation with maximum entropy (LIME) and member-
ship coe±cients to HMM. Furthermore, we present a general formula for output
probability estimation, which provides a way to develop new HMM. Experimental
results show that the performance of some existing HMMs can be improved by
integrating the proposed nonparametric kernel method and parameters adaption
formula.
In conclusion, this thesis develops three di?erent graphical models by (i) combining exemplar-based model with graphical models, (ii) introducing temporal
and categorical information into Latent Dirichlet Allocation (LDA) topic model,
and (iii) increasing the discrimination ability of HMM at both hidden state level
and class level.published_or_final_version
Computer Science
Doctoral
Doctor of Philosophy
6
Feffer, Michael A. (Michael Anthony). "Personalized machine learning for facial expression analysis." Thesis, Massachusetts Institute of Technology, 2018. http://hdl.handle.net/1721.1/119763.
Full textAbstract:
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 35-36).
For this MEng Thesis Project, I investigated the personalization of deep convolutional networks for facial expression analysis. While prior work focused on population-based ("one-size-fits-all") models for prediction of affective states (valence/arousal), I constructed personalized versions of these models to improve upon state-of-the-art general models through solving a domain adaptation problem. This was done by starting with pre-trained deep models for face analysis and fine-tuning the last layers to specific subjects or subpopulations. For prediction, a "mixture of experts" (MoE) solution was employed to select the proper outputs based on the given input. The research questions answered in this project are: (1) What are the effects of model personalization on the estimation of valence and arousal from faces? (2) What is the amount of (un)supervised data needed to reach a target performance? Models produced in this research provide the foundation of a novel tool for personalized real-time estimation of target metrics.
by Michael A. Feffer.
M. Eng.
7
Shenoy, A. "Computational analysis of facial expressions." Thesis, University of Hertfordshire, 2010. http://hdl.handle.net/2299/4359.
Full textAbstract:
This PhD work constitutes a series of inter-disciplinary studies that use biologically plausible computational techniques and experiments with human subjects in analyzing facial expressions. The performance of the computational models and human subjects in terms of accuracy and response time are analyzed. The computational models process images in three stages. This includes: Preprocessing, dimensionality reduction and Classification. The pre-processing of face expression images includes feature extraction and dimensionality reduction. Gabor filters are used for feature extraction as they are closest biologically plausible computational method. Various dimensionality reduction methods: Principal Component Analysis (PCA), Curvilinear Component Analysis (CCA) and Fisher Linear Discriminant (FLD) are used followed by the classification by Support Vector Machines (SVM) and Linear Discriminant Analysis (LDA). Six basic prototypical facial expressions that are universally accepted are used for the analysis. They are: angry, happy, fear, sad, surprise and disgust. The performance of the computational models in classifying each expression category is compared with that of the human subjects. The Effect size and Encoding face enable the discrimination of the areas of the face specific for a particular expression. The Effect size in particular emphasizes the areas of the face that are involved during the production of an expression. This concept of using Effect size on faces has not been reported previously in the literature and has shown very interesting results. The detailed PCA analysis showed the significant PCA components specific for each of the six basic prototypical expressions. An important observation from this analysis was that with Gabor filtering followed by non linear CCA for dimensionality reduction, the dataset vector size may be reduced to a very small number, in most cases it was just 5 components. The hypothesis that the average response time (RT) for the human subjects in classifying the different expressions is analogous to the distance measure of the data points from the classification hyper-plane was verified. This means the harder a facial expression is to classify by human subjects, the closer to the classifying hyper-plane of the classifier it is. A bi-variate correlation analysis of the distance measure and the average RT suggested a significant anti-correlation. The signal detection theory (SDT) or the d-prime determined how well the model or the human subjects were in making the classification of an expressive face from a neutral one. On comparison, human subjects are better in classifying surprise, disgust, fear, and sad expressions. The RAW computational model is better able to distinguish angry and happy expressions. To summarize, there seems to some similarities between the computational models and human subjects in the classification process.
8
Wang, Jing. "Reconstruction and Analysis of 3D Individualized Facial Expressions." Thesis, Université d'Ottawa / University of Ottawa, 2015. http://hdl.handle.net/10393/32588.
Full textAbstract:
This thesis proposes a new way to analyze facial expressions through 3D scanned faces of real-life people. The expression analysis is based on learning the facial motion vectors that are the differences between a neutral face and a face with an expression. There are several expression analysis based on real-life face database such as 2D image-based Cohn-Kanade AU-Coded Facial Expression Database and Binghamton University 3D Facial Expression Database. To handle large pose variations and increase the general understanding of facial behavior, 2D image-based expression database is not enough. The Binghamton University 3D Facial Expression Database is mainly used for facial expression recognition and it is difficult to compare, resolve, and extend the problems related detailed 3D facial expression analysis. Our work aims to find a new and an intuitively way of visualizing the detailed point by point movements of 3D face model for a facial expression.
In our work, we have created our own 3D facial expression database on a detailed level, which each expression model has been processed to have the same structure to compare differences between different people for a given expression.
The first step is to obtain same structured but individually shaped face models. All the head models are recreated by deforming a generic model to adapt a laser-scanned individualized face shape in both coarse level and fine level. We repeat this recreation method on different human subjects to establish a database. The second step is expression cloning. The motion vectors are obtained by subtracting two head models with/without expression. The extracted facial motion vectors are applied onto a different human subject’s neutral face. Facial expression cloning is proved to be robust and fast as well as easy to use. The last step is about analyzing the facial motion vectors obtained from the second step. First we transferred several human subjects’ expressions on a single human neutral face. Then the analysis is done to compare different expression pairs in two main regions: the whole face surface analysis and facial muscle analysis.
Through our work where smiling has been chosen for the experiment, we find our approach to analysis through face scanning a good way to visualize how differently people move their facial muscles for the same expression. People smile in a similar manner moving their mouths and cheeks in similar orientations, but each person shows her/his own unique way of moving. The difference between individual smiles is the differences of movements they make.
9
Mourão, André Belchior. "Robust facial expression analysis for affect-based interaction." Master's thesis, Faculdade de Ciências e Tecnologia, 2012. http://hdl.handle.net/10362/8292.
Full textAbstract:
Dissertação para obtenção do Grau de Mestre em Engenharia InformáticaInteraction is moving towards new and more natural approaches. Human Computer Interaction (HCI) is increasingly expanding towards more modalities of human expression such as gestures, body movements and other natural interactions. In this thesis, we propose to extend existing interaction paradigms by including the face as an affect-based input. Affective interaction methods can greatly change the way computers interact with humans; these methods can detect displays of user moods, such as frustration or engagement and adapt the experience accordingly. We have created an affect-based framework that encompasses face detection, face recognition and facial expression recognition and applied it in a computer game. ImEmotion is a two-player game where the player who best mimics an expression wins. The game combines face detection with facial expression recognition to recognize and rate an expression in real time. A controlled evaluation of the framework algorithms and a game trial with 46 users showed the potential of the framework and success of the usage of affect-based interaction based on facial expressions in the game. Despite the novelty of the interaction approach and the limitations of computer vision algorithms, players adapted and became competitive easily.
10
Yin, Lijun. "Facial expression analysis and synthesis for model based coding." Thesis, National Library of Canada = Bibliothèque nationale du Canada, 2000. http://www.collectionscanada.ca/obj/s4/f2/dsk1/tape3/PQDD_0011/NQ59702.pdf.
Full text
11
Wei, Xiaozhou. "3D facial expression modeling and analysis with topographic information." Diss., Online access via UMI:, 2008.
Find full text
12
Gupta, Ankit. "Live Performance and Emotional Analysis of MathSpring Intelligent Tutor System Students." Digital WPI, 2020. https://digitalcommons.wpi.edu/etd-theses/1372.
Full textAbstract:
An important goal of Educational Data Mining is to provide data and visualization about students’ state of knowledge and students’ affective states. The combination of these provides an understanding of the easiness or hardness of the concepts being taught and the student’s comfortability in it. While various studies have been conducted on estimating students’ knowledge and affect, little research has been done to transform this collected (Raw) data into meaningful information that is more relatable to teachers, parents and other stakeholders, i.e. Non-Researchers. This research seeks to enhance existing Teacher Tools (An application designed within the MathSpring - An Intelligent Tutoring system) to generate a live dashboard for teachers to use in the classroom, as their students are using MathSpring. The system captures student performance and detects students’ facial expressions, which highlight students emotion and engagement, using a deep learning model that detects facial expressions. The live dashboard enables teachers to understand and juxtapose the state of knowledge and corresponding affect of students as they practice math problem solving. This should help teachers understand students’ state of mind better, and feed this information back to act and alter their instruction or interaction with each student in a personalized way. We present results of teachers' perceptions of the usefulness of the Live Dashboard, through a qualitative and quantitative survey.
13
Saeed, Anwar Maresh Qahtan [Verfasser]. "Automatic facial analysis methods : facial point localization, head pose estimation, and facial expression recognition / Anwar Maresh Qahtan Saeed." Magdeburg : Universitätsbibliothek, 2018. http://d-nb.info/1162189878/34.
Full text
14
Arnade, Elizabeth Amalia. "Measuring Consumer Emotional Response to Tastes and Foods through Facial Expression Analysis." Thesis, Virginia Tech, 2014. http://hdl.handle.net/10919/54538.
Full textAbstract:
Emotions are thought to play a crucial role in food behavior. Non-rational emotional decision making may be credited as the reason why consumers select what, how, and when they choose to interact with a food product. In this research, three experiments were completed for the overall goal of understanding the usefulness and validity of selected emotional measurement tools, specifically emotion questionnaire ballots and facial expression analysis, as compared to conventional sensory methods in developing a holistic view of product interest and engagement. Emotional response to 1% low-fat unflavored and chocolate-flavored milk was evaluated by using an emotion-based questionnaire as well as facial expression analysis software, to measure post-experience cognitive and in-the-moment intrinsic (implicit) emotional response, respectively. The software correlated facial movements of participants to associated basic emotions to estimate with what degree consumers were expressing these measured emotions upon presentation of each sample. Finally, the adapted facial expression method was compared to expected measurements from previous studies by measuring emotional facial response to four (sweet, salt, sour, and bitter) basic tastes. The cognitive emotion ballot and implicit facial analysis were able to differentiate between milk samples and offer a greater understanding of the consumer experience. Validity of the facial expression method was lacking for reasons including high individual taste variability, social context, intensities of stimuli, quality of video data capture, calibration settings, sample size number, analysis duration, and software sensitivity limitations. To better validate automatic facial expression methodology, further study is needed to investigate and minimize method limitations.Master of Science in Life Sciences
15
Wong, Ka-wai Teresa. "Event-related potential analysis of facial emotion processing." Click to view the E-thesis via HKUTO, 2007. http://sunzi.lib.hku.hk/HKUTO/record/B3955773X.
Full text
16
Wong, Ka-wai Teresa, and 黃嘉慧. "Event-related potential analysis of facial emotion processing." Thesis, The University of Hong Kong (Pokfulam, Hong Kong), 2007. http://hub.hku.hk/bib/B3955773X.
Full text
17
Leitch, Kristen Allison. "Evaluating Consumer Emotional Response to Beverage Sweeteners through Facial Expression Analysis." Thesis, Virginia Tech, 2015. http://hdl.handle.net/10919/73695.
Full textAbstract:
Emotional processing and characterization of internal and external stimuli is believed to play an integral role in consumer acceptance or rejection of food products. In this research three experiments were completed with the ultimate goal of adding to the growing body of research pertaining to food, emotions and acceptance using traditional affective sensory methods in combination with implicit (uncontrollable) and explicit (cognitive) emotional measures. Sweetness equivalence of several artificial (acesulfame potassium, saccharin and sucralose) and natural (42% high fructose corn syrup and honey) sweeteners were established to a 5% sucrose solution. Differences in consumer acceptability and emotional response to sucrose (control) and four equi-sweet alternatives (acesulfame potassium, high fructose corn syrup, honey, and sucralose) in tea were evaluated using a 9-point hedonic scale, check-all-that-apply (CATA) emotion term questionnaire (explicit), and automated facial expression analysis (AFEA) (implicit). Facial expression responses and emotion term categorization based on selection frequencies were able to adequately discern differences in emotional response as it related to hedonic liking between sweetener categories (artificial; natural). The potential influence of varying product information on consumer acceptance and emotional responses was then evaluated in relation to three sweeteners (sucrose, ace-k, HFCS) in tea solutions. Observed differences in liking and emotional term characterizations based on the validity of product information for sweeteners were attributed to cognitive dissonance. False informational cues had an observed dampening effect on the implicit emotional response to alternative sweeteners. Significant moderate correlations between liking and several basic emotions supported the belief that implicit emotions are contextually specific. Limitations pertaining to AFEA data collection and emotional interpretations to sweeteners include high panelist variability (within and across), calibration techniques, video quality, software sensitivity, and a general lack of consistency concerning methods of analysis. When used in conjunction with traditional affective methodology and cognitive emotional characterization, AFEA provides an additional layer of valued information about the consumer food experience.Master of Science in Life Sciences
18
Zhao, Xi. "3D face analysis : landmarking, expression recognition and beyond." Phd thesis, Ecole Centrale de Lyon, 2010. http://tel.archives-ouvertes.fr/tel-00599660.
Full textAbstract:
This Ph.D thesis work is dedicated to automatic facial analysis in 3D, including facial landmarking and facial expression recognition. Indeed, facial expression plays an important role both in verbal and non verbal communication, and in expressing emotions. Thus, automatic facial expression recognition has various purposes and applications and particularly is at the heart of "intelligent" human-centered human/computer(robot) interfaces. Meanwhile, automatic landmarking provides aprior knowledge on location of face landmarks, which is required by many face analysis methods such as face segmentation and feature extraction used for instance for expression recognition. The purpose of this thesis is thus to elaborate 3D landmarking and facial expression recognition approaches for finally proposing an automatic facial activity (facial expression and action unit) recognition solution.In this work, we have proposed a Bayesian Belief Network (BBN) for recognizing facial activities, such as facial expressions and facial action units. A StatisticalFacial feAture Model (SFAM) has also been designed to first automatically locateface landmarks so that a fully automatic facial expression recognition system can be formed by combining the SFAM and the BBN. The key contributions are the followings. First, we have proposed to build a morphable partial face model, named SFAM, based on Principle Component Analysis. This model allows to learn boththe global variations in face landmark configuration and the local ones in terms of texture and local geometry around each landmark. Various partial face instances can be generated from SFAM by varying model parameters. Secondly, we have developed a landmarking algorithm based on the minimization an objective function describing the correlation between model instances and query faces. Thirdly, we have designed a Bayesian Belief Network with a structure describing the casual relationships among subjects, expressions and facial features. Facial expression oraction units are modelled as the states of the expression node and are recognized by identifying the maximum of beliefs of all states. We have also proposed a novel method for BBN parameter inference using a statistical feature model that can beconsidered as an extension of SFAM. Finally, in order to enrich information usedfor 3D face analysis, and particularly 3D facial expression recognition, we have also elaborated a 3D face feature, named SGAND, to characterize the geometry property of a point on 3D face mesh using its surrounding points.The effectiveness of all these methods has been evaluated on FRGC, BU3DFEand Bosphorus datasets for facial landmarking as well as BU3DFE and Bosphorus datasets for facial activity (expression and action unit) recognition.
19
Shreve, Matthew Adam. "Automatic Macro- and Micro-Facial Expression Spotting and Applications." Scholar Commons, 2013. http://scholarcommons.usf.edu/etd/4770.
Full textAbstract:
Automatically determining the temporal characteristics of facial expressions has extensive application domains such as human-machine interfaces for emotion recognition, face identification, as well as medical analysis. However, many papers in the literature have not addressed the step of determining when such expressions occur. This dissertation is focused on the problem of automatically segmenting macro- and micro-expressions frames (or retrieving the expression intervals) in video sequences, without the need for training a model on a specific subset of such expressions. The proposed method exploits the non-rigid facial motion that occurs during facial expressions by modeling the strain observed during the elastic deformation of facial skin tissue. The method is capable of spotting both macro expressions which are typically associated with emotions such as happiness, sadness, anger, disgust, and surprise, and rapid micro- expressions which are typically, but not always, associated with semi-suppressed macro-expressions. Additionally, we have used this method to automatically retrieve strain maps generated from peak expressions for human identification. This dissertation also contributes a novel 3-D surface strain estimation algorithm using commodity 3-D sensors aligned with an HD camera. We demonstrate the feasibility of the method, as well as the improvements gained when using 3-D, by providing empirical and quantitative comparisons between 2-D and 3-D strain estimations.
20
Paknikar, Gayatri Suhas. "Facial Image Based Expression Classification System Using Committee Neural Networks." University of Akron / OhioLINK, 2008. http://rave.ohiolink.edu/etdc/view?acc_num=akron1210699575.
Full text
21
Maurel, Pierre. "Shape gradients, shape warping and medical application to facial expression analysis." Paris 7, 2008. http://www.theses.fr/2008PA077151.
Full textAbstract:
Cette thèse porte sur le domaine des statistiques de formes. Une forme peut être une courbe plane en 2D ou une surface en 3D. Afin de pouvoir définir ces statistiques (moyenne, modes de variation), nous avons étudié plus précisément, dans une première partie plutôt théorique, le recalage et la mise en correspondance de deux formes entre elle. Cela consiste à développer des moyens de déformer une forme sur une autre. Des distances sont définies entre deux formes et une descente de gradient est effectuée pour déformer la première en la seconde. Nous avons donc défini la notion de gradient sur l'espace des formes et généralisé cette définition pour définir des champs de déformations qui ne dérivent plus d'un gradient. Cette notion a été appliquée pour construire une méthode permettant de déformer une courbe en une autre en étant guidé par des points d'amers définissant des correspondances entre ces deux courbes. Dans une seconde partie, nous présentons une application de ces méthodes à l'analyse d'expressions faciales de patients épileptiques en collaboration avec l'équipe du Professeur Patrick Chauvel à l'hôpital de La Timone à Marseille. Nous avons développé des techniques pour quantifier ces expressions faciales, et ainsi pouvoir les comparer entre elles. Nous avons ensuite étudié un moyen de mettre en relation ces expressions faciales (enregistrées pendant des crises d'épilepsies) avec le signal électrique enregistré simultanément dans le cerveau des patients. Cette mise en relation répond à une demande de l'équipe médicale qui se sert de cette information parmi d'autres pour affiner leur diagnosticThis work focuses on the issue of modeling prior knowledge about shapes, an essential problem in Computer Vision. A shape can be a planar curve in 2D or a surface in 3D. In order to model shape statistics, we studied in a first part, rather theoretical, shape warping and matching. We start by defining distances between shapes? Then, in order to deform a shape onto another, we define the gradient of this shape functional and apply a gradient descent scheme. We also developed a generalization of the gradient notion which can take priors into account and which do not derive from any inner product. We used this new notion for defining an extension of the very well-known level set method that can handle landmarks knowledge. On the application side and in collaboration with professor Patrick Chauvel at La Timone Hospital, Marseille, we worked on the task of correlating facial expressions and the electrical activity in the brain during the epileptic seizures. Therefore, we developed a method for fitting a three-dimensional face model under uncontrolled imaging conditions and used this method for analyzing facial expressions of epileptic patients. Finally we present a first step in the direction of being able to interrelate electrical activity produced by the brain during the seizure (and recorded by stereoelectroencephalography electrodes) and the facial expressions
22
Clark, Elizabeth A. "Application of Automated Facial Expression Analysis and Facial Action Coding System to Assess Affective Response to Consumer Products." Diss., Virginia Tech, 2020. http://hdl.handle.net/10919/97341.
Full textAbstract:
Sensory and consumer sciences seek to comprehend the influences of sensory perception on consumer behaviors such as product liking and purchase. The food industry assesses product liking through hedonic testing but often does not capture affectual response as it pertains to product-generated (PG) and product-associated (PA) emotions. This research sought to assess the application of PA and PG emotion methodology to better understand consumer experiences. A systematic review of the existing literature was performed that focused on the Facial Action Coding System (FACS) and its use to investigate consumer affect and characterize human emotional response to product-based stimuli, which revealed inconsistencies in how FACS is carried out as well as how emotional response is inferred from Action Unit (AU) activation. Automatic Facial Expression Analysis (AFEA), which automates FACS and translates the facial muscular positioning into the basic universal emotions, was then used in a two-part study. In the first study (n=50 participants), AFEA, a Check-All-That-Apply (CATA) emotions questionnaire, and a Single-Target Implicit Association Test (ST-IAT) were used to characterize the relationship between PA as well as PG emotions and consumer behavior (acceptability, purchase intent) towards milk in various types of packaging (k=6). The ST-IAT did not yield significant PA emotions for packaged milk (p>0.05), but correspondence analysis of CATA data produced PA emotion insights including term selection based on arousal and underlying approach/withdrawal motivation related to packaging pigmentation. Time series statistical analysis of AFEA data provided increased insights on significant emotion expression, but the lack of difference (p>0.05) between certain expressed emotions that maintain no related AUs, such as happy and disgust, indicates that AFEA software may not be identifying AUs and determining emotion-based inferences in agreement with FACS. In the second study, AFEA data from the sensory evaluation (n=48 participants) of light-exposed milk stimuli (k=4) stored in packaging with various light-blocking properties) underwent time series statistical analysis to determine if the sensory-engaging nature of control stimuli could impact time series statistical analysis of AFEA data. When compared against the limited sensory engaging (blank screen) control, contempt, happy, and angry were expressed more intensely (p<0.025) and with greater incidence for the light-exposed milk stimuli; neutral was expressed exclusively in the same manner for the blank screen. Comparatively, intense neutral expression (p<0.025) was brief, fragmented, and often accompanied by intense (albeit fleeting) expressions of happy, sad, or contempt for the sensory engaging control (water); emotions such as surprised, scared, and sad were expressed similarly for the light-exposed milk stimuli. As such, it was determined that care should be taken while comparing the control and experimental stimuli in time series analysis as facial activation of muscles/AUs related to sensory perception (e.g., chewing, smelling) can impact the resulting interpretation. Collectively, the use of PA and PG emotion methodology provided additional insights on consumer-product related behaviors. However, it is hard to conclude whether AFEA is yielding emotional interpretations based on true facial expression of emotion or facial actions related to sensory perception for consumer products such as foods and beverages.Doctor of Philosophy
Sensory and consumer sciences seek to comprehend the influences of sensory perception on consumer behaviors such as product liking and purchase. The food industry assesses product liking through consumer testing but often does not capture consumer response as it pertains to emotions such as those experienced while directly interacting with a product (i.e., product-generated emotions, PG) or those attributed to the product based on external information such as branding, marketing, nutrition, social environment, physical environment, memories, etc.( product-associated emotions, PA). This research investigated the application of PA and PG emotion methodology to better understand consumer experiences. A systematic review of the existing scientific literature was performed that focused on the Facial Action Coding System (FACS), a process used determine facially expressed emotion from facial muscular positioning, and its use to investigate consumer behavior and characterize human emotional response to product-based stimuli; the review revealed inconsistencies in how FACS is carried out as well as how emotional response is determined from facial muscular activation. Automatic Facial Expression Analysis (AFEA), which automates FACS, was then used in a two-part study. In the first study (n=50 participants), AFEA, a Check-All-That-Apply (CATA) emotions questionnaire, and a Single-Target Implicit Association Test (ST-IAT) were used to characterize the relationship between PA as well as PG emotions and consumer behavior (acceptability, purchase intent) towards milk in various types of packaging (k=6). While the ST-IAT did not yield significant results (p>0.05), CATA data produced illustrated term selection based on motivation to approach and/or withdrawal from milk based on packaging color. Additionally, the lack of difference (p>0.05) between emotions that do not produce similar facial muscle activations, such as happy and disgust, indicates that AFEA software may not be determining emotions as outlined in the established FACS procedures. In the second study, AFEA data from the sensory evaluation (n=48 participants) of light-exposed milk stimuli (k=4) stored in packaging with various light blocking properties underwent time series statistical analysis to determine if the nature of the control stimulus itself could impact the analysis of AFEA data. When compared against the limited sensory engaging control (a blank screen), contempt, happy, and angry were expressed more intensely (p<0.025) and consistently for the light-exposed milk stimuli; neutral was expressed exclusively in the same manner for the blank screen. Comparatively, intense neutral expression (p<0.025) was brief, fragmented, and often accompanied by intense (although fleeting) expressions of happy, sad, or contempt for the sensory engaging control (water); emotions such as surprised, scared, and sad were expressed similarly for the light-exposed milk stimuli. As such, it was determined that care should be taken as facial activation of muscles/AUs related to sensory perception (e.g., chewing, smelling) can impact the resulting interpretation. Collectively, the use of PA and PG emotion methodology provided additional insights to consumer-product related behaviors. However, it is hard to conclude whether AFEA is yielding emotional interpretations based on true facial expression of emotion or facial actions related to sensory perception for sensory engaging consumer products such as foods and beverages.
23
Derkach, Dmytro. "Spectrum analysis methods for 3D facial expression recognition and head pose estimation." Doctoral thesis, Universitat Pompeu Fabra, 2018. http://hdl.handle.net/10803/664578.
Full textAbstract:
Al llarg de les últimes dècades, l'anàlisi facial ha atret un interès creixent i considerable per part de la comunitat investigadora amb l’objectiu de millorar la interacció i la cooperació entre les persones i les màquines. Aquest interès ha propiciat la creació de sistemes automàtics capaços de reaccionar a diversos estímuls com ara els moviments del cap o les emocions d’una persona. Més enllà, les tasques automatitzades s’han de poder realitzar amb gran precisió dins d’entorns no controlats, fet que ressalta la necessitat d'algoritmes que aprofitin al màxim els avantatges que proporcionen les dades 3D. Aquests sistemes poden ser útils en molts àmbits com ara la interacció home-màquina, tutories, entrevistes, atenció sanitària, màrqueting, etc. En aquesta tesi, ens centrem en dos aspectes de l'anàlisi facial: el reconeixement d'expressions i l'estimació de l'orientació del cap. En ambdós casos, ens enfoquem en l’ús de dades 3D i presentem contribucions que tenen com a objectiu la identificació de representacions significatives de la geometria facial mitjançant mètodes basats en la descomposició espectral:
1. Proposem una tecnologia basada en la representació espectral per al reconeixement d’expressions facials utilitzant exclusivament la geometria 3D, la qual ens permet una descripció completa de la superfície subjacent que pot ser ajustada al nivell de detall desitjat. Dita tecnologia, es basa en la descomposició de fragments locals de la superfície en les seves components de freqüència espacial, d’una manera semblant a la transformada de Fourier, que estan relacionades amb característiques intrínseques de la superfície. Concretament, proposem la utilització de les Graph Laplacian Features (GLFs) que resulten de la projecció dels fragments locals de la superfície a una base comuna obtinguda a partir del Graph Laplacian eigenspace. El mètode proposat s’ha avaluat en termes de reconeixement d’expressions i Action Units (activacions musculars facials), i els resultats obtinguts confirmen que les GLFs produeixen taxes de reconeixement comparables a l’estat de l’art.
2. Proposem un mètode per a l’estimació de l’orientació del cap que permet modelar el manifold subjacent que formen les rotacions generals en 3D. En primer lloc, construïm un sistema completament automàtic que combina la detecció de landmarks (punts facials rellevants) i característiques basades en diccionari, el qual ha obtingut els millors resultats al FG2017 Head Pose Estimation Challenge. Posteriorment, utilitzem una representació basada en tensors i la seva descomposició en els valors singulars d’ordre més alt per tal de separar els subespais de cada factor de rotació i mostrar que cada un d’ells té una estructura clara que pot ser modelada amb funcions trigonomètriques. Aquesta representació proporciona un coneixement detallat del comportament de les dades i pot ser utilitzada per millorar l’estimació de les orientacions dels angles del cap.Facial analysis has attracted considerable research efforts over the last decades, with a growing interest in improving the interaction and cooperation between people and computers. This makes it necessary that automatic systems are able to react to things such as the head movements of a user or his/her emotions. Further, this should be done accurately and in unconstrained environments, which highlights the need for algorithms that can take full advantage of 3D data. These systems could be useful in multiple domains such as human-computer interaction, tutoring, interviewing, health-care, marketing etc. In this thesis, we focus on two aspects of facial analysis: expression recognition and head pose estimation. In both cases, we specifically target the use of 3D data and present contributions that aim to identify meaningful representations of the facial geometry based on spectral decomposition methods: 1. We propose a spectral representation framework for facial expression recognition using exclusively 3D geometry, which allows a complete description of the underlying surface that can be further tuned to the desired level of detail. It is based on the decomposition of local surface patches in their spatial frequency components, much like a Fourier transform, which are related to intrinsic characteristics of the surface. We propose the use of Graph Laplacian Features (GLFs), which result from the projection of local surface patches into a common basis obtained from the Graph Laplacian eigenspace. The proposed approach is tested in terms of expression and Action Unit recognition and results confirm that the proposed GLFs produce state-of-the-art recognition rates. 2. We propose an approach for head pose estimation that allows modeling the underlying manifold that results from general rotations in 3D. We start by building a fully-automatic system based on the combination of landmark detection and dictionary-based features, which obtained the best results in the FG2017 Head Pose Estimation Challenge. Then, we use tensor representation and higher order singular value decomposition to separate the subspaces that correspond to each rotation factor and show that each of them has a clear structure that can be modeled with trigonometric functions. Such representation provides a deep understanding of data behavior, and can be used to further improve the estimation of the head pose angles.
24
Nordén, Frans, and Reis Marlevi Filip von. "A Comparative Analysis of Machine Learning Algorithms in Binary Facial Expression Recognition." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-254259.
Full textAbstract:
In this paper an analysis is conducted regarding whether a higher classification accuracy of facial expressions are possible. The approach used is that the seven basic emotional states are combined into a binary classification problem. Five different machine learning algorithms are implemented: Support vector machines, Extreme learning Machine and three different Convolutional Neural Networks (CNN). The utilized CNN:S were one conventional, one based on VGG16 and transfer learning and one based on residual theory known as RESNET50. The experiment was conducted on two datasets, one small containing no contamination called JAFFE and one big containing contamination called FER2013. The highest accuracy was achieved with the CNN:s where RESNET50 had the highest classification accuracy. When comparing the classification accuracy with the state of the art accuracy an improvement of around 0.09 was achieved on the FER2013 dataset. This dataset does however include some ambiguities regarding what facial expression is shown. It would henceforth be of interest to conduct an experiment where humans classify the facial expressions in the dataset in order to achieve a benchmark.
25
Bilbao, María de los Ángeles, Elza Techio, and Darío Páez. "Acknowledgement of emotional facial expression in Mexican college students." Pontificia Universidad Católica del Perú, 2012. http://repositorio.pucp.edu.pe/index/handle/123456789/102344.
Full textAbstract:
The aim of this study is to explore the patterns of emotion recognition in Mexican bilinguals using the JACFEE (Matsumoto & Ekman, 1988). Previous cross cultural research has documented high agreement in judgments of facial expressions of emotion, however, none of the previous studies has included data from Mexican culture. Participants were 229 Mexican college students (mean age 21.79). Results indicate that each of the seven universal emotions: anger, contempt, disgust, fear, happiness, sadness and surprise was recognized by the participants above chance levels (p < .001), regardless of the gender or ethnicity of the posers. These findings replicate reported data on the high cross cultural agreement in emo- tion recognition (Ekman, 1994) and contribute to the increasing body of evidence regardingthe universality of emotions.Este estudio presenta un meta-análisis sobre la relación entre los valores de Schwartz y el bienestar subjetivo en distintos contextos culturales, con estudiantes, sus familiares e inmigrantes en España. Los resultados confirman una asociación significativa entre los valores y el bienestar. Auto trascendencia y apertura al cambio, y con menor intensidad, conservación, se asocian positivamente con mayor bienestar. Auto trascendencia se asocia con felicidad y satisfacción de forma positiva no homogénea, siendo los inmigrantes quienes presentan medias más bajas. Apertura al cambio se asocia con felicidad, siendo más fuerte la asociación en inmigrantes que en estudiantes. Los valores conservacionistas se asocian homogéneamente. Un segundo estudio sobre criterios de salud psicosocial y bienestar subjetivo -analizando un país sudamericano colectivista y jerárquico como Brasil, y otro europeo más individualista e igualitario como España- confirma que los valores conservacionistas, así como los de apertura al cambio y auto trascendencia, son deseables y favorecen el bienestar.
26
Jottrand, Matthieu. "Support Vector Machines for Classification applied to Facial Expression Analysis and Remote Sensing." Thesis, Linköping University, Department of Electrical Engineering, 2005. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-2938.
Full textAbstract:
The subject of this thesis is the application of Support Vector Machines on two totally different applications, facial expressions recognition and remote sensing.
The basic idea of kernel algorithms is to transpose input data in a higher dimensional space, the feature space, in which linear operations on the data can be processed more easily. These operations in the feature space can be expressed in terms of input data thanks to the kernel functions. Support Vector Machines is a classifier using this kernel method by computing, in the feature space and on basis of examples of the different classes, hyperplanes that separate the classes. The hyperplanes in the feature space correspond to non linear surfaces in the input space.
Concerning facial expressions, the aim is to train and test a classifier able to recognise, on basis of some pictures of faces, which emotion (among these six ones: anger, disgust, fear, joy, sad, and surprise) that is expressed by the person in the picture. In this application, each picture has to be seen has a point in an N-dimensional space where N is the number of pixels in the image.
The second application is the detection of camouflage nets hidden in vegetation using a hyperspectral image taken by an aircraft. In this case the classification is computed for each pixel, represented by a vector whose elements are the different frequency bands of this pixel.
27
Vadapalli, Hima Bindu. "Recognition of facial action units from video streams with recurrent neural networks : a new paradigm for facial expression recognition." University of the Western Cape, 2011. http://hdl.handle.net/11394/5415.
Full textAbstract:
Philosophiae Doctor - PhDThis research investigated the application of recurrent neural networks (RNNs) for recognition of facial expressions based on facial action coding system (FACS). Support vector machines (SVMs) were used to validate the results obtained by RNNs. In this approach, instead of recognizing whole facial expressions, the focus was on the recognition of action units (AUs) that are defined in FACS. Recurrent neural networks are capable of gaining knowledge from temporal data while SVMs, which are time invariant, are known to be very good classifiers. Thus, the research consists of four important components: comparison of the use of image sequences against single static images, benchmarking feature selection and network optimization approaches, study of inter-AU correlations by implementing multiple output RNNs, and study of difference images as an approach for performance improvement. In the comparative studies, image sequences were classified using a combination of Gabor filters and RNNs, while single static images were classified using Gabor filters and SVMs. Sets of 11 FACS AUs were classified by both approaches, where a single RNN/SVM classifier was used for classifying each AU. Results indicated that classifying FACS AUs using image sequences yielded better results than using static images. The average recognition rate (RR) and false alarm rate (FAR) using image sequences was 82.75% and 7.61%, respectively, while the classification using single static images yielded a RR and FAR of 79.47% and 9.22%, respectively. The better performance by the use of image sequences can be at- tributed to RNNs ability, as stated above, to extract knowledge from time-series data. Subsequent research then investigated benchmarking dimensionality reduction, feature selection and network optimization techniques, in order to improve the performance provided by the use of image sequences. Results showed that an optimized network, using weight decay, gave best RR and FAR of 85.38% and 6.24%, respectively. The next study was of the inter-AU correlations existing in the Cohn-Kanade database and their effect on classification models. To accomplish this, a model was developed for the classification of a set of AUs by a single multiple output RNN. Results indicated that high inter-AU correlations do in fact aid classification models to gain more knowledge and, thus, perform better. However, this was limited to AUs that start and reach apex at almost the same time. This suggests the need for availability of a larger database of AUs, which could provide both individual and AU combinations for further investigation. The final part of this research investigated use of difference images to track the motion of image pixels. Difference images provide both noise and feature reduction, an aspect that was studied. Results showed that the use of difference image sequences provided the best results, with RR and FAR of 87.95% and 3.45%, respectively, which is shown to be significant when compared to use of normal image sequences classified using RNNs. In conclusion, the research demonstrates that use of RNNs for classification of image sequences is a new and improved paradigm for facial expression recognition.
28
Maalej, Ahmed. "3D Facial Expressions Recognition Using Shape Analysis and Machine Learning." Thesis, Lille 1, 2012. http://www.theses.fr/2012LIL10025/document.
Full textAbstract:
La reconnaissance des expressions faciales est une tâche difficile, qui a reçu un intérêt croissant au sein de la communauté des chercheurs, et qui impacte les applications dans des domaines liés à l'interaction homme-machine (IHM). Dans le but de construire des systèmes IHM approchant le comportement humain et émotionnellement intelligents, les scientifiques essaient d'introduire la composante émotionnelle dans ce type de systèmes. Le développement récent des capteurs d'acquisition 3D a fait que les données 3D deviennent de plus en plus disponibles, et ce type de données vient pour remédier à des problèmes inhérents aux données 2D tels que les variations d'éclairage, de pose et d'échelle et de faible résolution. Plusieurs bases de données 3D du visage sont publiquement disponibles pour les chercheurs dans le domaine de la reconnaissance d'expression faciale leur permettant ainsi de valider et d'évaluer leurs approches. Cette thèse traite le problème la reconnaissance d'expression faciale et propose une approche basée sur l'analyse de forme pour la reconnaissance d'expression dans un cadre statique (relatif à une seule image) et dynamique (relatif à une séquence vidéo). Tout d'abord, une représentation du modèle 3D du visage basée sur les courbes est proposée pour décrire les traits du visage. Puis, une fois ces courbes sont extraites, l'information de forme qui leur est liée est quantifiée en utilisant un cadre de travail basé sur la géométrie Riemannienne. Nous obtenons, par la suite, des scores de similarité entre les différentes formes locales du visage. Nous constituons, alors, un vecteur de caractéristiques associées à chaque surface faciale. Ensuite, ces caractéristiques sont utilisées comme paramètres d'entrée à des algorithmes d'apprentissage automatique et de classification pour la reconnaissance d'expressions. Des expérimentations exhaustives sont alors entreprises pour valider notre approche et des résultats sont présentés et comparés aux résultats des travaux de l'état de l'artFacial expression recognition is a challenging task, which has received growing interest within the research community, impacting important applications in fields related to human machine interaction (HMI). Toward building human-like emotionally intelligent HMI devices, scientists are trying to include the essence of human emotional state in such systems. The recent development of 3D acquisition sensors has made 3D data more available, and this kind of data comes to alleviate the problems inherent in 2D data such as illumination, pose and scale variations as well as low resolution. Several 3D facial databases are publicly available for the researchers in the field of face and facial expression recognition to validate and evaluate their approaches. This thesis deals with facial expression recognition (FER) problem and proposes an approach based on shape analysis to handle both static and dynamic FER tasks. Our approach includes the following steps: first, a curve-based representation of the 3D face model is proposed to describe facial features. Then, once these curves are extracted, their shape information is quantified using a Riemannain framework. We end up with similarity scores between different facial local shapes constituting feature vectors associated with each facial surface. Afterwards, these features are used as entry parameters to some machine learning and classification algorithms to recognize expressions. Exhaustive experiments are derived to validate our approach and results are presented and compared to the related work achievements
29
Chung, Koon Yin C. "Facial Expression Recognition by Using Class Mean Gabor Responses with Kernel Principal Component Analysis." Ohio University / OhioLINK, 2010. http://rave.ohiolink.edu/etdc/view?acc_num=ohiou1260468428.
Full text
30
Ghayoumi, Mehdi. "FACIAL EXPRESSION ANALYSIS USING DEEP LEARNING WITH PARTIAL INTEGRATION TO OTHER MODALITIES TO DETECT EMOTION." Kent State University / OhioLINK, 2017. http://rave.ohiolink.edu/etdc/view?acc_num=kent1501273062260458.
Full text
31
Chu, Wen-Sheng. "Automatic Analysis of Facial Actions: Learning from Transductive, Supervised and Unsupervised Frameworks." Research Showcase @ CMU, 2017. http://repository.cmu.edu/dissertations/929.
Full textAbstract:
Automatic analysis of facial actions (AFA) can reveal a person’s emotion, intention, and physical state, and make possible a wide range of applications. To enable reliable, valid, and efficient AFA, this thesis investigates automatic analysis of facial actions through transductive, supervised and unsupervised learning. Supervised learning for AFA is challenging, in part, because of individual differences among persons in face shape and appearance and variation in video acquisition and context. To improve generalizability across persons, we propose a transductive framework, Selective Transfer Machine (STM), which personalizes generic classifiers through joint sample reweighting and classifier learning. By personalizing classifiers, STM offers improved generalization to unknown persons. As an extension, we develop a variant of STM for use when partially labeled data are available. Additional challenges for supervised learning include learning an optimal representation for classification, variation in base rates of action units (AUs), correlation between AUs and temporal consistency. While these challenges could be partly accommodated with an SVM or STM, a more powerful alternative is afforded by an end-to-end supervised framework (i.e., deep learning). We propose a convolutional network with long short-term memory (LSTM) and multi-label sampling strategies. We compared SVM, STM and deep learning approaches with respect to AU occurrence and intensity in and between BP4D+ [282] and GFT [93] databases, which consist of around 0.6 million annotated frames. Annotated video is not always possible or desirable. We introduce an unsupervised Branch-and-Bound framework to discover correlated facial actions in un-annotated video. We term this approach Common Event Discovery (CED). We evaluate CED in video and motion capture data. CED achieved moderate convergence with supervised approaches and enabled discovery of novel patterns occult to supervised approaches.
32
Crist, Courtney Alissa. "Application of Automated Facial Expression Analysis and Qualitative Analysis to Assess Consumer Perception and Acceptability of Beverages and Water." Diss., Virginia Tech, 2016. http://hdl.handle.net/10919/79718.
Full textAbstract:
Sensory and consumer sciences aim to understand the influences of product acceptability and purchase decisions. The food industry measures product acceptability through hedonic testing but often does not assess implicit or qualitative response. Incorporation of qualitative research and automated facial expression analysis (AFEA) may supplement hedonic acceptability testing to provide product insights. The purpose of this research was to assess the application of AFEA and qualitative analysis to understand consumer experience and response. In two studies, AFEA was applied to elucidate consumers emotional response to dairy (n=42) and water (n=46) beverages. For dairy, unflavored milk (x=6.6±1.8) and vanilla syrup flavored milk (x=5.9±2.2) (p>0.05) were acceptably rated (1=dislike extremely; 9=like extremely) while salty flavored milk (x=2.3±1.3) was least acceptable (p<0.05). Vanilla syrup flavored milk generated emotions with surprised intermittently present over time (10 sec) (p<0.025) compared to unflavored milk. Salty flavored milk created an intense disgust response among other emotions compared to unflavored milk (p<0.025). Using a bitter solutions model in water, an inverse relationship existed with acceptability as bitter intensity increased (rs=-0.90; p<0.0001). Facial expressions characterized as disgust and happy emotion increased in duration as bitter intensity increased while neutral remained similar across bitter intensities compared to the control (p<0.025). In a mixed methods analysis to enumerate microbial populations, assess water quality, and qualitatively gain consumer insights regarding water fountains and water filling stations, results inferred that water quality differences did not exist between water fountains and water filling stations (metals, pH, chlorine, and microbial) (p>0.05). However, the exterior of water fountains were microbially (8.8 CFU/cm^2) and visually cleaner than filling stations (10.4x10^3 CFU/cm^2) (p<0.05). Qualitative analysis contradicted quantitative findings as participants preferred water filling stations because they felt they were cleaner and delivered higher quality water. Lastly, The Theory of Planned Behavior was able to assist in understanding undergraduates' reusable water bottle behavior and revealed 11 categories (attitudes n=6; subjective norms n=2; perceived behavioral control n=2; intentions n=1). Collectively, the use of AFEA and qualitative analysis provided additional insight to consumer-product interaction and acceptability; however, additional research should include improving the sensitivity of AFEA to consumer product evaluation.Ph. D.
33
Peschka-Daskalos, Patricia Jean. "An Intercultural Analysis of Differences in Appropriateness Ratings of Facial Expressions Between Japanese and American Subjects." PDXScholar, 1993. https://pdxscholar.library.pdx.edu/open_access_etds/4700.
Full textAbstract:
In 1971 Paul Ekman posited his Neuro-Cultural Theory of Emotion which stated that expressions of emotion are universal but controlled by cultural display rules. This thesis tests the Neuro-Cultural Theory by having subjects from two cultures, Japan and the United States, judge the perceived appropriateness facial expressions in social situations. Preliminary procedures resulted in a set of scenarios in which socially appropriate responses were deemed to be either "Happy", "Angry" or "Surprised". Data in the experimental phase of the study were collected using a questionnaire format. Through the use of a 5-point Likert scale, each subject rated the appropriateness of happy, anger and surprise expressions in positive, negative and ambiguous social situations. Additionally, the subjects were asked to label each expression in each situation. The responses were analyzed statistically using Analysis of Variance procedures. Label percentages were also calculated for: the second task in the study. No support was found for two of the three research hypotheses, and only partial support was found for a third research hypothesis. These results were discussed in terms of the need for greater theoretical and methodological refinements.
34
Aina, Segun. "Loughborough University Spontaneous Expression Database and baseline results for automatic emotion recognition." Thesis, Loughborough University, 2015. https://dspace.lboro.ac.uk/2134/19524.
Full textAbstract:
The study of facial expressions in humans dates back to the 19th century and the study of the emotions that these facial expressions portray dates back even further. It is a natural part of non-verbal communication for humans to pass across messages using facial expressions either consciously or subconsciously, it is also routine for other humans to recognize these facial expressions and understand or deduce the underlying emotions which they represent. Over two decades ago and following technological advances, particularly in the area of image processing, research began into the use of machines for the recognition of facial expressions from images with the aim of inferring the corresponding emotion. Given a previously unknown test sample, the supervised learning problem is to accurately determine the facial expression class to which the test sample belongs using the knowledge of the known class memberships of each image from a set of training images. The solution to this problem building an effective classifier to recognize the facial expression is hinged on the availability of representative training data. To date, much of the research in the area of Facial Expression Recognition (FER) is still based on posed (acted) facial expression databases, which are often exaggerated and therefore not representative of real life affective displays, as such there is a need for more publically accessible spontaneous databases that are well labelled. This thesis therefore reports on the development of the newly collected Loughborough University Spontaneous Expression Database (LUSED); designed to bolster the development of new recognition systems and to provide a benchmark for researchers to compare results with more natural expression classes than most existing databases. To collect the database, an experiment was set up where volunteers were discretely videotaped while they watched a selection of emotion inducing video clips. The utility of the new LUSED dataset is validated using both traditional and more recent pattern recognition techniques; (1) baseline results are presented using the combination of Principal Component Analysis (PCA), Fisher Linear Discriminant Analysis (FLDA) and their kernel variants Kernel Principal Component Analysis (KPCA), Kernel Fisher Discriminant Analysis (KFDA) with a Nearest Neighbour-based classifier. These results are compared to the performance of an existing natural expression database Natural Visible and Infrared Expression (NVIE) database. A scheme for the recognition of encrypted facial expression images is also presented. (2) Benchmark results are presented by combining PCA, FLDA, KPCA and KFDA with a Sparse Representation-based Classifier (SRC). A maximum accuracy of 68% was obtained recognizing five expression classes, which is comparatively better than the known maximum for a natural database; around 70% (from recognizing only three classes) obtained from NVIE.
35
Ener, Emrah. "Recognition Of Human Face Expressions." Master's thesis, METU, 2006. http://etd.lib.metu.edu.tr/upload/3/12607521/index.pdf.
Full textAbstract:
In this study a fully automatic and scale invariant feature extractor which does not require manual initialization or special equipment is proposed. Face location and size is extracted using skin segmentation and ellipse fitting. Extracted face region is scaled to a predefined size, later upper and lower facial templates are used for feature extraction. Template localization and template parameter calculations are carried out using Principal Component Analysis. Changes in facial feature coordinates between analyzed image and neutral expression image are used for expression classification. Performances of different classifiers are evaluated. Performance of proposed feature extractor is also tested on sample video sequences. Facial features are extracted in the first frame and KLT tracker is used for tracking the extracted features. Lost features are detected using face geometry rules and they are relocated using feature extractor. As an alternative to feature based technique an available holistic method which analyses face without partitioning is implemented. Face images are filtered using Gabor filters tuned to different scales and orientations. Filtered images are combined to form Gabor jets. Dimensionality of Gabor jets is decreased using Principal Component Analysis. Performances of different classifiers on low dimensional Gabor jets are compared. Feature based and holistic classifier performances are compared using JAFFE and AF facial expression databases.
36
Wang, Ding. "The systematic analysis and innovative design of the essential cultural elements with Peking Opera Painted Faces (POPF)." Thesis, Brunel University, 2016. http://bura.brunel.ac.uk/handle/2438/14785.
Full textAbstract:
Peking Opera (‘Jingju’) is one of the most iconic traditional theatres in China, marketed as a global signifier of Chinese theatre and national identity. The research considers current recognised illustrations of Peking Opera Painted Faces (POPF). Through both new cultural-based product design solutions and design inspired visual communication solutions, the purpose of the new design is to apply the semantic features of Chinese Traditional POPF to the modern design, and establish close contact with all aspects of social life. Also to promote a series of developable plans including product design, interaction design, system design and service design in China and Western countries proceeding from POPF, along with the integration of other elements of traditional Chinese cultures and arts. *POPF is short for Peking Opera Painted Faces.
37
Arango, Duque Carlos. "Analysis of Micro-Expressions based on the Riesz Pyramid : Application to Spotting and Recognition." Thesis, Lyon, 2018. http://www.theses.fr/2018LYSES062/document.
Full textAbstract:
Les micro-expressions sont des expressions faciales brèves et subtiles qui apparaissent et disparaissent en une fraction de seconde. Ce type d'expressions reflèterait "l'intention réelle" de l'être humain. Elles ont été étudiées pour mieux comprendre les communications non verbales et dans un contexte médicale lorsqu'il devient presque impossible d'engager une conversation ou d'essayer de traduire les émotions du visage ou le langage corporel d'un patient. Cependant, détecter et reconnaître les micro-expressions est une tâche difficile pour l'homme. Il peut donc être pertinent de développer des systèmes d'aide à la communication exploitant les micro-expressions. De nombreux travaux ont été réalisés dans les domaines de l'informatique affective et de la vision par ordinateur pour analyser les micro-expressions, mais une grande majorité de ces méthodes repose essentiellement sur des méthodes de vision par ordinateur classiques telles que les motifs binaires locaux, les histogrammes de gradients orientés et le flux optique. Étant donné que ce domaine de recherche est relativement nouveau, d'autres pistes restent à explorer. Dans cette thèse, nous présentons une nouvelle méthodologie pour l'analyse des petits mouvements (que nous appellerons par la suite mouvements subtils) et des micro-expressions. Nous proposons d'utiliser la pyramide de Riesz, une approximation multi-échelle et directionnelle de la transformation de Riesz qui a été utilisée pour l'amplification du mouvement dans les vidéos à l'aide de l'estimation de la phase 2D locale. Pour l'étape générale d'analyse de mouvements subtils, nous transformons une séquence d'images avec la pyramide de Riesz, extrayons et filtrons les variations de phase de l'image. Ces variations de phase sont en lien avec le mouvement. De plus, nous isolons les régions d'intérêt où des mouvements subtils pourraient avoir lieu en masquant les zones de bruit à l'aide de l'amplitude locale. La séquence d'image est transformée en un signal ID utilisé pour l'analyse temporelle et la détection de mouvement subtils. Nous avons créé notre propre base de données de séquences de mouvements subtils pour tester notre méthode. Pour l'étape de détection de micro-expressions, nous adaptons la méthode précédente au traitement de certaines régions d'intérêt du visage. Nous développons également une méthode heuristique pour détecter les micro-événements faciaux qui sépare les micro-expressions réelles des clignotements et des mouvements subtils des yeux. Pour la classification des micro-expressions, nous exploitons l'invariance, sur de courtes durées, de l'orientation dominante issue de la transformation de Riesz afin de moyenner la séquence d'une micro-expression en une paire d'images. A partir de ces images, nous définissons le descripteur MORF (Mean Oriented Riesz Feature) constitué d'histogrammes d'orientation. Les performances de nos méthodes sont évaluées à l'aide de deux bases de données de micro-expressions spontanéesMicro-expressions are brief and subtle facial expressions that go on and off the face in a fraction of a second. This kind of facial expressions usually occurs in high stake situations and is considered to reflect a humans real intent. They have been studied to better understand non-verbal communications and in medical applications where is almost impossible to engage in a conversation or try to read the facial emotions or body language of a patient. There has been some interest works in micro-expression analysis, however, a great majority of these methods are based on classically established computer vision methods such as local binary patterns, histogram of gradients and optical flow. Considering the fact that this area of research is relatively new, much contributions remains to be made. ln this thesis, we present a novel methodology for subtle motion and micro-expression analysis. We propose to use the Riesz pyramid, a multi-scale steerable Hilbert transformer which has been used for 2-D phase representation and video amplification, as the basis for our methodology. For the general subtle motion analysis step, we transform an image sequence with the Riesz pyramid, extract and lifter the image phase variations as proxies for motion. Furthermore, we isolate regions of intcrcst where subtle motion might take place and mask noisy areas by thresholding the local amplitude. The total sequence is transformed into a ID signal which is used fo temporal analysis and subtle motion spotting. We create our own database of subtle motion sequences to test our method. For the micro-expression spotting step, we adapt the previous method to process some facial regions of interest. We also develop a heuristic method to detect facial micro-events that separates real micro-expressions from eye blinkings and subtle eye movements. For the micro-expression classification step, we exploit the dominant orientation constancy fom the Riesz transform to average the micro-expression sequence into an image pair. Based on that, we introduce the Mean Oriented Riesz Feature descriptor. The accuracy of our methods are tested in Iwo spontaneous micro-expressions databases. Furthermore, wc analyse the parameter variations and their effect in our results
38
Hariri, Walid. "Contribution à la reconnaissance/authentification de visages 2D/3D." Thesis, Cergy-Pontoise, 2017. http://www.theses.fr/2017CERG0905/document.
Full textAbstract:
L’analyse de visages 3D y compris la reconnaissance des visages et des expressions faciales 3D est devenue un domaine actif de recherche ces dernières années. Plusieurs méthodes ont été développées en utilisant des images 2D pour traiter ces problèmes. Cependant, ces méthodes présentent un certain nombre de limitations dépendantes à l’orientation du visage, à l’éclairage, à l’expression faciale, et aux occultations. Récemment, le développement des capteurs d’acquisition 3D a fait que les données 3D deviennent de plus en plus disponibles. Ces données 3D sont relativement invariables à l’illumination et à la pose, mais elles restent sensibles à la variation de l’expression. L’objectif principal de cette thèse est de proposer de nouvelles techniques de reconnaissance/vérification de visages et de reconnaissance d’expressions faciales 3D. Tout d’abord, une méthode de reconnaissance de visages en utilisant des matrices de covariance comme des descripteurs de régions de visages est proposée. Notre méthode comprend les étapes suivantes : le prétraitement et l’alignement de visages, un échantillonnage uniforme est ensuite appliqué sur la surface faciale pour localiser un ensemble de points de caractéristiques. Autours de chaque point, nous extrayons une matrice de covariance comme un descripteur de région du visage. Deux méthodes d’appariement sont ainsi proposées, et différentes distances (géodésiques / non-géodésique) sont appliquées pour comparer les visages. La méthode proposée est évaluée sur troisbases de visages GAVAB, FRGCv2 et BU-3DFE. Une description hiérarchique en utilisant trois niveaux de covariances est ensuite proposée et validée. La deuxième partie de cette thèse porte sur la reconnaissance des expressions faciales 3D. Pour ce faire, nous avons proposé d’utiliser les matrices de covariances avec les méthodes noyau. Dans cette contribution, nous avons appliqué le noyau de Gauss pour transformer les matrices de covariances en espace d’Hilbert. Cela permet d’utiliser les algorithmes qui sont déjà implémentés pour l’espace Euclidean (i.e. SVM) dans cet espace non-linéaire. Des expérimentations sont alors entreprises sur deux bases d’expressions faciales 3D (BU-3DFE et Bosphorus) pour reconnaître les six expressions faciales prototypiques3D face analysis including 3D face recognition and 3D Facial expression recognition has become a very active area of research in recent years. Various methods using 2D image analysis have been presented to tackle these problems. 2D image-based methods are inherently limited by variability in imaging factors such as illumination and pose. The recent development of 3D acquisition sensors has made 3D data more and more available. Such data is relatively invariant to illumination and pose, but it is still sensitive to expression variation. The principal objective of this thesis is to propose efficient methods for 3D face recognition/verification and 3D facial expression recognition. First, a new covariance based method for 3D face recognition is presented. Our method includes the following steps : first 3D facial surface is preprocessed and aligned. A uniform sampling is then applied to localize a set of feature points, around each point, we extract a matrix as local region descriptor. Two matching strategies are then proposed, and various distances (geodesic and non-geodesic) are applied to compare faces. The proposed method is assessed on three datasetsincluding GAVAB, FRGCv2 and BU-3DFE. A hierarchical description using three levels of covariances is then proposed and validated. In the second part of this thesis, we present an efficient approach for 3D facial expression recognition using kernel methods with covariance matrices. In this contribution, we propose to use Gaussian kernel which maps covariance matrices into a high dimensional Hilbert space. This enables to use conventional algorithms developed for Euclidean valued data such as SVM on such non-linear valued data. The proposed method have been assessed on two known datasets including BU-3DFE and Bosphorus datasets to recognize the six prototypical expressions
39
Jain, Varun. "Visual Observation of Human Emotions." Thesis, Université Grenoble Alpes (ComUE), 2015. http://www.theses.fr/2015GRENM006/document.
Full textAbstract:
Cette thèse a pour sujet le développement de méthodes et de techniques permettant d'inférer l'état affectif d'une personne à partir d'informations visuelles. Plus précisement, nous nous intéressons à l'analyse d'expressions du visage, puisque le visage est la partie la mieux visible du corps, et que l'expression du visage est la manifestation la plus évidente de l'affect. Nous étudions différentes théories psychologiques concernant affect et émotions, et différentes facons de représenter et de classifier les émotions d'une part et la relation entre expression du visage et émotion sousjacente d'autre part. Nous présentons les dérivées Gaussiennes multi-échelle en tant que descripteur dímages pour l'estimation de la pose de la tête, pour la détection de sourire, puis aussi pour la mesure de l'affect. Nous utilisons l'analyse en composantes principales pour la réduction de la dimensionalité, et les machines à support de vecteur pour la classification et la regression. Nous appliquons cette même architecture, simple et efficace, aux différents problèmes que sont l'estimation de la pose de tête, la détection de sourire, et la mesure d'affect. Nous montrons que non seulement les dérivées Gaussiennes multi-échelle ont une performance supérieure aux populaires filtres de Gabor, mais qu'elles sont également moins coûteuses en calculs. Lors de nos expérimentations nous avons constaté que dans le cas d'un éclairage partiel du visage les dérivées Gaussiennes multi-échelle ne fournissent pas une description d'image suffisamment discriminante. Pour résoudre ce problème nous combinons des dérivées Gaussiennes avec des histogrammes locaux de type LBP (Local Binary Pattern). Avec cette combinaison nous obtenons des résultats à la hauteur de l'état de l'art pour la détection de sourire dans le base d'images GENKI qui comporte des images de personnes trouvées «dans la nature» sur internet, et avec la difficile «extended YaleB database». Pour la classification dans la reconnaissance de visage nous utilisons un apprentissage métrique avec comme mesure de similarité une distance de Minkowski. Nous obtenons le résultat que les normes L1 and L2 ne fournissent pas toujours la distance optimale; cet optimum est souvent obtenu avec une norme Lp où p n'est pas entier. Finalement, nous développons un système multi-modal pour la détection de dépressions nerveuses, avec en entrée des informations audio et vidéo. Pour la détection de mouvements intra-faciaux dans les données vidéo nous utilisons de descripteurs de type LBP-TOP (Local Binary Patterns -Three Orthogonal Planes), alors que nous utilisons des trajectoires denses pour les mouvements plus globaux, par exemple de la tête ou des épaules. Nous avons trouvé que les descripteurs LBP-TOP encodés avec des vecteurs de Fisher suffisent pour dépasser la performance de la méthode de reférence dans la compétition «Audio Visual Emotion Challenge (AVEC) 2014». Nous disposons donc d'une technique effective pour l'evaluation de l'état dépressif, technique qui peut aisement être étendue à d'autres formes d'émotions qui varient lentement, comme l'humeur (mood an Anglais)In this thesis we focus on the development of methods and techniques to infer affect from visual information. We focus on facial expression analysis since the face is one of the least occluded parts of the body and facial expressions are one of the most visible manifestations of affect. We explore the different psychological theories on affect and emotion, different ways to represent and classify emotions and the relationship between facial expressions and underlying emotions. We present the use of multiscale Gaussian derivatives as an image descriptor for head pose estimation, smile detection before using it for affect sensing. Principal Component Analysis is used for dimensionality reduction while Support Vector Machines are used for classification and regression. We are able to employ the same, simple and effective architecture for head pose estimation, smile detection and affect sensing. We also demonstrate that not only do multiscale Gaussian derivatives perform better than the popular Gabor Filters but are also computationally less expensive to compute. While performing these experiments we discovered that multiscale Gaussian derivatives do not provide an appropriately discriminative image description when the face is only partly illuminated. We overcome this problem by combining Gaussian derivatives with Local Binary Pattern (LBP) histograms. This combination helps us achieve state-of-the-art results for smile detection on the benchmark GENKI database which contains images of people in the "wild" collected from the internet. We use the same description method for face recognition on the CMU-PIE database and the challenging extended YaleB database and our results compare well with the state-of-the-art. In the case of face recognition we use metric learning for classification, adopting the Minkowski distance as the similarity measure. We find that L1 and L2 norms are not always the optimum distance metrics and the optimum is often an Lp norm where p is not an integer. Lastly we develop a multi-modal system for depression estimation with audio and video information as input. We use Local Binary Patterns -Three Orthogonal Planes (LBP-TOP) features to capture intra-facial movements in the videos and dense trajectories for macro movements such as the movement of the head and shoulders. These video features along with Low Level Descriptor (LLD) audio features are encoded using Fisher Vectors and finally a Support Vector Machine is used for regression. We discover that the LBP-TOP features encoded with Fisher Vectors alone are enough to outperform the baseline method on the Audio Visual Emotion Challenge (AVEC) 2014 database. We thereby present an effective technique for depression estimation which can be easily extended for other slowly varying aspects of emotions such as mood
40
Zhang, Yuyao. "Non-linear dimensionality reduction and sparse representation models for facial analysis." Thesis, Lyon, INSA, 2014. http://www.theses.fr/2014ISAL0019/document.
Full textAbstract:
Les techniques d'analyse du visage nécessitent généralement une représentation pertinente des images, notamment en passant par des techniques de réduction de la dimension, intégrées dans des schémas plus globaux, et qui visent à capturer les caractéristiques discriminantes des signaux. Dans cette thèse, nous fournissons d'abord une vue générale sur l'état de l'art de ces modèles, puis nous appliquons une nouvelle méthode intégrant une approche non-linéaire, Kernel Similarity Principle Component Analysis (KS-PCA), aux Modèles Actifs d'Apparence (AAMs), pour modéliser l'apparence d'un visage dans des conditions d'illumination variables. L'algorithme proposé améliore notablement les résultats obtenus par l'utilisation d'une transformation PCA linéaire traditionnelle, que ce soit pour la capture des caractéristiques saillantes, produites par les variations d'illumination, ou pour la reconstruction des visages. Nous considérons aussi le problème de la classification automatiquement des poses des visages pour différentes vues et différentes illumination, avec occlusion et bruit. Basé sur les méthodes des représentations parcimonieuses, nous proposons deux cadres d'apprentissage de dictionnaire pour ce problème. Une première méthode vise la classification de poses à l'aide d'une représentation parcimonieuse active (Active Sparse Representation ASRC). En fait, un dictionnaire est construit grâce à un modèle linéaire, l'Incremental Principle Component Analysis (Incremental PCA), qui a tendance à diminuer la redondance intra-classe qui peut affecter la performance de la classification, tout en gardant la redondance inter-classes, qui elle, est critique pour les représentations parcimonieuses. La seconde approche proposée est un modèle des représentations parcimonieuses basé sur le Dictionary-Learning Sparse Representation (DLSR), qui cherche à intégrer la prise en compte du critère de la classification dans le processus d'apprentissage du dictionnaire. Nous faisons appel dans cette partie à l'algorithme K-SVD. Nos résultats expérimentaux montrent la performance de ces deux méthodes d'apprentissage de dictionnaire. Enfin, nous proposons un nouveau schéma pour l'apprentissage de dictionnaire adapté à la normalisation de l'illumination (Dictionary Learning for Illumination Normalization: DLIN). L'approche ici consiste à construire une paire de dictionnaires avec une représentation parcimonieuse. Ces dictionnaires sont construits respectivement à partir de visages illuminées normalement et irrégulièrement, puis optimisés de manière conjointe. Nous utilisons un modèle de mixture de Gaussiennes (GMM) pour augmenter la capacité à modéliser des données avec des distributions plus complexes. Les résultats expérimentaux démontrent l'efficacité de notre approche pour la normalisation d'illuminationFace analysis techniques commonly require a proper representation of images by means of dimensionality reduction leading to embedded manifolds, which aims at capturing relevant characteristics of the signals. In this thesis, we first provide a comprehensive survey on the state of the art of embedded manifold models. Then, we introduce a novel non-linear embedding method, the Kernel Similarity Principal Component Analysis (KS-PCA), into Active Appearance Models, in order to model face appearances under variable illumination. The proposed algorithm successfully outperforms the traditional linear PCA transform to capture the salient features generated by different illuminations, and reconstruct the illuminated faces with high accuracy. We also consider the problem of automatically classifying human face poses from face views with varying illumination, as well as occlusion and noise. Based on the sparse representation methods, we propose two dictionary-learning frameworks for this pose classification problem. The first framework is the Adaptive Sparse Representation pose Classification (ASRC). It trains the dictionary via a linear model called Incremental Principal Component Analysis (Incremental PCA), tending to decrease the intra-class redundancy which may affect the classification performance, while keeping the extra-class redundancy which is critical for sparse representation. The other proposed work is the Dictionary-Learning Sparse Representation model (DLSR) that learns the dictionary with the aim of coinciding with the classification criterion. This training goal is achieved by the K-SVD algorithm. In a series of experiments, we show the performance of the two dictionary-learning methods which are respectively based on a linear transform and a sparse representation model. Besides, we propose a novel Dictionary Learning framework for Illumination Normalization (DL-IN). DL-IN based on sparse representation in terms of coupled dictionaries. The dictionary pairs are jointly optimized from normally illuminated and irregularly illuminated face image pairs. We further utilize a Gaussian Mixture Model (GMM) to enhance the framework's capability of modeling data under complex distribution. The GMM adapt each model to a part of the samples and then fuse them together. Experimental results demonstrate the effectiveness of the sparsity as a prior for patch-based illumination normalization for face images
41
Weber, Marlene. "Automotive emotions : a human-centred approach towards the measurement and understanding of drivers' emotions and their triggers." Thesis, Brunel University, 2018. http://bura.brunel.ac.uk/handle/2438/16647.
Full textAbstract:
The automotive industry is facing significant technological and sociological shifts, calling for an improved understanding of driver and passenger behaviours, emotions and needs, and a transformation of the traditional automotive design process. This research takes a human-centred approach to automotive research, investigating the users' emotional states during automobile driving, with the goal to develop a framework for automotive emotion research, thus enabling the integration of technological advances into the driving environment. A literature review of human emotion and emotion in an automotive context was conducted, followed by three driving studies investigating emotion through Facial-Expression Analysis (FEA): An exploratory study investigated whether emotion elicitation can be applied in driving simulators, and if FEA can detect the emotions triggered. The results allowed confidence in the applicability of emotion elicitation to a lab-based environment to trigger emotional responses, and FEA to detect those. An on-road driving study was conducted in a natural setting to investigate whether natures and frequencies of emotion events could be automatically measured. The possibility of assigning triggers to those was investigated. Overall, 730 emotion events were detected during a total driving time of 440 minutes, and event triggers were assigned to 92% of the emotion events. A similar second on-road study was conducted in a partially controlled setting on a planned road circuit. In 840 minutes, 1947 emotion events were measured, and triggers were successfully assigned to 94% of those. The differences in natures, frequencies and causes of emotions on different road types were investigated. Comparison of emotion events for different roads demonstrated substantial variances of natures, frequencies and triggers of emotions on different road types. The results showed that emotions play a significant role during automobile driving. The possibility of assigning triggers can be used to create a better understanding of causes of emotions in the automotive habitat. Both on-road studies were compared through statistical analysis to investigate influences of the different study settings. Certain conditions (e.g. driving setting, social interaction) showed significant influence on emotions during driving. This research establishes and validates a methodology for the study of emotions and their causes in the driving environment through which systems and factors causing positive and negative emotional effects can be identified. The methodology and results can be applied to design and research processes, allowing the identification of issues and opportunities in current automotive design to address challenges of future automotive design. Suggested future research includes the investigation of a wider variety of road types and situations, testing with different automobiles and the combination of multiple measurement techniques.
42
Husseini, Orabi Ahmed. "Multi-Modal Technology for User Interface Analysis including Mental State Detection and Eye Tracking Analysis." Thesis, Université d'Ottawa / University of Ottawa, 2017. http://hdl.handle.net/10393/36451.
Full textAbstract:
We present a set of easy-to-use methods and tools to analyze human attention, behaviour, and physiological responses. A potential application of our work is evaluating user interfaces being used in a natural manner. Our approach is designed to be scalable and to work remotely on regular personal computers using expensive and noninvasive equipment.
The data sources our tool processes are nonintrusive, and captured from video; i.e. eye tracking, and facial expressions. For video data retrieval, we use a basic webcam. We investigate combinations of observation modalities to detect and extract affective and mental states.
Our tool provides a pipeline-based approach that 1) collects observational, data 2) incorporates and synchronizes the signal modality mentioned above, 3) detects users' affective and mental state, 4) records user interaction with applications and pinpoints the parts of the screen users are looking at, 5) analyzes and visualizes results.
We describe the design, implementation, and validation of a novel multimodal signal fusion engine, Deep Temporal Credence Network (DTCN). The engine uses Deep Neural Networks to provide 1) a generative and probabilistic inference model, and 2) to handle multimodal data such that its performance does not degrade due to the absence of some modalities. We report on the recognition accuracy of basic emotions for each modality. Then, we evaluate our engine in terms of effectiveness of recognizing basic six emotions and six mental states, which are agreeing, concentrating, disagreeing, interested, thinking, and unsure.
Our principal contributions include the implementation of a 1) multimodal signal fusion engine, 2) real time recognition of affective and primary mental states from nonintrusive and inexpensive modality, 3) novel mental state-based visualization techniques, 3D heatmaps, 3D scanpaths, and widget heatmaps that find parts of the user interface where users are perhaps unsure, annoyed, frustrated, or satisfied.
43
Dagnes, Nicole. "3D human face analysis for recognition applications and motion capture." Thesis, Compiègne, 2020. http://www.theses.fr/2020COMP2542.
Full textAbstract:
Cette thèse se propose comme une étude géométrique de la surface faciale en 3D, dont le but est de fournir un ensemble d'entités, issues du contexte de la géométrie différentielle, à utiliser comme descripteurs faciaux dans les applications d'analyse du visage, comme la reconnaissance faciale et la reconnaissance des expressions faciales. En effet, bien que chaque visage soit unique, tous les visages sont similaires et leurs caractéristiques morphologiques sont les mêmes pour tous les individus. Par conséquent, il est primordial pour l'analyse des visages d'extraire les caractéristiques faciales les plus appropriées. Tous les traits du visage, proposés dans cette étude, sont basés uniquement sur les propriétés géométriques de la surface faciale. En effet, l'objectif final de cette recherche est de démontrer que la géométrie différentielle est un outil complet pour l'analyse des visages et que les caractéristiques géométriques conviennent pour décrire et comparer des visages et, en général, pour extraire des informations pertinentes pour l'analyse faciale dans les différents domaines d'application. Enfin, ce travail se concentre aussi sur l'analyse des troubles musculo-squelettiques en proposant une quantification objective des mouvements du visage pour aider la chirurgie maxillo-faciale et la rééducation des mouvements du visage. Ce travail de recherche explore le système de capture du mouvement 3D, en adoptant la plateforme Technologie, Sport et Santé, située au Centre d'Innovation de l'Université de Technologie de Compiègne, au sein du Laboratoire de Biomécanique et Bioingénierie (BMBI)This thesis is intended as a geometrical study of the three-dimensional facial surface, whose aim is to provide an application framework of entities coming from Differential Geometry context to use as facial descriptors in face analysis applications, like FR and FER fields. Indeed, although every visage is unique, all faces are similar and their morphological features are the same for all mankind. Hence, it is primary for face analysis to extract suitable features. All the facial features, proposed in this study, are based only on the geometrical properties of the facial surface. Then, these geometrical descriptors and the related entities proposed have been applied in the description of facial surface in pattern recognition contexts. Indeed, the final goal of this research is to prove that Differential Geometry is a comprehensive tool oriented to face analysis and geometrical features are suitable to describe and compare faces and, generally, to extract relevant information for human face analysis in different practical application fields. Finally, since in the last decades face analysis has gained great attention also for clinical application, this work focuses on musculoskeletal disorders analysis by proposing an objective quantification of facial movements for helping maxillofacial surgery and facial motion rehabilitation. At this time, different methods are employed for evaluating facial muscles function. This research work investigates the 3D motion capture system, adopting the Technology, Sport and Health platform, located in the Innovation Centre of the University of Technology of Compiègne, in the Biomechanics and Bioengineering Laboratory (BMBI)
44
Dapogny, Arnaud. "A walk through randomness for face analysis in unconstrained environments." Thesis, Paris 6, 2016. http://www.theses.fr/2016PA066662/document.
Full textAbstract:
L'analyse automatique des expressions faciales est une étape clef pour le développement d'interfaces intelligentes ou l'analyse de comportements. Toutefois, celle-ci est rendue difficile par un grand nombre de facteurs, pouvant être d'ordre morphologiques, liés à l'orientation du visage ou à la présence d'occultations. Nous proposons des adaptations des Random Forest permettant d' adresser ces problématiques:- Le développement des Pairwise Conditional Random Forest, consistant en l'apprentissage de modèles à partir de paires d'images expressives. Les arbres sont de plus conditionnés par rapport à l'expression de la première image afin de réduire la variabilité des transitions. De plus, il est possible de conditionner les arbres en rapport avec une estimation de la pose du visage afin de permettre la reconnaissance quel que soit le point de vue considéré.- L'utilisation de réseaux de neurones auto-associatifs pour modéliser localement l'apparence du visage. Ces réseaux fournissent une mesure de confiance qui peut être utilisée dans le but de pondérer des Random Forests définies sur des sous-espaces locaux du visage. Ce faisant, il est possible de fournir une prédiction d'expression robuste aux occultations partielles du visage.- Des améliorations du récemment proposé algorithme des Neural Decision Forests, lesquelles consistent en une procédure d'apprentissage simplifiée, ainsi qu'en une évaluation "greedy" permettant une évaluation plus rapide, avec des applications liées à l'apprentissage en ligne de représentations profondes pour la reconnaissance des expressions, ainsi que l'alignement de points caractéristiquesAutomatic face analysis is a key to the development of intelligent human-computer interaction systems and behavior understanding. However, there exist a number of factors that makes face analysis a difficult problem. This include morphological differences between different persons, head pose variations as well as the possibility of partial occlusions. In this PhD, we propose a number of adaptations of the so-called Random Forest algorithm to specifically adress those problems. Mainly, those improvements consist in:– The development of a Pairwise Conditional Random Forest framework, that consists in training Random Forests upon pairs of expressive images. Pairwise trees are conditionned on the expression label of the first frame of a pair to reduce the ongoing expression transition variability. Additionnally, trees can be conditionned upon a head pose estimate to peform facial expression recognition from an arbitrary viewpoint.– The design of a hierarchical autoencoder network to model the local face texture patterns. The reconstruction error of this network provides a confidence measurement that can be used to weight Randomized decision trees trained on spatially-defined local subspace of the face. Thus, we can provide an expression prediction that is robust to partial occlusions.– Improvements over the very recent Neural Decision Forests framework, that include both a simplified training procedure as well as a new greedy evaluation procedure, that allows to dramatically improve the evaluation runtime, with applications for online learning and, deep learning convolutional neural network-based features for facial expression recognition as well as feature point alignement
45
MIRCOLI, ALEX. "Lexicon- and Learning-based Techniques for Emotion Recognition in Social Contents." Doctoral thesis, Università Politecnica delle Marche, 2019. http://hdl.handle.net/11566/263357.
Full textAbstract:
In tempi recenti, la diffusione massiva dei social network ha reso disponibili grandi quantità di contenuti generati dagli utenti, i quali spesso contengono informazioni autentiche in merito alle emozioni ed ai pensieri delle persone. L'analisi di tali contenuti attraverso tecniche di emotion recognition offre informazioni preziose in merito alla percezione di prodotti, servizi ed eventi, permettendo di estendere i tradizionali processi di Business Intelligence. A tal fine, nella presente tesi sono proposte tecniche innovative, basate sia sull'uso di risorse lessicali (lexicon-based) che di algoritmi di machine learning (learning-based), per l'emotion recognition, in particolare con applicazioni a contenuti social. Per quanto riguarda gli approcci lexicon-based, vengono estese le tecniche classiche introducendo due algoritmi, rispettivamente per la disambiguazione delle parole polisemiche e l'analisi delle frasi contenenti negazioni. Il primo algoritmo individua la variante semantica di una parola polisemica più adatta al contesto cercando il percorso più breve, all'interno di una risorsa lessicale, fra la parola polisemica e le parole vicine. Il secondo, invece, individua lo scope della negazione mediante analisi dell'albero sintattico.
La tesi presenta inoltre la progettazione e l'implementazione di una piattaforma basata su approcci lexicon-based per l'analisi delle opinioni espresse dagli utenti in vari social network.
Per quanto concerne gli approcci learning-based, è stata definita una metodologia per la creazione autoamtica di corpora annotati attraverso l'analisi delle espressioni facciali in video sottotitolati. La metodologia propone l'utilizzo di numerose tecniche di video preprocessing, per il filtraggio dei frame non rilevanti, e di un classificatore di espressioni facciali, implementabile mediante due approcci differenti.
Le tecniche proposte sono state valutate sperimentalmente attraverso numerosi dataset e i risultati sono promettenti.In recent years, the massive diffusion of social networks has made available large amounts of user-generated content, which often contains authentic information about people's emotions and thoughts. The analysis of such content through emotion recognition provides valuable insights into people's feeling about products, services and events, and allows to extend traditional processes of Business Intelligence. To this purpose, in the present work we propose novel techniques for lexicon- and learning-based emotion recognition, in particular for the analysis of social content. For what concerns lexicon-based approaches, the present work extends traditional techniques by introducing two algorithms for the disambiguation of polysemous words and the correct analysis of negated sentences. The former algorithm detects the most suitable semantic variant of a polysemous word with respect of its context, by searching for the shortest path in a lexical resource from the polysemous word to its nearby words. The latter detects the right scope of negation through the analysis of parse trees. Moreover, the paper describes the design and implementation of an application of the lexicon-based approach, that is a full-fledged platform for information discovery from multiple social networks, which allows for the analysis of users' opinions and characteristics and is based on Exploratory Data Analysis. For what concerns learning-based approaches, a methodology has been defined for the automatic creation of annotated corpora through the analysis of facial expressions in subtitled videos. The methodology is composed of several video preprocessing techniques, with the purpose of filtering out irrelevant frames, and a facial expression classifier, which can be implemented using two different approaches. The proposed techniques have been experimentally evaluated using several real-world datasets and the results are promising.
46
Rivera, Samuel. "Computational Methods for the Study of Face Perception." The Ohio State University, 2012. http://rave.ohiolink.edu/etdc/view?acc_num=osu1354650651.
Full text
47
Cheng, Xin. "Nonrigid face alignment for unknown subject in video." Thesis, Queensland University of Technology, 2013. https://eprints.qut.edu.au/65338/1/Xin_Cheng_Thesis.pdf.
Full textAbstract:
Non-rigid face alignment is a very important task in a large range of applications but the existing tracking based non-rigid face alignment methods are either inaccurate or requiring person-specific model. This dissertation has developed simultaneous alignment algorithms that overcome these constraints and provide alignment with high accuracy, efficiency, robustness to varying image condition, and requirement of only generic model.
48
Mascaró, Oliver Miquel. "Expresión de emociones de alegría para personajes virtuales mediante la risa y la sonrisa." Doctoral thesis, Universitat de les Illes Balears, 2014. http://hdl.handle.net/10803/145970.
Full textAbstract:
La animación facial es uno de los tópicos todavía no resueltos tanto en el campo de la
interacción hombre máquina como en el de la informática gráfica. Las expresiones de
alegría asociadas a risa y sonrisa son por su significado e importancia, parte
fundamental de estos campos. En esta tesis se hace una aproximación a la
representación de los diferentes tipos de risa en animación facial a la vez que se
presenta un nuevo método capaz de reproducir todos estos tipos. El método se valida
mediante la recreación de secuencias cinematográficas y mediante la utilización de
bases de datos de expresiones faciales genéricas y específicas de sonrisa.
Adicionalmente se crea una base de datos propia que recopila los diferentes tipos de
risas clasificados y generados en este trabajo. De acuerdo a esta base de datos propia
se generan las expresiones más representativas de cada una de las risas y sonrisas consideradas en el estudio.L'animació facial és un dels tòpics encara no resolts tant en el camp de la interacció home màquina com en el de la informàtica gràfica. Les expressions d'alegria associades a riure i somriure són pel seu significat i importància, part fonamental d'aquests camps. En aquesta tesi es fa una aproximació a la representació dels diferents tipus de riure en animació facial alhora que es presenta un nou mètode capaç de reproduir tots aquests tipus. El mètode es valida mitjançant la recreació de seqüències cinematogràfiques i mitjançant la utilització de bases de dades d'expressions facials genèriques i específiques de somriure. Addicionalment es crea una base de dades pròpia que recull els diferents tipus de rialles classificats i generats en aquest treball. D'acord a aquesta base de dades pròpia es generen les expressions més representatives de cadascuna de les rialles i somriures considerades en l'estudi.
Nowadays, facial animation is one of the most relevant research topics still unresolved both in the field of human machine interaction and in the computer graphics. Expressions of joy associated with laughter and smiling are a key part of these fields mainly due to its meaning and importance. In this thesis an approach to the representation of different types of laughter in facial animation is done while a new method to reproduce all these types is proposed. The method is validated by recreating movie sequences and using databases of generic and specific facial smile expressions. Additionally, a proprietary database that lists the different types of classified and generated laughs in this work is created. According to this proprietary database the most representative of every smile expression considered in the study is generated.
49
Bezerra, Giuliana Silva. "A framework for investigating the use of face features to identify spontaneous emotions." Universidade Federal do Rio Grande do Norte, 2014. http://repositorio.ufrn.br/handle/123456789/19595.
Full textAbstract:
Submitted by Automa??o e Estat?stica (sst@bczm.ufrn.br) on 2016-01-14T18:48:05Z
No. of bitstreams: 1
GiulianaSilvaBezerra_DISSERT.pdf: 12899912 bytes, checksum: 413f2be6aef4a909500e6834e7b0ae63 (MD5)Approved for entry into archive by Arlan Eloi Leite Silva (eloihistoriador@yahoo.com.br) on 2016-01-15T18:57:11Z (GMT) No. of bitstreams: 1 GiulianaSilvaBezerra_DISSERT.pdf: 12899912 bytes, checksum: 413f2be6aef4a909500e6834e7b0ae63 (MD5)
Made available in DSpace on 2016-01-15T18:57:11Z (GMT). No. of bitstreams: 1 GiulianaSilvaBezerra_DISSERT.pdf: 12899912 bytes, checksum: 413f2be6aef4a909500e6834e7b0ae63 (MD5) Previous issue date: 2014-12-12
Emotion-based analysis has raised a lot of interest, particularly in areas such as forensics, medicine, music, psychology, and human-machine interface. Following this trend, the use of facial analysis (either automatic or human-based) is the most common subject to be investigated once this type of data can easily be collected and is well accepted in the literature as a metric for inference of emotional states. Despite this popularity, due to several constraints found in real world scenarios (e.g. lightning, complex backgrounds, facial hair and so on), automatically obtaining affective information from face accurately is a very challenging accomplishment. This work presents a framework which aims to analyse emotional experiences through naturally generated facial expressions. Our main contribution is a new 4-dimensional model to describe emotional experiences in terms of appraisal, facial expressions, mood, and subjective experiences. In addition, we present an experiment using a new protocol proposed to obtain spontaneous emotional reactions. The results have suggested that the initial emotional state described by the participants of the experiment was different from that described after the exposure to the eliciting stimulus, thus showing that the used stimuli were capable of inducing the expected emotional states in most individuals. Moreover, our results pointed out that spontaneous facial reactions to emotions are very different from those in prototypic expressions due to the lack of expressiveness in the latter.
Emotion-based analysis has raised a lot of interest, particularly in areas such as forensics, medicine, music, psychology, and human-machine interface. Following this trend, the use of facial analysis (either automatic or human-based) is the most common subject to be investigated once this type of data can easily be collected and is well accepted in the literature as a metric for inference of emotional states. Despite this popularity, due to several constraints found in real world scenarios (e.g. lightning, complex backgrounds, facial hair and so on), automatically obtaining affective information from face accurately is a very challenging accomplishment. This work presents a framework which aims to analyse emotional experiences through naturally generated facial expressions. Our main contribution is a new 4-dimensional model to describe emotional experiences in terms of appraisal, facial expressions, mood, and subjective experiences. In addition, we present an experiment using a new protocol proposed to obtain spontaneous emotional reactions. The results have suggested that the initial emotional state described by the participants of the experiment was different from that described after the exposure to the eliciting stimulus, thus showing that the used stimuli were capable of inducing the expected emotional states in most individuals. Moreover, our results pointed out that spontaneous facial reactions to emotions are very different from those in prototypic expressions due to the lack of expressiveness in the latter.
50
Lemaire, Pierre. "Contributions à l'analyse de visages en 3D : approche régions, approche holistique et étude de dégradations." Phd thesis, Ecole Centrale de Lyon, 2013. http://tel.archives-ouvertes.fr/tel-01002114.
Full textAbstract:
Historiquement et socialement, le visage est chez l'humain une modalité de prédilection pour déterminer l'identité et l'état émotionnel d'une personne. Il est naturellement exploité en vision par ordinateur pour les problèmes de reconnaissance de personnes et d'émotions. Les algorithmes d'analyse faciale automatique doivent relever de nombreux défis : ils doivent être robustes aux conditions d'acquisition ainsi qu'aux expressions du visage, à l'identité, au vieillissement ou aux occultations selon le scénario. La modalité 3D a ainsi été récemment investiguée. Elle a l'avantage de permettre aux algorithmes d'être, en principe, robustes aux conditions d'éclairage ainsi qu'à la pose. Cette thèse est consacrée à l'analyse de visages en 3D, et plus précisément la reconnaissance faciale ainsi que la reconnaissance d'expressions faciales en 3D sans texture. Nous avons dans un premier temps axé notre travail sur l'apport que pouvait constituer une approche régions aux problèmes d'analyse faciale en 3D. L'idée générale est que le visage, pour réaliser les expressions faciales, est déformé localement par l'activation de muscles ou de groupes musculaires. Il est alors concevable de décomposer le visage en régions mimiques et statiques, et d'en tirer ainsi profit en analyse faciale. Nous avons proposé une paramétrisation spécifique, basée sur les distances géodésiques, pour rendre la localisation des régions mimiques et statiques le plus robustes possible aux expressions. Nous avons également proposé une approche régions pour la reconnaissance d'expressions du visage, qui permet de compenser les erreurs liées à la localisation automatique de points d'intérêt. Les deux approches proposées dans ce chapitre ont été évaluées sur des bases standards de l'état de l'art. Nous avons également souhaité aborder le problème de l'analyse faciale en 3D sous un autre angle, en adoptant un système de cartes de représentation de la surface 3D. Nous avons ainsi proposé de projeter sur le plan 2D des informations liées à la topologie de la surface 3D, à l'aide d'un descripteur géométrique inspiré d'une mesure de courbure moyenne. Les problèmes de reconnaissance faciale et de reconnaissance d'expressions 3D sont alors ramenés à ceux de l'analyse faciale en 2D. Nous avons par exemple utilisé SIFT pour l'extraction puis l'appariement de points d'intérêt en reconnaissance faciale. En reconnaissance d'expressions, nous avons utilisé une méthode de description des visages basée sur les histogrammes de gradients orientés, puis classé les expressions à l'aide de SVM multi-classes. Dans les deux cas, une méthode de fusion simple permet l'agrégation des résultats obtenus à différentes échelles. Ces deux propositions ont été évaluées sur la base BU-3DFE, montrant de bonnes performances tout en étant complètement automatiques. Enfin, nous nous sommes intéressés à l'impact des dégradations des modèles 3D sur les performances des algorithmes d'analyse faciale. Ces dégradations peuvent avoir plusieurs origines, de la capture physique du visage humain au traitement des données en vue de leur interprétation par l'algorithme. Après une étude des origines et une théorisation des types de dégradations potentielles, nous avons défini une méthodologie permettant de chiffrer leur impact sur des algorithmes d'analyse faciale en 3D. Le principe est d'exploiter une base de données considérée sans défauts, puis de lui appliquer des dégradations canoniques et quantifiables. Les algorithmes d'analyse sont alors testés en comparaison sur les bases dégradées et originales. Nous avons ainsi comparé le comportement de 4 algorithmes de reconnaissance faciale en 3D, ainsi que leur fusion, en présence de dégradations, validant par la diversité des résultats obtenus la pertinence de ce type d'évaluation.