Dissertations / Theses on the topic 'Face recognition. eng'

Orientador: Aparecido Nilceu Marana
Banca: Roberto Marcondes Cesar Junior
Banca: Ivan Rizzo Guilherme
Resumo: O reconhecimento facial é uma das principais formas de identificação humana. Apesar das pesquisas em reconhecimento facial automático terem crescido substancialmente ao longo dos últimos 35 anos, identificar pessoas a partir da face continua sendo um desafio para as áreas de Visão Computacional e Reconhecimento de Padrões. Em função dos cenários variarem desde a identificação a partir de fotografias até o reconhecimento baseado em vídeos sem nenhum tipo de controle ao serem gravados, os maiores desafios estão relacionados à independência contra diferentes tipos de iluminação, pose e expressão. O objetivo desta dissertação é propor técnicas que possam contribuir para a melhoria dos sistemas de reconhecimento facial. A primeira técnica endereça o problema da iluminação através da fusão dos espectros visível e infravermelho da face. Através desta abordagem, as taxas de reconhecimento foram melhoradas em 2.07% enquanto a taxa de erro igual (EER) foi reduzida em 45.47%. A segunda técnica trata do caso da extração e classificação de características faciais. Ela propõe um novo modelo para reconhecimento facial através do uso de características extraídas por Histogramas Census e de uma técnica de reconhecimento de padrões baseada em Máquinas de Vetores de Suporte (SVMs). Este outro grupo de experimentos nos possibilitou aumentar a precisão do reconhecimento no teste FERET fa/fb em 0.5%. Além destes resultados, algumas contribuições adicionais deste trabalho que merecem ser destacadas são a análise da dependência estatística entre classificadores de espectros diferentes e considerações sobre o comportamento de uma única C-SVC SVM para identificação de pessoas de forma eficaz.
Abstract: Face recognition is one of the primary ways of human identification. Although researches on automated face recognition have broadly increased along the last 35 years, it remains a challenging task in the fields of Computer Vision and Pattern Recognition. As the scenarios varies from static and constrained photographs to uncontrolled video images, the challenging issues on automatic face recognition are usually related with variations in illumination, pose and expressions. The goal of this master thesis is to propose techniques for the improvement of face recognition systems. The first technique addresses the problem of illumination by fusing the visible and the infrared spectra of the face. With this approach the recognition rates were improved in 2.07% while the Equal Error Rate (EER) were reduced in 45.47%. The second technique addresses the issue of face features extraction and classification. It proposes a new framework for face recognition by using features extracted by Census Histograms and a pattern recognition technique based on Support Vector Machines (SVMs). This other group of experiments enabled us to increase the recognition accuracy in the FERET fa/fb test in 0.5%. Beyond these results, additional contributions of this work that deserve to be highlighted are the statistical dependency analysis between face recognition systems based on different spectra and a better comprehension about the behavior of a single C-SVC SVM to reliably predict faces identities.
Mestre

Orientador: Aparecido Nilceu Elias
Banca: Agma Juci Machado Traina
Banca: Wilson Massashiro Yonezawa
Resumo: Nos últimos anos tem sido observado um crescimento exponencial na oferta de cursos a distância realizados pela Internet, decorrente de suas vantagens e características (menores custos de distribuição e atualização de conteúdo, gerenciamento de grandes turmas, aprendizado assíncrono e geograficamente independente, etc.), bem como de sua regulamentação e apoio governamental. Entretanto, a falta de mecanismos eficazes para assegurar a autenticação dos alunos neste tipo de ambiente é apontada como uma séria deficiência, tanto no acesso ao sistema quanto durante a participação do usuário nas atividades do curso. Atualmente, a autenticação baseada em senhas continua predominante. Porém, estudos têm sido conduzidos sobre possíveis aplicações da Biometria para autenticação em ambientes Web. Com a popularização e conseqüente barateamento de hardware habilitado para coleta biométrica (como webcams, microfone e leitores de impressão digital embutidos), a Biometria passa a ser considerada uma forma segura e viável de autenticação remota de indivíduos em aplicações Web. Baseado nisso, este trabalho propõe uma arquitetura distribuída para um ambiente de e-Learning, explorando as propriedades de um sistema Web para a autenticação biométrica tanto no acesso ao sistema quanto de forma contínua, durante a realização do curso. Para análise desta arquitetura, é avaliada a performance de técnicas de reconhecimento de faces a partir de vídeo capturadas on-line por uma webcam em um ambiente de Internet, simulando a interação natural de um indivíduo em um sistema de e- Learning. Para este fim, foi criada uma base de dados de vídeos própria, contando com 43 indivíduos navegando e interagindo com páginas Web. Os resultados obtidos mostram que os métodos analisados, consolidados na literatura, podem ser aplicados com sucesso nesse tipo de aplicação... (Resumo completo, clicar acesso eletrônico abaixo)
Abstract: In the last years it has been observed an exponential growth in the offering of Internet-enabled distance courses, due to its advantages and features (decreased distribution and content updates costs, management of large groups of students, asynchronous and geographically independent learning) as well as its regulation and governmental support. However, the lack of effective mechanisms that assure user authentication in this sort of environment has been pointed out as a serious deficiency, both in the system logon and during user attendance in the course assignments. Currently, password based authentication still prevails. Nevertheless, studies have been carried out about possible biometric applications for Web authentication. With the popularization and resultant decreasing costs of biometric enabled devices, such as webcams, microphones and embedded fingerprint sensors, Biometrics is reconsidered as a secure and viable form of remote authentication of individuals for Web applications. Based on that, this work presents a distributed architecture for an e-Learning environment, by exploring the properties of a Web system for biometric authentication both in the system logon and in continuous monitoring, during the course attendance. For the analysis of this architecture, the performance of techniques for face recognition from video, captured on-line by a webcam in an Internet environment, is evaluated, simulating the natural interaction of an individual in an e-Learning system. For that, a private database was created, with 43 individuals browsing and interacting with Web pages. The results show that the methods analyzed, though consolidated in the literature, can be successfully applied in this kind of application, with recognition rates up to 97% in ideal conditions, with low execution times and with short amount of information transmitted between client and server, with templates sizes of about 30KB.
Mestre

In a criminal investigation, witnesses may get asked by the police to provide a perpetrator’s description or to generate a composite image of the perpetrator’s face. Due to their elevated vulnerability to victimisation people with a learning disability (LD) may be more likely than other members of the wider community to find themselves in such situations. Research regarding face recognition and description abilities of this group has been to some extent neglected in the eyewitness research literature. Consequently, guidance for practitioners on how to effectively generate facial composite images with LD witnesses is limited. The current research addresses this issue, by investigating basic and applied face recognition and description abilities in individuals with mild learning disabilities (mLD) during a series of experimental studies. Moreover, potential facilitating measures are introduced and assessed. Five studies were conducted during the course of this thesis. In the first study a survey was designed to collect information on currently used composite systems by UK law enforcement agencies and how operators perceive and treat witnesses with LD. The survey findings confirmed the initial assumption that individuals with LD may indeed find themselves in the situation of having to describe a perpetrator’s face to an investigative officer. Furthermore, the results emphasised the lack of guidance available to operators on how to best meet the special needs of this particular witness population. Study 2 investigated basic face recognition and description abilities in people with mLD and revealed that overall they performed at a lower level than the non-LD controls. Despite this finding, mLD individuals as a group performed above chance levels and they displayed variability in performance depending on the introduced measures. iv Studies 3 and 5 investigated these abilities in a more applied setting, namely during the construction of facial composites with contemporary facial composite systems. Study 3 revealed that composites generated with the E-FIT system, a featural system, were considerably poorer than those created by their non-LD counterparts. Studies 4 and 5 attempted to improve mLD individuals’ performance by applying visual prompts and by using a more holistic facial composite system, i.e. EvoFIT. There was little evidence of the former being advantageous for witnesses with mLD, however, EvoFIT significantly enhanced composite construction abilities in the mLD participants. Finally, the practical and theoretical implications of the main findings are discussed.

Traditional authentication methods like Personal Identification Number (PIN) are getting obsolete and insecure for electronic-payments while mobile-payments are becoming more and more popular. Biometrics such as fingerprint and face recognition authentication methods seem to be a solution to this security issue as they are becoming a regular and integrated part of an average smartphone end-users purchase. However, for mobile-payments to be authenticated by biometrics, end-users acceptability of both technologies must be high. In this research, fingerprint and face recognition authentication methods are being tested with end-users and their current acceptability level is being determined based on interviews which are conducted upon finishing each testing scenario. The interview is using 39 questions which are determining previous usage of the technologies, their likeability, positives, negatives, and feelings about various features biometrics provide such as ease-of-use, stress-free method of payment, security, and many others. Additionally, one more authentication method is tested, namely two factor authentication consisting of one biometric method (fingerprint) and one traditional method (PIN) of authentication. The main goal for testing this method is to find out whether implementing (as currently it is not available) such technology into mobile-payments would be beneficial and how it scored in user-acceptance next to fingerprint and face recognition authentication methods. Once the user-acceptance level is determined the main reasons for it are presented. Last but not least, suggestions for improvements in this domain are presented so that biometrics are even more accepted by end-users who are performing mobile-payments on their smartphones.

Dual-process models of recognition memory propose that recognition memory can be supported by either a general sense of familiarity or the recollection of the encoding context. One source of evidence supporting dual-process models comes from event-related potential (ERP) studies of recognition memory, which have identified distinct patterns of neural activity associated with familiarity and recollection (the mid frontal and left parietal old/new effects, respectively). In this thesis, dual-process accounts of recognition memory were investigated in a series of ERP studies using three categories of stimulus: previously unknown faces, famous faces, and names. For previously unknown faces, familiarity was associated with activity over posterior scalp electrodes while recollection was associated with topographically dissociable activity over anterior electrodes. These dissociable patterns of activity support dual-process models. However, the typical pattern of old/new effects was only observed for stimuli associated with pre-existing representations (i.e., names and famous faces), suggesting that the presence/absence of pre-existing representations may determine the particular retrieval processes that support recognition memory. Furthermore, recollection was associated with two different patterns of activity (anterior and left parietal effects), suggesting that recollection is not a homogenous process. Dual-process theories may represent an important starting point for investigating recognition memory, but neither familiarity nor recollection appear to be functionally homogenous processes when theorizing is constrained by the analysis of scalp recorded electrophysiological activity.

Biometric authentication systems provide alternative solutions to traditional methods that are based on knowledge (e.g. password) or physical tokens (e.g., smart card). Many studies now focus on getting high accuracy rates for biometric verification. However,with advances in technology, biometric data (e.g. fingerprint, face, iris) can be captured/sniffed, duplicated, modified, and then resubmitted in the same or in other applications that utilize the same biometric features. Watermarking techniques can be used effectively to protect the genuine ownership of biometric data, either to accept or reject. This thesis presents a proposal for a suitable and viable combination of a face recognition algorithm and a watermarking technique, namely a Principal Component Analysis (PCA) and Discrete Cosine Transform (DCT) combination, that will ensure the authenticity of the data being transmitted in the face recognition system, which will then increase its level of security. The emphasis is on replay attack, which is recognizing and rejecting captured biometric data resubmitted into the system. The research begins with an analysis of biometric systems, with an emphasis on face recognition systems, and in particular with reference to the recorded threats on such systems. Biometric watermarking algorithms proposed by previous researchers within the face recognition environment are then studied, noting their proposed solutions to the said threats. This would then give a good idea towards a watermarking scheme to be proposed to enhance the security of face recognition systems, especially in terms of the authenticity of the data being transmitted. This proposed watermarking face recognition scheme is the main objective, which will be implemented in a PCA—DCT combination, followed by a check on all the 8 vulnerable positions where data may be captured and/or resubmitted. All the results produced are positive, apart from a few situations that will have to be left for future work. Non degradation of the individual PCA and DCT systems due to the combination is also checked and experimented on, again with positive results. Finally, the robustness of the watermarking scheme is experimented on to evaluate its resilience against attacks. The contributions from this research constitute a meaningful solution step to security problems associated with biometric techniques. The outcome of the research should also stimulate further research by opening up more research gaps in the area of combining biometric and watermarking techniques.

Orientador: Aparecido Nilceu Marana
Banca: Hélio Pedrini
Banca: Aledir Silveira Pereira
Resumo: As técnicas para identificação biométrica têm evoluído cada vez mais devido à necessidade que os seres humanos têm de identificar as pessoas em tempo real e de forma precisa para permitir o acesso a determinados recursos, como por exemplo, as aplicações e serviços WEB. O reconhecimento facial é uma técnica biométrica que apresenta várias vantagens em relação às demais, tais como: uso de equipamentos simples e baratos para a obtenção das amostras e a possibilidade de se realizar o reconhecimento em sigilo e à distância. O reconhecimento de faces a partir de vídeo é uma tendência recente na área de Biometria. Esta dissertação tem por objetivo principal comparar diferentes técnicas de reconhecimento facial a partir de vídeo para determinar as que apresentam um melhor compromisso entre tempo de processamento e precisão. Outro objetivo é a incorporação dessas melhores técnicas no sistema de autenticação biométrica em ambientes de E-Learning, proposto em um trabalho anterior. Foi comparado o classificador vizinho mais próximo usando as medidas de distância Euclidiana e Mahalanobis com os seguintes classificadores: Redes Neurais MLP e SOM, K Vizinhos mais Próximos, Classificador Bayesiano, Máquinas de Vetores de Suporte (SVM) e Floresta de Caminhos Ótimos (OPF). Também foi avaliada a técnica de Modelos Ocultos de Markov (HMM). Nos experimentos realizados com a base Recogna Video Database, criada especialmente para uso neste trabalho, e Honda/UCSD Video Database, os classificadores apresentaram os melhores resultados em termos de precisão, com destaque para o classificador SVM da biblioteca SVM Torch. A técnica HMM, que incorpora informações temporais, apresentou resultados melhores do que as funções de distância, em termos de precisão, mas inferiores aos classificadores
Abstract: The biometric identification techniques have evolved increasingly due to the need that humans have to identify people in real time to allow access to certain resources, such as applications and Web services. Facial recognition is a biometric technique that has several advantages over others. Some of these advantages are the use of simple and cheap equipment to obtain the samples and the ability to perform the recognition in covert mode. The face recognition from video is a recent approach in the area of Biometrics. The work in this dissertation aims at comparing different techniques for face recognition from video in order to find the best rates on processing time and accuracy. Another goal is the incorporation of these techniques in the biometric authentication system for E-Learning environments, proposed in an earlier work. We have compared the nearest neighbor classifier using the Euclidean and Mahalanobis distance measures with some other classifiers, such as neural networks (MLP and SOM), k-nearest neighbor, Bayesian classifier, Support Vector Machines (SVM), and Optimum Path Forest (OPF). We have also evaluated the Hidden Markov Model (HMM) approach, as a way of using the temporal information. In the experiments with Recogna Video Database, created especially for this study, and Honda/UCSD Video Database, the classifiers obtained the best accuracy, especially the SVM classifier from the SVM Torch library. HMM, which takes into account temporal information, presented better performance than the distance metrics, but worse than the classifiers
Mestre

Research studies into the reliability of forensic facial reconstruction have focused mainly on the accuracy of facial reconstruction from European or African ancestry skulls. Additionally the recognition of the human face by different racial groups and its effect on practitioners in facial reconstruction has not been considered in those studies. As a result, there is a paucity of research addressing the systemic approach to the accuracy of reconstructed faces produced by practitioners of a ‘different race’ (from the origin of the skull) or the cross-race recognition of the reconstructed faces. The aims of this research are; to investigate whether the ‘cross-race effect’ has any influence on the accuracy of facial reconstruction using both experienced and inexperienced practitioners (‘cross-race effect on reconstructing faces’); to evaluate how much the recognition rate of the facial reconstructions is affected by the cross-race effect (‘cross-race effect on recognizing reconstructed faces’). Fifteen facial reconstructions from three racial type skulls were produced by experienced or trained practitioners who had different racial backgrounds in order to explore the aims. The results demonstrated that the reconstructions from the same race skulls to the practitioners were recognised more accurately by the assessors from three racial groups, and that training and experience in recognising and reconstructing the face from other race can reduce the cross-race effect.

The accurate identification of children from facial photographs could provide a great attribute in the fight against child sexual exploitation, and may also aid in the detection of missing juveniles where comparative material is available. The European Commission is actively pursuing a global alliance for the identification of the victims of child sexual abuse; a task which is considered to be of the utmost importance. Images of child sexual abuse are shared, copied, and distributed online and their origin can be difficult to trace. Current investigations attempting to identify the children within such images appear to focus on the determination of places or geographical regions depicted in these images, from which victims can subsequently be tracked down and identified. Cutting edge technology is also used to detect duplicate images in order to decrease the workload of human operators and dedicate more time to the identification of new victims. Present investigations do not appear to focus on facial information for victim identification. Methods of facial identification already exist for adult individuals, consisting of both automated facial recognition algorithms and manual facial comparison techniques carried out by human operators. Human operator image comparison is presently the only method considered accurate enough to verify a face identity. It is only recently that researchers involved in automated facial recognition have begun to concern themselves with identification spanning childhood. Methods focus on age simulation to match query images with the age of the target database, rather than discrimination of individual faces over age progression. As far as can be determined, this is the first attempt to assess the manual comparison of juvenile faces. This study aimed to create a database of children’s faces from which identification accuracy could be tested using both automated facial recognition and manual facial comparison methods, which already exist for the identification of adults. A state-of-the-art facial recognition algorithm was employed and manual facial comparison was based on current recommendations by the Facial Identification Scientific Working Group (FISWG). It was not known if methods based on adult faces could be successfully extrapolated to juvenile faces, particularly as facial identification is highly susceptible to errors when there is an age difference between images of an individual. In children, the face changes much more rapidly than adults over ageing, due to the rapid growth and development of the juvenile face. The results of this study are in agreement with comparisons of automated and human performance in the identification of adult faces. Overall the automated facial recognition algorithm superseded human ability for identification of juvenile faces, however human performance was higher for the most difficult face pairs. The average accuracy for human image comparison was 61%. There was no significant difference in juvenile identification between individuals with prior experience of adult facial comparison and those with no prior experience. For automated facial recognition a correct identification rate of 71% was achieved at a false acceptance rate of 9%. Despite using methods created for adult facial identification, the results of this study are promising, particularly as they are based on a set of images acquired under uncontrolled conditions, which is known to increase error rates. With further augmentation of the database and investigation into child-specific identification techniques, the ability to accurately identify children from facial images is certainly a future possibility.

In this thesis, we present three main contributions to laser-based detection and tracking of dynamic objects, from both a model-based point of view and a model-free point of view, with an emphasis on applications to autonomous driving. A segmentation-based detector is first proposed to provide an end-to-end detection of the classes car, pedestrian and bicyclist in 3D laser data amongst significant background clutter. We postulate that, for the particular classes considered, solving a binary classification task outperforms approaches that tackle the multi-class problem directly. This is confirmed using custom and third-party datasets gathered of urban street scenes. The sliding window approach to object detection, while ubiquitous in the Computer Vision community, is largely neglected in laser-based object detectors, possibly due to its perceived computational inefficiency. We give a second thought to this opinion in this thesis, and demonstrate that, by fully exploiting the sparsity of the problem, exhaustive window searching in 3D can be made efficient. We prove the mathematical equivalence between sparse convolution and voting, and devise an efficient algorithm to compute exactly the detection scores at all window locations, processing a complete Velodyne scan containing 100K points in less than half a second. Its superior performance is demonstrated on the KITTI dataset, and compares commensurably with state of the art vision approaches. A new model-free approach to detection and tracking of moving objects with a 2D lidar is then proposed aiming at detecting dynamic objects of arbitrary shapes and classes. Objects are modelled by a set of rigidly attached sample points along their boundaries whose positions are initialised with and updated by raw laser measurements, allowing a flexible, nonparametric representation. Dealing with raw laser points poses a significant challenge to data association. We propose a hierarchical approach, and present a new variant of the well-known Joint Compatibility Branch and Bound algorithm to handle large numbers of measurements. The system is systematically calibrated on real world data containing 7.5K labelled object examples and validated on 6K test cases. Its performance is demonstrated over an existing industry standard targeted at the same problem domain as well as a classical approach to model-free tracking.

La voie visuelle ventrale, s’étendant des régions occipitales aux régions temporales antérieures, est spécialisée dans la reconnaissance, par la modalité visuelle, des objets et personnes rencontrés au quotidien. De nombreuses études en imagerie par résonance magnétique fonctionnelle se sont intéressées aux bases cérébrales de la reconnaissance visuelle. Toutefois, la susceptibilité de cette technique aux artefacts magnétiques dans les régions du lobe temporal antérieur a conduit à sous-estimer le rôle de ces régions au sein de la voie ventrale. Le but de cette thèse est de mieux comprendre les mécanismes de reconnaissance visuelle au sein du cortex ventral occipito-temporal, et notamment de clarifier la contribution des structures temporales postérieures et antérieures dans la mise en œuvre des mécanismes de reconnaissance visuelle et de mise en lien avec la mémoire sémantique. Pour cela, nous nous appuyons sur une approche multimodale combinant neuropsychologie, stimulation visuelle périodique rapide (FPVS) et enregistrements en EEG de scalp et en EEG intracérébral (SEEG), chez des participants neurotypiques et des participants épileptiques. Nous rapportons cinq études empiriques dans lesquelles nous démontrons que (1) les patients avec une épilepsie temporale antérieure (i.e., le type d’épilepsie focale le plus fréquemment concerné par une procédure en SEEG) présentent des performances typiques en discrimination individuelle de visages, (2) la stimulation électrique du gyrus fusiforme antérieur droit peut entraîner un déficit transitoire spécifique à la reconnaissance des visages, même lorsqu’aucune dénomination n’est requise, (3) le processus de discrimination de visages familiers parmi des visages inconnus sollicite l’engagement d’un large réseau de structures ventrales bilatérales incluant les régions temporales antérieures et médiales, (4) certaines structures du lobe temporal antérieur ventral gauche sont impliquées dans l’intégration d’un visage familier et de son nom en une représentation unifiée, et (5) les régions temporales antérieures ventrales bilatérales sont engagées dans la mise en œuvre de représentations sémantiques associées à des mots écrits. Dans l’ensemble, nos travaux montrent que (1) le réseau de reconnaissance visuelle s’organise le long de la voie visuelle ventrale en suivant une hiérarchisation progressive selon l’axe postéro-antérieur, au sein duquel une transition graduelle s’effectue entre représentations majoritairement perceptives et représentations sémantiques de plus en plus abstraites, et (2) les régions impliquées dans la reconnaissance visuelle sont fortement latéralisées dans les régions postérieures ventrales, et deviennent bilatérales dans les régions temporales antérieures ventrales
The ventral visual stream extends from the occipital to the anterior temporal regions and is specialized in recognizing objects and people through vision. Numerous studies in functional magnetic resonance imaging have focused on the cerebral basis of visual recognition. However, this technique is susceptible to magnetic artefacts in anterior temporal regions and it has led to an underestimation of the role of these regions within the ventral visual stream. The aim of this thesis is to better understand the mechanisms of visual recognition within the ventral occipito-temporal cortex and, more specifically, to clarify the contribution of posterior and anterior temporal structures in the visual recognition of a stimulus and its association with semantic representations. For this purpose, we used a multimodal approach combining neuropsychology, fast periodic visual stimulation (FPVS), and scalp and intracerebral EEG (SEEG) recordings in neurotypical and epileptic participants. We report five empirical studies in which we demonstrate that (1) patients with anterior temporal epilepsy (i.e., the most frequent type of focal epilepsy that undergo intracerebral EEG recordings) show typical performance in individual face discrimination, (2) electrical stimulation of the right anterior fusiform gyrus can lead to a transient deficit that is specific to face recognition, even when no naming is required, (3) familiar face discrimination processes involve a large network of bilateral ventral structures including the anterior and medial temporal regions, (4) some structures of the left ventral anterior temporal lobe are involved in the integration of a familiar face and its name into a unified representation, and (5) the bilateral ventral anterior temporal regions are involved in representing semantic knowledge associated with written words. Overall, this work shows that (1) the visual recognition network follows a progressive hierarchization along the postero-anterior axis of the ventral visual stream, with a graded transition between perceptual representations and increasingly abstract semantic representations, and (2) the regions involved in visual recognition are strongly lateralized in the ventral posterior regions and become bilateral in the ventral anterior temporal regions

Das Konzept der interpersonellen Fähigkeiten bezieht sich auf Leistungsaufgaben der sozialen Kognition. Diese Aufgaben messen die Fähigkeiten Gesichter zu erkennen und sich diese zu merken sowie Emotionen zu erkennen und diese auszudrücken. Ziel dieser Dissertation war die Untersuchung des Einflusses von interpersonellen Fähigkeiten auf soziale Entscheidungen. Ein besonderer Fokus lag auf der Quantifizierung von individuellen Unterschieden in zugrundeliegenden neuronalen Mechanismen. Studie 1 erweiterte bestehende Evidenz zu Beziehungen zwischen psychometrischen Konstrukten der Gesichterkognition und Ereigniskorrelierten Potentialen, welche mit den verschiedenen Stadien der Gesichterverarbeitung (Enkodierung, Wahrnehmung, Gedächtnis) während einer Bekanntheitsentscheidung assoziiert sind. Unsere Ergebnisse bestätigen eine substantielle Beziehung zwischen der N170 Latenz und der Amplitude des frühen Wiederholungseffektes (ERE) mit drei Faktoren der Gesichterkognition. Je kürzer die N170 Latenz und je ausgeprägter die ERE Amplitude, umso genauer und schneller ist die Gesichterkognition. Studie 2 ergab, dass die Fähigkeit ängstliche Gesichter zu erkennen sowie die generelle spontane Expressivität während der sozialen Interaktion mit prosozialen Entscheidungen korreliert. Sensitivität für das Leid anderer sowie emotionale Expressivität scheinen reziproke Interaktionen mit Gleichgesinnten zu fördern. Studie 3 bestätigte das Modell der starken Reziprozität, da Prosozialität die negative Reziprozität im Ultimatum Spiel beeinflusste. Unter der Verwendung von Strukturgleichungsmodellen entdeckten wir, dass Menschen mit ausgeprägter Reziprozität eine größere Amplitude der relativen feedback-negativity auf das Gesicht von Spielpartnern zeigen. Insgesamt sprechen die Ergebnisse dafür, dass die etablierten individuellen Unterschiede in den Verhaltensmaßen der interpersonellen Fähigkeiten zum Teil auf individuelle Unterschiede in neuronalen Mechanismen zurückzuführen sind.
The concept of interpersonal abilities refers to performance measures of social cognition such as the abilities to perceive and remember faces and the abilities to recognize and express emotions. The aim of this dissertation was to examine the influence of interpersonal abilities on social decisions. A particular focus lay on the quantification of individual differences in brain-behavior relationships associated with processing interpersonally relevant stimuli. Study 1 added to existing evidence on brain-behavior relationships, specifically between psychometric constructs of face cognition and event-related potentials associated with different stages of face processing (encoding, perception, and memory) in a familiarity decision. Our findings confirm a substantial relationship between the N170 latency and the early-repetition effect (ERE) amplitude with three established face cognition ability factors. The shorter the N170 latency and the more pronounced the ERE amplitude, the better is the performance in face perception and memory and the faster is the speed of face cognition. Study 2 found that the ability to recognize fearful faces as well as the general spontaneous expressiveness during social interaction are linked to prosocial choices in several socio-economic games. Sensitivity to the distress of others and spontaneous expressiveness foster reciprocal interactions with prosocial others. Study 3 confirmed the model of strong reciprocity in that prosociality drives negative reciprocity in the ultimatum game. Using multilevel structural equation modeling in order to estimate brain-behavior relationships of fairness preferences, we found strong reciprocators to show more pronounced relative feedback-negativity amplitude in response to the faces of bargaining partners. Thus, the results of this dissertation suggest that established individual differences in behavioral measures of interpersonal ability are partly due to individual differences in brain mechanisms.

La présente thèse s'intéresse à l'utilisation de méta-données textuelles pour l'analyse d'image. Nous cherchons à utiliser ces informations additionelles comme supervision faible pour l'apprentissage de modèles de reconnaissance visuelle. Nous avons observé un récent et grandissant intérêt pour les méthodes capables d'exploiter ce type de données car celles-ci peuvent potentiellement supprimer le besoin d'annotations manuelles, qui sont coûteuses en temps et en ressources. Nous concentrons nos efforts sur deux types de données visuelles associées à des informations textuelles. Tout d'abord, nous utilisons des images de dépêches qui sont accompagnées de légendes descriptives pour s'attaquer à plusieurs problèmes liés à la reconnaissance de visages. Parmi ces problèmes, la vérification de visages est la tâche consistant à décider si deux images représentent la même personne, et le nommage de visages cherche à associer les visages d'une base de données à leur noms corrects. Ensuite, nous explorons des modèles pour prédire automatiquement les labels pertinents pour des images, un problème connu sous le nom d'annotation automatique d'image. Ces modèles peuvent aussi être utilisés pour effectuer des recherches d'images à partir de mots-clés. Nous étudions enfin un scénario d'apprentissage multimodal semi-supervisé pour la catégorisation d'image. Dans ce cadre de travail, les labels sont supposés présents pour les données d'apprentissage, qu'elles soient manuellement annotées ou non, et absentes des données de test. Nos travaux se basent sur l'observation que la plupart de ces problèmes peuvent être résolus si des mesures de similarité parfaitement adaptées sont utilisées. Nous proposons donc de nouvelles approches qui combinent apprentissage de distance, modèles par plus proches voisins et méthodes par graphes pour apprendre, à partir de données visuelles et textuelles, des similarités visuelles spécifiques à chaque problème. Dans le cas des visages, nos similarités se concentrent sur l'identité des individus tandis que, pour les images, elles concernent des concepts sémantiques plus généraux. Expérimentalement, nos approches obtiennent des performances à l'état de l'art sur plusieurs bases de données complexes. Pour les deux types de données considérés, nous montrons clairement que l'apprentissage bénéficie de l'information textuelle supplémentaire résultant en l'amélioration de la performance des systèmes de reconnaissance visuelle.

The performance of face recognition systems depends heavily on facial representation, which is naturally coupled with many types of face variations, such as views, illuminations, and expressions. Many existing works have been proposed to account for such variations. These face recognition systems either use handcrafted features or learn features discriminatively to improve recognition accuracy. In this thesis, we focus on learning effective and robust representations using deep neural networks for face recognition.
Firstly, we propose a novel deep neural network to learn identity-preserving representations for face recognition. The learned face representations are effective for face recognition and are also capable of reconstructing face images in their frontal views. Classical face recognition methods can be improved if applied on the reconstructed face images.
We further extend the first work by proposing a new deep learning framework that can recover the canonical view of face images which are taken in the wild environment. This approach directly learns the transformation between face images with a complex set of variations and their canonical views. As an application, this face recovery approach is used for face verification.
Moreover, we propose a deep model that can disentangle identities and viewpoints,and infer a full spectrum of multi-view images in the meanwhile, given a single 2D face image. The model is also capable of interpolating and predicting images under viewpoints that are unobserved in the training set. Experiments show that the representations learned by multi-view perception are more discriminative than those learned
by a single view.
Lastly, we propose a model compression method to enable the deployment of deep neural networks based face recognition systems, which usually contain large number ofparameters and require extensive computation resources, on mobile and embedded devices.
Zhu, Zhenyao.
Thesis Ph.D. Chinese University of Hong Kong 2016.
Includes bibliographical references (leaves ).
Abstracts also in Chinese.
Title from PDF title page (viewed on …).

Deep learning has been widely and successfully applied to many difficult tasks in computer vision, such as image parsing, object detection, and object recognition, where various deep learning architectures such as deep neural networks, convolutional deep neural networks, and deep belief networks have achieved impressive performance and significantly outperformed state-of-the-art methods. However, the potential of deep learning in face related problems has not be fully explored yet. In this thesis, we fully explore different deep learning methods and proposes new network architectures and learning algorithms on face related applications, such as face parsing, face attribute inference, and face recognition.
For face parsing, we propose a novel face parser, which recasts segmentation of face components as a cross-modality data transformation problem, i.e., transforming an image patch to a label map. Specifically, a face is represented hierarchically by parts, components, and pixel-wise labels. With this representation, this approach first detects faces at both the part- and component-levels, and then computes the pixel-wise label maps. The part-based and component-based detectors are generatively trained with the deep belief network (DBN), and are discriminatively tuned by logistic regression. The segmentators transform the detected face components to label maps, which are obtained by learning a highly nonlinear mapping with the deep autoencoder. The proposed hierarchical face parsing is not only robust to partial occlusions but also provide richer information for face analysis and face synthesis compared with face keypoint detection and face alignment.
For face attribute inference, the proposed approach captures the interdependencies of local regions for each attribute, as well as the high-order correlations between different attributes, which makes it more robust to occlusions and misdetection of face regions. First, we have modeled region interdependencies with a discriminative decision tree, where each node consists of a detector and a classifier trained on a local region. The detector allows us to locate the region, while the classifier determines the presence or absence of an attribute. Second, correlations of attributes and attribute predictors are modeled by organizing all of the decision trees into a large sum-product network (SPN), which is learned by the EM algorithm and yields the most probable explanation (MPE) of the facial attributes in terms of the region’s localization and classification. Experimental results on a large data set with 22,400 images show the effectiveness of the proposed approach.
For face recognition, this thesis addresses this challenge by proposing a new deep learning framework that can recover the canonical view of face images. It dramatically reduces the intra-person variances, while maintaining the inter-person discriminativeness. Unlike the existing face reconstruction methods that were either evaluated in controlled 2D environment or employed 3D information, our approach directly learns the transformation between face images with a complex set of variations and their canonical views. At the training stage, to avoid the costly process of labeling canonical-view images from the training set by hand, we have devised a new measurement and algorithm to automatically select or synthesize a canonical-view image for each identity. The recovered canonical-view face images are matched by using a facial component-based convolutional neural network. Our approach achieves the best performance on the LFW dataset under the unrestricted protocol. We also demonstrate that the performance of existing methods can be improved if they are applied to our recovered canonical-view face images.
近年來，深度學習算法被成功應用於解決各種困難的計算機視覺問題，例如圖像分割、物體識別和檢測等。深度學習算法，如深度神經網絡、深度卷積神經網絡、和深度置信度網絡在上述方面取得重要突破，並且算法性能超過了傳統計算機視覺算法。然而，人臉圖片，作為人的視覺認知最重要的環節之一，還沒有在深度學習框架下進行研究。本文以人臉圖片分析為背景，深入探討了適用的深度學習算法與不同的深度網絡結構。主要關注以下幾個應用，包括人臉分割、人臉屬性判斷、和人臉識別。
對於人臉分割問題，我們把傳統的計算機視覺分割問題變成一個高維空間數據轉換問題，即把人臉圖片轉換為分割圖。一張人臉圖片可以層次化的表示為像素塊、人臉關鍵點（五官）、和人臉區域。通過使用該人臉表示，我們的方法先檢測人臉的區域，其次檢測人臉關鍵點，最後根據人臉關鍵點位置把像素塊轉換為分割圖。本文提出的方法包括兩個步驟：關鍵點檢測和圖元轉換為分割圖。本文使用深度置信度網絡進行關鍵點檢測；使用深度編碼器進行像素點到分割圖的轉換。該方法對人臉遮擋也具有魯棒性。
對於人臉屬性判斷，本文提出的方法對兩種相關性進行建模，包括人臉關鍵區域相關性和人臉屬性之間的相關性。我們使用決策樹對人臉關鍵區域相關性進行建模。通過把尋找與決策樹一一對應的Sum-Product樹對屬性之間的相關性進行建模。通過對22400張人臉圖片進行實驗，驗證本文提出的方法的有效性與魯棒性。
對於人臉識別問題，本論文提出了一種新的人臉表示方法，稱爲人臉身份保持性特徵。該特徵能夠保持不同身份人臉之間的判別性，同時減少同一身份人臉間的變化。該特徵還可以恢復輸入人臉圖片的正臉。使用該正臉圖片進行人臉歸一化，可以使現有人臉識別算法的準確率都能得到提高。
Luo, Ping.
Thesis (Ph.D.)--Chinese University of Hong Kong, 2014.
Includes bibliographical references (leaves 83-95).
Abstracts also in Chinese.
Title from PDF title page (viewed on 27, October, 2016).
Detailed summary in vernacular field only.
Detailed summary in vernacular field only.
Detailed summary in vernacular field only.
Detailed summary in vernacular field only.

Trabalho de Projeto do Mestrado Integrado em Engenharia Biomédica apresentado à Faculdade de Ciências e Tecnologia
A perceção de rostos é um dos aspetos básicos para as interações sociais. Desde muito cedo, aprendemos a interagir com outras pessoas, expressando várias emoções e sentimentos que podem ser vistos pelas expressões nos rostos das pessoas. Estudar os correlatos neurais do reconhecimento de emoção é meio caminho andado para ajudar as pessoas com dificuldades neste importante domínio cognitivo e afetivo.Estudos neurofisiológicos anteriores relativos às respostas do cérebro às emoções conduzem a resultados controversos e muitas vezes não replicáveis. Teremos duas tarefas principais para este trabalho.A primeira tarefa é tentar replicar alguns estudos já feitos na área de reconhecimento de emoções e tentar encontrar algumas diferenças estatísticas entre três emoções (Triste, Neutra e Feliz) para o N170.O segundo objetivo desta tese é compreender e explicar os motivos pelos quais alguns estudos levam a resultados controversos e não replicáveis. Supondo que a tarefa seja realizada de maneira adequada, um dos motivos para isso é a utilização de métodos diferentes no pré-processamento dos dados.A nossa tarefa utilizou 10 sujeitos (4 mulheres, 6 homens), onde cada sujeito fez 4 execuções de EEG e cada corrida teve cerca de 6 minutos de registo. A tarefa consistia em mostrar uma face neutra ao sujeito e após um GAP mostrar a instrução. A instrução era um rosto com uma emoção olhando para um dos lados. Depois disso, eles deviam completar uma sacada ou anti-sacada.Depois de obter os dados, usamos diferentes métodos de pré-processamento: 1) interpolação; 2) Re-referência; 3) filtros, correção de linha de base e rejeição de épocas e 4) Análise de Componentes Independentes (ICA).Para o primeiro objetivo, poderíamos replicar alguns estudos anteriores e encontramos diferenças estatísticas entre as diferentes emoções para o N170.Para o segundo objetivo concluímos que todo método de pré-processamento tem influência significativa nos resultados, dando diferentes amplitudes e latências para ERP’s. Também recomendamos que, para um resultado mais confiável, usemos todos os métodos de pré-processamento mencionados neste documento.
The perception of faces is one of the basic aspects for social interactions. From very young age we learned to interact with others by expressing various emotions and feelings that can be seen by expressions in people’s faces. Studying the neural correlates of emotion recognition is halfway to help people with difficulties in this important cognitive and affective domain. Previous neurophysiological studies concerning brain responses to emotions lead controversial and often nonreplicable results. We will have two main tasks for this work.The first task is to try to replicate some studies already made in the area of emotion recognition and try to find some statistical differences between three emotions (Sad, Neutral and Happy) for the N170.The second objective of this thesis is to understand and explain the reasons why some studies lead to controversial results and nonreplicable results. Assuming that the task is done properly, one of the reasons for that is the use of different methods in the pre-processing data. Our task used 10 subjects (4 females, 6 males), where each subject done 4 EEG runs and each run had around 6 minutes of recording. The task consisted in showing a neutral face to the subject and after a GAP show the instruction. The instruction was a face with an emotion looking to on side. After that they should complete a saccade or an anti-saccade. After getting the data we used different methods of pre-processing: 1) interpolation; 2) Re-reference; 3) filters, baseline correction and epochs rejection and 4) Independent Component Analysis (ICA).For the first objective we could replicate some previous studies and we found statistical differences between the different emotions for the N170.For the second objective we concluded that every method of pre-processing has significant influence in the results, giving different amplitudes and latencies for ERP’s. We also recommend that for a more reliable result we should use every method of pre-processing that we refer on this paper.

The application of Brain Computer Interfaces techniques to vital crime witnesses could and probably will be a key feature in the justice system. Features from the electroencephalogram signals were extracted with information detailing their domain (time or frequency), and their spacial scalp and time placement. For both domains, two different classification pipelines were applied in order to select the most relevant features: one to rank and select the top features and another to recursively eliminate the least relevant feature. The Support Vector Machine (linear and non-linear) is the classification model included in the pipeline. Further observations on the selected features by the applied techniques were performed and discussed in relation to the available knowledge about face recognition. The present work provides an experimental study on the electroencephalogram signals acquired from an experiment in which an array of subjects were asked to identify both culprit and distractor being the culprit related to a previously shown crime scene video.
A aplicação de técnicas de Interfaces Cérebro-Computador a testemunhas vitais de um crime pode e provavelmente será uma funcionalidade chave no sistema de justiça. Características de sinais provenientes de eletroencefalograma foram extraídas com informações sobre o seu domínio (tempo ou frequência), e a sua localização espacial e temporal. Para ambos os domínios, dois modelos de classificação diferentes foram aplicados com vista a selecionar as características mais relevantes: um para classificar, ordenar e selecionar as características mais importantes e outro para eliminar recursivamente a característica menos relevante. O modelo utilizado para classificação foi o Support Vector Machine (linear e não linear). Outras observações sobre as características selecionadas pelas técnicas aplicadas foram realizadas e discutidas tendo em conta o conhecimento disponível sobre reconhecimento facial. O presente trabalho fornece um estudo experimental sobre os sinais de eletroencefalograma adquiridos numa experiência na qual foi pedido a um grupo de indivíduos para identificar tanto culpado como distrator, sendo que o culpado estava relacionado a um vídeo de cenário de crime mostrado anteriormente.
Mestrado em Engenharia de Computadores e Telemática

La reconnaissance visuelle est un processus temporel : d’abord, l’information visuelle est reçue sur notre rétine de manière continue à travers le temps; ensuite, le traitement de l’information visuelle par notre cerveau prend un certain temps à s’effectuer; finalement, notre perception est toujours fonction autant des expériences acquises dans le passé que de l’input sensoriel présent. Les interactions entre ces aspects temporels de la reconnaissance sont rarement abordées dans la littérature. Dans cette thèse, nous évaluons l’échantillonnage de l’information visuelle à travers le temps pendant une tâche de reconnaissance, comment il se traduit dans le cerveau et comment il est modulé par des attentes spécifiques. Plusieurs études indiquent que nos attentes modulent notre perception. Comment l’attente d’un objet spécifique influence nos représentations internes demeure cependant largement inconnu. Dans le premier article de cette thèse, nous utilisons une variante de la technique Bubbles pour retrouver avec précision le décours temporel de l’utilisation d’information visuelle pendant la reconnaissance d’objets, lorsque les observateurs s’attendent à voir un objet spécifique ou non. Nous observons que les attentes affectent la représentation de différents attributs différemment et qu’elles ont un effet distinct à différents moments pendant la réception d’information visuelle. Dans le deuxième article, nous utilisons une technique similaire en conjonction avec l’électroencéphalographie (EEG) afin de révéler pour la première fois le traitement, à travers le temps, de l’information reçue à un moment spécifique pendant une fixation oculaire. Nous démontrons que l’information visuelle n’est pas traitée de la même manière selon le moment auquel elle est reçue sur la rétine, que ces différences ne sont pas explicables par l’adaptation ou l’amorçage, qu’elles sont d’origine au moins partiellement descendante et qu’elles corrèlent avec le comportement. Finalement, dans le troisième article, nous approfondissons cette investigation en utilisant la magnétoencéphalographie (MEG) et en examinant l’activité dans différentes régions cérébrales. Nous démontrons que l’échantillonnage de l’information visuelle est hautement variable selon le moment d’arrivée de l’information sur la rétine dans de larges parties des lobes occipitaux et pariétaux. De plus, nous démontrons que cet échantillonnage est rythmique, oscillant à diverses fréquences entre 7 et 30 Hz, et que ces oscillations varient en fréquences selon l’attribut échantillonné.
Visual recognition is a temporal process: first, visual information is continuously received through time on our retina; second, the processing of visual information by our brain takes time; third, our perception is function of both the present sensory input and our past experiences. Interactions between these temporal aspects have rarely been discussed in the literature. In this thesis, we assess the sampling of visual information through time during recognition tasks, how it is translated in the brain, and how it is modulated by expectations of specific objects. Several studies report that expectations modulate perception. However, how the expectation of a specific object modulates our internal representations remains largely unknown. In the first article of this thesis, we use a variant of the Bubbles technique to uncover the precise time course of visual information use during object recognition when specific objects are expected or not. We show that expectations modulate the representations of different features differently, and that they have distinct effects at distinct moments throughout the reception of visual information. In the second article, we use a similar method in conjunction with electroencephalography (EEG) to reveal for the first time the processing, through time, of information received at a specific moment during an eye fixation. We show that visual information is not processed in the same way depending on the moment at which it is received on the retina, that these differences cannot be explained by simple adaptation or repetition priming, that they are of at least partly top- down origin, and that they correlate with behavior. Finally, in a third article, we push this investigation further by using magnetoencephalography (MEG) and examining brain activity in different brain regions. We show that the sampling of visual information is highly variable depending on the moment at which information arrives on the retina in large parts of the occipital and parietal lobes. Furthermore, we show that this sampling is rhythmic, oscillating at multiple frequencies between 7 and 30 Hz, and that these oscillations vary according to the sampled feature.