Dissertations / Theses on the topic 'Multi-modal interface'

This work introduces a new UIMS (User Interface Management System), which aims to solve numerous problems in the field of user-interface development arising from hard-coded use of user interface toolkits. The presented solution is a concrete system architecture based on the abstract ARCH model consisting of an interface abstraction-layer, a dialog definition language called GIML (Generalized Interface Markup Language) and pluggable interface rendering modules. These components form an interface toolkit called GITK (Generalized Interface ToolKit). With the aid of GITK (Generalized Interface ToolKit) one can build an application, without explicitly creating a concrete end-user interface. At runtime GITK can create these interfaces as needed from the abstract specification and run them. Thereby GITK is equipping one application with many interfaces, even kinds of interfaces that did not exist when the application was written. It should be noted that this work will concentrate on providing the base infrastructure for adaptive/adaptable system, and does not aim to deliver a complete solution. This work shows that the proposed solution is a fundamental concept needed to create interfaces for everyone, which can be used everywhere and at any time. This text further discusses the impact of such technology for users and on the various aspects of software systems and their development. The targeted main audience of this work are software developers or people with strong interest in software development.

Computers are developed continuously to satisfy the human demands, and typical tools used everywhere for ranging from daily life usage to all kinds of research. Virtual Reality (VR), a virtual environment simulated to present physical presence in the real word and imaginary worlds, has been widely applied to simulate the virtual environment. People’s feeling is limited to visual perception when only computers are applied for simulations, since computers are limited to display visualization of data, while human senses include sight, smell, hearing, taste, touch and so on. Other devices can be applied, such as haptics, a device for sense of touch, to enhance the human perception in virtual environment. A good way to apply VR applications is to place them in a virtual display system, a system with multiply tools displays a virtual environment with experiencing different human senses, to enhance the people’s feeling of being immersed in a virtual environment. Such virtual display systems include VR dome, recursive acronym CAVE, VR workbench, VR workstation and so on. Menus with lots of advantages in manipulating applications are common in conventional systems, operating systems or other systems in computers. Normally a system will not be usable without them. Although VR applications are more natural and intuitive, they are much less or not usable without menus. But very few studies have focused on user interfaces in VR. This situation motivates us working further in this area. We want to create two models on different purposes. One is inspired from menus in conventional system and the sense of touch. And the other one is designed based on the spatial presence of VR. The first model is a two-dimensional pie menu in pop-up style with spring force feedback. This model is in a pie shape with eight options on the root menu. And there is a pop-up style hierarchical menu belongs to each option on the root menu. When the haptics device is near an option on the root menu, the spring force will force the haptics device towards to the center of the option and that option will be selected, and then the sub menu with nine options will pop up. The pie shape together with the spring force effect is expected to both increase the speed of selection and decrease the error rate of selection. The other model is a semiautomatic three-dimensional cube menu. This cube menu is designed with a aim to provide a simple, elegant, efficient and accurate user interface approach. This model is designed with four faces, including the front, back, left and right faces of the cube. Each face represents a category and has nine widgets. Users can make selections in different categories. An efficient way to change between categories is to rotate the cube automatically. Thus, a navigable rotation animation system is built and is manipulating the cube rotate horizontally for ninety degrees each time, so one of the faces will always face users. These two models are built under H3DAPI, an open source haptics software development platform with UI toolkit, a user interface toolkit. After the implementation, we made a pilot study, which is a formative study, to evaluate the feasibility of both menus. This pilot study includes a list of tasks for each menu, a questionnaire regards to the menu performance for each subject and a discussion with each subject. Six students participated as test subjects. In the pie menu, most of the subjects feel the spring force guides them to the target option and they can control the haptics device comfortably under such force. In the cube menu, the navigation rotation system works well and the cube rotates accurately and efficiently. The results of the pilot study show the models work as we initially expected. The recorded task completion time for each menu shows that with the same amount of tasks and similar difficulties, subjects spent more time on the cube menu than on the pie menu. This may implicate that pie menu is a faster approach comparing to the cube menu. We further consider that both the pie shape and force feedback may help reducing the selection time. The result for the option selection error rate test on the cube menu may implicates that option selection without any force feedback may also achieve a considerable good effect. Through the answers from the questionnaire for each subject, both menus are comfortable to use and in good control.

We present a set of easy-to-use methods and tools to analyze human attention, behaviour, and physiological responses. A potential application of our work is evaluating user interfaces being used in a natural manner. Our approach is designed to be scalable and to work remotely on regular personal computers using expensive and noninvasive equipment. The data sources our tool processes are nonintrusive, and captured from video; i.e. eye tracking, and facial expressions. For video data retrieval, we use a basic webcam. We investigate combinations of observation modalities to detect and extract affective and mental states. Our tool provides a pipeline-based approach that 1) collects observational, data 2) incorporates and synchronizes the signal modality mentioned above, 3) detects users' affective and mental state, 4) records user interaction with applications and pinpoints the parts of the screen users are looking at, 5) analyzes and visualizes results. We describe the design, implementation, and validation of a novel multimodal signal fusion engine, Deep Temporal Credence Network (DTCN). The engine uses Deep Neural Networks to provide 1) a generative and probabilistic inference model, and 2) to handle multimodal data such that its performance does not degrade due to the absence of some modalities. We report on the recognition accuracy of basic emotions for each modality. Then, we evaluate our engine in terms of effectiveness of recognizing basic six emotions and six mental states, which are agreeing, concentrating, disagreeing, interested, thinking, and unsure. Our principal contributions include the implementation of a 1) multimodal signal fusion engine, 2) real time recognition of affective and primary mental states from nonintrusive and inexpensive modality, 3) novel mental state-based visualization techniques, 3D heatmaps, 3D scanpaths, and widget heatmaps that find parts of the user interface where users are perhaps unsure, annoyed, frustrated, or satisfied.

Urban transportation networks are inefficient due to sub-optimal use by travelers. One approach to counter the increase in urban transportation demand is to provide better information to travelers, which would allow them to make better use of the network. Existing traveler information systems do this to a certain extent, but are limited by the data available and the scope of their implementation. These systems are vertically integrated and closed so that using any external elements for analysis, user interfacing etc. is difficult. The effects of such traveler information systems are reviewed via a comparative analysis of case studies available in the literature. It is found that information availability has a definite positive effect, but the social and environmental benefits are difficult to quantify. It is also seen that combining data by integrating systems can lead to additional uses for the same data and result on better quality of service and information. In this thesis, a regional platform for multi-modal traveler information is proposed that would support the development of traveler information systems. The architecture incorporates a central processing and storage module, which acts as an information clearinghouse and supports receiving, managing and sending data to and from multiple sources and interfaces. This setup allows sharing of data for analysis or application development, but with access control. The components are loosely coupled to minimize inter-dependencies. Due to this, the source, analysis, user interface and storage components can be developed independently of each other. To better develop the requirements and understand the challenges of the proposed concept, a limited implementation of the system is designed for the midtown Atlanta region, incorporating multiple data sources and user interfaces. The individual elements of the system are described in detail as is the testing and evaluation of the system.

The original VTQuest web-based software system requires users to interact using a mouse or a keyboard, forcing the usersâ hands and eyes to be constantly in use while communicating with the system. This prevents the user from being able to perform other tasks which require the userâ s hands or eyes at the same time. This restriction on the userâ s ability to multitask while using VTQuest is unnecessary and has been eliminated with the creation of the VTQuest Voice web-based software system. VTQuest Voice extends the original VTQuest functionality by providing the user with a voice interface to interact with the system using the Speech Application Language Tags (SALT) technology. The voice interface provides the user with the ability to navigate through the site, submit queries, browse query results, and receive helpful hints to better utilize the voice system. Individuals with a handicap that prevents them from using their arms or hands, users who are not familiar with the mouse and keyboard style of communication, and those who have their hands preoccupied need alternative communication interfaces which do not require the use of their hands. All of these users require and benefit from a voice interface being added onto VTQuest. Through the use of the voice interface, all of the systemâ s features can be accessed exclusively with voice and without the use of a userâ s hands. Using a voice interface also frees the userâ s eyes from being used during the process of selecting an option or link on a page, which allows the user to look at the system less frequently. VTQuest Voice is implemented and tested for operation on computers running Microsoft Windows using Microsoft Internet Explorer with the correct SALT and Adobe Scalable Vector Graphics (SVG) Viewer plug-ins installed. VTQuest Voice offers a variety of features including an extensive grammar and out-of-turn interaction, which are flexible for future growth. The grammar offers ways in which users may begin or end a query to better accommodate the variety of ways users may phrase their queries. To accommodate for abbreviations of building names and alternate pronunciations of building names, the grammar also includes nicknames for the buildings. The out-of-turn interaction combines multiple steps into one spoken sentence thereby shortening the interaction and also making the process more natural for the user. The addition of a voice interface is recommended for web applications which a user may need to use his or her eyes and hands to multitask. Additional functionality which can be added later to VTQuest Voice is touch screen support and accessibility from cell phones, Personal Digital Assistants (PDAs), and other mobile devices.
Master of Science

Insider threat is one of the greatest concerns for information security that could cause more significant financial losses and damages than any other attack. However, implementing an efficient detection system is a very challenging task. It has long been recognized that solutions to insider threats are mainly user-centric and several psychological and psychosocial models have been proposed. A user's psychophysiological behavior measures can provide an excellent source of information for detecting user's malicious behaviors and mitigating insider threats. In this dissertation, we propose a multi-modal framework based on the user's psychophysiological measures and computer-based behaviors to distinguish between a user's behaviors during regular activities versus malicious activities. We utilize several psychophysiological measures such as electroencephalogram (EEG), electrocardiogram (ECG), and eye movement and pupil behaviors along with the computer-based behaviors such as the mouse movement dynamics, and keystrokes dynamics to build our framework for detecting malicious insiders. We conduct human subject experiments to capture the psychophysiological measures and the computer-based behaviors for a group of participants while performing several computer-based activities in different scenarios. We analyze the behavioral measures, extract useful features, and evaluate their capability in detecting insider threats. We investigate each measure separately, then we use data fusion techniques to build two modules and a comprehensive multi-modal framework. The first module combines the synchronized EEG and ECG psychophysiological measures, and the second module combines the eye movement and pupil behaviors with the computer-based behaviors to detect the malicious insiders. The multi-modal framework utilizes all the measures and behaviors in one model to achieve better detection accuracy. Our findings demonstrate that psychophysiological measures can reveal valuable knowledge about a user's malicious intent and can be used as an effective indicator in designing insider threat monitoring and detection frameworks. Our work lays out the necessary foundation to establish a new generation of insider threat detection and mitigation mechanisms that are based on a user's involuntary behaviors, such as psychophysiological measures, and learn from the real-time data to determine whether a user is malicious.

Insider threat is one of the greatest concerns for information security that could cause more significant financial losses and damages than any other attack. However, implementing an efficient detection system is a very challenging task. It has long been recognized that solutions to insider threats are mainly user-centric and several psychological and psychosocial models have been proposed. A user's psychophysiological behavior measures can provide an excellent source of information for detecting user's malicious behaviors and mitigating insider threats. In this dissertation, we propose a multi-modal framework based on the user's psychophysiological measures and computer-based behaviors to distinguish between a user's behaviors during regular activities versus malicious activities. We utilize several psychophysiological measures such as electroencephalogram (EEG), electrocardiogram (ECG), and eye movement and pupil behaviors along with the computer-based behaviors such as the mouse movement dynamics, and keystrokes dynamics to build our framework for detecting malicious insiders. We conduct human subject experiments to capture the psychophysiological measures and the computer-based behaviors for a group of participants while performing several computer-based activities in different scenarios. We analyze the behavioral measures, extract useful features, and evaluate their capability in detecting insider threats. We investigate each measure separately, then we use data fusion techniques to build two modules and a comprehensive multi-modal framework. The first module combines the synchronized EEG and ECG psychophysiological measures, and the second module combines the eye movement and pupil behaviors with the computer-based behaviors to detect the malicious insiders. The multi-modal framework utilizes all the measures and behaviors in one model to achieve better detection accuracy. Our findings demonstrate that psychophysiological measures can reveal valuable knowledge about a user's malicious intent and can be used as an effective indicator in designing insider threat monitoring and detection frameworks. Our work lays out the necessary foundation to establish a new generation of insider threat detection and mitigation mechanisms that are based on a user's involuntary behaviors, such as psychophysiological measures, and learn from the real-time data to determine whether a user is malicious.

In order to leverage the use of various modalities such as audio-visual-touch in instilling learning behaviour, we present an intuitive approach of annotation based hapto-audio-visual interaction with the traditional digital learning materials such as eBooks. By integrating the traditional home entertainment system and respective media in the user's reading experience combined with haptic interfaces, we examine whether such augmentation of modalities influence the user's reading experience in terms of attention, entertainment and retention. The proposed Haptic E-Book (HE-Book) system leverages the haptic jacket, haptic arm band as well as haptic sofa interfaces to receive haptic emotive signals wirelessly in the form of patterned vibrations of the actuators and expresses the learning material by incorporating audio-video based augmentation in order to pave ways for intimate reading experience in the popular eBook platform. We have designed and developed desktop, mobile/tablet based HE-Book system as well as a semi-automated annotation authoring tool. Our system also supports multimedia based diverse quiz augmentations, which can help in learning tracking. We have conducted quantitative and qualitative tests using the developed prototype systems. We have adopted the indirect objective based performance analysis methodology, which is commonly used for multimedia based learning investigation. The user study shows that, there is a positive tendency of accepting multimodal interactions including haptics with traditional eBook reading experience. Though our limited number of laboratory tests reveal, that haptics can be an influencing media in eBook reading experience, but it requires large scale real life tests to provide a concluding remarks.

Current studies done using a learning test for children have problems as they only make evaluations of Physically and Neurologically Impaired (PNI) children who can succeed in the test and can be considered as a PASS/FAIL test. This pilot study takes a holistic view of cognitive testing of PNI children using a user-test-device triad model and provides a framework using non-PNI children and adults as controls. Comparisons using adapted off-the-shelf novel interfaces to the computer, in particular, an Electroencephalograph (EEG) head-set, an eye-tracker and a head-tracker and a common mouse were carried out. In addition, two novel multi-modal technologies were developed based on the use of brain-waves and eye-tracking as well as head-tracking technologies to support the study. The devices were used on three tests with increasing cognitive complexity. A self-developed measure based on success streaks (consecutive outcomes) was introduced to improve evaluations of PNI children. A theoretical model regarding a fit of ability to devices was initially setup and finally modified to fit the view of the empirical model that emerged from the outcomes of the study. Results suggest that while multi-modal technologies can address weaknesses of the individual component modes, a compromise is made between the user’s ability for multi-tasking between the modes and the benefits of a multi-modal device but the sample size is very small. Results also show children failing a test with a mouse but passing it subsequently when direct communication is used suggesting that a device can affect a test for children who are of a developing age. This study provides a framework for a more meaningful conversation between educational psychologists as well as other professionals and PNI parents because it provides more discrimination of outcomes in cognitive tests for PNI children. The framework provides a vehicle that addresses scientifically the concerns of parents and schools.

Dans le processus de conception des véhicules, la vibration et le bruit sont des sujets d’étude très importants. En effet, les vibrations sont susceptibles d’affecter le comportement dynamique des structures et le bruit dégrade le confort acoustique des passagers. L’objectif principal de la thèse est de proposer un ensemble de méthodes pour l’optimisation du comportement dynamique des systèmes complexes afin de réduire les vibrations des structures et le bruit dans l’habitacle. À cet effet, on s’intéresse à des stratégies de contrôle des interfaces, comme le collage de couches viscoélastiques sur les zones les plus déformées, ou l’introduction de dispositifs frottants calibrés pour ajouter de l’amortissement à certaines fréquences de résonance. Les structures assemblées résultantes sont étudiées numériquement par une méthode de synthèse modale généralisée. La méthode de synthèse modale proposée contient plusieurs niveaux de condensation. Le premier concerne les degrés de libertés (DDL) internes de chaque sous-structure. La deuxième condensation s’effectue sur les modes de branches, de sorte à réduire le nombre de DDL aux interfaces entre les sous-structures. Pour les systèmes couplés fluide/structure, une troisième condensation portant sur les DDL du fluide est proposée. Suite à ces condensations, la dimension du système est fortement réduite. Cette méthode permet alors d’obtenir aussi bien la réponse forcée de la structure que les fluctuations du champ de pression dans le fluide. Les chemins de transmission acoustiques et vibratoires peuvent également être déduits des contributions modales intermédiaires. On montre que ces paramètres modaux peuvent être utilisés comme fonctions objectif pour une démarche d’optimisation des interfaces. Le front de Pareto des conceptions optimales est obtenu avec un algorithme génétique multi-objectif élitiste, appliqué à une approximation par krigeage de la fonction objectif. Cette approche modale est étendue à l’étude de systèmes non-linéaires. L’hypothèse fondamentale est que les modes non-linéaires sont faiblement couplés. Les paramètres modaux non-linéaires (fréquences propres, amortissements...), dépendent des amplitudes modales. L’idée est alors de calculer des modes normaux non-linéaires en fonction de leur amplitude et de superposer leurs réponses pour obtenir celle de la structure. La méthode est appliquée à des systèmes incorporant des non-linéarités de type Duffing et de frottement sec. Le cas particulier du frottement sec est considéré à travers un modèle de Masing généralisé. Deux approches modales sont développées : l’une basée sur les modes complexes, et l’autre basée sur les modes réels. L’utilisation de modes complexes ou réels dans la synthèse modale conduit à des termes d’amortissement par frottement différents. On montre que la synthèse modale non-linéaire combinée au modèle de Masing généralisé aboutit à une méthode numérique simple, rapide et efficace pour décrire le comportement non-linéaire de structures soumise à du frottement sec
Noise and vibration are important topics in the automotive industry for several reasons, including passenger comfort and structural integrity. The main objective of this thesis is to propose a series of appropriate methods to optimize structural system characteristics, so that the vibration and noise can be reduced. To achieve this goal, interface control strategies are employed, including bonding viscoelastic layers onto the most heavily deformed zones and introducing frictional damping devices calibrated on certain resonance frequencies. Such built-up structural systems are numerically investigated via a generalized modal synthesis approach that incorporates several groups of modes. The employed modal synthesis approach consists of several levels of condensation. The first one is on the internal degrees of freedoms (DOFs) of each substructure, and the second condensation is on the branch modes so as to reduce the boundary DOFs among substructures. For coupled fluid-structural systems, a third condensation on the fluid DOFs is suggested. With these condensation techniques, the system dimension can be significantly reduced. The method allows us to obtain the forced response of the structures as well as the pressure variation of the fluids. Additionally, modal parameters characterizing vibration and noise transmission paths can be deduced as mid-stage results. We show that these modal parameters can be used as optimization objective during the interface configuration design. The Pareto front of the optimal design is achieved by employing Kriging approximations followed with an elitist multi-objective genetic algorithm. Another advantage of the modal approach is that a modal overview on the system characteristics is provided by analyzing the natural frequencies, modal damping ratios and the aforementioned modal parameters. The modal synthesis approach is further extended to study nonlinear systems. The basic assumption is that the nonlinear modes are weakly coupled. Nonlinear modal parameters, such as modal frequency and modal damping ratio, contain the essential nonlinear information and depend on modal amplitude. The main idea is to compute nonlinear normal modes according to their modal amplitude and superimpose the response of several nonlinear modes to obtain the overall forced response. The method is applied to systems involving Duffing and dry friction nonlinearities. In the case of dry friction, a generalized Masing model is considered to capture the dry friction nature. Both complex modes and real modes are used in the modal synthesis, leading to different frictional damping terms. We show that the nonlinear modal synthesis combined with the generalized Masing model yields a simple, fast and efficient numerical method to describe nonlinear performance of structures with dry friction

Providing carrier grade characteristics for Internet Protocol (IP) communication applications is a significant problem for IP application providers in order to offer integrated services that span IP 
and telecommunication networks. This thesis addresses the provision of life-cycle management, which is only one carrier grade characteristic, for a SoftBridge application, which is an example of IP communication applications. A SoftBridge provides semi-synchronous multi-modal IP-based communication. The work related to IP-Telecommunication integrated services and the SoftBridge is analyzed with respect to life-cycle management in a literature review. It is suggested to use an Application Server in a Next Generation Network (NGN) to provide life-cyclemanagement functionality for IP-Telecommunication applications. In this thesis, the Application Server is represented by a JAIN Service Logic Execution Environment(JSLEE), in which 
a SoftBridge application can be deployed, activated, deactivated, uninstalled and upgraded online.Two methodologies are applied in this research: exploratory prototyping, which evolves the development of a SoftBridge application, and empirical comparison, which is concerned with the empirical evaluation of a SoftBridge application in terms of carriergrade capabilities. A SoftBridge application called SIMBA 
provides a Deaf Telephony service similar to aprevious Deaf Telephony SoftBridge, However, SIMBA&rsquo
s SoftBridge design and implementation are unique to this thesis. In order to test the life-cycle 
management ability of SIMBA, an empirical evaluation is carried out including the experiments oflife-cycle management and call-processing performance. The final experimental results of the evaluation show that a JSLEE is able to provide life-cycle management for SIMBA without causing a significant decrease in performance. In conclusion, the life-cycle management can be provided 
or a SoftBridge application by using an Application Server such as a JSLEE. Futhermore, the results indicate that 
approach of using Application Server (JSLEE) integration should be 
sufficiently general to provide life cycle management, and indeed other carrier grade capabilities, for other IP communication applications. This allows IP communication applications to be 
 
 
integrated into an NGN.

This thesis explores the use of multimodality to communicate retrieved results of Internet search engines. The investigation aimed to investigate suitable multimodal metaphors which would increase the level of usability of Internet search engine interfaces and enhance users` experience in the search activity. The study consisted of three experiments based on questionnaires and Internet search activities with a set of text-based and multimodal interfaces. These interfaces were implemented in two browsing platforms, named AVBRO and AVBRO II. In the first experiment, the efficiency of specific multimodal metaphors to communicate additional information of retrieved results was investigated. The experiment also sought to obtain users` views of these metaphors with a questionnaire. An experimental multimodal interface of the AVBRO platform, which communicated additional information with a combination of three 2D graphs and musical stimuli, was used as a basis for the experiment, together with the Google search engine. The results obtained led to the planning of a second experiment. The aim of this experiment was to obtain and compare the level of usability of four different experimental multimodal interfaces and one traditional text-based interface, all implemented in the AVBRO II platform. Effectiveness, efficiency and users` satisfaction were used as criteria to evaluate the usability of these interfaces. In the third and final experiment the usability analysis of a traditional text-based interface and the two most suitable experimental multimodal interfaces of the AVBRO II platform was further investigated. Learnability, errors rate, efficiency, memorability and users` satisfaction were used as criteria to evaluate the usability of these interfaces. The analysis of the results obtained from these experiments provided the basis for a set of design guidelines for the development of usable interfaces based on a multimodal approach.

Les systèmes de freins sont parfois sujets au crissement qui sont des vibrations auto-entretenues induites par le frottement et caractérisées par un contenu fréquentiel formé de raies à hautes fréquences supérieures à 1kHz. Ces vibrations et bruits intenses sont une source de gêne pour les usagers automobile, un problème de santé publique pour les riverains de gares lors du freinage de TGV et peuvent amener à l'endommagement du train d'atterrissage sur les avions.Afin de comprendre ce phénomène et pour le reproduire numériquement, une stratégie complète d'étude est développée. Elle se base sur l'observation expérimentale d'essais de crissement sur un banc d'essais qui permet de formuler des hypothèses de modélisation. Ces dernières sont un guide pour la construction d'un modèle numérique de frein simple. Une méthode de Double Synthèse Modale est appliquée au modèle afin d'en réduire la taille et de permettre des simulation numériques en temps raisonnable et ne nécessitant pas trop de ressources informatiques.La démarche numérique qui suit commence classiquement par une analyse de stabilité par la méthode CEA où les valeurs propres du modèle linéarisé autour de la position d'équilibre glissante sont évaluées. Puis une intégration temporelle est effectuée dans les cas détectés comme instables afin de calculer les niveaux de vibrations. L'étude se termine par une estimation du champ acoustique rayonné par la structure complète.Dans chacune des phases de l'analyse numérique, des outils spécifiques sont utilisés pour comparer le modèle de référence aux modèles obtenus par les deux étapes de réduction. Un critère d'erreur sur les valeurs propres et un critère de MAC sont utilisés pour l'analyse de stabilité. Pour l'étude temporelle, les allures des signaux sont comparées, ainsi que leurs cycles limites et leurs spectrogrammes. Les participations des modes instables sont également calculées pour observer le régime transitoire. En ce qui concerne la partie acoustique, les signaux sont comparés dans un premier temps de façon qualitative pour observer les différences entre les champs émis en fonction des différentes tailles de bases de réduction. Puis un outil basé sur une décomposition en 2D par wavelet des motifs acoustiques est introduit et appliqué pour estimer de façon quantitative les convergences des champs rayonnés
Brake systems are sometimes prone to squeal noise, which is due to friction-induced self-sustained vibrations, characterized by a set of frequencies above 1kHz. Those vibrations and resulting noises are a source of perturbations for car occupants, which can be nowadays considered as a health issue.This thesis deals with a global strategy to better understand this phenomenon from an experimental point of view and to propose the prediction of squeal noise by numerical approaches. Moreover, experimental observations of squeal occurrences are analyzed to lead to assumptions about the modelisation of numerical finite element models for squeal prediction. A Double Modal Synthesis is also applied to reduce the size of the discrete finite element model of brake system and to save computational time and ressources. The proposed numerical approach starts with a stability analysis with the classical CEA method. Then the determination of nonlinear self-excited vibrations are performed for the unstable cases detected via the CEA method. Finally the acoustic field emitted by the brake system is computed to predict squeal noise.Specific tools are applied for each computational step to assess the efficiency of reduced model versus the reference model: criteria based on the mean error on eigenvalues and the Modal Assurance Criterion analysis (MAC) are used for the stability analysis; comparisons of the limit cycles, spectrograms and the modal contributions of unstable modes are undertaken for the transient responses; patterns of the acoustic intensity are computed on several observations surfaces and a decomposition based on the theory of 2D wavelets is introduced and applied to assess the convergence of patterns

Ce travail de thèse de doctorat, effectué au sein de Laboratoire de Simulation et Modélisation Électromagnétique (LSME) du CEA List, s’intègre dans le cadre du projet européen « NDTonAir » financé sous l'action « H2020-MSCA-ITN-2016- GRANT 722134 ». Le principal objectif est le développement d’un outil de simulation rapide et précis dédié au contrôle non destructif par courants de Foucault des matériaux composites homogénéisés. Comme cas d’application, on s’intéresse particulièrement à l’orientation des fibres d’une part, et d’autre part, à des défauts de type délaminage et ondulation des fibres qui se manifestent par une déformation géométrique locale des interfaces. Les méthodes semi-analytiques existantes dans la littérature, basées sur le formalisme des Dyades de Green, sont limitées jusqu’au là à des structures planes multicouches. Pour introduire des variations locales de géométrie aux interfaces, nous proposons une approche innovante basée sur un changement de coordonnées adapté au profil de la pièce et des interfaces. On propose un modèle numérique performant basé sur le formalisme covariant des équations de Maxwell. Ce formalisme unificateur englobe l'anisotropie du spécimen et les déformations locales des interfaces. La méthode de coordonnées curvilignes est usuellement utilisée pour résoudre des problèmes de diffraction sur des surfaces rugueuses dans le domaine des hautes fréquences (diffraction sur des réseaux). Ce travail de thèse s’inspire des méthodes de Fourier modale et propose de nouveaux outils adaptés au domaine des courants de Foucault. L’extension de la méthode des coordonnées curvilignes au domaine du contrôle des composites par courants de Foucault constitue l’innovation de ce travail. Deux modèles numériques ont été développés pour le calcul de l’interaction du champ émis par un capteur à courants de Foucault avec un matériau composite multicouches. Le modèle numérique développé pour le contrôle des composites plans exploite les structures particulières des matrices creuses pour réduire le temps de calcul sans limitation de nombre de modes utilisés pour la représentation du champ. Dans le cas des profils curvilignes des interfaces, le modèle permet de traiter des interfaces parallèles et quelques cas particuliers des profils non parallèles. Ce cas général présente quelques limitations qui nécessitent le développement des outils numériques complémentaires. Enfin, plusieurs configurations de contrôle ont été envisagées et les résultats numériques produits par les modèles ont été confrontés à des données de simulation par éléments finis. Quelques expérimentations ont été effectuées dans des laboratoires partenaires étrangers pour accroître notre expérience sur la validation expérimentale
This doctoral thesis work, carried out within the Laboratory of Simulation and Modeling for Electromagnetics (LSME) of CEA List, is part of the “NDTonAir” European project funded under the action “H2020-MSCA-ITN -2016- GRANT 722134”. The main goal of the project is the development of a fast and accurate simulation tool for the non-destructive eddy current testing of homogenized composite materials. As an application case, we are particularly interested in the orientation of the fibers on the one hand, and on the other hand, in defects as delamination which are manifested by a local geometrical deformation of the interfaces. The semi-analytical methods existing in the literature, based on Green's Dyad formalism, have been limited so far to multilayered and planar structures. To introduce local variations in geometry at the interfaces, we propose an innovative approach based on a change of coordinates adapted to the profile of the local perturbation. We propose a powerful numerical model based on the covariant formalism of Maxwell's equations. This unifying formalism takes in the anisotropy of specimen and the local deformations of the interfaces. The curvilinear coordinate method is usually used to solve diffraction problems on rough interfaces in the high frequency domain (diffraction on gratings). This thesis work is inspired by Fourier Modal Methods and proposes new tools which have been adapted to the field of eddy currents. The extension of the curvilinear coordinate method to the field of eddy currents non-destructive testing technique of composites constitutes the innovation of this work. Two numerical models have been developed to calculate the interaction of the field emitted by an eddy current probe with a multilayered composite material. The numerical model developed for the evaluation of planar composite exploits the particular structures of sparse matrices to reduce the computation time without limiting the number of modes used for the modal expansion of the field. In the case of the curvilinear profiles of the interfaces, the model makes it possible to treat parallel interfaces and some particular cases of non-parallel profiles. The general case of non-identical profiles presents some limitations which require the development of complementary numerical tools. Finally, several testing configurations were considered and the numerical results produced by the models were compared to finite element simulated data. Some experiments were carried out in foreign partner laboratories to increase our experience on experimental validation

碩士
中原大學
電子工程研究所
98
Technology device comes from Human concern which promoting and providing the quality of life for the disable and the aging society. The assistance of modern technology it is not only be replaced by laborer, but also providing dignity of the disable and the elder to become self- sufficient. Therefore, the technology device not only provide convenience but also assist the disable. Technology products is able to assist people with cognition disable. Modern technologies such as computers, Internet, mobile phones, and PDAs save times for office workers and simplify their daily tasks. However, for the physically and mentally disables, these technology products-termed "assistive devices"-are not just valuable, but may be their only medium to communicate with the outside world. Considering the requirement for cognition disable, the design of navigation interface have to conform with user-friendly, easy operation accessibility in mode exchanging, as well as having easy personal setup process, such as faster velocity, shorter distance or to have better consideration in security condition, and to keep away from danger and dark street, it will progressively telling you how to arrive the destination through its voicing system and its arrow sign, if you missed last voice, you can repeat the voice again and again .In the indoor space, cognition disables can be assisted by the real-simulation navigation system which can be used instead of finding their own way by themselves, and to help them to find the shortest and most appropriate path to their destination. Respect life and to believe that the cognition disable has the same right to pursuit happiness as average people. By realizing this idea requires more than just patience and compassion. Through the performance of this research, the expectation can apply the latest information for communication science and technology, constructing a humanization smart environments.

Brain-computer interfaces (BCI) have been used to control the gait of a virtual self-avatar with the aim of being used in gait rehabilitation. A BCI decodes the brain signals representing a desire to do something and transforms them into a control command for controlling external devices. The feelings described by the participants when they control a self-avatar in an immersive virtual environment (VE) demonstrate that humans can be embodied in the surrogate body of an avatar (ownership illusion). It has recently been shown that inducing the ownership illusion and then manipulating the movements of one’s self-avatar can lead to compensatory motor control strategies. In order to maximize this effect, there is a need for a method that measures and monitors embodiment levels of participants immersed in virtual reality (VR) to induce and maintain a strong ownership illusion. This is particularly true given that reaching a high level of both BCI performance and embodiment are inter-connected. To reach one of them, the second must be reached as well. Some limitations of many existing systems hinder their adoption for neurorehabilitation: 1- some use motor imagery (MI) of movements other than gait; 2- most systems allow the user to take single steps or to walk but do not allow both, which prevents users from progressing from steps to gait; 3- most of them function in a single BCI mode (cue-paced or self-paced), which prevents users from progressing from machine-dependent to machine-independent walking. Overcoming the aforementioned limitations can be done by combining different control modes and options in one single system. However, this would have a negative impact on BCI performance, therefore diminishing its usefulness as a potential rehabilitation tool. In this case, there will be a need to enhance BCI performance. For such purpose, many techniques have been used in the literature, such as providing modified feedback (whereby the presented feedback is not consistent with the user’s MI), sequential training (recalibrating the classifier as more data becomes available). This thesis was developed over 3 studies. The objective in study 1 was to investigate the possibility of measuring the level of embodiment of an immersive self-avatar, during the performing, observing and imagining of gait, using electroencephalogram (EEG) techniques, by presenting visual feedback that conflicts with the desired movement of embodied participants. The objective of study 2 was to develop and validate a BCI to control single steps and forward walking of an immersive virtual reality (VR) self-avatar, using mental imagery of these actions, in cue-paced and self-paced modes. Different performance enhancement strategies were implemented to increase BCI performance. The data of these two studies were then used in study 3 to construct a generic classifier that could eliminate offline calibration for future users and shorten training time. Twenty different healthy participants took part in studies 1 and 2. In study 1, participants wore an EEG cap and motion capture markers, with an avatar displayed in a head-mounted display (HMD) from a first-person perspective (1PP). They were cued to either perform, watch or imagine a single step forward or to initiate walking on a treadmill. For some of the trials, the avatar took a step with the contralateral limb or stopped walking before the participant stopped (modified feedback). In study 2, participants completed a 4-day sequential training to control the gait of an avatar in both BCI modes. In cue-paced mode, they were cued to imagine a single step forward, using their right or left foot, or to walk forward. In the self-paced mode, they were instructed to reach a target using the MI of multiple steps (switch control mode) or maintaining the MI of forward walking (continuous control mode). The avatar moved as a response to two calibrated regularized linear discriminant analysis (RLDA) classifiers that used the μ power spectral density (PSD) over the foot area of the motor cortex as features. The classifiers were retrained after every session. During the training, and for some of the trials, positive modified feedback was presented to half of the participants, where the avatar moved correctly regardless of the participant’s real performance. In both studies, the participants’ subjective experience was analyzed using a questionnaire. Results of study 1 show that subjective levels of embodiment correlate strongly with the power differences of the event-related synchronization (ERS) within the μ frequency band, and over the motor and pre-motor cortices between the modified and regular feedback trials. Results of study 2 show that all participants were able to operate the cued-paced BCI and the selfpaced BCI in both modes. For the cue-paced BCI, the average offline performance (classification rate) on day 1 was 67±6.1% and 86±6.1% on day 3, showing that the recalibration of the classifiers enhanced the offline performance of the BCI (p < 0.01). The average online performance was 85.9±8.4% for the modified feedback group (77-97%) versus 75% for the non-modified feedback group. For self-paced BCI, the average performance was 83% at switch control and 92% at continuous control mode, with a maximum of 12 seconds of control. Modified feedback enhanced BCI performances (p =0.001). Finally, results of study 3 show that the constructed generic models performed as well as models obtained from participant-specific offline data. The results show that there it is possible to design a participant-independent zero-training BCI.
Les interfaces cerveau-ordinateur (ICO) ont été utilisées pour contrôler la marche d'un égo-avatar virtuel dans le but d'être utilisées dans la réadaptation de la marche. Une ICO décode les signaux du cerveau représentant un désir de faire produire un mouvement et les transforme en une commande de contrôle pour contrôler des appareils externes. Les sentiments décrits par les participants lorsqu'ils contrôlent un égo-avatar dans un environnement virtuel immersif démontrent que les humains peuvent être incarnés dans un corps d'un avatar (illusion de propriété). Il a été récemment démontré que provoquer l’illusion de propriété puis manipuler les mouvements de l’égo-avatar peut conduire à des stratégies de contrôle moteur compensatoire. Afin de maximiser cet effet, il existe un besoin d'une méthode qui mesure et surveille les niveaux d’incarnation des participants immergés dans la réalité virtuelle (RV) pour induire et maintenir une forte illusion de propriété. D'autre part, atteindre un niveau élevé de performances (taux de classification) ICO et d’incarnation est interconnecté. Pour atteindre l'un d'eux, le second doit également être atteint. Certaines limitations de plusieurs de ces systèmes entravent leur adoption pour la neuroréhabilitation: 1- certains utilisent l'imagerie motrice (IM) des mouvements autres que la marche; 2- la plupart des systèmes permettent à l'utilisateur de faire des pas simples ou de marcher mais pas les deux, ce qui ne permet pas à un utilisateur de passer des pas à la marche; 3- la plupart fonctionnent en un seul mode d’ICO, rythmé (cue-paced) ou auto-rythmé (self-paced). Surmonter les limitations susmentionnées peut être fait en combinant différents modes et options de commande dans un seul système. Cependant, cela aurait un impact négatif sur les performances de l’ICO, diminuant ainsi son utilité en tant qu'outil potentiel de réhabilitation. Dans ce cas, il sera nécessaire d'améliorer les performances des ICO. À cette fin, de nombreuses techniques ont été utilisées dans la littérature, telles que la rétroaction modifiée, le recalibrage du classificateur et l'utilisation d'un classificateur générique. Le projet de cette thèse a été réalisé en 3 études, avec objectif d'étudier dans l'étude 1, la possibilité de mesurer le niveau d'incarnation d'un égo-avatar immersif, lors de l'exécution, de l'observation et de l'imagination de la marche, à l'aide des techniques encéphalogramme (EEG), en présentant une rétroaction visuelle qui entre en conflit avec la commande du contrôle moteur des sujets incarnés. L'objectif de l'étude 2 était de développer un BCI pour contrôler les pas et la marche vers l’avant d'un égo-avatar dans la réalité virtuelle immersive, en utilisant l'imagerie motrice de ces actions, dans des modes rythmés et auto-rythmés. Différentes stratégies d'amélioration des performances ont été mises en œuvre pour augmenter la performance (taux de classification) de l’ICO. Les données de ces deux études ont ensuite été utilisées dans l'étude 3 pour construire des classificateurs génériques qui pourraient éliminer la calibration hors ligne pour les futurs utilisateurs et raccourcir le temps de formation. Vingt participants sains différents ont participé aux études 1 et 2. Dans l'étude 1, les participants portaient un casque EEG et des marqueurs de capture de mouvement, avec un avatar affiché dans un casque de RV du point de vue de la première personne (1PP). Ils ont été invités à performer, à regarder ou à imaginer un seul pas en avant ou la marche vers l’avant (pour quelques secondes) sur le tapis roulant. Pour certains essais, l'avatar a fait un pas avec le membre controlatéral ou a arrêté de marcher avant que le participant ne s'arrête (rétroaction modifiée). Dans l'étude 2, les participants ont participé à un entrainement séquentiel de 4 jours pour contrôler la marche d'un avatar dans les deux modes de l’ICO. En mode rythmé, ils ont imaginé un seul pas en avant, en utilisant leur pied droit ou gauche, ou la marche vers l’avant . En mode auto-rythmé, il leur a été demandé d'atteindre une cible en utilisant l'imagerie motrice (IM) de plusieurs pas (mode de contrôle intermittent) ou en maintenir l'IM de marche vers l’avant (mode de contrôle continu). L'avatar s'est déplacé en réponse à deux classificateurs ‘Regularized Linear Discriminant Analysis’ (RLDA) calibrés qui utilisaient comme caractéristiques la densité spectrale de puissance (Power Spectral Density; PSD) des bandes de fréquences µ (8-12 Hz) sur la zone du pied du cortex moteur. Les classificateurs ont été recalibrés après chaque session. Au cours de l’entrainement et pour certains des essais, une rétroaction modifiée positive a été présentée à la moitié des participants, où l'avatar s'est déplacé correctement quelle que soit la performance réelle du participant. Dans les deux études, l'expérience subjective des participants a été analysée à l'aide d'un questionnaire. Les résultats de l'étude 1 montrent que les niveaux subjectifs d’incarnation sont fortement corrélés à la différence de la puissance de la synchronisation liée à l’événement (Event-Related Synchronization; ERS) sur la bande de fréquence μ et sur le cortex moteur et prémoteur entre les essais de rétroaction modifiés et réguliers. L'étude 2 a montré que tous les participants étaient capables d’utiliser le BCI rythmé et auto-rythmé dans les deux modes. Pour le BCI rythmé, la performance hors ligne moyenne au jour 1 était de 67±6,1% et 86±6,1% au jour 3, ce qui montre que le recalibrage des classificateurs a amélioré la performance hors ligne du BCI (p <0,01). La performance en ligne moyenne était de 85,9±8,4% pour le groupe de rétroaction modifié (77-97%) contre 75% pour le groupe de rétroaction non modifié. Pour le BCI auto-rythmé, la performance moyenne était de 83% en commande de commutateur et de 92% en mode de commande continue, avec un maximum de 12 secondes de commande. Les performances de l’ICO ont été améliorées par la rétroaction modifiée (p = 0,001). Enfin, les résultats de l'étude 3 montrent que pour la classification des initialisations des pas et de la marche, il a été possible de construire des modèles génériques à partir de données hors ligne spécifiques aux participants. Les résultats montrent la possibilité de concevoir une ICO ne nécessitant aucun entraînement spécifique au participant.

碩士
國立臺北科技大學
互動設計系
107
Upon the artificial intelligent trend in recent years, the world has gradually evolved from the era of information digitalization to the era of comprehensive intellectualization, with the technical development of Internet of Things, Big Data and Cloud Service, etc. All of the concerned parties take great interest in the beneficial impact of artificial intelligence regarding their own industries. From the perspective of technology development, the purpose of progress has changed from solving people’s inconvenience and providing single-aspect function to the customization and intellectualization in product service. Not only did this movement provide tailored service experience but also integrate the service system onto the cloud, facilitating business organization to increase their management efficiency by central control system, to plan strategy by data analysis and to explore more opportunities. Along with technological development, the digital signage industry in Taiwan has progressed from Non-touch display with single visualization to the single touch and multi-touch. By the Natural User Interface approach of gesture, voice and mouse over, the interaction between the signages is become more personal, intuitive and diverse. The research was trying to find out the method that could introduce the technicality of Natural User Interface to carry artificial intelligence into digital signage, which can inspire and guide people to have a vision about intelligent service, to create a innovative application for integration service and to provide mass business data analysis, in order to derive more advanced technology and business model for the opportunities to a win-win situation of intelligent service. In the pilot study, the research was aimed at the analysis on the current progress of Natural User Interface development and digital signage industry in Taiwan, therefore, it included three phases: first, the research method, conducted experimental test using smart TV system with multi-mode Natural User Interface(by Voice, Gesture and Mouse over operation approach), and evaluated user’s acceptance regarding the operation of Natural User Interface and the limitation on current technique; second, interviewed the focus group of digital signage company in Taiwan to grasp the latest progress and technique trend within the industry, and to apply the result into “voice” operation; third, conducted preliminary survey on the user experience of voice service. After the analysis of the pilot study, the research, by targeting “combination of digital signage with voice control” through a cross-discipline design workshop that used POEMS (People, Object, Environment, Message, Service) system as a design method, further studied the application area in future digital signage with voice service operation. At last, the research decided to apply “voice service system” as a research development on the next stage of service scenario under the structure of the future unmanned restaurant. For the considerations on the design element required, the research included both the Macro perspective on “overall service process” and the Micro perspective on “voice service system”. At first, in order to define Persona, to find the core service need and to provide the creative planning of user journey map and service blueprint, the research had conducted a user interview. Based on this, the research applied two-directional questionnaire of Kano’s model to analyze the Macro and Micro perspectives in the service function, which targeted at the interactive scenario design of the creative prototype, and also took gender analysis and user’s level of satisfaction into consideration, furthermore, to complete the evaluation on overall scenario and service function application, the research had applied QFD House of Quality analysis to prioritize the importance of service functions. At the final stage, the research had discussed the effects and potential development in technique integration-related overall service mechanism and function by using fast voice prototype design in voice service system to evaluate user experience for the purpose of understanding user needs, level of satisfaction and experience design. The analysis result from the aforementioned has reached a conclusion. The research scenario has positively improved dining experience and contributed in solving restaurant’s current problem. The “Overall Service Process” in “Unmanned Restaurant Voice Service” macro perspective, the service item “On-Line Streaming With the Chief” had the most charismatic attraction; while in the micro perspective of “Voice Service System”, the “Real-Time Voice Order Comment” had drew most of the attention. From the results of gender analysis, the two have different quality attributes for different service functions. The research has verified that the voice service system has showed positive availability and potential development regarding the application aspect in unmanned restaurant, by evaluating the user experience in voice service system. The conclusion above could serve as a reference to the business organization for future development in restaurant voice service scenario, and also provide as an academic reference to discuss the potential function requirement of overall service process and voice system.

碩士
國立臺南大學
數位學習科技學系碩士班
100
Recently, educationists have indicated that emotions are one of the most crucial factors which might affect learning. However, the application of affective computing techniques to digital learning is still at the initial stage. On the other hand, digital arts are known to be both subjective and highly sensitive. What’s more, they are innovative and have gradually become the mainstream. As a result, it is necessary that we should educate and promote digital arts by integrating affective factors with them. Resent research have further set up the intelligent tutoring system to assist and invigorate students’ learning effects. In view of this, this study will apply both affective recognition and digital art education to intelligent tutoring system. By observing the learners’ emotions, it is able to estimate whether the learning process is smooth, and then give timely feedback. Therefore, it can be more interactive so as to achieve its final goal—increase the utility of the system, enhance the motivation of the learners, and improve the learning outcome. Therefore, it can be more interactive so as to achieve its final goal—increase the utility of the system, enhance the motivation of the learners, and improve the learning outcome. This study adopts both facial expression recognition and semantic emotion recognition. By using the dual-mode mechanism as the foundation of affective computing, we can not only complement the shortcomings of the single-mode recognizing mechanism but also increase the variety of emotion recognition, so we can be aware of the learners’ learning condition. Facial expression recognition is carried out by the training of image processing template, while semantic emotion recognition receive the emotions by detecting emotional key words and calculating with syntactical logic. The system will know the learning condition through emotion recognition, so it can arrange proper teaching strategies and courses. In addition, the teaching assistant can also serve as the communication channel between the system and the user, so the user of this system can enhance his/her learning motivation and learns well. This study adopts triangulation designs: The system would be evaluated by observation, questionnaire, and interview. These assessment methods emphasize both quality and quantity. By these assessments, it would be clear that whether the system can conform to the aim of this study or not. Finally, the result of assessment shows that the application of the intelligent tutoring system, which makes use of affective computing techniques, to the learning of digital arts courses will not only have good utility, but also enhance the motivation and improve the learning outcome.

Due to the world’s rapidly growing elderly population, dementia is becoming increasingly prevalent. This poses considerable health, social, and economic concerns as it impacts individuals, families and healthcare systems. Current research has shown that cognitive interventions may slow the decline of or improve brain functioning in older adults. This research investigates the use of intelligent socially assistive robots to engage individuals in person-centered cognitively stimulating activities. Specifically, in this thesis, a novel learning-based control architecture is developed to enable socially assistive robots to act as social motivators during an activity. A hierarchical reinforcement learning approach is used in the architecture so that the robot can learn appropriate assistive behaviours based on activity structure and personalize an interaction based on the individual’s behaviour and user state. Experiments show that the control architecture is effective in determining the robot’s optimal assistive behaviours for a memory game interaction and a meal assistance scenario.