Dissertations / Theses on the topic 'Haptic Interaction'

Historically, haptics—all different aspects of the sense of touch and its study—has developed around very technical and scientific inquiries. Despite considerable haptic research advances and the obviousness of haptics in everyday life, this modality remains mostly foreign and unfamiliar to designers. The guiding motif of this research relates to a desire to reverse the situation and have designers designing for and with the haptic sense, for human use and looking beyond technical advances. Consequently, this thesis aims to nurture the development of haptics from a designerly perspective, leading to a new field of activities labeled haptic interaction design. It advances that haptic attributes and characteristics are increasingly part of the qualities that make up the interactions and the experiences we have with objects and the interfaces that surround us, and that these considerations can and ought to be knowingly and explicitly designed by designers. The book encompasses an annotated research through design exploration of the developing field of haptic interaction design, building on a considerable account of self-initiated individual design activities and empirical-style group activities with others. This extensive investigation of designing haptic interactions leads to the Simple Haptics proposition, an approach to ease the discovery and appropriation of haptics by designers. Simple Haptics consists in a simplistic, rustic approach to the design of haptic interactions, and advocates an effervescence of direct perceptual experiences in lieu of technical reverence. Simple Haptics boils down to three main traits: 1) a reliance on sketching in hardware to engage with haptics; 2) a fondness for basic, uncomplicated, and accessible tools and materials for the design of haptic interactions; and 3) a strong focus on experiential and directly experiencable perceptual qualities of haptics.  Ultimately, this thesis offers contributions related to the design of haptic interactions. The main knowledge contribution relates to the massification of haptics, i.e. the intentional realization and appropriation of haptics—with its dimensions and qualities—as a non-visual interaction design material. Methodologically, this work suggests a mixed longitudinal approach to haptics in a form of a well-grounded interplay between personal inquiries and external perspectives. The book also presents design contributions as ways to practically, physically and tangibly access, realize and explore haptic interactions. Globally these contributions help make haptics concrete, graspable, sensible and approachable for designers. The hope is to inspire design researchers, students and practitioners to discover and value haptics as a core component of any interaction design activities.

The integration of haptics into virtual environments has triggered a new era by allowing interaction with virtual objects through force feedback in a number of fields. Medicine has been the field with highest potential benefit through improved realism and immersion. Not only have virtual environments become superior to traditional medical training methods due to cost-efficiency, repeatability, and objective assessment but the idea of surgery rehearsal by using patient specific data has been raised as well. Achieving sufficient realism in haptics has been a significant challenge due to performance requirements. In order to provide a stable and smooth feedback to the user, the update rates of force feedback need to be in the range of 1~kHz, which restricts the solution time for real-time interactive applications. Realism, on the other hand, demands advanced algorithms capable of simulating physical properties. These advanced algorithms have a high computational burden, taking significant amounts of time and their real-time use, therefore, mostly requires simplification of the virtual scene affecting realism. During palpation, information is transferred to the hand from the local neighbourhood of contact. In deformation simulations, it is therefore common to use a multiresolution scheme, where the local region is modelled with a higher resolution than more distant regions, and at higher update rates. This approach saves computational power, however the less elaborate modelling in the more remote regions affects accuracy. This thesis presents a pipeline to analyse the error introduced by multiresolution techniques. The idea is to estimate how simulation parameters lead to different error magnitudes, as a preprocessing step. This information can subsequently be used for monitoring the error in real-time, or for adjusting simulation parameters to keep the error under a desired limit. There is a trade-off between accuracy/error and computation time required. In an ideal situation, this error should be kept under perceivable levels. Levels of perception is a topic that has been surveyed in psychophysics among other aspects of touch. It has been shown that differences smaller than a ratio of a reference signal, such as force or stiffness, cannot be perceived. Evaluating the exact value of this ratio, however, is nontrivial since there are many secondary factors having a significant impact, such as the multimodal input. This thesis presents the analysis of some factors affecting the sense of touch that were shown to have such impact. Effects of exploratory procedures on stiffness perception were examined through user studies, followed by another study indicating the significant effects of stiffness gradient. Medical data, such as MR and CT, has much higher resolution than is practically used for deformable meshes. It has been common practice to model deformation behaviour by a mesh with lower resolution than is used for visual representation. Lastly, this thesis presents an approach to introduce high-resolution information. The proposed algorithm allows for the detection of inhomogeneous structures beneath a surface. This can be applied in situations similar to the diagnosis of tumours by palpation. The approach is independent of mesh structure and resolution, and can be integrated into any proxybased haptic rendering algorithm. This makes the algorithm a complementary choice for deformation simulation.

The haptic exploration of 2-D images is a challenging problem in computer haptics. Research on the topic has primarily been focused on the exploration of maps and curves. This thesis describes the design and implementation of a system for the haptic exploration of photographs. The system builds on various research directions related to assistive technology, computer haptics, and image segmentation. An object-level segmentation hierarchy is generated from the source photograph to be rendered haptically as a contour image at multiple levels-of-detail. A tool for the authoring of object-level hierarchies was developed, as well as an innovative type of user interaction by region selection for accurate and efficient image segmentation. According to an objective benchmark measuring how the new method compares with other interactive image segmentation algorithms shows that our region selection interaction is a viable alternative to marker-based interaction. The hierarchy authoring tool combined with precise algorithms for image segmentation can build contour images of the quality necessary for the images to be understood by touch with our system. The system was evaluated with a user study of 24 sighted participants divided in different groups. The first part of the study had participants explore images using haptics and answer questions about them. The second part of the study asked the participants to identify images visually after haptic exploration. Results show that using a segmentation hierarchy supporting multiple levels-of-detail of the same image is beneficial to haptic exploration. As the system gains maturity, it is our goal to make it available to blind users.

With a spatial haptic interface device and a suitable haptic rendering algorithm, users can explore and modify virtual geometries in three dimensions with the aid of their haptic (touch) sense. Designers of surgery simulators, anatomy exploration tools and applications that involve assembly of complex objects should consider employing this technology. However, in order to know how the technology behaves as a design material, the designer needs to become well acquainted with its material properties. This presents a significant challenge today, since the haptic devices are presented as black boxes, and implementation of advanced rendering algorithms represent highly specialized and time consuming development activities. In addition, it is difficult to imagine what an interface will feel like until it has been fully implemented, and important design trade-offs such as the virtual object's size and stability gets neglected. Traditional user-centered design can be interpreted as that the purpose of the field study phase is to generate a set of specifications for an interface, and only solutions that cover these specifications will be considered in the design phase. The designer might miss opportunities to create solutions that uses e.g. lower cost devices since that might require reinterpretation of the overarching goal of the situation with starting point in the technical possibilities, which is unlikely without significant material knowledge. As an example, a surgery simulator designed in this thesis required a high cost haptic device to render adequate forces on the scale of human teeth, but if the design goal is reinterpreted as creating a tool for learning anatomical differences and surgical steps, an application more suitable for the lower cost haptic devices could be crafted. This solution is as much informed by the haptic material "speaking back to" the designer as by field studies. This licentiate thesis will approach a perspective of spatial haptic interface design that is grounded in contemporary design theory. These theories emphasizes the role of the designer, who is not seen as an objective actor but as someone who has a desire to transform a situation into a preferred one as a service to a client or greater society. It also emphasizes the need for crafting skills in order to innovate, i.e. make designed objects real. Further, it considers aesthetic aspects of a design, which includes the subtle differences in friction as you move the device handle, and overall attractiveness of the device and system. The thesis will cover a number of design cases which will be related to design theory and reflected upon. Particular focus will be placed on the most common class of haptic devices which can give force feedback in three dimensions and give input in six (position and orientation). Forces will be computed and objects deformed by an volume sampling algorithm which will be discussed. Important design properties such as stiffness, have been identified and exposed as a material for design. A tool for tuning these properties interactively has been developed to assist designers to become acquainted with the spatial haptic material and to craft the material for a particular user experience. Looking forward, the thesis suggests the future work of making spatial haptic interfaces more design ready, both in software and hardware. This is proposed to be accomplished through development of toolkits for innovation which encapsulate complexities and exposes design parameters. A particular focus will be placed on enabling crafting with the haptic material whose natural limitations should be seen as suggestions rather than hinders for creating valuable solutions.

QC 20130916

En Réalité Virtuelle (RV), le sens haptique accroît l’immersion d’un utilisateur dans un Environnement Virtuel (EV) avec lequel il interagit en temps réel. Dans cette thèse, nous proposons des approches pour améliorer l’interaction haptique à deux mains avec des EV. Nous abordons d’abord des problèmes avec l’interaction bimanuelle dans des EV avec des interfaces à effecteur unique. Nous proposons une technique d’interaction nommée la double bulle pour l’exploration d’EV avec une combinaison de contrôle en position et en vitesse. Nous présentons aussi une technique de manipulation nommée prise magnétique qui facilite la saisie d’objets virtuels avec des proxys rigides simples. Des modes de contrôle communs sont utilisés pour améliorer la saisie et l’exploration simultanées. Une évaluation utilisateur a été réalisée pour mesurer l’efficacité de ces techniques. Nous nous intéressons ensuite au calcul de surfaces de contact. Nous proposons une technique nommée god-finger pour rendre des surfaces similaires à celles générées par des doigts à partir d’un unique point de contact. Elle est basée sur un simple parcours de la géométrie locale de l’objet en contact, et est donc moins coûteuse que des méthodes de simulation de corps souples. La méthode est adaptée pour l’interaction avec des proxys rigides simples ou plus complexes, ainsi qu’avec des objets rigides ou déformables, y compris avec des surfaces rugueuses. Une méthode de rendu visuel donne un retour à l’utilisateur sur la forme de la surface de contact. Enfin, nous abordons la résolution de contacts durant la manipulation dextre d’objets virtuels avec des doigts souples. Le calcul des mécaniques de contact est amélioré en agrégeant les multiples contraintes de contact concernées. Une méthode de distribution de pression non uniforme sur la surface de contact adapte la réponse lors d’un contact contre des arêtes pointues. Nous utilisons le modèle de frottement de Coulomb- Contensou pour simuler efficacement le frottement en torsion. L’approche est évaluée avec un modèle de main déformable pour de l’interaction en temps réel. Les travaux présentés dans ce manuscrit ouvrent de nouvelles perspectives dans le contexte de l’haptique bimanuelle et de la RV, en permettant une interaction plus naturelle avec des EV plus complexes
In Virtual Reality (VR), the haptic sense increases the immersion of users in a Virtual Environment (VE) with which they interact in real time. In this Ph.D thesis, we propose contributions to improve two-handed interaction in haptics with VEs. We first address issues with bimanual interaction in VEs using haptic devices with single effectors. We propose an interaction technique called double bubble for exploration of VEs through a combination of position and rate control. We also present a manipulation technique called magnetic pinch which facilitates the grasping of virtual objects with simple rigid proxies. Simultaneous grasping and exploration of the VE is enhanced using common control modes. A user evaluation was conducted to assess the efficiency of these techniques. We then focus on improving the computation of contact surfaces. We propose a god-finger method to render finger padlike surfaces from a single contact point. It relies on a simple scan of the local geometry of the object in contact, and is this less costly than soft body simulation methods. The method is adapted for interaction using simple or more complex rigid virtual proxies, and with rigid or deformable objects, including rough surfaces. A visual rendering method provides feedback on the shape of the contact surface. Finally, we address the resolution of contacts during dexterous manipulation of virtual objects through soft fingers. The computation of contact mechanics is improved by aggregating the multiple contact constraints involved. A method for nonuniform pressure distribution over the contact surface adapts the response when touching sharp edges. We use the Coulomb-Contensou friction model to efficiently simulate torsional friction. The approach is evaluated with a deformable hand model for real time interaction. The contributions of this manuscript open novel perspectives in the context of bimanual haptics and VR, allowing more natural interaction with more complex VEs

Consumers evaluate products in the market place using their senses and often form mental representations of product properties. These mental representations have been studied extensively. Imagery has been shown to interact with perception within many perceptual modalities including vision, auditory, olfactory, and motor. This dissertation draws on the vast visual imagery literature to examine imagery in the haptic, or touch, modality. Two studies were undertaken to examine the relationship between haptic imagery and haptic perception The first study is based on studies from cognitive psychology that have used similar methods for examining visual imagery and visual perception. In study 1, sighted and visually impaired participants were asked to evaluate objects haptically, to form a haptic image of that object during a short interval, and then to compare the haptic image to a second object. In Study 2, sighted and visually impaired participants listened to five radio advertisements containing imagery phrases from multiple modalities. After listening to the advertisements, participants were asked to recall the ad content and assess both the ad and the product while haptically evaluating the product in the ad. Though results were mixed and further exploration will be necessary, these studies offer broad implications for consumer use of haptic imagery in shopping environments. The implications for both sighted and blind consumers are discussed.

The present document exposes a different approach for haptic rendering, defined as the simulation of force interactions to reproduce the sensation of surface relief in dense models. Current research shows open issues in timely haptic interaction involving large meshes, with several problems affecting performance and fidelity, and without a dominant technique to treat these issues properly. Relying in pure geometric collisions when rendering highly dense mesh models (hundreds of thousands of triangles) sensibly degrades haptic rates due to the sheer number of collisions that must be tracked between the mesh's faces and a haptic probe. Several bottlenecks were identified in order to enhance haptic performance: software architecture and data structures, collision detection, and accurate rendering of surface relief. To account for overall software architecture and data structures, it was derived a complete component framework for transforming standalone VR applications into full-fledged multi-threaded Collaborative Virtual Reality Environments (CVREs), after characterizing existing implementations into a feature-rich superset. Enhancements include: a scalable arbitrated peer-to-peer topology for scene sharing; multi-threaded components for graphics rendering, user interaction and network communications; a collaborative user interface model for session handling; and interchangeable user roles with multi-camera perspectives, avatar awareness and shared annotations. We validate the framework by converting the existing ALICE VR Navigator into a complete CVRE, showing good performance in collaborative manipulation of complex models. To specifically address collision detection computation, we derive a conformal algebra treatment for collisions among points, segments, areas, and volumes, based on collision detection in conformal R{4,1} (5D) space, and implemented in GPU for faster parallel queries. Results show orders of magnitude time reductions in collisions computations, allowing interactive rates. Finally, the main core of the research is the haptic rendering of surface mesostructure in large meshes. Initially, a method for surface haptic rendering was proposed, using image-based Hybrid Rugosity Mesostructures (HRMs) of per-face heightfield displacements and normalmaps layered on top of a simpler mesh, adding greater surface detail than actually present. Haptic perception is achieved modulating the haptic probe's force response using the HRM coat. A usability testbed framework was built to measure experimental performance with a common set tests, meshes and HRMs. Trial results show the goodness of the proposed technique, rendering accurate 3D surface detail at high sampling rates. This local per-face method is extended into a fast global approach for haptic rendering, building a mesostructure-based atlas of depth/normal textures (HyRMA), computed out of surface differences of the same mesh object at two different resolutions: original and simplified. For each triangle in the simplified mesh, an irregular prism is considered defined by the triangle's vertices and their normals. This prism completely covers the original mesh relief over the triangle. Depth distances and surfaces normals within each prism are warped from object volume space to orthogonal tangent space, by means of a novel and fast method for computing barycentric coordinates at the prism, and storing normals and relief in a sorted atlas. Haptic rendering is effected by colliding the probe against the atlas, and effecting a modulated force response at the haptic probe. The method is validated numerically, statistically and perceptually in user testing controlled trials, achieving accurate haptic sensation of large meshes' fine features at interactive rendering rates, with some minute loss of mesostructure detail.
En aquesta tesi es presenta un novedós enfocament per a la percepció hàptica del relleu de models virtuals complexes mitjançant la simulació de les forces d'interacció entre la superfície i un element de contacte. La proposta contribueix a l'estat de l'art de la recerca en aquesta àrea incrementant l'eficiència i la fidelitat de la interacció hàptica amb grans malles de triangles. La detecció de col·lisions amb malles denses (centenars de milers de triangles) limita la velocitat de resposta hàptica degut al gran nombre d'avaluacions d'intersecció cara-dispositiu hàptic que s'han de realitzar. Es van identificar diferents alternatives per a incrementar el rendiment hàptic: arquitectures de software i estructures de dades específiques, algorismes de detecció de col·lisions i reproducció hàptica de relleu superficial. En aquesta tesi es presenten contribucions en alguns d'aquests aspectes. S'ha proposat una estructura completa de components per a transformar aplicacions de Realitat Virtual en Ambients Col·laboratius de Realitat Virtual (CRVEs) multithread en xarxa. L'arquitectura proposada inclou: una topologia escalable punt a punt per a compartir escenes; components multithread per a visualització gràfica, interacció amb usuaris i comunicació en xarxa; un model d'interfície d'usuari col·laboratiu per a la gestió de sessions; i rols intercanviables de l'usuari amb perspectives de múltiples càmeres, presència d'avatars i anotacions compartides. L'estructura s'ha validat convertint el navegador ALICE en un CVRE completament funcional, mostrant un bon rendiment en la manipulació col·laborativa de models complexes. Per a incrementar l'eficiència del càlcul de col·lisions, s'ha proposat un algorisme que treballa en un espai conforme R{4,1} (5D) que permet detectar col·lisions entre punts, segments, triangles i volums. Aquest algorisme s'ha implementat en GPU per obtenir una execució paral·lela més ràpida. Els resultats mostren reduccions en el temps de càlcul de col·lisions permetent interactivitat. Per a la percepció hàptica de malles complexes que modelen objectes rugosos, s'han proposat diferents algorismes i estructures de dades. Les denominades Mesoestructures Híbrides de Rugositat (HRM) permeten substituir els detalls geomètrics d'una cara (rugositats) per dues textures: de normals i d'alçades. La percepció hàptica s'aconsegueix modulant la força de resposta entre el dispositiu hàptic i la HRM. Els tests per avaluar experimentalment l'eficiència del càlcul de col·lisions i la percepció hàptica utilitzant HRM respecte a modelar les rugositats amb geometria, van mostrar que la tècnica proposada va ser encertada, permetent percebre detalls 3D correctes a altes tases de mostreig. El mètode es va estendre per a representar rugositats d'objectes. Es proposa substituir l'objecte per un model simplificat i un atles de mesoestructures en el que s'usen textures de normals i de relleus (HyRMA). Aquest atles s'obté a partir de la diferència en el detall de la superfície entre dos malles del mateix objecte: l'original i la simplificada. A partir d'un triangle de la malla simplificada es construeix un prisma, definit pels vèrtexs del triangle i les seves normals, que engloba el relleu de la malla original sobre el triangle. Les alçades i normals dins del prisma es transformen des de l'espai de volum a l'espai ortogonal tangent, amb mètode novedós i eficient que calcula les coordenades baricèntriques relatives al prisma, per a guardar el mapa de textures transformat en un atles ordenat. La percepció hàptica s'assoleix detectant les col·lisions entre el dispositiu hàptic i l'atles, i modulant la força de resposta d'acord al resultat de la col·lisió. El mètode s'ha validat numèricament, estadística i perceptual en tests amb usuaris, aconseguint una correcta i interactiva sensació tàctil dels objectes simulats mitjançant la mesoestructura de les malles
En esta tesis se presenta un enfoque novedoso para la percepción háptica del relieve de modelos virtuales complejos mediante la simulación de las fuerzas de interacción entre la superficie y un elemento de contacto. La propuesta contribuye al estado del arte de investigación en este área incrementando la eficiencia y fidelidad de interacción háptica con grandes mallas de triángulos. La detección de colisiones con mallas geométricas densas (cientos de miles de triángulos) limita la velocidad de respuesta háptica debido al elevado número de evaluaciones de intersección cara-dispositivo háptico que deben realizarse. Se identificaron diferentes alternativas para incrementar el rendimiento háptico: arquitecturas de software y estructuras de datos específicas, algoritmos de detección de colisiones y reproducción háptica de relieve superficial. En esta tesis se presentan contribuciones en algunos de estos aspectos. Se ha propuesto una estructura completa de componentes para transformar aplicaciones aisladas de Realidad Virtual en Ambientes Colaborativos de Realidad Virtual (CRVEs) multithread en red. La arquitectura propuesta incluye: una topología escalable punto a punto para compartir escenas; componentes multithread para visualización gráfica, interacción con usuarios y comunicación en red; un modelo de interfaz de usuario colaborativo para la gestión de sesiones; y roles intercambiables del usuario con perspectivas de múltiples cámaras, presencia de avatares y anotaciones compartidas. La estructura se ha validado convirtiendo el navegador ALICE en un CVRE completamente funcional, mostrando un buen rendimiento en la manipulación colaborativa de modelos complejos. Para incrementar la eficiencia del cálculo de colisiones, se ha propuesto un algoritmo que trabaja en un espacio conforme R4,1 (5D) que permite detectar colisiones entre puntos, segmentos, triángulos y volúmenes. Este algoritmo se ha implementado en GPU a efectos de obtener una ejecución paralelamás rápida. Los resultadosmuestran reducciones en el tiempo de cálculo de colisiones permitiendo respuesta interactiva. Para la percepción háptica de mallas complejas que modelan objetos rugosos, se han propuesto diferentes algoritmos y estructuras de datos. Las denominadasMesoestructuras Híbridas de Rugosidad (HRM) permiten substituir los detalles geométricos de una cara (rugosidades) por una textura de normales y otra de alturas. La percepción háptica se consigue modulando la fuerza de respuesta entre el dispositivo háptico y la HRM. Los tests realizados para evaluar experimentalmente la eficiencia del cálculo de colisiones y la percepción háptica utilizando HRM respecto a modelar las rugosidades con geometría, mostraron que la técnica propuesta fue acertada, permitiendo percibir detalles 3D correctos a altas tasas de muestreo. Este método anterior es extendido a un procedimiento global para representar rugosidades de objetos. Para hacerlo se propone sustituir el objeto por un modelo simplificado y un atlas de mesostructuras usando texturas de normales y relieves (HyRMA). Este atlas se obtiene de la diferencia en detalle de superficie entre dos mallas del mismo objeto: la original y la simplificada. A partir de un triángulo de la malla simplificada se construye un prisma definido por los vértices del triángulo a lo largo de sus normales, que engloba completamente el relieve de la malla original sobre este triángulo. Las alturas y normales dentro de cada prisma se transforman del espacio de volumen al espacio ortoganal tangente, usando un método novedoso y eficiente que calcula las coordenadas baricéntricas relativas a cada prisma para guardar el mapa de texturas transformado en un atlas ordenado. La percepción háptica se consigue detectando directamente las colisiones entre el dispositivo háptico y el atlas, y modulando la fuerza de respuesta de acuerdo al resultado de la colisión. El procedmiento se ha validado numérica, estadística y perceptualmente en ensayos con usuarios, consiguiendo a tasas interactivas la correcta sensación táctil de los objetos simulados mediante la mesoestructura de las mallas, con alguna pérdida muy puntual de detalle

As a solution to mid-air haptic actuation with strong and continuous tactile force, Magnetic Rendering is presented as an intuitive haptic display method applying an electromagnet array to produce a magnetic field in mid-air where the force field can be felt as magnetic repulsive force exerted on the hand through the attached magnet discs. The magnetic field is generated by a specifically designed electromagnet array driven by direct current. By attaching small magnet discs on the hand, the tactile sensation can be perceived by the user. This method can provide a strong tactile force on multiple points covering user’s hand and avoid cumbersome attachments with wires, thus it is suitable for a co-located visual and haptic display. In my work, the detailed design of the electromagnet array for haptic rendering purposes is introduced, which is modelled and tested using Finite Element Method simulations. The model is characterized mathematically, and three methods for controlling the magnetic field are applied accordingly: direct control, system identification and adaptive control. The performance of the simulated model is evaluated in terms of magnetic field distribution, force strength, operation distance and force stiffness. The control algorithms are implemented and tested on a 3-by-3 and a 15-by-15 model, respectively. Simulations are performed on a 15-by-15 model to generate a haptic human face, which results in a smooth force field and accurate force exertion on the control points.

Artistic painting involves mastery of haptic interaction with tools. Each tool brings unique physical affordances which determines an aesthetic expression of the finished work. For instance, a pen offers an ability to make a precise stroke in a realism painting, whereas a thick brush or a sponge works perfectly with dynamic arm movement in the abstract art such as action painting. Yet the selection of a tool is just a beginning. It requires repetitive training to understand the full capability of the tool affordance and to master the painting of preferred aesthetic strokes. Such physical act of an artistic expression cannot be captured by the computational tools today. Due to the increasing market adoption of augmented reality and virtual reality, and the decades of studies in haptics, we see an opportunity for advancing 3D painting experiences in non-conventional approach. In this research, we focus on force haptic interaction for 3D painting art in a room-scale virtual reality. We explore virtual tangibility and tool affordance of its own medium. In addition to investigating the fidelity of a physical interactivity, we seek ways to extend the painting capabilities by computationally customized force feedback and metaphor design. This system consists of a wearable force feedback device that sits on user’s hand, a software for motor control and real-time 3D stroke generation, and their integration to VR platform. We work closely with an artist to refine the 3D painting application and to evaluate the system’s usability.

Thesis (M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1999.
Includes bibliographical references (p. 87-88).
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
In this thesis, I describe the design and implementation of a system integrating a force-reflecting haptic interface, the PHANToM, with a high-powered computer designed to efficiently compute cellular automata, the CAM-8. My goal was to build a system to allow users to interact with three-dimensional bitmapped simulations running on the CAM. Such simulations are far more computationally complex than can currently be computed on the PHANToM host. This system can also provide an intuitive user interface for manipulating data within CAM simulations. Methods for coping with network and computational latencies are described, and example applications are evaluated to explore the effectiveness of the system.
M.Eng.

Search and rescue operations are often undertaken in dark and noisy environment in which rescue team must rely on haptic feedback for exploration and safe exit. However, little attention has been paid specifically to haptic sensitivity in such contexts or the possibility of enhancing communicational proficiency in the haptic mode as a life-preserving measure. The potential of root swarms for search and rescue has been shown by the Guardians project (EU, 2006-2010); however the project also showed the problem of human robot interaction in smoky (non-visibility) and noisy conditions. The REINS project (UK, 2011-2015) focused on human robot interaction in such conditions. This research is a body of work (done as a part of he REINS project) which investigates the haptic interaction of a person wit a guide robot in zero visibility. The thesis firstly reflects upon real world scenarios where people make use of the haptic sense to interact in zero visibility (such as interaction among firefighters and symbiotic relationship between visually impaired people and guide dogs). In addition, it reflects on the sensitivity and trainability of the haptic sense, to be used for the interaction. The thesis presents an analysis and evaluation of the design of a physical interface (Designed by the consortium of the REINS project) connecting the human and the robotic guide in poor visibility conditions. Finally, it lays a foundation for the design of test cases to evaluate human robot haptic interaction, taking into consideration the two aspects of the interaction, namely locomotion guidance and environmental exploration.

The sense of touch is important in our everyday lives and its absence makes it difficult to explore and manipulate everyday objects. Existing online shopping practice lacks the opportunity for physical evaluation, that people often use and value when making product choices. However, with recent advances in haptic research and technology, it is possible to simulate various physical properties such as heaviness, softness, deformation, and temperature. The research described here investigates the use of haptic feedback interaction to enhance e-commerce product evaluation, particularly haptic weight and texture evaluation. While other properties are equally important, besides being fundamental to the shopping experience of many online products, weight and texture can be simulated using cost-effective devices. Two initial psychophysical experiments were conducted using free motion haptic exploration in order to more closely resemble conventional shopping. One experiment was to measure weight force thresholds and another to measure texture force thresholds. The measurements can provide better understanding of haptic device limitation for online shopping in terms of the availability of different stimuli to represent physical products. The outcomes of the initial psychophysical experimental studies were then used to produce various absolute stimuli that were used in a comparative experimental study to evaluate user experience of haptic product evaluation. Although free haptic exploration was exercised on both psychophysical experiments, results were relatively consistent with previous work on haptic discrimination. The threshold for weight force discrimination represented as downward forces was 10 percent. The threshold for texture force discrimination represented as friction forces was 14.1 percent, when using dynamic coefficient of friction at any level of static coefficient of friction. On the other hand, the comparative experimental study to evaluate user experience of haptic product information indicated that haptic product evaluation does not change user performance significantly. However, although there was an increase in the time taken to complete the task, the number of button click actions tended to decrease. The results showed that haptic product evaluation could significantly increase the confidence of shopping decision. Nevertheless, the availability of haptic product evaluation does not necessarily impose different product choices but it complements other selection criteria such as price and appearance. The research findings from this work are a first step towards exploring haptic-based environments in e-commerce environments. The findings not only lay the foundation for designing online haptic shopping but also provide empirical support to research in this direction.

This research explores how haptic feedback can increase the usefulness of microinteractions on touchscreen devices. With the importance of microinteraction in user daily lives and with the increase of availability of haptic generating devices, this thesis studies how a realistic user group perceive haptic feedback in microinteractions. It studied the perceived usefulness of haptic feedback in correlation to specific microinteractions. Microinteractions are defined as small actions responding to user interaction, examples tested in this study included, simple buttons, radio buttons, checkboxes and dropdown menus. The participants also test and evaluated generic haptic patterns currently available on iOS devices. A software prototype was produced in a prototype tool called Protopie and then tested on an iOS device (iPhone 11 pro). This study is of qualitative character meaning it relies on qualitative data. The collection method relied on a combination of two types of research methods, usability studies and in-depth interviews. Nine participants were interviewed and studied using these methods. This data was analysed using a qualitative analysis method called Applied Thematic Analysis (ATA). This method allows for the creation of themes to be used in the final discussion of the report. In this thesis, a number of four themes were found. These themes combined with previous haptic principles serve as the academic foundation on which answers to the research questions was made. The results show a general increase in the perceived usefulness of tested microinteractions. With microinteractions such as checkboxes and radio buttons benefitting greatly by the haptic enhancement.[A1]  The conclusion of the research found several future areas of research within HaXD.   [A1]Here are the results shortly describe as in previous version. Enough?

New Virtual Reality technologies provide the possibility of widening access to information in data. Haptics, the technology of touch, could be an interesting future aid and have large impact on medical applications. The use of haptic devices allows computer users to use their sense of touch, in order to feel virtual objects with a high degree of realism.

The aim of the thesis is to investigate the potential deployment and the benefits of using haptic force feedback instruments in maxillo-facial surgery. Based on a produced test application, the thesis includes suggested recommendations for future haptic implementations.

At the Department of Maxillo-Facial Surgery, at the Karolinska Hospital in Stockholm, Virtual Reality technologies are used as an aid to a limited extent during the production of physical medical models. The physical medical models are produced with Rapid Prototyping techniques. This process is examined and described in the thesis. Moreover, the future of the physical medical models is outlined, and a future alternative visualizing patient data in 3D and use haptics as an interaction tool, is described. Furthermore, we have examined the present use of haptic technology in medicine, and the benefits of using the technology as an aid for diagnostic and treatment planning.

Based on a presented literature study and an international outlook, we found that haptics could improve the management of medical models. The technology could be an aid, both for physical models as well as for virtual models. We found three different ways of implementing haptics in maxillo-facial surgery. A haptic system could be developed in order to only manage virtual medical models and be an alternative solution to the complete Rapid Prototyping process. A haptic system could serve as a software, handling the image processing and interfacing from a medical scanner to an Rapid Prototyping system. A haptic system could be developed as an alternative interaction tool, which could be implemented as an additional function in currently used image processing software, in order to improve the management of virtual medical models before the Rapid Prototyping process.

An implementation for planning and examination in maxillo-facial surgery, using haptic force feedback interaction, is developed and evaluated. The test implementation is underlying our aim of investigating the potential deployment and the benefits of using haptic force feedback instruments in maxillo-facial surgery.

After discussing the possible future of our implementation and the future of haptic force feedback in maxillo-facial surgery, a recommendation is given as a conclusion of our total work.

The purpose of this report is to cover the procedure of creating and explaining how to use a tool kit that allows the haptic Application Programming Interface (API) H3D from SenseGraphics to be used in conjunction with an advanced physics simulator from Meqon. Both haptic applications and physics engines have developed rapidly the last couple of years but they are rarely used together. If such a connection would be created it would be possible to interact with complex environments in a new way and a variety of haptic applications can be produced.

The physics engine from Meqon has gained recognition for its abilities to produce realistic results due to efficient implementation of collision detection system, friction models and collision handling, among other things. H3D is a completely open source API that is based on standards such as OpenGL and X3D. H3D consists of a data base containing nodes, an XML parser to extract a scene graph from the data base and functionality to produce a graphic and haptic interface.

The tool kit produced in this thesis is an extension to H3D. A fundamental function of the tool kit is to communicate with the Meqon system and still be a part of the H3D structure. The Meqon system has a modular structure where each module has its own abilities. Only the rigid body module is utilised by the tool kit, which however is the most important module. It is possible to define global settings of the engine and rigid body module, add rigid bodies with several elements and insert constraints on the motion of the rigid bodies into the engine. All of these operations are done from the X3D file format that H3D uses, thus letting all functionality of the H3D system available.

Emotion communication is an important aspect of social interaction. Affect display research from psychology as well as social human-robot interaction has focused primarily on facial or vocal behaviors, as these are the predominant means of expression for humans. Much less attention, however, has been on emotion communication through touch, which, though unique among the senses, can be methodologically and technologically difficult to study. Our thesis investigated the role of affective touch in the social interaction between human and robot. Through a process of design and controlled user evaluation, we examined the display, recognition, and emotional influence of affective touch. To mitigate issues inherent in touch research, we drew from interaction models not between humans but between human and animal, whereby the robot assumes the role of companion animal. We developed the Haptic Creature, a small, zoomorphic robot novel in its sole focus on touch for both affect sensing and display. The robot perceives movement and touch, and it expresses emotions through ear stiffness, modulated breathing, and vibrotactile purring. The Haptic Creature was employed in three user studies, each exploring a different aspect of affective touch interaction. Our first study examined emotion display from the robot. We detail the design of the Haptic Creature's affect display, which originated from animal models, then was enhanced through successive piloting. A formal study demonstrated the robot was more successful communicating arousal than valence. Our second study investigated affect display from the human. We compiled a touch dictionary from psychology and human-animal interaction research. Participants first rated the likelihood of using these touch gestures when expressing a variety of emotions, then performed likely gestures communicating specific emotions for the Haptic Creature. Results provided properties of human affect display through touch and high-level categorization of intent. Our final study explored the influence of affective touch. Results empirically demonstrated the human's emotional state was directly influenced from affective touch interactions with the robot. Our research has direct significance to the field of socially interactive robotics and, further, any domain interested in human use of affective touch: psychology, mediated social touch, human-animal interaction.

Les interactions tactiles avec les écrans tactiles tels que les smartphones et les tablettes sont devenues de plus en plus omniprésentes dans notre vie quotidienne. Ces dispositifs tactiles commerciaux fournissent rarement un retour haptique convaincant aux doigts humains malgré l'utilisation du toucher comme entrée principale; le retour haptique est généralement limité à la vibration. Par conséquent, différentes technologies ont été explorées pour générer un retour haptique dynamique afin d'améliorer la saisie sur les appareils à écran tactile. Dans cette thèse, nous nous intéressons plus particulièrement à une catégorie de retour haptique qui exploite les vibrations ultrasoniques pour créer un film d'air entre le doigt d'un utilisateur et l'écran pour réduire la friction lorsqu'il est activé, ce phénomène est appelé effet squeeze film. En effet, la perception tactile de l'utilisateur joue un rôle crucial dans l'interaction avec les dispositifs haptiques. Dans cette thèse, nous explorons d'abord la limitation de la perception tactile des doigts de l'utilisateur sur les écrans tactiles haptiques ultrasoniques pour des explorations tactiles à un doigt et à plusieurs doigts au moyen d'expériences psychophysiques. Nous proposons ensuite un nouveau concept, appelé taxel, concernant la perception par l'utilisateur des éléments tactiles sur les écrans tactiles haptiques à ultrasons. En outre, nous décrivons comment optimiser les performances d'interaction de l'utilisateur dans des tâches d'interaction communes en utilisant les vibrations ultrasoniques. Enfin, nous étudions comment le signal tactile peut être combiné avec des signaux auditifs pour améliorer la perception de l'utilisateur dans les interactions musicales
Touch interactions with tactile displays such as smart-phones and tablets, have become more and more ubiquitous in our daily life. These commercial touchscreen devices rarely provide a compelling haptic feedback to human fingers despite the use of touch as primary input; haptic feedback is typically limited to vibration. Therefore, different technologies have been explored to generate dynamic haptic feedback to enhance input on touchscreen devices. In this dissertation, we are are particularly interested in a category of haptic feedback which leverages ultrasonic vibrations to create an air-gap between a user's finger and the display to reduce friction when activated, a phenomenon called the squeeze film effect. Indeed, user's tactile perception plays a crucial role for interacting with haptic displays. In this thesis, we first explore user's fingers limitation of tactile perception on ultrasonic haptic displays for both one-finger and multi-finger touch explorations by means of psychophysical experiments. We then propose a novel concept, called taxel concerning user's perception of tactile elements on ultrasonic haptic touchscreens. Furthermore, we describe how to optimize user's interaction performances in common interaction tasks by leveraging ultrasonic lubrication. Finally, we study how tactile signal can be combined with auditory signals to enhance user's perception in musical interactions

Real-time point-based rendering and interaction with virtual objects is gaining popularity and importance as different haptic devices and technologies increasingly provide the basis for realistic interaction. Haptic Interaction is being used for a wide range of applications such as medical training, remote robot operators, tactile displays and video games. Virtual object visualization and interaction using haptic devices is the main focus; this process involves several steps such as: Data Acquisition, Graphic Rendering, Haptic Interaction and Data Modification. This work presents a framework for Haptic Interaction using the GPU as a hardware accelerator, and includes an approach for enabling the modification of data during interaction. The results demonstrate the limits and capabilities of these techniques in the context of volume rendering for haptic applications. Also, the use of dynamic parallelism as a technique to scale the number of threads needed from the accelerator according to the interaction requirements is studied allowing the editing of data sets of up to one million points at interactive haptic frame rates.

This thesis investigates the assumed affordance of haptic technology in a wearable context. This work position itself within the internet of things where wearable and connected objects have established themselves as a sub-domain in the area. Some argued that haptic technologies provide certain benefits when creating interfaces for wearable technologies. However, their seams to be a lack of studies investigating the natural responses to haptic technology from a user perspective. To investigate the assumed benefits of haptic technology, we have developed a prototype of a wearable haptic system. This system consists of two jacket were each jacket contains 16 touch sensitive sensor and haptic actuators. All interactions with the sensor of one jacket are mirrored with a haptic feedback in the other jacket. The purpose of this system is to investigate the initial responses and accord of haptic technology by users. The reason for doing this is to develop guidelines for further investigation into haptic technology as a means for creating wearable objects with a non-screen based interfaces.The study concludes there to be some merit to the use of rhythm for creating haptic interaction patterns as well as haptic technology affording a simple and natural dialog in human computer interaction. Further, we also conclude that there is some merit to haptic affording faster learning curves in novel non-screen based interaction and are suitable for providing clear feedback. This study also shows limitations for haptic technology. Haptics seem less proficient for interfaces existing in a system as well as handling errors. This study also proved problematic because of the lack in clearly defined methods for investigating novel non-screen based interfaces.In chapter two of this thesis we first present a theoretical overview of wearable’s generally and how haptic technology position itself within wearable technology. In chapter three, we move on by introducing our methods of research. Based on our theory we then frame the technological outline of the project and practical implementation. We follow this by presenting the results in chapter five from our user testing conducted with the prototype. We end our thesis with a discussion in chapter six with presenting the principal findings together with our discussion and future developments in chapter seven.

De nombreux travaux ont montré que l’utilisation de systèmes de guidage haptique pouvait être utile pour l’apprentissage de l'écriture manuscrite. Au cours de cette thése, quatre expériences ont été construites afin de trouver la meilleure façon d’utiliser le guidage haptique pour apprendre l'écriture chinoise. Nous avons utilisé un bras robotique avec un retour d'effort contrôlé par un logiciel afin de montrer aux utilisateurs le bon mouvement à produire. Ces résultats de ces quatre expériences suggèrent que le guidage visuo-haptique basé sur une méthode d'apprentissage passive et successive est bénéfique à l’apprentissage de l’écriture chinoise pour les débutants. Il s'agit de la première étape de nos travaux. Nous envisageons à présent de poursuive ces travaux avec un plus grand nombre de participants, plus de temps de formation, et en utilisant des caractères chinois plus complexes
It has been confirmed in lots of researches that haptic guidance systems can help users learning handwriting. In our study, four experiments have been undertaken respectively so that a good method can be designed in using haptic guidance to learn Chinese handwriting. A force-feedback arm controlled by software has been used in the experiments to show the correct movement to produce. Five parameters were used to assess Chinese handwriting: shape and size of the Chinese character, movement average speed, inair time (the pause time during writing each strokes of the character) and order of each stroke of the character. Results of these four experiments suggest that visual-haptic guidance under a passive and successive learning method is good for teaching beginners to learn Chinese handwriting. This is the first step of our study and larger number of other participants, more training time, and more complex Chinese characters will be used in future work

Information Visualisation aims to provide a means for the easy understanding of data through pictures. However, using the visual sense alone may limit the amount of information that can be understood. Haptics is the science of applying touch sensations to human interaction with computers. The addition of haptics provides more ways in which the user can understand the data. We define a visual-haptic information system to be any system that uses haptic and visual feedback to convey information. There are many domains where the data has some attributes which are spatially referenced to the earth. Visualisation of data-sets with geographic attributes often use a map as the background and overlay visualisation features on the map. Graph drawing is concerned with the geometric representation of relational information. Interactive graph drawing allows the user to have some control over the geometric representation. In this thesis, we investigate the use of haptic feedback in representing geographic information on a map and in interactive graph drawing. This involves designing and implementing textured maps and an interactive graph drawing system called GRAPPLE. In both these designs, users receive haptic and visual information helping them to understand the information and perform their tasks. Through user experiments, we evaluate our techniques and determine the usefulness of different haptic modes for different types of data. Our results provide evidence that by adding the sense of touch to visualisations we can provide a greater foundation for insights into the data. We also define a framework and guidelines for the use and design of visualhaptic information systems.

Humans use all their senses when they explore and interact with the environment. In human-machine interaction (HMI) vision is the dominant sense, followed by audition. Haptic information - information concerning the sense of touch - is not commonly available. The overall aim of this thesis was to investigate how haptic feedback affects visual-manual HMI. In the experiment presented in paper I the spatial haptic properties shape and location were compared. Shape encoding often relies on users sequentially exploring differently shaped knobs, levers or buttons. The experiment revealed that physical shapes available through a shape-changing device can be as efficient as adjacently located push-buttons to encode functions in an interface. The experiment presented in paper II investigated the extent to which interface information can be transferred between the haptic and visual modalities. The feedback - rendered textures - was displayed haptically through a rotary device and visually through a computer monitor. There was a cross-modal transfer between the modalities, although not effortless, and the transfer from haptics to vision seemed to be easier than the transfer from vision to haptics. The asymmetry of the cross-modal transfer and the enhanced visual performance might be a result of the visual information being more useful for the task at hand. Paper III presents an experiment carried out in a car simulator. The experiment was conducted to investigate how haptic feedback in an in-car interface affects driver behaviour. Visual feedback was provided on a screen at the centre panel of the simulator. Haptic feedback was provided through the interaction device - a rotary device. The results revealed that, although driving performance degradation did not differ between the different haptic and visual feedback conditions, all conditions caused a degradation in driving performance. Visual behaviour did not differ between conditions including visual feedback. It is therefore apparent that the haptic feedback was not actively used when visual interface information was provided. Using haptic feedback only was shown to be more time-consuming. In addition it was revealed that tasks with only haptic feedback induce a cognitive load on the driver. It was apparent in studies II and III that the haptic information is not actively used if the visual information is more easily achieved.

Godkänd; 2007; 20070919 (biem)

Acoustic models driven by real-time velocity signals can suffer unduly from quality issues due to sampling and differentiation, especially at high sampling rates. In audio-haptic friction interaction, as found in a bowed stringsimulation for example, this noise appears as a gritty or dry feel, and is audible in the sound.In this thesis, two approaches to this problem are proposed: firstly, reduction of the sensitivity of the model to velocity noise by the application of a position-dependent friction model; secondly, the improvement of velocity estimation by means of filtering and enhanced sensing.Several estimators are compared, by means of parameter optimisation, to direct velocity measurement in order to find a good trade-off between filter-imposed delay and noise rejection. Optimised estimators are then compared by subjects in an online scenario to test their respective effect on the impedance range and noise qualities of a bowed string friction display.
Les modèles acoustiques commandés en temps réel par des signaux de vitesse peuvent souffrir indûment de problèmes de qualité dus à l'échantillonnage et à la différenciation de la position, en particulier à des taux d'échantillonnage élevé. Dans l'interaction audio-haptique basée sur la friction, que l'on trouve dans la simulation d'une corde frottée par exemple, ce bruit se présente comme une sensation granuleuse ou sèche, et est audible dans le son. Nous proposons dans cette thèse deux solutions à ce problème: d'une part, la réduction de la sensibilité au bruit du modèle de simulation par l'application d'un modèle de friction dépendant de la position; d'autre part, l'amélioration de l'estimation de la vitesse en utilisant différents filtres ainsi que des capteurs supplémentaires. Plusieurs estimateurs sont comparées entre eux et à la mesure directe de la vitesse, dans le but de trouver un compromis acceptable entre le retard imposé par le filtrage et la réduction du bruit.Ces estimateurs optimisés sont ensuite évalués par des participants afin de tester leurs effets sur la gamme d'impédance obtenue ainsi que sur la qualité du bruit dans la simulation de corde frottée.

Virtual Reality (VR) is a rapidly advancing scientific field which enables humans to experience environments other than that which they physically inhabit. Humans use all their senses to interact in the real world and there should be no difference when using VR. This thesis explores the current state-of-the-art in Multi-Sensory Virtual Reality (MSVR) and presents new techniques for improving the level of interaction and realism in touch enabled MSVR. It is shown that, of the three senses which are commonly included in MSVR: vision, audition and haptics (touch), haptics is the least well represented. Further, it is observed that haptic interaction is particularly lacking in two areas: natural object manipulation and large workspace interaction. Object manipulation is limited by the number of contact points a haptic device can provide and there are both hardware and software challenges related to this. It is also limited by the realism of simulated object motion which is more complex for haptics than purely visual-auditory simulations. A novel haptic rendering algorithm, called the xFCA, has been designed to improve multi-finger manipulation of arbitrarily shaped objects. Also, a software platform known as MUSI has been developed which integrates the xFCA into a dynamic rigid-body simulator to allow the natural manipulation of virtual objects. The challenges in the development of MUSI, along with its advantages and limitations are discussed. Two new approaches to increasing the workspace of haptic devices have been investigated. The first, a novel haptic rendering technique which provides force feedback related to velocity is applied to a virtual shopping trolley. The second, a novel method of chaining devices together, is used to create a multi-finger haptic interface for both large and fast movements. Finally, both systems have been integrated into an MSVR simulator and the results of this are also discussed.

Dissertação para obtenção do Grau de Mestre em Engenharia Eletrotécnica e de Computadores
This dissertation presents a system for haptic interaction and self-supervised learning mechanisms to ascertain navigation affordances from depth cues. A simple pan-tilt telescopic arm and a structured light sensor, both fitted to the robot’s body frame, provide the required haptic and depth sensory feedback. The system aims at incrementally develop the ability to assess the cost of navigating in natural environments. For this purpose the robot learns a mapping between the appearance of objects, given sensory data provided by the sensor, and their bendability, perceived by the pan-tilt telescopic arm. The object descriptor, representing the object in memory and used for comparisons with other objects, is rich for a robust comparison and simple enough to allow for fast computations. The output of the memory learning mechanism allied with the haptic interaction point evaluation prioritize interaction points to increase the confidence on the interaction and correctly identifying obstacles, reducing the risk of the robot getting stuck or damaged. If the system concludes that the object is traversable, the environment change detection system allows the robot to overcome it. A set of field trials show the ability of the robot to progressively learn which elements of environment are traversable.

Thesis (Ph. D.)--Mechanical Engineering, Georgia Institute of Technology, 2006.
Arkin, Ronald, Committee Member ; DeWeerth, Steve, Committee Member ; Vito, Raymond, Committee Member ; Ebert-Uphoff, Imme, Committee Member ; Book, Wayne, Committee Chair. Includes bibliographical references.

Além de um canal para adquirir informações sobre o ambiente ao nosso redor, o sentido do tato é também o nosso sentido mais social. No entanto, a interação háptica é geralmente implementada como chamariz nas interfaces modernas. Embora a comunicação multimodal seja comum em Ambientes Virtuais, as tecnologias de Realidade Virtual mais acessíveis nem sequer incluem o componente háptico como parte fundamental. Esta tese apresenta estudos sobre percepção, desempenho do usuário, e experiência do usuário com dispositivos de comunicação vibrotátil construídos para suportar diferentes tarefas interativas em ambientes virtuais e físicos. Foram avaliados diferentes atuadores hápticos, configurações de exibição tátil, locais do corpo, perfis de usuário, e métodos para se projetar uma plataforma tátil robusta. Tal plataforma foi finalmente construída como uma tela vibrotátil para ser usada ao redor da cabeça e para suportar tarefas de consciência espacial e comunicação em ambientes virtuais e físicos. Durante a pesquisa foi observado que, apesar de sua importância para a comunicação, o uso proativo de háptica para intercomunicação é surpreendentemente negligenciado. Portanto, foi dada especial atenção aos elementos presentes na articulação da fala para introduzir a articulação háptica proativa como uma nova abordagem para intercomunicação. Foi proposto que a habilidade de usar uma interface háptica como uma ferramenta para comunicação implícita pode suplementar a comunicação e suportar tarefas colaborativas próximas e remotas em diferentes contextos. Além disso, uma interface articulatória pode fornecer um modo direto e expressivo de se comunicar através de sinais táteis. Para demonstrar isso, os resultados dessa pesquisa foram aplicados ao projeto de uma tela montada na cabeça com vibração, especialmente feita para interação com ambientes virtuais imersivos. Tal aparato mostrou-se útil não apenas para orientação no espaço 3D, mas também para intercomunicação em ambientes virtuais colaborativos. Além de nossas contribuições técnicas em relação à construção de uma tela tátil totalmente testada para múltiplas tarefas e contextos, nossa principal contribuição é a concepção e demonstração de um novo paradigma de interação tátil. Tal paradigma se concentra em fornecer maneiras simples e diretas para que indivíduos se expressem através de sinais táteis em aplicações mediadas por computador para interair com seu ambiente e com outros indivíduos. Esse paradigma envolve os usuários finais e permite que eles se tornem interlocutores ao invés de meros receptores do feedback tátil.
The sense of touch not only is a channel for acquiring information about the environment around us, it is also our most social sense. However, haptic interaction is usually implemented as a gimmick feature in modern interfaces. Although multimodal communication is commonplace in Virtual Environments, the most accessible Virtual Reality technologies do not even include the haptic component as a fundamental part. This thesis presents studies on perception, user performance, and user experience with vibrotactile communication devices built to support different interactive tasks in virtual and physical environments. We have assessed different haptic actuators, tactile display configurations, body sites, user profiles and methods to design a robust tactile platform. Such platform was finally built as a vibrotactile display to be worn around the head and to support spatial awareness and communication in both virtual and physical environments. During our research, we particularly notice that the proactive use of touch for intercommunication is surprisingly neglected regardless of its importance for communication. Therefore, we have also directed our attention to elements present in speech articulation to introduce proactive haptic articulation as a novel approach for intercommunication. We propose that the ability to use a haptic interface as a tool for implicit communication can supplement communication and support near and remote collaborative tasks in different contexts. In addition, an articulatory interface can provide a direct and expressive way for communicating through tactile cues. To demonstrate that, our results were applied to the design of a vibrotactile head-mounted display especially made for interaction with immersive virtual environments. Such apparatus was shown not only to support guidance in 3D space but also to support intercommunication in collaborative virtual environments. In addition to our technical contributions regarding the construction of a fully tested tactile display for multiple tasks and contexts, our main contribution is the conception and demonstration of a new paradigm for tactile interaction. Such paradigm focuses on providing simple and direct ways for individuals to express themselves through tactile cues in computer-mediated interaction with their environment and with others. Such paradigm embraces the final users and allows them to become interlocutors rather than just receivers of the haptic feedback.

Through qualitative and exploratory research, this thesis project investigates how body stimulation from haptic feedback affects user’s feeling of presence in VR environments. It identifies that in current time,the development of haptic feedback in VR lags severely behind the advancements made in visual and auditory feedback, and that somecompanies disregard its importance. Simultaneously, new companies are emerging which focus entirely on haptics in VR. Since development is still an early stage, this thesis highlights now as a unique opportunity to explore the thoughts professionals have on the topic, as well as try to find out which exact haptics are important for feeling presence to serve as a guide to those developing such systems.Finally, to tackle this issue, it is imperative to understand certain theoretical concepts such as affordances and embodiment, and how they change in the world of VR. This understanding can contribute to Interaction Design knowledge.

On-the-go users' interaction with mobile devices often requires high visual attention that can overtax limited human resources. For example, while attending information displayed on a mobile device, on-the-go users who are driving a car or walking in the street can easily fail to see a dangerous situation. This dissertation explores the benefits of wearable tactile displays (WTDs) to support eyes-free interaction for on-the-go users. The design and implementation of the WTDs are motivated by two principles in mobile user interaction that have been proven both commercially and academically: wristwatch interfaces that reduce the time for device acquisition and tactile interfaces that eliminate the need for visual attention. In this dissertation, I present three phases of design iteration on WTDs to provide the design rationale and challenges. The result of the iterative design is evaluated through in-depth formal investigations with novice users in two experiments: user perception of the tactile stimuli and information throughput in association with multiple tactile parameters, and perception of the tactile stimuli and information throughput when the user is visually distracted. The first experiment explores general human capabilities in perceiving tactile stimuli on the wrist. It reveals that subjects could discriminate 24 tactile patterns with 98% accuracy after 40 minutes of training. Of the four parameters (intensity, starting point, rhythm, direction) that were configured to design the 24 patterns, intensity was the most difficult parameter to distinguish, and temporal variation was the easiest. The second experiment explores users' abilities to perceive incoming alerts from two mobile devices (WTD and mobile phone) with and without visual distraction. The second experiment reveals that when the user was distracted visually, reaction time to perceive the incoming alerts became slower with the mobile phone alert but not with the WTD.

Human-scale mobile robots that manipulate objects (mobile manipulators) have the potential to perform a variety of useful roles in healthcare. Many promising roles for robots require physical contact with patients and caregivers, which is fraught with both psychological and physical implications. In this thesis, we used a human factors approach to evaluate system performance and participant responses when potential end users performed a healthcare task involving physical contact with a robot. We performed four human-robot interaction studies with 100 people who were not experts in robotics (naive participants). We show that physical contact between naive participants and human-scale mobile manipulators can be acceptable and effective in a variety of healthcare contexts. In this thesis, we investigated two forms of touch-based (haptic) interaction relevant to healthcare. First, we studied how participants responded to physical contact initiated by an autonomous robotic nurse. On average, people responded favorably to robot-initiated touch when the robot indicated that it was a necessary part of a healthcare task. However, their responses strongly depended on what they thought the robot's intentions were, which suggests that this will be an important consideration for future healthcare robots. Second, we investigated the coordination of whole-body motion between human-scale robots and people by the application of forces to the robot's hands and arms. Nurses found this haptic interaction to be intuitive and preferred it over a standard gamepad interface. They also navigated the robot through a cluttered healthcare environment in less time, with fewer collisions, and with less cognitive load via haptic interaction. Through a study with expert dancers, we demonstrated the feasibility of robots as dance-based exercise partners. The experts rated a robot that used only haptic interaction to be a good follower according to subjective measures of dance quality. We also determined that healthy older adults were accepting of using a robot for partner dance-based exercise. On average, they found the robot easy and enjoyable to use and that it performed a partnered stepping task well. The findings in this work make several impacts on the design of robots in healthcare. We found that the perceived intent of robot-initiated touch significantly influenced people's responses. Thus, we determined that autonomous robots that initiate touch with patients can be acceptable in some contexts. This result highlights the importance of considering the psychological responses of users when designing physical human-robot interactions in addition to considering the mechanics of performing tasks. We found that naive users across three user groups could quickly learn how to effectively use physical interaction to lead a robot during navigation, positioning, and partnered stepping tasks. These consistent results underscore the value of using physical interaction to enable users of varying backgrounds to lead a robot during whole-body motion coordination across different healthcare contexts.

Computer interfaces commonly make large demands on our visual and auditory attention, which can make multi-tasking with multiple systems difficult. In cases where a primary task demands constant, unbroken attention from the user, it is often implausible for such a user to employ a system for a secondary task, even when desirable. The haptic modality has been suggested as a conduit for the appropriately-intrusive delivery of information from computer systems. Furthermore, physiological signals can be used to infer the affective state of a user without requiring attention. Combining these underexplored channels for implicit system command, control and display, we envision an automated, intelligent and emotionally aware interaction paradigm. We call this paradigm the Haptic-Affect Loop (HALO). This work investigates the potential for the HALO paradigm in a specific use case (portable audio consumption). It uses three experimental techniques to gather requirements for the paradigm, validate its technological feasibility, and develop the feedback-supported language of interaction with a HALO-enabled portable audio system. A focus group is first conducted to identify the perceived utility of the paradigm with a diverse – albeit technologically conservative – group of portable audio users, and to narrow its scope. Results of this focus group indicate that participants are sceptical of its technological feasibility (in particular, context resolution) and are unwilling to relinquish control over their players. This scepticism was alleviated somewhat by the conclusion of the sessions. Next, technological validation of online affect classification is undertaken via an exploratory, but formally controlled, experiment. Galvanic skin response measures provided a means to make introductory measures of interruption and, in some cases, musical engagement. A richer signal array is necessary to make the full array of required affect identifications for this paradigm, and is under development. The final phase of work involves an iterative participatory design process with a single participant who was enthusiastic but practical about technology to better define system requirements and to evaluate input and output mechanisms using a variety of devices and signals. The outcome of this design effort was a functioning prototype, a set of initial system requirements and an exemplar interaction language for HALO.

Pressure input and thermal feedback are two under-researched aspects of touch in mobile human-computer interfaces. Pressure input could provide a wide, expressive range of continuous input for mobile devices. Thermal stimulation could provide an alternative means of conveying information non-visually. This thesis research investigated 1) how accurate pressure-based input on mobile devices could be when the user was walking and provided with only audio feedback and 2) what forms of thermal stimulation are both salient and comfortable and so could be used to design structured thermal feedback for conveying multi-dimensional information. The first experiment tested control of pressure on a mobile device when sitting and using audio feedback. Targeting accuracy was >= 85% when maintaining 4-6 levels of pressure across 3.5 Newtons, using only audio feedback and a Dwell selection technique. Two further experiments tested control of pressure-based input when walking and found accuracy was very high (>= 97%) even when walking and using only audio feedback, when using a rate-based input method. A fourth experiment tested how well each digit of one hand could apply pressure to a mobile phone individually and in combination with others. Each digit could apply pressure highly accurately, but not equally so, while some performed better in combination than alone. 2- or 3-digit combinations were more precise than 4- or 5-digit combinations. Experiment 5 compared one-handed, multi-digit pressure input using all 5 digits to traditional two-handed multitouch gestures for a combined zooming and rotating map task. Results showed comparable performance, with multitouch being ~1% more accurate but pressure input being ~0.5sec faster, overall. Two experiments, one when sitting indoors and one when walking indoors tested how salient and subjectively comfortable/intense various forms of thermal stimulation were. Faster or larger changes were more salient, faster to detect and less comfortable and cold changes were more salient and faster to detect than warm changes. The two final studies designed two-dimensional structured ‘thermal icons’ that could convey two pieces of information. When indoors, icons were correctly identified with 83% accuracy. When outdoors, accuracy dropped to 69% when sitting and 61% when walking. This thesis provides the first detailed study of how precisely pressure can be applied to mobile devices when walking and provided with audio feedback and the first systematic study of how to design thermal feedback for interaction with mobile devices in mobile environments.

Micromanipulation is considered a challenging task which requires high precision motion and measurement at the micro scale. When micromanipulation is concerned with living organisms important considerations need to be addressed. These include the physical or chemical properties of micro-organisms, living conditions, responses to the environment and achieving suitably delicate manipulation. Bio-micromanipulation can include micro surgery or cell injection operations, or to determine interaction forces as the basis to investigate behavior and properties of living micro-organisms. In order to achieve suitable bio-micromanipulation appropriate processes and/or sensory systems need to be investigated. This thesis aims to look into the force interaction and sensing addressing two distinctive challenges in the field of bio-micromanipulation. To this end, this thesis presents two major contributions to advancing bio-micromanipulation. Firstly, a novel Haptic Microrobotic Cell Injection System is introduced which is able to assist a bio-operator through haptic interaction. The system introduces a mapping framework which provides an intuitive method for the bio-operator to maneuver the micropipette in a manner similar to handheld needle insertion. To accurately control the microrobot, a neuro-fuzzy modeling and control scheme has been developed. Volumetric, axial and planar haptic virtual fixtures are introduced to guide the bio-operator during cell injection. Aside from improving real-time operator performance using the physical system, the system is novel in facilitating virtual offline operator training. Secondly, a first-of-its-kind micro-pillar based on-chip system for dynamic force measurement of C. elegans motion is introduced. The system comprises a microfabricated PDMS device to direct C. elegans into a matrix of micropillars within a channel mimicking its dwelling environment. An image processing algorithm is able to track the interaction of the C. elegans with the pillars and estimate contact forces based on micropillar deflections. The developed micropillar system is capable of measuring the force with sub-micron resolution while providing a continuous force output spectrum.

There are many current and future scenarios that require teams of air, ground or humanoid robots to gather information in complex and often dangerous environments, where it would be unreasonable or impossible for humans to be physically present [1-6]. The current state of the art involves a single robot being monitored by one or many human operators [7], but a single operator managing a team of autonomous robots is preferred as long as effective and time-efficient management of the team is maintained [8-9]. This is limited by the operator's ability to command actions of multiple robots, be aware of robot states, and respond to less important tasks, while accomplishing a primary objective defined by the application scenario. The operator's ability to multi-task could be improved with the use of a multimodal interface, using both visual and haptic feedback. This thesis investigates the use of haptic feedback in developing intuitive, shape-based interaction to maintain heads-up control and increase an operator's situation awareness (SA) while managing a robot team.In this work, the autonomous behavior of the team is modeled after a patrol and cordon scenario, where the team travels to and surrounds buildings of interest. A novel approach that involves treating the team as a moldable volume is presented, where deformations of this volume correspond to changes in team shape. During surround mode, the operator may explore or manipulate the team shape to create custom formations around a building. A spacing interaction method also allows the operator to adjust how robots are spaced within the current shape. Separate haptic feedback is developed for each method to allow the operator to "feel" the shape or spacing manipulation. During travel mode, the operator chooses desired travel locations and receives feedback to help identify how and where the team travels. RoTHSim, an experimental robot team haptic simulator, was developed and used as a test bed for single-operator management of a robot team in a multitasking reconnaissance and surveillance scenario. Using RoTHSim, a human subject experiment was conducted with 19 subjects to determine the effects of haptic feedback and task demand difficulty on levels of performance, SA and workload. Results from the experiment suggest that haptic feedback significantly improves operator performance in a reconnaissance task when task demand is higher, but may slightly increase operator workload. Due to the experimental setup, these results suggest that haptic feedback may make it easier for the operator to experience heads-up control of a team of autonomous robots. There were no significance differences on SA scores due to haptic feedback in this study.

Supplying haptic or force feedback to operators using hydraulic machinery such as excavators has the potential to increase operator capabilities. Haptic, robotic, human-machine interfaces enable several enhancing features including coordinated motion control and programmable haptic feedback. Coordinated or resolved motion control supplies a more intuitive means of specifying the equipment's motion. Haptic feedback is used to relay meaningful information back to the user in the form of force signals about digging force acting on the bucket, programmable virtual constraints and system limitations imposed by the mechanism, maximum pressure or maximum flow. In order to make this technology economically viable, the benefits must offset the additional cost associated with implementation. One way to minimize this cost is to not use high-end hydraulic components. For smaller backhoes and mini-excavators this means that the hydraulic systems are comprised of a constant displacement pump and proportional direction control valves. Hydraulic and haptic control techniques suitable for backhoes/excavators are developed and tested on a small backhoe test-bed. A virtual backhoe simulator is created for controller design and human evaluation. Not only is the virtual simulator modeled after the test-bed, but the control algorithm used in the simulator is the same as the actual backhoe test-bed. Data from human subject tests are presented that evaluate the control strategies on both the real and virtual backhoe. The end goal of this project is to incorporate coordinated haptic control algorithms that work with low-cost systems and maximize the enhancement of operator capabilities.

Touch is an expressive and powerful modality in affect conveyance. A simple touch like a hug can elicit strong feelings of affection both in the touch initiator and recipient. Therefore delivering touch over a distance to a long-distance family member or significant other has been an appealing concept for both researchers and designers. However compared to the development of audio, video channels which allow the transmission of voice, facial expression and gesture, digitally mediated touch (Remote Touch) has not received much attention. We believe that this is partially due to the lack of understanding of the capabilities and communication possibilities that remote touch brings. This dissertation presents a review of relevant psychological and sociological literature of touch and proposes a model of immediacy of the touch channel for affect conveyance. We advance three hypotheses regarding the possibility of remote touch in immediate affect conveyance: presence, fidelity and context. We posit that remote touch with relatively low touch fidelity can convey meaningful immediate affect when it is accompanied by a contextualizing channel. To test the hypothesis, two sets of remote touch devices are designed and prototyped which allow users to send/receive a squeeze on the upper arm to/from others effectively. Three in-lab user studies are conducted to investigate the role of remote touch in affect conveyance. These studies showed clearly that remote touch, when contextualized, can influence the affective component in communication. Our results demonstrated that remote touch can afford a rich spectrum of meanings and affects. Three major categories of the usage are identified as positive affect touch which serves to convey affects such as affection, sympathy and sharing, comfort etc., playful touch which serves to lighten the conversations, and conversational touch which serves to regulate the dynamics in the discourse. Our interview results also provide insights of how people use this new channel in their communication.
Ph. D.

Ce travail se situe dans le domaine de l’Interaction Personne-Système, et plus particulièrement dans celui de la simulation instrumentale multisensorielle. La première partie de cette thèse présente les différents degrés d’intégration du geste dans l’ordinateur et propose une approche fonctionnelle des technologies pour le retour d’effort. Nous présentons ensuite une étude des différents composants matériels et logiciels nécessaires à la chaîne de simulation haptique, ainsi que les différentes approches utilisées pour connecter un système à retour d’effort à un processus de simulation en temps réel. Nous formalisons le concept de « Haptic Processor Unit ». La dernière partie présente la validation de la plate-forme de simulation développée. Elle est essentiellement orientée vers l’implantation de nouveaux modèles, utilisés dans le cadre d’un travail de recherche sur la situation instrumentale médiatisée
This work is related to the field of Human-Computer Interaction, and particularly °to the field of multisensory instrumental simulation, as conceptualized by the research group ACROE & ICA, and which needs a strong coupling between the human and the instrument. The fust part of this thesis presents various degrees of the integration of gesture in computer uses, then develops a functional approach of force feedback technologies. This analysis elicits the mainstreams tbat are currently sharing the field of haptics research. We then present a study of the hardware and software components that are used in haptic simulation, and the various approaches used to connect a force feedback device to a real time modelling system. The analysis of the role of each of the components in the simulation chain and their relationships allowed us to conceptualize the "Haptic Processor Unit". This component guarantees in particular the conditions of reactivity that are requ'Î. Red for multisensory simulation. The new simulation architecture that we designed in this work, named ERGON_X, implements the concept of HPU. ERGON_X is a compact and transportable simulator, and handies simulation frequencies up to 44 100Hz. The third part presents the validation of the simulation platfotUl ERGON_X. It mainly focuses on the design of new models, which were used in the framework of the research carried on by ACROE & ICA about instrumental interaction. The "E" is a model demonstrating the capabilities of the ERGOS technology, which is now fully exploitable thanks to this new simulation architecture. The models of tapping and of deformable paste allowed us to bring new results on human-object interaction, and validate the simulator as a 1001 for psychophysical experimentation. The Enactive Emblematic Scenarii "Ergotic Sounds" and "Pebble Box' illustrate the conception of Enaction. They validate the use of our simulation architecture as an experimental platfotUl and lead us to a paradigm shift trom "instrumental interaction" to "enactive interaction"

Interaction with mobile devices suffers from a number of shortcomings, most of which are linked to the small size of screens. Artificial tactile feedback promises to be particularly well suited to the mobile interaction context. To be practical, tactile transducers for mobile devices must be small and light, and yet be capable of displaying a rich set of expressive stimuli. This thesis introduces a tactile transducer for mobile interaction that is capable of distributed skin stimulation on the fingertip. The transducer works on a principle that was first investigated because of its potential application to the display of Braille. A preliminary study was conducted on an earlier version of the transducer. It concluded that subjects were able to identify simple Braille characters with a high rate of success. Then, a complete re-design of the transducer addressed the goal of integration in a handheld prototype for mobile interaction. The resulting device comprises a liquid crystal graphic display co-located with the miniature, low-power, distributed tactile transducer. Next, it was needed to measure the perceptual differences between the stimuli that the device could display. Our experiences with one evaluation approach raised questions relating to the methodology for data collection. Therefore, an analysis of the process was carried out using a stimulus set obtained with the device. By means of multidimensional scaling analysis, both the perceptual parameters forming the stimuli space and the evaluation technique were validated. Finally, two experiments were carried out with the objective to develop new mobile interactions paradigms that combined visual and tactile feedback. Both experiments modeled a list scrolling task on the device. The first experiment found a marginal improvement in performance when tactile feedback was employed. It also came at a higher attentional cost dedicated to operating the device. For the second experiment, the scrolling paradigm and the tactile feedback were improved. This lead to a decrease in the reliance on vision when tactile feedback was enabled. Results showed a 28% decrease in the number of key presses that controlled the visibility state of the scroll list.