Dissertations / Theses on the topic 'Paralinguistik'

Paralinguistic events are useful indicators of the affective state of a speaker. These cues, in children's speech, are used to form social bonds with their caregivers. They have also been found to be useful in the very early detection of developmental disorders such as autism spectrum disorder (ASD) in children's speech. Prior work on children's speech has focused on the use of a limited number of subjects which don't have sufficient diversity in the type of vocalizations that are produced. Also, the features that are necessary to understand the production of paralinguistic events is not fully understood. To account for the lack of an off-the-shelf solution to detect instances of laughter and crying in children's speech, the focus of the thesis is to investigate and develop signal processing algorithms to extract acoustic features and use machine learning algorithms on various corpora. Results obtained using baseline spectral and prosodic features indicate the ability of the combination of spectral, prosodic, and dysphonation-related features that are needed to detect laughter and whining in toddlers' speech with different age groups and recording environments. The use of long-term features were found to be useful to capture the periodic properties of laughter in adults' and children's speech and detected instances of laughter to a high degree of accuracy. Finally, the thesis focuses on the use of multi-modal information using acoustic features and computer vision-based smile-related features to detect instances of laughter and to reduce the instances of false positives in adults' and children's speech. The fusion of the features resulted in an improvement of the accuracy and recall rates than when using either of the two modalities on their own.

Paralinguistic cues are powerful non-verbal communication elements that have the ability to make even good news sound horrible or bad news more palatable. Drawing upon the hermeneutical phenomenology communication theory of Dilthey (1961) and Shannon's (1949) communication channel model, this thesis targeted the six primary paralinguistic cues of fluidity, articulation, sonority, tempo, emotion, and dynamic intensity to discover how these cues interact and affect the credibility and palatability of a leader's message. To facilitate remembrance of these cues, the acronym F.A.S.T.E.D. was coined and used. Three different groups consisting of managers, non-managers, and media and public relations professionals were surveyed to provide a triangulation of the data that helped identify these relationships. Research showed that these vocal cues were every bit as essential to an accurate understanding of the message as the actual words the leader employed to explain and compel both internal and external stakeholders. Further, the study indicates the importance of learning and applying these skills as well as a universality of their existence that transcends language barriers and cultural differences.

I dagens samhälle är det av största vikt att kunna kommunicera med varandra vare sig det är på ett personligt plan eller ett företag med sina kunder. Inom informatik är en del av vårt arbete att optimera dessa sätt att kommunicera varav ljuset denna gång faller på infografik i dess audiovisuella form. Detta arbete utforskar de paralingvistiska element inom infografikens olika berättarröster och hur dessa påverkar dess publik. Med en litterär studie förtydligas innebörden av just paralingvistiska element för att kunna applicera dessa på berättarröster som skapats för en fallstudie där 13 respondenter visas en infografisk film benämnd “Alma”. Dessa berättarröster framförs med tre olika sinnesstämningar av såväl en man som en kvinna som alla följer samma manus vilket resulterar i totalt sex olika varianter. Detta är till grund för att utvärdera huruvida berättarröstens kön så väl som de paralingvistiska elementen påverkar dess trovärdighet. Genom fallstudien har jag kunnat bekräfta att paralingvistiska element sannerligen har en mycket stor påverkan huruvida en berättarröst uppfattas som trovärdig eller ej. Däremot huruvida kön direkt påverkar en berättarrösts trovärdighet och om det bör räknas som ett paralingvistiskt element i sig går ej ännu med säkerhet att bekräfta.

We live in a world which bombards us with a huge amount of sensory information, even if we are not always aware of it. To successfully navigate, function and ultimately survive in our environment we use all of the cues available to us. Furthermore, we actually combine this information: doing so allows us not only to construct a richer percept of the objects around us, but actually increases the reliability of our decisions and sensory estimates. However, at odds with our naturally multisensory awareness of our surroundings, the literature addressing unisensory processes has always far exceeded that which examines the multimodal nature of perception. Arguably the most salient and relevant stimuli in our environment are other people. Our species is not designed to operate alone, and so we have evolved to be especially skilled in all those things which enable effective social interaction – this could be engaging in conversation, but equally as well recognising a family member, or understanding the current emotional state of a friend, and adjusting our behaviour appropriately. In particular, the face and the voice both provide us with a wealth of hugely relevant social information - linguistic, but also non-linguistic. In line with work conducted in other fields of multisensory perception, research on face and voice perception has mainly concentrated on each of these modalities independently, particularly face perception. Furthermore, the work that has addressed integration of these two sources by and large has concentrated on the audiovisual nature of speech perception. The work in this thesis is based on a theoretical model of voice perception which not only proposed a serial processing pathway of vocal information, but also emphasised the similarities between face and voice processing, suggesting that this information may interact. Significantly, these interactions were not just confined to speech processing, but rather encompassed all forms of information processing, whether this was linguistic or paralinguistic. Therefore, in this thesis, I concentrate on the interactions between, and integration of face-voice paralinguistic information. In Chapter 3 we conducted a general investigation of neural face-voice integration. A number of studies have attempted to identify the cerebral regions in which information from the face and voice combines; however, in addition to a large number of regions being proposed as integration sites, it is not known whether these regions are selective in the binding of these socially relevant stimuli. We identified firstly regions in the bilateral superior temporal sulcus (STS) which showed an increased response to person-related information – whether this was faces, voices, or faces and voices combined – in comparison to information from objects. A subsection of this region in the right posterior superior temporal sulcus (pSTS) also produced a significantly stronger response to audiovisual as compared to unimodal information. We therefore propose this as a potential people-selective, integrative region. Furthermore, a large portion of the right pSTS was also observed to be people-selective and heteromodal: that is, both auditory and visual information provoked a significant response above baseline. These results underline the importance of the STS region in social communication. Chapter 4 moved on to study the audiovisual perception of gender. Using a set of novel stimuli – which were not only dynamic but also morphed in both modalities – we investigated whether different combinations of gender information in the face and voice could affect participants’ perception of gender. We found that participants indeed combined both sources of information when categorising gender, with their decision being reflective of information contained in both modalities. However, this combination was not entirely equal: in this experiment, gender information from the voice appeared to dominate over that from the face, exerting a stronger modulating effect on categorisation. This result was supported by the findings from conditions which directed to attention, where we observed participants were able to ignore face but not voice information; and also reaction times results, where latencies were generally a reflection of voice morph. Overall, these results support interactions between face and voice in gender perception, but demonstrate that (due to a number of probable factors) one modality can exert more influence than another. Finally, in Chapter 5 we investigated the proposed interactions between affective content in the face and voice. Specifically, we used a ‘continuous carry-over’ design – again in conjunction with dynamic, morphed stimuli – which allowed us to investigate not only ‘direct’ effects of different sets of audiovisual stimuli (e.g., congruent, incongruent), but also adaptation effects (in particular, the effect of emotion expressed in one modality upon the response to emotion expressed in another modality). Parallel to behavioural results, which showed that the crossmodal context affected the time taken to categorise emotion, we observed a significant crossmodal effect in the right pSTS, which was independent of any within-modality adaptation. We propose that this result provides strong evidence that this region may be composed of similarly multisensory neurons, as opposed to two sets of interdigitised neurons responsive to information from one modality or the other. Furthermore, an analysis investigating stimulus congruence showed that the degree of incongruence modulated activity across the right STS, further inferring neural response in this region can be altered depending on the particular combination of affective information contained within the face and voice. Overall, both behavioural and cerebral results from this study suggested that participants integrated emotion from the face and voice.

Parallel with the orthographic streams of words in conversation are multiple layered epiphenomena, short in duration and with a communicativepurpose. These paralinguistic events regulate the interaction flow via gaze,gestures and intonation. This thesis focus on how to compute, model, discoverand analyze prosody and it’s applications for spoken dialog systems.Specifically it addresses automatic classification and analysis of conversationalcues related to turn-taking, brief feedback, affective expressions, their crossrelationshipsas well as their cognitive and neurological basis. Techniques areproposed for instantaneous and suprasegmental parameterization of scalarand vector valued representations of fundamental frequency, but also intensity and voice quality. Examples are given for how to engineer supervised learned automata’s for off-line processing of conversational corpora as well as for incremental on-line processing with low-latency constraints suitable as detector modules in a responsive social interface. Specific attention is given to the communicative functions of vocal feedback like "mhm", "okay" and "yeah, that’s right" as postulated by the theories of grounding, emotion and a survey on laymen opinions. The potential functions and their prosodic cues are investigated via automatic decoding, data-mining, exploratory visualization and descriptive measurements.

QC 20120914

The framework developed and tested in this study argues that the success of mediated persuasive appeals can be partially explained by the interaction among linguistic and extra-linguistic variables. It was predicted that sources would be most persuasive when the topic was of little importance and the source was liked by the viewer and that sources using intense language would be evaluated as believing the arguments presented more than sources using less intense language. These predictions were supported. The study also supported a predicted interaction between familiarity and liking such that familiarity works to the advantage of liked sources and to the disadvantage of disliked sources. It was also posited that intensity would have a differentially effective role for liked and disliked sources such that it works to the advantage of liked sources and to the disadvantage of disliked sources. This hypothesis was solely supported in proattitudinal appeals when intensity was operationalized using structural criteria and in counterattitudinal appeals when intensity was operationalized as perceived by the viewer. Finally, a 3-way interaction was predicted suggesting that familiarity and intensity would combine such that high familiarity results in increases in attitude change with the use of language that is more intense than expected for liked sources and decreases in attitude change for disliked sources. This hypothesis was not supported. In fact, both highly familiar sources benefited from use of language that was less intense than expected.

This study has two major purposes: (1) to investigate if and how punctuation conventions have been rewritten in text messages to compensate for lack of paralinguistic cues and (2) the sociolinguistic implications of these findings. Data for this study was collected through an online, anonymous questionnaire in which participants gave their judgments about the meanings and function of punctuation used in sample text messages. The results show that punctuation is used to convey differences in meaning in direct and indirect ways and most are dependent on the context. Furthermore, age showed to be a factor in punctuation style and interpretation. The results here challenge the notion that texting is detrimental to one’s literacy skills and is in fact a site of linguistic innovation.

This convergent mixed methods study explores prosodic and paralinguistic features of nine English speaker origins pertinent to Korean learners of English and how language attitude formation and English listening comprehension may be affected. The study found prosodic features (tone, clarity, rhythm, pausing, filled pausing) and paralinguistic features (talking to oneself, crying) interact with language attitude formation and English listening comprehension, which has implications for practice, pedagogy and policy in Korea and the wider ELF context.

Вивчення вербалізації мови тіла на матеріалі творів Френсіса Скотта Фіцджеральда "Великий Гетсбі" та "Загадкова історія Бенджаміна Баттона" можуть слугувати гарним прикладом вербалізації мови тіла у лінгвістиці, зокрема у паралінгвістиці та кінесиці.

The purpose of this thesis is to deepen the understanding of grown-up blind people’s non-verbal communication, including body expressions and paralinguistic (voice) expressions. More specifically, the thesis includes the following three studies: Blind people’s different forms of body expressions, blind people’s non-verbal conversation regulation and blind people’s experience of their own non-verbal expressions. The focus has been on the blind participants’ competence and on their subjective perspectives. I have also compared congenitally and adventitiously blind in all of the studies. The approach is mainly phenomenological and the qualitative empirical phenomenological psychological method is the primary methodological source of inspiration. Fourteen blind persons (and also some sigthed persons) participated. They have no other obvious disability than the blindness and their ages vary between 18 and 54. Data in the first two studies consisted of video recordings and data in the last study consisted of interviews. The overall results can be summarized in the following three points: 1. There are (almost) only similarities between the congenitally blind and adventitiously blind persons concerning their paralinguistic expressions. 2. There are mainly similarities between the two groups with respect to the occurrences of different body expressive forms. 3. There are also some differences between the groups. For example, the congenitally blind persons seem to have a limited ability to use the body in an abstract and symbolic way and they often mentioned that they have been told that their body expressions deviate from sighted people’s norms. But the persons in both groups also struggle to see themselves as unique persons who express themselves on the basis of their conditions and their previous experiences.

Much interactive media development, especially commercial development, implies the dominance of the visual modality, with sound as a limited supporting channel. The development of multimedia technologies such as augmented reality and virtual reality has further revealed a distinct partiality to visual media. Sound, however, and particularly voice, have many aspects which have yet to be adequately investigated. Exploration of these aspects may show that sound can, in some respects, be superior to graphics in creating immersive and expressive interactive experiences. With this in mind, this thesis investigates the use of non-speech voice characteristics as a complementary input mechanism in controlling multimedia applications. It presents a number of projects that employ the paralinguistic elements of voice as input to interactive media including both screen-based and physical systems. These projects are used as a means of exploring the factors that seem likely to affect users' preferences and interaction patterns during non-speech voice control. This exploration forms the basis for an examination of potential roles for paralinguistic voice input. The research includes the conceptual and practical development of the projects and a set of evaluative studies. The work submitted for Ph.D. comprises practical projects (50 percent) and a written dissertation (50 percent). The thesis aims to advance understanding of how voice can be used both on its own and in combination with other input mechanisms in controlling multimedia applications. It offers a step forward in the attempts to integrate the paralinguistic components of voice as a complementary input mode to speech input applications in order to create a synergistic combination that might let the strengths of each mode overcome the weaknesses of the other.

Several studies have examined the use of emoji and emoticons in computer mediated communication among peers and colleges but there is yet academic research on the impact of businesses using these paralinguistic cues when responding to online consumer reviews. This research is examining the influence these paralinguistic cues have on the consumers perception of the companies quality of the response to an online consumer review, brand relationship, purchase intent. Using an online survey, participants are asked to answer general questions about the brand, the quality of the response, brand relationship, and purchase intent after seeing a random condition of both a low and high involvement product. Result support previous eWOM research as valence of the review had the largest impact on the consumers perception of the companies quality of the response to an online consumer review, brand relationship, purchase intent with an interesting finding where in most cases the addition of an emoji in positive valence message attributes to the strongest findings. With the increase in emoji usage in marketing and advertising, it is important that business are utilizing these tools in effective means otherwise the use of these paralinguistic cues could negatively impact the companies quality of the response to an online consumer review, brand relationship, purchase intent.

У статті розглядаються невербальні особливості міжкультурної комунікації. При цитуванні документа, використовуйте посилання http://essuir.sumdu.edu.ua/handle/123456789/32479
В статье рассматриваются невербальные особенности межкультурной коммуникации. При цитировании документа, используйте ссылку http://essuir.sumdu.edu.ua/handle/123456789/32479
The article deals with non-verbal features of intercultural communication. When you are citing the document, use the following link http://essuir.sumdu.edu.ua/handle/123456789/32479

Креолізовані тексти – це тексти, фактура котрих складається з двох негомогенних частин: вербальної (мовленнєвої/мовної) та невербальної (тої, що відноситься до інших знакових системам, на відміну від природньої мови) [1, c.180]. У текстах такого типу поєднуються паралінгвістичні засоби та вербальне повідомлення, що разом утворюють єдине змістовне ціле.

The general aim of this thesis was to test the effects of paralinguistic (emotional) and prior contextual (topical) cues on perception of poorly specified visual, auditory, and audiovisual speech. The specific purposes were to (1) examine if facially displayed emotions can facilitate speechreading performance; (2) to study the mechanism for such facilitation; (3) to map information-processing factors that are involved in processing of poorly specified speech; and (4) to present a comprehensive conceptual framework for speech perception, with specification of the signal being considered. Experi¬mental and correlational designs were used, and 399 normal-hearing adults participated in seven experiments. The main conclusions are summarised as follows. (a) Speechreading can be facilitated by paralinguistic information as constituted by facial displayed emotions. (b) The facilitatory effect of emitted emotional cues is mediated by their degree of specification in transmission and ambiguity as percepts; and by how distinct the perceived emotions combined with topical cues are as cues for lexical access. (c) The facially displayed emotions affect speech perception by conveying semantic cues; no effect via enhanced articulatory distinctiveness, nor of emotion-related state in the perceiver is needed for facilitation. (d) The combined findings suggest that emotional and topical cues provide constraints for activation spreading in the lexicon. (e) Both bottom-up and top-down factors are associated with perception of poorly specified speech, indicating that variation in information-processing abilities is a crucial factor for perception if there is paucity in sensory input. A conceptual framework for speech perception, comprising specification of the linguistic and paralinguistic information, as well as distinctiveness of primes, is presented. Generalisations of the findings to other forms of paralanguage and language processing are discussed.

Il presente elaborato è dedicato allo studio della trasmissione vocale non-verbale (paraverbale) delle emozioni in interpretazione simultanea per la televisione di intrattenimento. Il primo capitolo si pone l’obiettivo di esplorare il mondo delle emozioni umane: di darne una definizione, di descriverne natura e funzioni, nonché di analizzarne brevemente l’espressione tramite parole (livello verbale), gesti (livello non-verbale) e voce (livello paraverbale). Il secondo capitolo rappresenta un approfondimento del livello paraverbale e consiste nello studio dei tratti vocali, della loro suddivisione in informativi (extralinguistici) e comunicativi (paralinguistici), nonché del raggruppamento di questi ultimi in un codice dotato di regole proprie. Il terzo capitolo funge da ponte tra lo studio delle emozioni e della loro espressione vocale e quello della televisione di intrattenimento: in questa parte verranno delineate le principali caratteristiche della Conversazione-Spettacolo dal punto di vista discorsivo e interpretativo. Il quarto capitolo riguarda la presentazione di un esperimento pratico di interpretazione simultanea dal russo in italiano di un’intervista a un ospite russofono invitato negli studi di un talk show. Un elemento interessante del presente capitolo consiste nell’uso, per la trascrizione del testo dell’esperimento e delle rese dei candidati, del codice dei segni vocali paralinguistici ideato nel secondo capitolo. Il quinto e ultimo capitolo è dedicato, invece, alla rielaborazione di quattro interviste a interpreti professionisti nell’ambito della televisione di intrattenimento. Partendo dalle conclusioni dell’esperimento, l’obiettivo è quello di passare in rassegna le strategie che possono essere messe in pratica per produrre una resa quanto più fruibile possibile per una platea di telespettatori.

This document describes the three methods for the detection and classification of paralinguistic expressions such as laughing and crying from usual speech by analysis of the audio signal. The database of records was originally designed for this purpose. When analyzing everyday dialogs, music might be included, so the database was extended by four new classes as speech, music, singing with music and usual speech with background music. Feature extraction, feature reduction and classification are common steps in recognizing for all three methods. Difference of the methods is given by classification process in detail. One classification of all six classes at once is proposed in the first method called straight approach. In the second method called decision tree oriented approach we are using five intuitive sub classifiers in the tree structure and the final method uses for classification emotion coupling approach. The best features were reduced by feature evaluation using F-ratio and GMM classifiers were used for the each classification part.

El objetivo de nuestra investigación es demostrar la importancia de incluir en el comentario de texto tradicional el análisis paralingüístico y los sistemas comunicativos no verbales. Hemos realizado nuestra investigación a tres niveles. En el primer nivel, hemos desarrollado un Modelo Semiótico de Transcrpición del Paralenguaje en Literatura y lo hemos aplicado a la obra teatral La Celestina. De forma gráfica y panorámica, hemos recogido toda la información sobre las categorías paralingüísticas y los sistemas comunicativos no verbales. En el segundo nivel, hemos extraído del texto de La Celestina una muestra representativa de cada categoría paralingüística. Hemos identificado los recursos paralingüísticos utilizados por el autor y los sistemas comunicativos no verbales presentes en el texto. A partir de esta información, hemos clasificado, analizado e interpretado los sonidos paralingüísticos emitidos por el personaje. En el tercer nivel, tomamos como punto de partida el análisis de texto tradicional y realizamos dos casos prácticos de análisis de textos literarios. Nuestros comentarios de textos añaden un apartado más al comentario de texto tradicional: el análisis e interpretación de los sonidos paralingüísticos presentes en el texto y los sistemas comunicativos no verbales.
The aim of this research is to demonstrate the importance of including paralinguistic analysis and non-verbal communication systems in traditional text commentary. We have undertaken this research at three levels. At the first level, we have developed a Semiotic transcription model of Paralanguage in Literature and we have applied it to the play Celestine. In a graphic and panoramic way, we have brought together all the information about paralinguistic categories and non-verbal communicative systems. At the second level, we have extracted from the text of Celestine a representative sample of each paralinguistic category. We have identified the paralinguistic resources employed by the author and the non-verbal communicative systems present in the text. Using this information as a point of departure, we have classified, analysed and interpreted the paralinguistic sounds emmited by the character. At the thid level, we take as a starting point the analysis of traditional texts and we undertake two practical cases of analysis of literary texts. Our text commentaries add a further section to the commentary on traditional text: the analysis and interpretation of paralinguistic sounds present in the text and non-verbal communication.

Les travaux de recherche de cette thèse portent sur l'amélioration des capacités sociales d'un système conversationnel en interaction avec un humain. Lorsque le système n'est pas dédié à une tâche particulière, il doit tenir compte de difficultés relevant de l'interaction sociale elle-même. L'humour est un mécanisme naturel dans les interactions sociales. Nous considérons un humour-machine comme une simulation de comportements simplifiés des capacités humoristiques humaines: dérision, blagues, jeux de mots. Les travaux de cette thèse s'appuient sur des théories issues de domaines variés en sociologie, psychologie, neurosciences et linguistique pour l'intégration de cet humour-machine dans un système robotique. Implémentées dans certains systèmes de dialogue, ces capacités humoristiques sont cependant rarement utilisées pour pouvoir choisir les comportements à générer du robot. Dans nos travaux, la mise en œuvre des comportements humoristiques du système en conversation est réalisée en utilisant la théorie des rites d'interaction. L'estimation de la face de l'interlocuteur permet de diriger le comportement du robot dans la conversation casuelle humoristique. Les facteurs expressifs d'une perte ou d'une valorisation de la face sont liés à des états mentaux exprimés dans une situation donnée. Pour réaliser cette estimation de la face, nous étudions, à partir de corpus créés à cet effet, les réactions comportementales, affectives et expressives des participants à différents types d'humour réalisés par le robot et ayant des impacts variables sur celle-ci (l'humour prenant pour cible le robot, le participant ou un autre sujet). Les réactions des participants à l'humour sont établies sur une représentation multi-niveaux d'indices émotionnels, comportementaux et linguistiques, extraits à partir de l'audio.Des règles sont ainsi construites à partir de l'apprentissage automatique de ces indices issus des corpus, concernant l'appréciation de la réaction des participants à l'humour et la mise à jour de l'estimation de la face présentée du participant. Leur implémentation dans un système automatique nous permet de les évaluer. De nombreuses expérimentations ont été menées avec des publics variés : personnes âgées, adultes, adolescents. Enfin, l'utilisation des préférences du participant à l'humour dans la conversation fait émerger des questions éthiques, notamment face au pouvoir persuasif et manipulateur de l'humour
This doctoral dissertation is about the improvement of social capacities of a conversationnal system to interact with humans. When the system is not dedicated to one particular task, it must take into account the inherent difficulties of social interaction it-self. Humor is a natural mechanism present in social interactions. We consider humour in a robotic system as a simulation of simplified behaviors from human humor : derision, jokes, puns. This work is based on theories issued from various research domains as sociology, psychology, neurosciences and linguistics to enable integration of humor in a robotic system. Implemented in some dialog systems, humorous capacities are however rarely used when programming the robot’s behavior. In our study, the humourous behavior is implemented in the system by using the ritual theory of face-work. The face analysis of the interlocutor can be used to direct the robot’s reactions during a casual humorous talk. In order to evaluate the faces of participants in interaction, we study, using data collections created for this purpose, the participant’s behavior, emotionnal and expressive responses to different types of humor (humorous act targeting the robot, the participant or a neutral subject). Participant's reaction to humor are made upon a multi-level processing of emotionnal, linguistic and behavioral cues.Machine learning is used to extract rules defining appreciation or not and update the participant's face evaluation in regards of the humorous act produced by the robot. An implementation of these rules in an automatic dialog system allows us to evaluate their accuracy. Numerous experiments were carried out on various populations : elderly persons, adults, teenagers. Finally, the use of the participant’s preferences in the conversation raises ethical questions, in particular against the persuasive and manipulative power of humor

This thesis explores, through a number of production andperception studies, the nature of the voice source signal andhow it varies in spoken communication. Research is alsopresented that deals with the techniques and methodologies foranalysing and synthesising the voice source. The main analytictechnique involves interactive inverse filtering for obtainingthe source signal, which is then parameterised to permit thequantification of source characteristics. The parameterisationis carried by means of model matching, using the four-parameterLF model of differentiated glottal flow.

The first three analytic studies focus on segmental andsuprasegmental determinants of source variation. As part of theprosodic variation of utterances, focal stress shows for theglottal excitation an enhancement between the stressed voweland the surrounding consonants. At a segmental level, the voicesource characteristics of a vowel show potentially majordifferences as a function of the voiced/voiceless nature of anadjacent stop. Cross-language differences in the extent anddirectionality of the observed effects suggest differentunderlying control strategies in terms of the timing of thelaryngeal and supralaryngeal gestures, as well as in thelaryngeal tensions settings. Different classes of voicedconsonants also show differences in source characteristics:here the differences are likely to be passive consequences ofthe aerodynamic conditions that are inherent to the consonants.Two further analytic studies present voice source correlatesfor six different voice qualities as defined by Laver'sclassification system. Data from stressed and unstressedcontexts clearly show that the transformation from one voicequality to another does not simply involve global changes ofthe source parameters. As well as providing insights into theseaspects of speech production, the analytic studies providequantitative measures useful in technology applications,particularly in speech synthesis.

The perceptual experiments use the LF source implementationin the KLSYN88 synthesiser to test some of the analytic resultsand to harness them to explore the paralinguistic dimension ofspeech communication. A study of the perceptual salience ofdifferent parameters associated with breathy voice indicatesthat the source spectral slope is critically important andthat, surprisingly, aspiration noise contributes relativelylittle. Further perceptual tests using stimuli with differentvoice qualities explore the mapping between voice quality andits paralinguistic function of expressing emotion, mood andattitude. The results of these studies highlight the crucialrole of voice quality in expressing affect as well as providingpointers to how it combines withf₀for this purpose.

The last section of the thesis focuses on the techniquesused for the analysis and synthesis of the source. Asemi-automatic method for inverse filtering is presented, whichis novel in that it optimises the inverse filter by exploitingthe knowledge that is typically used by the experimenter whencarrying out manual interactive inverse filtering. A furtherstudy looks at the properties of the modified LF model in theKLSYN88 synthesiser: it highlights how it differs from thestandard LF model and discusses the implications forsynthesising the glottal source signal from LF model data.Effective and robust source parameterisation for the analysisof voice quality is the topic of the final paper: theeffectiveness of global, amplitude-based, source parameters isexamined across speech tokens with large differences inf₀. Additional amplitude-based parameters areproposed to enable a more detailed characterisation of theglottal pulse.

Keywords:Voice source dynamics, glottal sourceparameters, source-filter interaction, voice quality,phonation, perception, affect, emotion, mood, attitude,paralinguistic, inverse filtering, knowledge-based, formantsynthesis, LF model, fundamental frequency,f₀.

Є публікація в журналі: “Science and Education a New Dimension” м. Будапешт, Угорщина (online version ISSN 2308-1996, printed version ISSN 2308-5258) назва статті: “Лінгвістика та паралінгвістика в контексті інтернет комунікації” (Linguistics and paralinguistics in the context of Internet communication), дата публікації: грудень, 2020 рік.
Мета: полягає у дослідженні та аналізі лінгвістичних та паралінгвістичних аспектів англомовного інтернет спілкування. Теоретичне значення: полягає у тому, що в роботі здійснюється подальший розвиток теорії англомовного дискурсу, зокрема, на рівні діалогічного спілкування, з урахуванням взаємодії вербальних і невербальних засобів передачі інформації та специфіки цього мовного регістру; розширюються та поглиблюються наукові знання про мовні особливості електронного спілкування, єдність та взаємодію мови і мовлення у цій формі комунікації. Особлива увага приділяється питанням співвідношення усного та писемного мовлення в процесі реалізації електронних повідомлень, що мають інтерактивний характер. Мова електронного спілкування виступає у якості особливої форми існування природної мови, що реалізується в специфічних умовах і відображає процес взаємодії усної та письмової форм сучасної англійської мови.
Цель: заключается в исследовании и анализе лингвистических и паралингвистических аспектов англоязычного интернет общения. Теоретическое значение: заключается в том, что в работе осуществляется дальнейшее развитие теории англоязычного дискурса, в частности, на уровне диалогического общения, с учетом взаимодействия вербальных и невербальных средств передачи информации и специфики этого языкового регистра; расширяются и углубляются научные знания о языковых особенностях электронного общения, единство и взаимодействие языка и речи в этой форме коммуникации. Особое внимание уделяется вопросам соотношения устной и письменной речи в процессе реализации электронных сообщений, имеющих интерактивный характер. Язык электронного общения выступает в качестве особой формы существования естественного языка, реализуется в специфических условиях и отражает процесс взаимодействия устной и письменной форм современного английского языка.

Notre thèse focalise sur une analyse phonostylistique du discours politique oral de M. Dominique de Villepin. Une étude qui aborde les variations de son style vocal dans plusieurs de ces discours oraux, d'une manière qui nous permet d'analyser les éléments discursifs et prosodiques tout au long de ses discours politiques. L'étude portera sur le double codage (linguistique et paralinguistique) caché derrière ses propos politiques. Notre corpus d'étude est l'ensemble d'échantillons tirés d'entretien, débat, conférence, interview. Cette étude, par ailleurs, soulève sa compétence discursive et communicationnelle manifestée quant à l'emploi des unités linguistiques, prosodiques, et discursive perçus comme un rôle praxéologique de pratiques sociales, politiques et de l'expérience dans son discours. Au niveau discursif, l'étude analyse des éléments pragmatiques tels que le tour de la parole les temps verbaux et les effets persuasifs. Au niveau phonostylistique, plusieurs phénomènes prosodiques de valeurs phonostylistiques (intonation, mélodie, accent et pause) seront analysés à l'aide du logiciel Praat pour connaître les différentes rôles pragmatiques et énonciatives en termes de stratégie de persuasion de cet homme politique. La thèse consiste à démontrer le rôle de ces phénomènes prosodiques dans l'établissement des informations nouvelles capables de transmettre un deuxième message vocal à son public. Ce message phonostylistique, qui renforce le premier message discursif des forces illocutoires, rajoute au style de Dominique de Villepin un nouveau procédé d'influence et de convaincre sur un public auditoire
This thesis studies the phonostylistical analysis of Mr. Dominique de Villepin's political oral discourse. A study which requires several of his vocal style in his different oral speeches in a way that allows us to analyze the discourse and prosodic elements throughout his political speeches. The study will focus on the dual coding (linguistic and paralinguistic) hidden behind his political purposes. The corpus of study is the set of samples taken from his different speeches: debate, conference, interviews. This study also rises to explain his discursive and communicative competence which is manifested by using linguistic, prosodic and discursive units in order to disclose the praxeological role of social practices, political roles and the role of experience in his speeches. Concerning the discursive level, the present study aims to analyze the pragmatic elements such as the tour of speech and verbal and persuasive effects. Concerning the phonostylistical level, several of phonostylistic phenomena which have prosodic values (intonation, melody, accent and pause) are analyzed under the Praat software to identify different pragmatic and enunciative roles in terms of persuasive strategy of this political man. The thesis is to demonstrate the role of prosodic phenomena in the establishment of new information, capable of transmitting a second vocal message to his audience. This phonostylistical message, which supports the first message carried through discursive illocutionary forces, adds to the style of Dominique de Villepin a new method to influence and persuade the public audience

This thesis is about the effectiveness of speech features for cognitive load assessment, with particular attention being paid to new perspectives of this research area. A new cognitive load database, called CoLoSS, is introduced containing speech recordings of users who performed a learning task. Various acoustic features from different categories including prosody, voice quality, and spectrum are investigated in terms of their relevance. Moreover, Teager energy parameters, which have proven highly successful in stress detection, are introduced for cognitive load assessment and it is demonstrated how automatic speech recognition technology can be used to extract potential indicators. The suitability of the extracted features is systematically evaluated by recognition experiments with speaker-independent systems designed for discriminating between three levels of load. Additionally, a novel approach to speech-based cognitive load modelling is introduced, whereby the load is represented as a continuous quantity and its prediction can thus be regarded as a regression problem.
Die vorliegende Arbeit befasst sich mit der automatischen Erkennung von kognitiver Belastung auf Basis menschlicher Sprachmerkmale. Der Schwerpunkt liegt auf der Effektivität von akustischen Parametern, wobei die aktuelle Forschung auf diesem Gebiet um neuartige Ansätze erweitert wird. Hierzu wird ein neuer Datensatz – als CoLoSS bezeichnet – vorgestellt, welcher Sprachaufzeichnungen von Nutzern enthält und speziell auf Lernprozesse fokussiert. Zahlreiche Parameter der Prosodie, Stimmqualität und des Spektrums werden im Hinblick auf deren Relevanz analysiert. Darüber hinaus werden die Eigenschaften des Teager Energy Operators, welche typischerweise bei der Stressdetektion Verwendung finden, im Rahmen dieser Arbeit berücksichtigt. Ebenso wird gezeigt, wie automatische Spracherkennungssysteme genutzt werden können, um potenzielle Indikatoren zu extrahieren. Die Eignung der extrahierten Merkmale wird systematisch evaluiert. Dabei kommen sprecherunabhängige Klassifikationssysteme zur Unterscheidung von drei Belastungsstufen zum Einsatz. Zusätzlich wird ein neuartiger Ansatz zur sprachbasierten Modellierung der kognitiven Belastung vorgestellt, bei dem die Belastung eine kontinuierliche Größe darstellt und eine Vorhersage folglich als ein Regressionsproblem betrachtet werden kann.

It is widely accepted in our society that people’s paralinguistic (i.e., non-semantic characteristics of the voice) and nonverbal (i.e., posture, gestures, and facial expressions) behaviours play an important role in conveying information about their personality traits. Two particularly relevant traits include one’s preferred levels of dominance and affiliation, which are the two major axes of the interpersonal circumplex. The current study investigates how dominance and affiliation are conveyed through paralinguistic and nonverbal behaviour using a lens model framework. Two major issues addressed by this framework include: 1) How do observers make inferences about people’s dominance and affiliation using paralinguistic and nonverbal behaviours and 2) How do people’s trait dominance and affiliation relate to these behavioural cues? To examine these two questions, we collected data from 114 opposite-sex dyads who worked together to complete a relatively unstructured collaborative task. The videotaped interactions were coded for specific paralinguistic (e.g., pitch, volume, resonance) and nonverbal (e.g., hand gestures, trunk posture, facial expressions) behaviours, in addition to coding more global displays of dominance and affiliation. Participants also completed several measures of trait dominance and affiliation, which tapped both their relatively conscious (i.e., explicit) and their relatively unconscious (i.e., implicit) levels of these traits. Our findings suggest that observers used mainly paralinguistic behaviour to infer dominance and mainly nonverbal behaviour to infer affiliation. In comparison to observers’ perceptions, there were fewer significant relations between individuals’ self-reported trait dominance and affiliation and the nonverbal and paralinguistic behaviours they expressed during the interaction, suggesting that people may have limited conscious awareness of how these behaviours convey information about their trait dominance and affiliation. In line with this idea, several behaviours showed relations to implicit measures of trait dominance and affiliation. We also conducted factor analyses of the measured paralinguistic and nonverbal behaviours, to examine whether or not these behaviours might co-occur as subsets or factors. We found that paralinguistic and nonverbal behaviours can be captured by overarching factors which relate meaningfully to measures of dominance and affiliation. Finally, we demonstrated that dyad members’ paralinguistic and nonverbal behaviours become interdependent as they interact with one another.

The thesis is dedicated to the specifics of Paralinguistics. In the research there discussed the features of paralinguistic devices in different types of mixed texts in English and Chinese, their impact on the verbal communication in both oral and written language. The structure of the work consists of two chapters: theoretical and practical. The theoretical part examines the theoretical background of the research, the basic concepts and essence of Paralinguistics, classification of paralinguistic devices, concept and essence of mixed texts and their variety. In the second chapter, on the basis of practical materials, there determined and analyzed, cases, the use of certain paralinguistic devices, their functions, interaction with the verbal component of the text and results. The main resultss of the thesis are presented and published in the scientific article "Non-verbal Communication in Business Relations with China" at the International Scientific and Practical Conference "Discovery Science Research" (Petrozavodsk, Russia, 2020). Awarded by Diploma I.
Данное магистерское диссертационное исследование посвящено специфике науки «паралингвистика». В ходе исследования рассматриваются особенности паралингвистических средств в разных типах смешанных текстов на английском и китайском языках, их влияние на вербальную сторону коммуникации как в устной, так и в письменной речи. Структура работы состоит из двух глав: теоретической и практической. В теоретической части рассматриваются теоретические предпосылки изучения, основные понятия и сущность паралингвистики, классификация паралингвистических средств, понятие и сущность смешанных текстов, и их разновидность. Во второй главе на основе практических материалов определяются и анализируются ситуации, применение тех или иных паралингвистических средств, их функции, взаимодействие с вербальной составляющей текста и результаты. Основные положения диссертационного исследования представлены и опубликованы в научной статье «Невербальная коммуникация в деловых отношениях с Китаем» на Международной научно-практической конференции «Discovery Science Research» (г. Петрозаводск, 2020 г.). Диплом I степени.

The topic of the thesis is to monitor differences in communication between men and women. The theoretical part summarizes the information about communication in general and outlines the differences in communication between men and women. The empirical part is based on interview. We examine possible differences of verbal communication between men and women with university education. Following characteristics were examined: formality, expressions, vulgar, addressing, topics, and the role in communication and interruptions. The results provide information about relatively specific differences in verbal communication between men and women. It was found that men use significantly more technical terms, use more vulgar words, while women choose such expressions exeptionally. Men do not interrupt the other speaker, women interrupt in more than 50% cases.. The results confirm previously published findings on this topic.

La voix est tout sauf un stimulus auditif ordinaire. Pour cause, elle prend son importance de manière très précoce chez l’Homme lorsque, dans l’environnement amniotique, le fœtus entend pour la toute première fois la voix de sa mère. C’est en quelque sorte par l’intermédiaire de cette voix que les premiers contacts avec le monde extérieur, mais également avec l’Autre, s’effectuent. Le statut particulier de la voix humaine perdure au fil du développement, devenant plus tard le principal médium véhiculant le langage oral, si significatif pour l’Homme. En parallèle, et de manière tout aussi adaptative, elle permet la transmission d’informations non langagières renseignant sur l’identité, l’état émotionnel mais également le statut social de chaque individu. C’est ainsi que simplement en entendant la voix d’une personne inconnue, il est généralement possible d’en extrapoler son âge, son genre, mais également d’avoir une idée assez précise de l’état émotionnel dans lequel elle se trouve. Les capacités permettant d’extraire de la voix les divers éléments informationnels qu’elle contient ne seraient toutefois pas stables au fil du temps. Ainsi, le vieillissement normal semble associé à des difficultés de traitement des informations vocales de nature langagière, mais également non langagière. De nombreuses études se sont intéressées au déclin des capacités de traitement du discours avec l’âge. Beaucoup moins de travaux ont cependant considéré les conséquences du vieillissement sur le domaine paralinguistique et, lorsque des travaux s’y sont attardés, c’est essentiellement la sphère affective qui a été investiguée. En raison de ce peu d’études, mais également de leur focus portant spécifiquement sur la sphère émotionnelle, il est extrêmement ardu de généraliser les résultats obtenus au traitement vocal général. La présente thèse s’est donc intéressée aux capacités de traitement de la voix dans le vieillissement normal. La première étude de cette thèse (Article 1) avait pour objectif d’évaluer l’impact du vieillissement normal sur les capacités comportementales de traitement paralinguistique vocal non émotionnel. Pour ce faire, une batterie informatisée composée de quatre tâches a été élaborée : la batterie d’évaluation de la perception vocale (Batterie EPV; tâches de catégorisation de genre, de discrimination de sources sonores, adaptative de discrimination et de mémorisation). Cette batterie permettait de comparer les performances d’adultes jeunes et âgés lors du traitement de stimuli vocaux et non vocaux, mais également lors du traitement de divers stimuli vocaux. Cette première étude met en évidence, pour trois des quatre tâches comportementales, des performances inférieures chez les adultes âgés et ce, malgré le contrôle statistique des contributions du déclin auditif et cognitif. Pour les aînés, le traitement de stimuli vocaux, en comparaison au traitement de stimuli non vocaux, n’était toutefois pas systématiquement inférieur à celui des jeunes adultes. Sans que les performances ne puissent être prédites par la mesure cognitive utilisée comme covariable (performances au MoCA), il appert que les demandes cognitives inhérentes aux tâches participent à ces différences intergroupes. Le second article de ce travail visait quant à lui à explorer à l’aide de l’imagerie par résonnance magnétique fonctionnelle (IRMf), l’influence du vieillissement normal sur les réseaux neuronaux sous-tendant le traitement de l’information vocale, une telle investigation n’ayant jamais été effectuée auparavant. Pour ce faire, une tâche d’écoute passive (permettant le contraste de blocs de sons vocaux et non vocaux) ainsi qu’une tâche adaptative de discrimination ont été utilisées. La tâche adaptative, basée sur une type de protocole psycoacoustique « up-down », assurait l’obtention de niveau de performance équivalent entre les deux groupes, une condition nécessaire pour la comparaison de groupe au niveau neurofonctionnel. La comparaison des adultes jeunes et âgés n’a mis en évidence aucune disparité quant au recrutement des aires répondant préférentiellement à la voix : les aires vocales temporales (AVT). Ce résultat suggère que l’âge n’affecte pas la mobilisation des aires spécialisées dans le traitement de la voix. Néanmoins, à l’extérieur des AVT et chez les aînés, le sous recrutement d’une portion du cortex auditif a été observé, en parallèle au recrutement additionnel de régions pariétale, temporale et frontale (Article 2 – Étude 1). Lors de la réalisation d’une tâche adaptative de discrimination, contrairement à ce qui était attendu, les seuils de discrimination des deux groupes d’âges étaient comparables. Pour effectuer la tâche, les participants âgés ont cependant recruté un réseau neuronal plus étendu que celui des jeunes adultes, et pour les aînés, l’activation additionnelle de régions frontale et temporale sous-tendaient la réalisation de la tâche (Article 2 - Étude 2). Les données comportementales présentées dans cette thèse suggèrent que l’effet délétère que semble avoir le vieillissement normal sur les capacités de traitement paralinguistique vocal affectif est également retrouvé lors du traitement d’informations vocales émotionnellement neutres. En parallèle, la mise en place de phénomènes de plasticité cérébrale est objectivée. Ces derniers ne toucheraient cependant pas les réseaux spécialisés dans le traitement de la voix, qui seraient recrutés de manière comparable par les adultes jeunes et âgés. Néanmoins, la tâche d’écoute passive a mis en évidence la présence, chez les aînés, du recrutement sous-optimal d’une portion du cortex auditif (gyrus temporal transverse). En parallèle, et ce pour les deux tâches, des réseaux neuronaux surnuméraires étaient sollicitées par les adultes âgés, permettant potentiellement d’assurer, chez les ainés, le maintien de performances adéquates.
Amongst all the sounds present in our environment, the human voice is probably the most significant one. Indeed, it takes its importance very early on, when, in the amniotic environment, the foetus hears the voice of its mother. Through this voice, the first contacts with the World but also with others are made. This special status of voice remains during the development, since it later becomes the medium via which speech is carried. Indeed, not only does voice convey oral language, but it also passes on a wide array of paralinguistic information characterising each individual. In fact, simply by hearing a voice, we are generally able to determine the age, identity, social status but also the mood in which the speaker is at the moment. However, during normal aging, the processing of the information contained in voice, may it be linguistic or not, appears to become more difficult. In the paralinguistic domain, the deleterious effects of normal aging were mainly explored via affective prosody. It is thus quite difficult to generalize those results to the processing of vocal information in general. In addition, cerebral reorganization accompanying normal aging was characterized for speech perception, but the effects of age on the cerebral processing of paralinguistic vocal information still remain unclear. The main purpose of this work was to palliate to this shortcoming. The first study presented in this thesis (Article 1) evaluated the impact of normal aging on the behavioral processing of non affective paralinguistic vocal information by comparing the performances of young and older adults. To do so, a computerized battery was created: the Voice Perception Assessment battery, which comprised four behavioral tasks (VPA battery; gender categorization task, auditory source discrimination task, adaptive discrimination task and memory task). The tasks created allowed the comparison of vocal and non vocal processing, as well as the comparison of the processing of different types of vocal stimuli. Our results indicate that, while controlling for cognitive and auditory decline, older adults were less efficient than younger participants for three of the four VPA tasks. However, a systematic age-related decline for vocal processing was not observed. The task performances could not be predicted by the cognitive measure selected as a control variable (MoCA scores). However, the disparities in task demands seem to explain a portion of the paralinguistic vocal processing age-related decline observed. The main purpose of the second article was to empirically assess, using functional magnetic resonance imagery (fMRI), the impact of normal aging on the neural networks underlying the processing of human voice, as it has never been done yet. To do so, a passive listening task (contrasting vocal and non vocal blocks of sounds) and an adaptive discrimination task were used. The adaptive task, based on a transformed up-down protocol, allowed us to insure equivalent performance levels in both age-groups, a necessary condition to neurofunctional group comparison. Our results indicate that young and older adults recruited similarly the cortical areas preferentially responding to voice, the Temporal Voice Areas (TVA). In parallel, outside the TVA, an age-related under recruitment of a portion of the auditory cortex was observed, as well as an over recruitment of parietal, temporal and frontal regions (Article 2 – Study 1). For the adaptive discrimination task, contrary to what was expected, the discrimination thresholds of both age-groups were similar. Nevertheless, to be able to perform the task, older adults needed to engage cortical regions in the temporal and frontal lobes that younger adults did not recruit (Article 2 – Study 2). The behavioral data presented in this work suggest that normal aging can be associated with a decline of the capacity to process non affective paralinguistic vocal information. The observed age-related differences appear to be more salient when the cognitive demands of the task are high. In parallel, mechanisms of cerebral plasticity appear to take place as well. Those phenomena however do not seem to affect the way the cortical areas specialized in the processing of human voice are recruited, which remains the same between young and older adults. However, the passive listening task elicited an age-related under-recruitment for a portion of the auditory cortex (left transverse temporal gyrus). Both for the passive listening task and the adaptive discrimination task, an over-recruitment of additional networks were observed for older adults. This extended network could contribute to the preservation of adequate performance in older adults.