Academic literature on the topic 'Audiovisual Uncertainty'

The natural environments in which infants and children learn speech and language are noisy and multimodal. Adults rely on the multimodal nature of speech to compensate for noisy environments during speech communication. Multiple mechanisms underlie mature audiovisual benefit to speech perception, including reduced uncertainty as to when auditory speech will occur, use of correlations between the amplitude envelope of auditory and visual signals in fluent speech, and use of visual phonetic knowledge for lexical access. This paper reviews evidence regarding infants’ and children’s use of temporal and phonetic mechanisms in audiovisual speech perception benefit. The ability to use temporal cues for audiovisual speech perception benefit emerges in infancy. Although infants are sensitive to the correspondence between auditory and visual phonetic cues, the ability to use this correspondence for audiovisual benefit may not emerge until age four. A more cohesive account of the development of audiovisual speech perception may follow from a more thorough understanding of the development of sensitivity to and use of various temporal and phonetic cues.

At present, the analysis of the results of advertising and marketing studies is done qualitatively in terms of the experience of a marketing analyst, thus generating little certainty and uncertainty of the effectiveness of the feelings and the message emitted. Brands are connected with the idea that the sender wants to transmit. For this, the marketing has studied the behavior of the consumer when exposed to different advertising stimuli, in order to understand the behavior at the time of the stimulus and to achieve alignment of the message to be transmitted with what is actually perceived. In this paper we propose a vector model based on computational intelligence and Neuromarketing studies that allows the identification of four basic emotions: joy, fear, anger and sadness from the bioelectric brain activity recorded by a person exposed to a certain audiovisual advertising. The results of the model allowed the identification of emotions in audiovisual advertising, which constitutes a tool that allows companies to create audiovisual advertising that guarantees greater commitment and effectiveness of the advertising segments with which it wants to impact the marketKeywords: Neuromarketing, audiovisual advertising, basic emotions, neural networks, Emotiv-EPOC®, vector support machines

Background Risk information in patient decision aids (PDAs) is often difficult for older patients to process. Providing audiovisual and narrative information may enhance the understanding and use of health-related information. We studied the effects on patients’ information processing and use of audiovisual and narrative information of an early-stage non–small-cell lung cancer treatment decision aid explaining surgery and stereotactic ablative radiotherapy. We further investigated differences between older and younger patients. Methods We conducted a 2 (modality: textual v. audiovisual) × 2 (narration style: factual v. narrative) online experiment among cancer patients and survivors ( N = 305; Mage = 62.42, SD = 11.68 y). Age was included as a potential modifier: younger (<65 y) versus older (≥65 y) age. We assessed 1) perceived cognitive load, 2) satisfaction with information, 3) comprehension, 4) information recall, and 5) decisional conflict. Analysis of variance was used for data analysis. Results Irrespective of patient age, audiovisual information (compared with textual information) led to lower perceived cognitive load, higher satisfaction with information, and lower decisional conflict (subscale Effective Decision). Narrative information (compared with factual information) led to reduced decisional conflict (subscale Uncertainty) but only in younger patients. Combining audiovisual information with factual information also resulted in lower perceived cognitive load in younger patients as compared with older patients. Limitations Patients who actually face the decision, especially older patients, might be more motivated to process our decision-aid information than the present study participants who responded to a hypothetical situation online. Conclusions Providing participants with audiovisual information, irrespective of their age, improved their processing and use of information in a decision aid. Narratives did not clearly benefit information processing.

Quality assessment of audiovisual (AV) signals is important from the perspective of system design, optimization, and management of a modern multimedia communication system. However, automatic prediction of AV quality via the use of computational models remains challenging. In this context, machine learning (ML) appears to be an attractive alternative to the traditional approaches. This is especially when such assessment needs to be made in no-reference (i.e., the original signal is unavailable) fashion. While development of ML-based quality predictors is desirable, we argue that proper assessment and validation of such predictors is also crucial before they can be deployed in practice. To this end, we raise some fundamental questions about the current approach of ML-based model development for AV quality assessment and signal processing for multimedia communication in general. We also identify specific limitations associated with the current validation strategy which have implications on analysis and comparison of ML-based quality predictors. These include a lack of consideration of: (a) data uncertainty, (b) domain knowledge, (c) explicit learning ability of the trained model, and (d) interpretability of the resultant model. Therefore, the primary goal of this article is to shed some light into mentioned factors. Our analysis and proposed recommendations are of particular importance in the light of significant interests in ML methods for multimedia signal processing (specifically in cases where human-labeled data is used), and a lack of discussion of mentioned issues in existing literature.

Perceiving the temporal order of sensory events typically depends on participants' attentional state, thus likely on the endogenous fluctuations of brain activity. Using magnetoencephalography, we sought to determine whether spontaneous brain oscillations could disambiguate the perceived order of auditory and visual events presented in close temporal proximity, that is, at the individual's perceptual order threshold (Point of Subjective Simultaneity [PSS]). Two neural responses were found to index an individual's temporal order perception when contrasting brain activity as a function of perceived order (i.e., perceiving the sound first vs. perceiving the visual event first) given the same physical audiovisual sequence. First, average differences in prestimulus auditory alpha power indicated perceiving the correct ordering of audiovisual events irrespective of which sensory modality came first: a relatively low alpha power indicated perceiving auditory or visual first as a function of the actual sequence order. Additionally, the relative changes in the amplitude of the auditory (but not visual) evoked responses were correlated with participant's correct performance. Crucially, the sign of the magnitude difference in prestimulus alpha power and evoked responses between perceived audiovisual orders correlated with an individual's PSS. Taken together, our results suggest that spontaneous oscillatory activity cannot disambiguate subjective temporal order without prior knowledge of the individual's bias toward perceiving one or the other sensory modality first. Altogether, our results suggest that, under high perceptual uncertainty, the magnitude of prestimulus alpha (de)synchronization indicates the amount of compensation needed to overcome an individual's prior in the serial ordering and temporal sequencing of information.

The situation of uncertainty, which became a consequence of the pandemic, forced the audience to actively interact with the media. This effect is especially noticeable in the field of video information. Because of this, the ability to check the quality of such information and to detect a video fake is especially in high demand. Meanwhile, the researchers' interest is focused primarily on the verbal component of media content. In our study, we are trying to fill this gap and concretize the models of generating video fakes, their types, and reasons for their appearance.Based on the analysis of fake audiovisual content distributed in the media sphere in 2020 we identify two models of its appearance - synchronous and asynchronous. In the first case, fake is a result of distorting the video and audio of the work. In the second case, only one of the rows broadcasts a false idea. The latter case is dangerous in that the series of works that do not contain false information inspires confidence in the audience and makes it accept the media message as a whole.Also, in the study, we single out the most common types of fakes based on such characteristics as the degree of information distortion, the degree of reliability of spatial and temporal characteristics, and the degree of reliability of the source. We determine the most frequent markers of fakes in video works, namely: distortion of the shooting angle, concealment of the staged nature of filming, the use of animation and animation technologies that imitate newsreel footage, placement of inaccurate infographics in video work, fake news announcement, and publication.Among the most common reasons for the formation of video fakes, we note the desire to increase ratings, discriminate against specific individuals or organizations, draw attention to a real problem, and to entertain the audience.In the analysis of fake as a global phenomenon using the example of deepfake technology, we show how it can be used in constructively and destructively and emphasize the importance of developing media education to neutralize the negative consequences of the spread of fakes.

Keywords: video game, intellectual property law, computer program, software, digitalinformation, electronic information, multimedia work, audiovisual work The article is devoted to thestudy of the legal understanding of video games as an object of intellectual propertyrights. The author concludes that video games constitute digital information, whichgreatly facilitates their development, however, complicates the protection of intellectualproperty rights. There is a contradictory connection between the concepts of«electronic (digital) information», «audiovisual work», «computer program» and «literarywork», which establishes some uncertainty in the protection of video games,namely, what should be understood by these concepts in terms of law.The author found that the use of the latest technologies to improve the visual componentbrings the video game even closer to the game audiovisual object, which increasesthe complexity of the process of protection of intellectual property rights. However, thecurrent situation with the use of the term «computer program» in the context of protectionof intellectual property rights to video games does not cover all features of the latter,covering only the main program among the software recording and operation of thegame, such as audiovisual content. Therefore, there is a need to use the generalized conceptof «software» as opposed to the narrower concept of «computer program». In thiscase, in the absence of at least one of the characteristics of the multimedia work, the computerprogram should be referred to the scope of legal regulation of software.The author proposed to understand the video game as object of intellectual propertylaw in the following manner: multimedia work expressed in the form of digital(electronic) information, the principle of which is determined by the algorithms of thesoftware provided for installation in the memory of computer devices.In general, the digital nature of video games presupposes a revision of existing approachesto the protection of intellectual property rights, which may be aimed atweakening the regulatory function of the law where it is possible to regulate the relationshipby more flexible means.

Es presenta un esquema únic per a la síntesi i anàlisi audiovisual personalitzable realista de seqüències audiovisuals de cares parlants i seqüències visuals de llengua de signes en àmbit domèstic. En el primer cas, amb animació totalment sincronitzada a través d'una font de text o veu; en el segon, utilitzant la tècnica de lletrejar paraules mitjançant la ma. Les seves possibilitats de personalització faciliten la creació de seqüències audiovisuals per part d'usuaris no experts. Les aplicacions possibles d'aquest esquema de síntesis comprenen des de la creació de personatges virtuals realistes per interacció natural o vídeo jocs fins vídeo conferència des de molt baix ample de banda i telefonia visual per a les persones amb problemes d'oïda, passant per oferir ajuda a la pronunciació i la comunicació a aquest mateix col·lectiu. El sistema permet processar seqüències llargues amb un consum de recursos molt reduït, sobre tot, en el referent a l'emmagatzematge, gràcies al desenvolupament d'un nou procediment de càlcul incremental per a la descomposició en valors singulars amb actualització de la informació mitja. Aquest procediment es complementa amb altres tres: el decremental, el de partició i el de composició.
Se presenta un esquema único para la síntesis y análisis audiovisual personalizable realista de secuencias audiovisuales de caras parlantes y secuencias visuales de lengua de signos en entorno doméstico. En el primer caso, con animación totalmente sincronizada a través de una fuente de texto o voz; en el segundo, utilizando la técnica de deletreo de palabras mediante la mano. Sus posibilidades de personalización facilitan la creación de secuencias audiovisuales por parte de usuarios no expertos. Las aplicaciones posibles de este esquema de síntesis comprenden desde la creación de personajes virtuales realistas para interacción natural o vídeo juegos hasta vídeo conferencia de muy bajo ancho de banda y telefonía visual para las personas con problemas de oído, pasando por ofrecer ayuda en la pronunciación y la comunicación a este mismo colectivo. El sistema permite procesar secuencias largas con un consumo de recursos muy reducido gracias al desarrollo de un nuevo procedimiento de cálculo incremental para la descomposición en valores singulares con actualización de la información media.
A shared framework for realistic and personalizable audiovisual synthesis and analysis of audiovisual sequences of talking heads and visual sequences of sign language is presented in a domestic environment. The former has full synchronized animation using a text or auditory source of information; the latter consists in finger spelling. Their personalization capabilities ease the creation of audiovisual sequences by non expert users. The applications range from realistic virtual avatars for natural interaction or videogames to low bandwidth videoconference and visual telephony for the hard of hearing, including help to speech therapists. Long sequences can be processed with reduced resources, specially storing ones. This is allowed thanks to the proposed scheme for the incremental singular value decomposition with mean preservation. This scheme is complemented with another three: the decremental, the split and the composed ones.

While previous research on language development has highlighted the facilitating role of gesture (mainly deictic gesture) in children’s early access to meaning, little is known about the possible facilitating role of prosody and also of co-speech gestures in children’s pragmatic development in the preschool years. Previous work on developmental pragmatics has focused on the acquisition of morphosyntactic and lexical forms, and there is a need to adopt a more integrative multimodal perspective. The overarching aim of this thesis is to experimentally investigate, through a set of cross-sectional studies with preschool children (3- to 5-year-olds), whether prosodic and gestural cues serve as pragmatic precursors in the development of two key pragmatic phenomena, namely knowledge state (i.e., commitment to the status of information) and politeness (e.g., broadly speaking, adjusting one’s language). The first study uses a forced-choice paradigm to investigate preschool children’s understanding of another speaker’s knowledge state, presented in audio only, video only and audio-visual formats. Results show that overall children perform significantly better in detecting a speaker’s uncertainty when they have gestural cues present, and, importantly, the younger children were significantly better in detecting a speaker’s uncertainty when listening to a speaker’s intonation contour as compared to uncertainty expressed through a lexical epistemic adverb. The second study analyzes children’s (and adults’) multimodal expression of their knowledge state through an object guessing game by evaluating their production of prosodic, gestural and lexical cues and additionally assessing their self-assessment of their knowledge state. Results show that while preschool children are not yet able to self-report on their knowledge state, in the younger group, children encode their knowledge state through prosodic and gestural means only. And only in the older age group do children start to use a few lexical markers to signal their uncertainty. The third study uses a forced-choice paradigm to assess children’s understanding of a speaker’s politeness presented in audio only, video only and audio-visual formats. Results show that 3-year-old children detect a speaker’s polite stance significantly more through facial cues and intonation, highlighting children’s early ability to extract meaning from intonation when lexical cues are kept the same. Finally, the fourth study explores children’s multimodal production of politeness in semi-spontaneous requests in different sociopragmatic situations. Results show that regardless of the age group, children marked politeness through fine-grained gestural and prosodic means when producing requests to an adult with high social distance as compared to a classmate with low social distance, and this also depends on the cost of the request. Altogether the results of these studies demonstrate that children’s early pragmatic comprehension and expressive abilities rely strongly on prosodic and gestural marking, developing well before children master lexical and morphosyntactic markers. More specifically, the four studies presented in this thesis bring forth evidence that both prosody and co-speech gestures play a precursor role in children’s pragmatic development of knowledge state and politeness. Ultimately, the thesis highlights the importance of approaching the study of children’s pragmatic development from a multimodal perspective.
Tot i que les investigacions prèvies sobre el desenvolupament del llenguatge han destacat el paper facilitador del gest (principalment, del gest díctic o d’assenyalament) en l'accés primerenc dels nens al significat del llenguatge, se sap poc sobre els beneficis de la prosòdia i dels gestos de la parla en el desenvolupament pragmàtic dels nens en els anys preescolars. La recerca que s’ha fet sobre el desenvolupament de la pragmàtica s'ha centrat en l'adquisició de formes morfosintàctiques i lèxiques, però és important adoptar una perspectiva multimodal més integradora. L'objectiu general d'aquesta tesi consisteix a investigar experimentalment, a través de quatre estudis transversals amb nens d’edat preescolar (de 3 a 5 anys d'edat), si els senyals prosòdics i gestuals actuen com a precursors pragmàtics en el desenvolupament de dos fenòmens pragmàtics, concretament el posicionament epistèmic (i.e., el grau de certesa del parlant sobre la informació expressada) i la cortesia. El primer estudi utilitza un paradigma d'elecció forçada per investigar la comprensió dels nens en edat preescolar sobre el posicionament epistèmic d'un altre parlant, presentat en tres condicions (només en àudio, només en vídeo i en format audiovisual). Els resultats mostren que els nens detecten millor la incertesa d'un parlant quan estan exposats a indicis gestuals. Crucialment, els nens de tres anys detecten millor la la incertesa d'un parlant quan la incertesa s'expressa a través de la prosòdia que quan s'expressa a través d’adverbis epistèmics. El segon estudi analitza l'expressió multimodal de l’estat epistèmic dels nens (i dels adults) a través d’un joc d’endevinar objectes. L'anàlisi dels senyals prosòdics, gestuals i lèxics mostren que, mentre els nens d’edat preescolar encara no poden valorar el seu posicionament epistèmic en canvi sí que codifiquen el seu grau de certesa a través de senyals prosòdics i gestuals. Només en el grup d'edat més gran els nens comencen a utilitzar alguns marcadors lèxics. El tercer estudi utilitza un paradigma d'elecció forçada per avaluar la comprensió dels nens sobre la cortesia d'un parlant, presentada també en tres condicions (només en àudio, només en vídeo i en format audiovisual). Els resultats mostren que els nens de 3 anys poden detectar una actitud educada a través de senyals facials i entonatius quan es mantenen constants les marques lèxiques. Els resultats mostren la capacitat inicial dels nens per extreure el significat de l'entonació, de forma paral·lela al que passa amb la gestualitat. Finalment, el quart estudi explora l'expressió multimodal de la cortesia en peticions emprades en diferents situacions sociopragmàtiques. Els resultats mostren que, independentment del grup d'edat, els nens marquen la cortesia a través de senyals gestuals i prosòdics detallats quan es fan peticions a un adult amb més distància social, en comparació amb un company amb qui presenten una distància social menor. L'ús d'aquestes marques també depèn del cost de la petició. En resum, els resultats dels quatre estudis de la tesi demostren que les habilitats primerenques de comprensió i d’expressió pragmàtica es basen en el marcatge prosòdic i gestual. i que aquestes es desenvolupen molt abans que els nens controlin els marcadors lèxics i morfosintàctics. Així, els quatre estudis presentats en aquesta tesi posen de manifest que tant la prosòdia com els gestos de la parla actuen com a precursors en el desenvolupament pragmàtic dels llenguatge. En definitiva, la tesi posa de relleu la importància d’aproximar-se a l'estudi del desenvolupament pragmàtic infantil des d'una perspectiva multimodal.
A pesar de que las investigaciones previas sobre el desarrollo del lenguaje hayan destacado el papel facilitador del gesto (principalmente, del gesto deíctico o de señalamiento) en el acceso temprano de los niños al significado del lenguaje, poco se sabe aún sobre los beneficios de la prosodia y de los gestos del habla en el desarrollo pragmático de los niños en etapa preescolar. Hasta ahora, las investigaciones sobre el desarrollo de la pragmática se han centrado en la adquisición de formas morfosintácticas y léxicas, haciéndose necesario adoptar una perspectiva multimodal más integradora. El objetivo general de esta tesis consiste en investigar experimentalmente —a través de cuatro estudios transversales con niños de edad preescolar (de 3 a 5 años de edad)— si las señales prosódicas y gestuales actúan como precursores en el desarrollo de dos fenómenos pragmáticos, concretamente del posicionamiento epistémico (i.e., del grado de certeza del hablante sobre la información expresada) y de la cortesía. El primer estudio utiliza un paradigma de elección forzada para investigar cómo los niños en edad preescolar comprenden el posicionamiento epistémico de otro hablante, presentado los estímulos en tres condiciones (solo en audio, solo en vídeo y en formato audiovisual). Los resultados muestran que los niños detectan mejor la incertidumbre de un hablante cuando están expuestos a indicios gestuales. Crucialmente, los resultados también muestran que los niños de tres años detectan mejor la incertidumbre de un hablante cuando esta viene expresada a través de la prosodia que cuando lo hace a través de adverbios epistémicos. El segundo estudio analiza la expresión multimodal del estado epistémico de los niños (y de los adultos) a través de un juego de adivinar objetos. El análisis de las señales prosódicas, gestuales y léxicas muestra que, aunque los niños en edad preescolar todavía no pueden valorar su posicionamiento epistémico, los del grupo de menor edad ya son capaces de codificar su grado de certeza a través de señales prosódicas y gestuales, mientras que solo los niños del grupo de mayor edad empiezan a utilizar algunos marcadores léxicos. El tercer estudio utiliza también un paradigma de elección forzada para evaluar la comprensión de los niños sobre la cortesía de un hablante, presentado también los estímulos en tres condiciones (solo en audio, solo en vídeo y en formato audiovisual). Los resultados muestran que los niños de 3 años son capaces de detectar una actitud cortés significativamente mejor a través de señales faciales y entonativas, destacando además la capacidad temprana de los niños para extraer el significado de la entonación cuando las marcas léxicas se mantienen constantes. Finalmente, el cuarto estudio investiga la expresión multimodal de la cortesía en peticiones realizadas en diferentes situaciones socio-pragmáticas. Los resultados muestran, por un lado, que, independientemente del grupo de edad, los niños marcan la cortesía a través de señales gestuales y prosódicas diferenciadas en función de si las peticiones van dirigidas hacia un adulto con el que mantienen una mayor distancia social, o bien hacia un compañero con quien la distancia social es menor. Además, los resultados también muestran que el uso de estas marcas también depende del coste de la petición. En resumen, los resultados de los cuatro estudios de esta tesis demuestran que las habilidades tempranas de la comprensión y expresión de significados pragmáticos se basan en el marcaje prosódico y gestual, y que, además, estas habilidades se desarrollan mucho antes de que los niños dominen los marcadores léxicos y morfosintácticos. De este modo, los cuatro estudios presentados en esta tesis ponen de manifiesto que tanto la prosodia como los gestos del habla actúan como precursores en el desarrollo pragmático del lenguaje, subrayando así la necesidad de aproximarse al estudio del desarrollo pragmático infantil desde una perspectiva multimodal.

This chapter presents a comprehensive analysis of the audiovisual sector in a moment of change and disruption thereof. The different elements of the ecosystem (platforms and technologies, regulation, uses and business models) are analyzed by making a deep diagnosis of their evolution and future perspectives. The sector is heavily impacted by the emergence and integration of technologies that modify the economic status-quo, and the evolution of the sector is guided by the uncertainty of the consumers' behavior, requiring adaptive strategies and innovation on the business models, making them imaginative, flexible and collaborative between the consumer and the platforms. In this analysis of the audiovisual industry and its integration strategies from a technological, regulatory and consumer point of view, relevant phenomena of this market's evolution have been identified, opening new avenues of research to delve in the industrial and economic impact of the observed changes in order to understand the future of this challenging sector.

This chapter presents a comprehensive analysis of the audiovisual sector in a moment of change and disruption thereof. The different elements of the ecosystem (platforms and technologies, regulation, uses and business models) are analyzed by making a deep diagnosis of their evolution and future perspectives. The sector is heavily impacted by the emergence and integration of technologies that modify the economic status-quo, and the evolution of the sector is guided by the uncertainty of the consumers' behavior, requiring adaptive strategies and innovation on the business models, making them imaginative, flexible and collaborative between the consumer and the platforms. In this analysis of the audiovisual industry and its integration strategies from a technological, regulatory and consumer point of view, relevant phenomena of this market's evolution have been identified, opening new avenues of research to delve in the industrial and economic impact of the observed changes in order to understand the future of this challenging sector.

This report captures the various ways in which the cultural heritage sector is adapting, not only to cope with the uncertainty caused by the COVID-19 pandemic, but also to flourish in the future. It is the result of a joint virtual exchange between members of the International Association of Sound and Audiovisual Archives (IASA) and the International Federation of Television Archives (FIAT/IFTA). The aim was to gather professionals from the global archival community and to discuss how positive changes could be identified and sustained, to share best practices and individual experiences, and to collaboratively think out the best ways forward.