Tesi sul tema "Ambisonic"

This thesis explores the means of objectively measuring diffuse sound fields contained within music performance auditoria. Although the diffuse field is considered to be an important component of the reverberant sound field there is currently no widely accepted method for its measurement. A review of methods shows that attempts to characterize the field may be divided into those methods that seek to directly measure the state of the field and those that indirectly indicate the existence of the state. The primary focus of this thesis is the application of Ambisonic techniques to capture the spatial aspects of the sound field. Initial work explores the rotation of a directional microphone in three measurement spaces. The results and modeling in idealised simulated sound fields indicate that the method may have some efficacy. The method is extended through the application of signal processing to the output of an Ambisonic microphone array. The method is tested firstly in a reverberation room that is modified progressively to produce a series of room states with incrementally increasing reverberation time. The extents of the measurement system were tested by measuring the degrees of diffusivity reached in a reverberation room. Diffusing panels were progressively added in the expectation the increases in diffusivity would be detected. The measurement was carried out in conjunction with standard absorption coefficient measurements outlined in Appendix A of ISO 354. Comparison was made between the measured field and the standard method for achieving a diffuse field in a reverberation room test facility. The final stage attempts to find correlation between physical measures of diffuse fields and listener’s subjective assessment of those fields. To that end a paired comparison test was conducted where listeners were presented music samples rendered through simulated halls where the scattering coefficients and consequently the sound field diffusivity was varied. Subjects were asked to choose which pair they preferred.

Due to the recent advances in virtual and augmented reality, ambisonics has emerged as the de facto standard for immersive audio. Ambisonic audio can be captured using spherical microphone arrays, which are becoming increasingly popular. Yet, many methods for acoustic and microphone array signal processing are not speci cally tailored for spherical geometries. Therefore, there is still room for improvement in the eld of automatic analysis and description of ambisonic recordings. In the present thesis, we tackle this problem using methods based on the parametric analysis of the sound eld. Speci cally, we present novel contributions in the scope of blind reverberation time estimation, diffuseness estimation, and sound event localization and detection. Furthermore, several software tools developed for ambisonic dataset generation and management are also presented.

This research dissertation presents work undertaken to develop new performance techniques and mappings for the expressive control of spatial motion using Ambisonic projection. The dissertation reviews relevant research from the fields of Spatial Sound and Extended Instruments, and establishes playability as a useful set of criteria for a reflexive project methodology and evaluation. This reflexive research systematically investigates Trevor Wishart’s taxonomy of spatial motions through the development of new hardware, software, performance techniques and spatial motion analysis.

This thesis deals with cues used by the human auditory system to identify the location of sound and methods for sound localisation based these cues, namely, vector based amplitude panning and ambisonics, which are described in detail. These methods have been implemented as a VST plug-in module. This thesis also contains listening tests of second order ambisonics along with acquired data analysis.

There are several methods to get individualized HRTFs such as measurements, syntheses, or selection. The study first aims to select the fittest HRTFs in an existing database for listeners. The idea is developed based on the connection between anthropometric parameters and auditory localisation. By means of machine learning, the neural network designed for HRTF ranking produces a reliable prediction. The new approach has been verified by the anthropometry and listening perception of 24 subjects. The final selected dataset is used to synthesize a virtual audio scene. Two binaural ambisonic decoders are proposed to overcome the dilemma of improving sound localisation or enhancing audio quality. The first one is the equalization decoding, equalizing the root mean square (RMS) power in each 1/3 octave frequency bank, especially compensating the low-pass filtered components in a high-density speaker array. Therefore, the energy distribution of the treated signal is nearly uniform. The other proposed method is split-band decoding, selecting and then mixing the better reconstructed frequency components from different speaker arrays. Through several experiments and listening tests, there are no click noises in the real-time system, when the virtual auditory space is rapidly rotated. The split-band decoder presents comparable performance to a pure HRTF decoder.

La tesi si propone di studiare la percezione del senso di immersività nella musica in formato binaurale, con l’obiettivo di migliorare l’esperienza d’ascolto e metterla a confronto con la musica ascoltata in formato stereofonico. Attraverso un’analisi delle basi della fisica del suono e della psicoacustica, il saggio vuole illustrare quali sono le tecnologie che permettono alla musica immersiva di essere prodotta. A tal proposito verranno presi in esame sia la storia delle tecniche di riproduzione sonora, che gli strumenti attualmente disponibili, per permettere alla comunità scientifica di avere un quadro completo sulle risorse odierne. Il binomio tra musica e spatial audio è in continua crescita con il passare del tempo, per questo il testo si propone anche di evidenziare quali sono i principali problemi di ricerca e gli standard necessari sui quali concentrare eventuali prossimi lavori. Il saggio si conclude con un esperimento che mette a confronto due differenti tipologie di mix di una canzone: una versione stereofonica e una versione in formato binaurale, processata tramite la Digital Audio Workstation Logic Pro X e la Dolby Atmos Production Suite. I risultati porteranno delle evidenze interessanti a favore della versione in formato binaurale e diverse suggestioni sugli sviluppi futuri.
The thesis aims to study the perception of the sense of immersion in music in binaural format, with the target of improving the listening experience and comparing it with the music listened to in stereophonic format. Through an analysis of the basics of the physics of sound and psychoacoustics, the essay wants to illustrate which technologies allow immersive music to be produced. In this regard, both the history of sound reproduction techniques and the tools currently available will be examined to allow the scientific community to have a complete picture of today's resources. The combination of music and spatial audio is constantly growing with the passage of the years, which is why the text also aims to highlight the main research problems and the necessary standards on which to focus any future works. The essay ends with an experiment that compares two different types of song mixes: a stereo version and a binaural format version, processed through the Digital Audio Workstation Logic Pro X and the Dolby Atmos Production Suite. The results will bring interesting evidence in favor of the binaural format version and various suggestions on future developments.

Higher Order Ambisonics is a spatial audio technique that aims to recreate a sound image over as large a listening area as possible. Only limited investigation has taken place into localisation with Ambisonics and Higher Order Ambisonics at off-centre listening positions. This thesis presents the results of three psychoacoustic localisation experiments Investigating off-centre localisation of first and third order Ambisonics under different conditions: for studio and concert hall sized arrays, and for transient vs ongoing stimuli. A detailed analysis of the results of each experiment is carried out 'to determine the robustness of the tested systems. Comparisons are made between the results of the three experiments to determine the influence of changing the stimulus or increasing the arrival time delay between loudspeakers, where the relative gains of the loudspeakers was found to be perceptually more important than increased time differences between them. The usefulness of these results can be increased by comparison with models for prediction of human localisation, where a robust model would afford fast evaluation of ambisonic systems and allow system optimisation for off-centre positions. Therefore, evaluation Is performed for two binaural models on their ability to predict the results of the psychoacoustic experiments. A model by Dietz and a modified version of the Undemann model are evaluated. Finally, the energy vector model, prevalent in the Ambisonics community, is extended to include elements of the precedence effect. The binaural models, the standard energy vector, and the precedence-extended model are evaluated by comparison to the perceptual results. The robustness of each of the binaural and vector models is discussed in the context of their use as predictors of localisation at off-centre listening positions. The predictions of the precedence~extended energy vector are found to exhibit the lowest deviation from the perceptual results.

Mestrado em Engenharia Electrónica e Telecomunicações
Este trabalho explora técnicas de auralização 3D binaural (reprodução por head-phones) em tempo-real. Foi criada uma aplicação que permite reproduzir, com base em Head-Related Transfer Functions (HRTF), várias fontes sonoras movimentado-se de forma independente em torno do ouvinte. A aplicação permite usar um sensor para capturar os movimentos da cabeça (head-tracking ) e incorpora um método para eliminação dos artefactos causados pela interpolação entre HRTF. Foram também efectuadas experiências práticas com 16 sujeitos para estudar o impacto perceptual da interpolação de HRTF. Não se encontrou diferença significativa no desempenho de localização espacial de fontes sonoras entre as situações com HRTF interpoladas e HRTF não interpoladas. Foi efectuado um estudo geral sobre Ambisonics e sua descodificação, culminando numa aplicação que permite a abertura de ficheiros Ambisonics e reprodução selectiva dos seus componentes. Permite também a conversão para formato binaural aplicando HRTF aos sinais descodificados (método de altifalantes virtuais)
This work studies different binaural 3D sound auralization techniques in realtime for reproduction with head-phones An application was developed and it allows to play, using Head-Related Transfer Functions (HRTFs), different sounds moving, independently from each-other, around the listener. The application provides a method to overcome de clicks and crackles caused by the HRTFs interpolation and it is possible to capture head movements with the aid of an head-tracking sensor. In order to study the perceptual impact of using HRTFs interpolation an experimental procedure, with sixteen subjects, took place. There was no meaningful diferences between interpolated HRTFs and no interpolated HRTFs in what spatial localization performance matters. Ambisonics and its decoding methods were subject of a broad study and a new application was made. This application can open and play the different channels of an Ambisonics file and can convert to a binaural format using HRTFs (virtual speakers method).

Résumé : La synthèse de champs sonores est un domaine de recherche actif trouvant de nombreuses applications musicales, multimédias ou encore industrielles. Dans ce dernier cas, la re- construction précise du champ sonore est souhaitée, ce qui implique de répondre à un certains nombre de questionnements scientifiques. À l’aide de réseaux de microphones et de haut-parleurs, la captation, la synthèse et la reconstruction précise de champs sonores sont théoriquement possibles. Seulement, pour des applications pratiques, la disposition des haut-parleurs et l’influence acoustique du lieu de restitution sont des facteurs cruciaux à prendre en compte pour s’assurer de la bonne reconstruction du champ sonore. Dans ce contexte, cette thèse de doctorat propose des méthodes et des techniques pour la captation, la transformation et la reconstruction précise de champs sonores en trois dimen- sions en se basant sur la méthode ambisonique d’ordre élevé. Une configuration sphérique pour le réseau de microphones et de haut-parleurs est proposée. Elle suit un maillage de Lebedev à cinquante points qui permet la captation et la reconstruction du champ sonore jusqu’à l’ordre 5 avec le formalisme ambisonique. Les limitations de cette approche, tel le repliement spatial, sont étudiés en détails. De plus, une opération de transformation du champ sonore est présentée. Elle est établie dans le domaine des harmoniques sphériques et permet d’effectuer un filtrage directionnel avant le décodage pour privilégier certaines directions dans le champ sonore, suivant une fonction de directivité choisie. Pour la re- construction, une approche originale, également établie dans le domaine des harmoniques sphériques, permet de prendre en compte l’influence acoustique du lieu de restitution, ainsi que les défauts du système de restitution. Ce traitement permet alors d’adapter la synthèse de champs sonores au lieu de restitution, en conservant le formalisme théorique établi en champ libre. Finalement, une validation expérimentale des méthodes et des tech- niques développées au cours de la thèse est faite. Dans ce contexte, une suite logicielle de synthèse et traitement en temps-réel des champs sonore est développée.
Abstract : Sound field synthesis is an active research domain with various musical, multimedia or industrial applications. In the latter case, the accurate reconstruction of the sound field is targeted, which involves answering several scientific questions. Using arrays of microphones and loudspeakers, the capture, synthesis and accurate reconstruction of sound fields are theoretically possible. However, for practical applications, the arrangement of the loud- speakers and the acoustic influence of the restitution room are critical factors to consider in order to ensure the accurate reconstruction of the sound field. In this context, this thesis proposes methods and techniques for the capture, transforma- tions and accurate reconstruction of sound fields in three dimensions based on the Higher Order Ambisonics (HOA) method. A spherical configuration for the array of microphones and loudspeakers is proposed. It follows a fifty-node Lebedev grid that enables the capture and reconstruction of the sound field up to order 5 with HOA formalism. The limitations of this approach, such as the spatial aliasing, are studied in detail. A transformation op- eration of the sound field is also proposed. The formulation is established in the spherical harmonics domain and enables a directional filtering on the sound field prior to the decod- ing step. For the reconstruction of the sound field, an original approach, also established in the spherical harmonics domain, can take into account the acoustic influence of the restitution room and the defects of the playback system. This treatment then adapts the synthesis of sound fields to the restitution room, maintaining the theoretical formalism established in free field. Finally, an experimental validation of methods and techniques developed in the thesis is made. In this context, a digital signal processing toolkit is de- veloped. It process in real-time the microphones, ambisonics, and loudspeaker signals for the sound field capture, transformations, and decoding.

Les principales techniques de spatialisation sonores sont pour la plupart orientées vers la reproduction de sources ponctuelles, l’étendue reste un sujet peu exploré. Cette thèse défend l’idée que les sources étendues sont pourtant des objets expressifs pouvant significativement contribuer à la richesse des pratiques de spatialisation, notamment en musique. Nous décomposerons cette thèse en trois hypothèses. La première, auditive, postulera que les sources étendues sont pertinentes perceptivement. C’est-à-dire qu’elles offrent la possibilité de faire varier de nouveaux attributs sonores et que l’auditeur est sensible à ces variations. La seconde, analytique, proposera que la polarisation des techniques de spatialisation les plus courantes vers la source ponctuelle n’est qu’arbitraire et que d’autres modèles de sources peuvent être considérés. La troisième, opérationnelle, suggèrera qu’il est possible de créer des outils permettant aux compositeurs de manier et de s’approprier les sources étendues à des fins musicales.Pour valider ces hypothèses, nous formaliserons les propriétés auditives et musicales de ces sources puis nous proposerons des méthodes concrètes pour les analyser et les synthétiser. Ces travaux seront considérés dans le cadre de la bibliothèque HOA, un ensemble d’outils de spatialisation de bas niveau que nous avons spécialement fondé à des fins d’expérimentations. Nous décrirons les spécificités de cette bibliothèque. Nous verrons notamment comment son architecture et ses différents modules permettent d’ouvrir l’ambisonie à de nouvelles pratiques éloignées du concept de ponctualité
Mainstream spatialization techniques are often oriented towards the reproduction of point sources; extension remains a relatively unexplored topic. This thesis advocates that extended sources are yet expressive objects that could contribute to the richness of spatialization practices, especially in the field of music. We’ll decompose this thesis in three hypotheses. A perceptive one, who postulates that extended sources are perceptually relevant, i.e., that they offer the possibility of varying new sound attributes and that the listener is sensitive to these variations. An analytical one, who proposes that the most common spatialization techniques focus to point sources is arbitrary and that other source’s models can be considered. And an operational one, who suggests that it’s possible to create tools for composers so they can handle and musicalize extended objects. To confirm these hypotheses, we’ll formalize the auditory and musical properties of extended sources and we’ll propose concrete methods for their analysis and synthesis. This work will be considered as part of the HOA library, a set of low-level spatialization tools we’ve founded for the purpose of experimentation. We’ll describe the specificities of this library and see how its architecture and its different modules allow the generalization of ambisonics to new practices away of punctuality

This thesis presents an investigation into the generation and perception of three-dimensional sound fields using Higher Order Ambisonics. The first part involves the design, construction and characterisation of a hemi-anechoic chamber, as well as the design, construction, calibration and characterisation of a three-dimensional sound field reproduction system. The reproduction system is capable of rendering fourth-order 3D HOA sound scenes. Objective measurements revealed that the system achieves an almost perfect reconstruction of the ITD cues, reasonable reconstruction of the ILD cues, and inaccurate reconstruction of the monaural cues. Sound localisation testing conducted showed reasonable lateral sound localisation was not severely affected, while elevation sound localisation was poor. The second part of the investigation focuses on the perception of the sound field generated using HOA. The measured limitations of the fourth-order system motivated an investigation into the Ambisonic order required to achieve perceptually accurate sound field reconstruction. Three methods for binaurally rendering HOA signals were studied and evaluated. Of the three methods examined, the Magnitude Optimised Reconstruction technique was rated consistently better. A discrimination test showed that the sensory threshold for perceptually accurate reconstruction was order-12 for sound sources located on the horizon and order-14 for sound sources located across the full sphere of space. The perceptual fidelity of mixed-order HOA signal encoding (Mixed Order Ambisonics or MOA) was investigated by conducting listening tests using headphones and loudspeakers. Complex sound scenes were encoded using both full HOA and MOA up to fourth order. Results from both evaluations showed that the perceived fidelity of the sounds improved with increasing order. Also, the addition of harmonic components on top of 2D HOA showed greater improvement in fidelity for scenes with sound sources in elevation.

Musik och rörelser har sedan urminnes tider varit nära bundna via dansen, men detta sambandet har för det mesta varit endimensionellt i den mening att en eller fler människor som reagerar till musik eller ljud av något slag. Här har vi sett att det finns ett intressant rumsligt, interaktivt och sinnligt utforskande att göra. Via ambisonics, en teknik för att panorera ljud i tre dimensioner och en Kinect kamera har vi utforskat vad som sker när en vänder på detta sambandet och låter ljudet reagera till människans rörelse istället. Men här finns mer att utforska än bara sambandet mellan musik och dans. En av de för oss mest intressanta delarna har varit hur det går att utveckla en ljudinstallation där ljuden som spelas upp motsvarar rörelserna som skapar dem. Med sensory ethnography som perspektiv har vi under ett antal workshops tillsammans med personer med en bakgrund inom dans och uppträde undersökt vilken roll kinestesi och agens har i detta möte mellan rörelse och ljud. Vad vi funnit utifrån diffrakterande analys är hur agens, förkunskap och precision spelar stor roll för upplevelsen av kontroll i en installation som våran; men även hur rumsligt utforskande och lekfullhet kan träda fram när ovannämnda kriterier uteblir.
Music and motion have shared an intimate connection through dance since ages past, however this bond has for the most part been one dimensional in the sense that people are the ones reacting to music. This connection is something we have found intriguing to explore through the aspects of interactivity, spaciousness and sensory experience. Using ambisonics, a technology used to recreate three dimensional sound fields and a Kinect camera, we have sought out to investigate what happens if you reverse this connection and let audio react to a person's movement instead. There is however more to this than just the bond between dance and music. One of the more interesting aspects has been how an audio installation can be developed to play back audio that corresponds to the movements that generate them. We have through the usage of sensory ethnography as our exploratory perspective enacted a number of workshops where we, together with people with a background in dance and performance, investigated the role that kinaesthesia and a person's sense of agency has in this interconnection of motion and sound. What we have found is how agency, precognition and precision plays a vital role in how well the sense of control is experienced in an installation such as ours; but also how a more spacious and playful exploration can emerge when the above mentioned criterias are absent.

Cette thèse s’inscrit dans le contexte de l’essor des contenus immersifs. Depuis quelques années, les technologies de captation et de restitution sonore immersive se sont développées de manière importante. Ce nouveau contenu a fait naître le besoin de créer de nouvelles méthodes dédiées à la compression audio spatialisée, notamment dans le domaine de la téléphonie et des services conversationnels. Il existe plusieurs manières de représenter l’audio spatialisé, dans cette thèse nous sommes intéressés à l’ambisonie d’ordre 1. Dans un premier temps, nos travaux ont porté sur la recherche d’une solution pour améliorer le codage multimono. Cette solution consiste en un traitement en amont du codec multimono pour décorréler les signaux des composantes ambisoniques. Une attention particulière a été portée à la garantie de continuité du signal entre les trames et à la quantification des métadonnées spatiales. Dans un second temps, nous avons étudié comment utiliser la connaissance de la répartition de l’énergie du signal dans l’espace, aussi appelée image spatiale, pour créer de nouvelles méthodes de codage. L’utilisation de cette image spatiale a permis d’élaborer deux méthodes de compression. La première approche proposée est basée sur la correction spatiale du signal décodé. Cette correction se base sur la différence entre les images spatiales du signal d’origine et du signal décodés pour atténuer les altérations spatiales. Ce principe a été étendu dans une seconde approche à une méthode de codage paramétrique. Dans une dernière partie de cette thèse, plus exploratoire, nous avons étudié une approche de compression par réseaux de neurones en nous inspirant de modèles de compression d’images par auto-encodeur variationnel
This thesis takes place in the context of the spread of immersive content. For the last couple of years, immersive audio recording and playback technologies have gained momentum and have become more and more popular. New codecs are needed to handle those spatial audio formats, especially for communication applications. There are several ways to represent spatial audio scenes. In this thesis, we focused on First Order Ambisonic. The first part of our research focused on improving multi-monocoding by decorrelated each ambisonic signal component before the multi-mono coding. To guarantee signal continuity between frames, efficient quantization new mechanisms are proposed. In the second part of this thesis, we proposed a new coding concept using a power map to recreate the original spatial image. With this concept, we proposed two compressing methods. The first one is a post-processing focused on limiting the spatial distortion of the decoded signal. The spatial correction is based on the difference between the original and the decoded spatial image. This post-processing is later extended to a parametric coding method. The last part of this thesis presents a more exploratory method. This method studied audio signal compression by neural networks inspired by image compression models using variational autoencoders

La synthèse de champs sonores est un domaine de recherche actif trouvant de nombreuses applications musicales, multimédias ou encore industrielles. Dans ce dernier cas, la reconstruction précise du champ sonore est souhaitée, ce qui implique de répondre à un certains nombre de questionnements scientifiques. À l'aide de réseaux de microphones et de haut-parleurs, la captation, la synthèse et la reconstruction précise de champs sonores sont théoriquement possibles. Seulement, pour des applications pratiques, la disposition des haut-parleurs et l'influence acoustique du lieu de restitution sont des facteurs cruciaux à prendre en compte pour s'assurer de la bonne reconstruction du champ sonore.Dans ce contexte, cette thèse de doctorat propose des méthodes et des techniques pour la captation, la transformation et la reconstruction précise de champs sonores en trois dimensions en se basant sur la méthode ambisonique d'ordre élevé. Une configuration sphérique pour le réseau de microphones et de haut-parleurs est proposée. Elle suit un maillage de Lebedev à cinquante points qui permet la captation et la reconstruction du champ sonore jusqu'à l'ordre 5 avec le formalisme ambisonique. Les limitations de cette approche, tel le repliement spatial, sont étudiés en détails.De plus, une opération de transformation du champ sonore est présentée. Elle est établie dans le domaine des harmoniques sphériques et permet d'effectuer un filtrage directionnel avant le décodage pour privilégier certaines directions dans le champ sonore, suivant une fonction de directivité choisie.Pour la reconstruction, une approche originale, également établie dans le domaine des harmoniques sphériques, permet de prendre en compte l'influence acoustique du lieu de restitution, ainsi que les défauts du système de restitution. Ce traitement permet alors d'adapter la synthèse de champs sonores au lieu de restitution, en conservant le formalisme théorique établi en champ libre.Finalement, une validation expérimentale des méthodes et des techniques développées au cours de la thèse est faite. Dans ce contexte, une suite logicielle de synthèse et traitement en temps-réel des champs sonore est développée
Sound field synthesis is an active research domain with various musical, multimedia or industrial applications. In the latter case, the accurate reconstruction of the sound field is targeted, which involves answering several scientific questions. Using arrays of microphones and loudspeakers, the capture, synthesis and accurate reconstruction of sound fields are theoretically possible. However, for practical applications, the arrangement of the loudspeakers and the acoustic influence of the restitution room are critical factors to consider in order to ensure the accurate reconstruction of the sound field.In this context, this thesis proposes methods and techniques for the capture, transformations and accurate reconstruction of sound fields in three dimensions based on the Higher Order Ambisonics (HOA) method. A spherical configuration for the array of microphones and loudspeakers is proposed. It follows a fifty-node Lebedev grid that enables the capture and reconstruction of the sound field up to order 5 with HOA formalism. The limitations of this approach, such as the spatial aliasing, are studied in detail.A transformation operation of the sound field is also proposed. The formulation is established in the spherical harmonics domain and enables a directional filtering on the sound field prior to the decoding step.For the reconstruction of the sound field, and original approach, also established in the spherical harmonics domain, can take into account the acoustic influence of the restitution room and the defects of the playback system. This treatment then adapts the synthesis of sound fields to the restitution room, maintaining the theoretical formalism established in free field.Finally, an experimental validation of methods and techniques developed in the thesis is made. In this context, a digital signal processing toolkit is developed. It process in real-time the microphones, ambisonics, and loudspeaker signals for the sound field capture, transformations, and decoding

O trabalho envolve a pesquisa de técnicas para espacialização sonora musical e o desenvolvimento de um sistema de espacialização sonora sobre uma arquitetura de áudio multicanal aberta (AUDIENCE) para a auralização imersiva. Utilizando-se a técnica Ambisonics, o sistema construído permite a recriação de um campo sonoro tridimensional. O sistema reproduz virtualmente peças musicais espacializadas, por meio de um arranjo de alto-falantes em torno do ouvinte, onde as fontes sonoras podem assumir posições incomuns e movimentar-se no espaço. Através dos resultados dos experimentos musicais realizados com o sistema, chega-se a conclusão de que o sistema pode ser utilizado por compositores, regentes e produtores para a tarefa de espacialização musical.
This work involves the research of techniques for musical spatialization, and the development of a spatialization system, based on an open multichannel audio architecture (AUDIENCE) for immersive auralization. The constructed system uses the Ambisonics technique, which allows the recreation of a three-dimensional sound field. The system reproduces spatial music through an arrangement of loudspeakers around the listener where sound sources can assume some uncommon positions, and also be put into motion. From the results of experiments with a musical piece it follows that the system can be used by composers, conductors, and producers to the musical spatialization task.

Parmi les systèmes de reproduction 3D, les technologies Ambisonics et Higher Order Ambisonics (HOA) sont basées sur une décomposition spatiale du champ. Tronquée à un ordre fini - ordre HOA -, cette décomposition retient un nombre de signaux caractérisant le champ sonore encodé avec une résolution spatiale donnée. Plus l’ordre utilisé est élevé, plus le champ sonore sera reproduit avec exactitude. Le SoundField, microphone ambisonique d'ordre 1 et trois prototypes HOA d'ordre 2, 3 et 4 développés au sein de Orange Labs sont étudiés. Dans un premier temps, les fonctions de directivité de ces quatre dispositifs microphoniques sont mesurées. Leur capacité à reconstruire les composantes ambisoniques, et par conséquent à encoder le champ enregistré, est caractérisée. La capacité des microphones à reproduire le champ enregistré est alors évaluée perceptivement. Un test de localisation est mené sur ces systèmes d'ordre 1 à 4. Les résultats confirment, sur le plan perceptif, l’apport des ordres supérieurs sur la résolution spatiale du champ reproduit, ainsi que l'importance de la reconstruction des composantes ambisoniques par le système microphonique. Afin d'approcher l'évaluation d'une application de conférence audio, des scènes de synthèse intégrant des signaux "réels" sont alors utilisées. Les mêmes systèmes ambisoniques sont évalués grâce à une procédure de test MUSHRA, généralement utilisée pour l'évaluation de la qualité des codeurs audio. Les résultats sont conformes à ceux du test de localisation. Les deux premiers tests sont axés sur l'analyse des systèmes de prise de son. Une troisième évaluation s'applique aux dispositifs de restitution. Pour se concentrer sur cet élément et éviter les dégradations possibles causées par un dispositif microphonique, des encodages idéaux (composantes ambisoniques de synthèse) sont utilisés. Ce dernier test compare différents ordres ambisoniques décodés pour deux configurations de haut-parleurs. L'influence du système de restitution est alors mise en évidence
Among the spatial audio reproduction techniques, the ambisonic approach is based on a spherical harmonics sound field decomposition. Using an intermediate format, this technology combines a spatial encoding sound field process and a decoding process reproducing the sound field on a loudspeaker setup. By truncating the decomposition to the Mth order, a finite number of ambisonic components that form the spatial ambisonic format remains. The higher the order M is, the larger the reproduced area for a given frequency. Our studies deal with four microphone arrays of different orders (1 to 4). At first, the directives of these microphone devices are measured. The ambisonic components recreated from the measures are characterized. A localisation test and a MUSHRA-like procedure show an influence of the order on the spatial resolution of a recreated sound source. A third experiment shows the influence of both the order and the number of loudspeakers on the reproduced scenes

La thèse s’inscrit dans un contexte d’essor de l’audio spatialisé (5.1, Dolby Atmos...). Parmi les formats audio 3D existants, l’ambisonie permet une représentation spatiale homogène du champ sonore et se prête naturellement à des manipulations : rotations, distorsion du champ sonore. L’objectif de cette thèse est de fournir un outil d’analyse et de manipulation de contenus audio (essentiellement vocaux) au format ambisonique. Un fonctionnement temps-réel et en conditions acoustiques réelles sont les principales contraintes à respecter. L’algorithme mis au point est basé sur une analyse en composantes indépendantes (ACI) appliquée trame à trame qui permet de décomposer le champ acoustique en un ensemble de contributions, correspondant à des sources (champ direct) ou à de la réverbération. Une étape de classification bayésienne, appliquée aux composantes extraites, permet alors l’identification et le dénombrement des sources sonores contenues dans le mélange. Les sources identifiées sont localisées grâce à la matrice de mélange obtenue par ACI, pour fournir une cartographie de la scène sonore. Une étude exhaustive des performances est menée sur des contenus réels en fonction de plusieurs paramètres : nombre de sources, environnement acoustique, longueur des trames, ou ordre ambisonique utilisé. Des résultats fiables en terme de localisation et de comptage de sources ont été obtenus pour des trames de quelques centaines de ms. L’algorithme, exploité comme prétraitement dans un prototype d’assistant vocal domestique, permet d’améliorer significativement les performances de reconnaissance, notamment en prise de son lointaine et en présence de sources interférentes
The context of this thesis is the development of spatialized audio (5.1 contents, Dolby Atmos...) and particularly of 3D audio. Among the existing 3D audio formats, Ambisonics and Higher Order Ambisonics (HOA) allow a homogeneous spatial representation of a sound field and allows basics manipulations, like rotations or distorsions. The aim of the thesis is to provides efficient tools for ambisonics and HOA sound scene analyse and manipulations. A real-time implementation and robustness to reverberation are the main constraints to deal with. The implemented algorithm is based on a frame-by-frame Independent Component Analysis (ICA), wich decomposes the sound field into a set of acoustic contributions. Then a bayesian classification step is applied to the extracted components to identify the real sources and the residual reverberation. Direction of arrival of the sources are extracted from the mixing matrix estimated by ICA, according to the ambisonic formalism, and a real-time cartography of the sound scene is obtained. Performances have been evaluated in different acoustic environnements to assess the influence of several parameters such as the ambisonic order, the frame length or the number of sources. Accurate results in terms of source localization and source counting have been obtained for frame lengths of a few hundred milliseconds. The algorithm is exploited as a pre-processing step for a speech recognition prototype and allows a significant increasing of the recognition results, in far field conditions and in the presence of noise and interferent sources

Cette thèse s'inscrit dans le contexte de l'essor des assistants vocaux mains libres. Dans un environnement domestique, l'appareil est généralement posé à un endroit fixe, tandis que le locuteur s'adresse à lui depuis diverses positions, sans nécessairement s'appliquer à être proche du dispositif, ni même à lui faire face. Cela ajoute des difificultés majeures par rapport au cas, plus simple, de la commande vocale en champ proche (pour les téléphones portables par exemple) : ici, la réverbération est plus importante ; des réflexions précoces sur les meubles entourant l'appareil peuvent brouiller le signal ; les bruits environnants sont également sources d'interférences. À ceci s'ajoutent de potentiels locuteurs concurrents qui rendent la compréhension du locuteur principal particulièrement difficile. Afin de faciliter la reconnaissance vocale dans ces conditions adverses, plusieurs pré-traitements sont proposés ici. Nous utilisons un format audio spatialisé, le format Ambisonique, adapté à l'analyse de scènes sonores. Dans un premier temps, nous présentons une méthode de localisation des sources sonores basée sur un réseau de neurones convolutif et récurrent. Nous proposons des descripteurs inspirés du vecteur d'intensité acoustique qui améliorent la performance de localisation, notamment dans des situations réelles où plusieurs sources sont présentes et l'antenne de microphones est posée sur une table. La technique de visualisation appelée layerwise relevance propagation (LRP) met en valeur les zones temps-fréquence positivement corrélées avec la localisation prédite par le réseau dans un cas donné. En plus d'être méthodologiquement indispensable, cette analyse permet d'observer que le réseau de neurones exploite principalement les zones dans lesquelles le son direct domine la réverbération et le bruit ambiant. Dans un second temps, nous proposons une méthode pour rehausser la parole du locuteur principal et faciliter sa reconnaissance. Nous nous plaçons dans le cadre de la formation de voies basée sur des masques temps-fréquence estimés par un réseau de neurones. Afin de traiter le cas où plusieurs personnes parlent à un volume similaire, nous utilisons l'information de localisation pour faire un premier rehaussement à large bande dans la direction du locuteur cible. Nous montrons que donner cette information supplémentaire au réseau n'est pas suffisant dans le cas où deux locuteurs sont proches ; en revanche, donner en plus la version rehaussée du locuteur concurrent permet au réseau de renvoyer de meilleurs masques. Ces masques permettent d'en déduire un filtre multicanal qui améliore grandement la reconnaissance vocale. Nous évaluons cet algorithme dans différents environnements, y compris réels, grâce à un moteur de reconnaissance de la parole utilisé comme boîte noire. Dans un dernier temps, nous combinons les systèmes de localisation et de rehaussement et nous évaluons la robustesse du second aux imprécisions du premier sur des exemples réels
This work was conducted in the fast-growing context of hands-free voice command. In domestic environments, smart devices are usually laid in a fixed position, while the human speaker gives orders from anywhere, not necessarily next to the device, or nor even facing it. This adds difficulties compared to the problem of near-field voice command (typically for mobile phones) : strong reverberation, early reflections on furniture around the device, and surrounding noises can degrade the signal. Moreover, other speakers may interfere, which make the understanding of the target speaker quite difficult. In order to facilitate speech recognition in such adverse conditions, several preprocessing methods are introduced here. We use a spatialized audio format suitable for audio scene analysis : the Ambisonic format. We first propose a sound source localization method that relies on a convolutional and recurrent neural network. We define an input feature vector inspired by the acoustic intensity vector which improves the localization performance, in particular in real conditions involving several speakers and a microphone array laid on a table. We exploit the visualization technique called layerwise relevance propagation (LRP) to highlight the time-frequency zones that are correlate positively with the network output. This analysis is of paramount importance to establish the validity of a neural network. In addition, it shows that the neural network essentially relies on time-frequency zones where direct sound dominates reverberation and background noise. We then present a method to enhance the voice of the main speaker and ease its recognition. We adopt a mask-based beamforming framework based on a time-frequency mask estimated by a neural network. To deal with the situation of multiple speakers with similar loudness, we first use a wideband beamformer to enhance the target speaker thanks to the associated localization information. We show that this additional information is not enough for the network when two speakers are close to each other. However, if we also give an enhanced version of the interfering speaker as input to the network, it returns much better masks. The filters generated from those masks greatly improve speech recognition performance. We evaluate this algorithm in various environments, including real ones, with a black-box automatic speech recognition system. Finally, we combine the proposed localization and enhancement systems and evaluate the robustness of the latter to localization errors in real environments

Ambisonics is a complete theory for spatial audio whose building blocks are the spherical harmonics. Some of the drawbacks of low order Ambisonics, like poor source directivity and small sweet-spot, are directly related to the properties of spherical harmonics. In this thesis we illustrate a novel spatial audio framework similar in spirit to Ambisonics that replaces the spherical harmonics by an alternative set of functions with compact support: the spherical wavelets. We develop a complete audio chain from encoding to decoding, using discrete spherical wavelets built on a multiresolution mesh. We show how the wavelet family and the decoding matrices to loudspeakers can be generated via numerical optimization. In particular, we present a decoding algorithm optimizing acoustic and psychoacoustic parameters that can generate decoding matrices to irregular layouts for both Ambisonics and the new wavelet format. This audio workflow is directly compared with Ambisonics.
Ambisonics és una teoria completa d’àudio espacial construïda a partir dels harmònics esfèrics. Alguns dels inconvenients d'Ambisonics de baix ordre, com ara una localització pobra i una àrea petita d’escolta òptima, estan directament relacionats amb les propietats dels harmònics esfèrics. En aquesta tesi presentem un nou formalisme d’àudio espacial basat en Ambisonics substituint però els harmònics esfèrics per les ondetes esfèriques. Desenvolupem una cadena d’àudio completa, des de la codificació fins a la descodificació, a través de l'ús de ondetes discretes construïdes en una malla de multirresolució. Mostrem com es pot generar la família de ondetes i les matrius de descodificació a altaveus mitjançant una optimització numèrica. Presentem un algorisme de descodificació que pot generar matrius de descodificació a conjunts irregulars d'altaveus tant per a Ambisonics com per al nou format basat en ondetes. Finalment, comparem aquest nou formalisme d’àudio amb Ambisonics.

With the development of technology over the past decade and the former challenges of virtual environments mitigated, the need for further study of human interaction with these environments has grown apparent. The visual and interaction components for virtual reality applications have been comprehensively studied, but a lack of spatial audio fidelity leaves a need for understanding how well humans can localize aural cues and discern audio-visual disparity in these virtual environments. In order for development of accurate and efficient levels of audio fidelity, a human study was conducted with 18 participants to see how far a bimodal audio and visual cue need to separate for someone to notice. As suspected, having a visual component paired with an auditory one led to biasing toward the visual component. The average participant noticed a disparity when the audio component was 33.7° apart from the visual one, pertaining to the azimuth. There was no significant evidence to suggest that speed or direction of audio component disparity led to better localization performance by participant. Presence and prior experience did not have an effect on localization performance; however, a larger participant base may be needed to draw further conclusions. Increase in localization ability was observed within a few practice rounds for participants. Overall, performance in virtual reality was parallel to augmented reality performance when a visual source biased sound localization, and can this be a both tool and design constraint for virtual environment developers.
Master of Science

This thesis describes a system that can be used for the decoding of a three dimensional audio recording over headphones or two, or more, speakers. A literature review of psychoacoustics and a review (both historical and current) of surround sound systems is carried out. The need for a system which is platform independent is discussed, and the proposal for a system based on an amalgamation of Ambisonics, binaural and transaural reproduction schemes is given. In order for this system to function optimally, each of the three systems rely on providing the listener with the relevant psychoacoustic cues. The conversion from a five speaker ITU array to binaural decode is well documented but pair-wise panning algorithms will not produce the correct lateralisation parameters at the ears of a centrally seated listener. Although Ambisonics has been well researched, no one has, as yet, produced a psychoacoustically optimised decoder for the standard irregular five speaker array as specified by the ITU as the original theory, as proposed by Gerzon and Barton (1992) was produced (known as a Vienna decoder), and example solutions given, before the standard had been decided on. In this work, the original work by Gerzon and Barton (1992) is analysed, and shown to be suboptimal, showing a high/low frequency decoder mismatch due to the method of solving the set of non-linear simultaneous equations. A method, based on the Tabu search algorithm, is applied to the Vienna decoder problem and is shown to provide superior results to those shown by Gerzon and Barton (1992) and is capable of producing multiple solutions to the Vienna decoder problem. During the write up of this report Craven (2003) has shown how 4th order circular harmonics (as used in Ambisonics) can be used to create a frequency independent panning law for the five speaker ITU array, and this report also shows how the Tabu search algorithm can be used to optimise these decoders further. A new method is then demonstrated using the Tabu search algorithm coupled with lateralisation parameters extracted from a binaural simulation of the Ambisonic system to be optimised (as these are the parameters that the Vienna system is approximating). This method can then be altered to take into account head rotations directly which have been shown as an important psychoacoustic parameter in the localisation of a sound source (Spikofski et al., 2001) and is also shown to be useful in differentiating between decoders optimised using the Tabu search form of the Vienna optimisations as no objective measure had been suggested. Optimisations for both Binaural and Transaural reproductions are then discussed so as to maximise the performance of generic HRTF data (i.e. not individualised) using inverse filtering methods, and a technique is shown that minimises the amount of frequency dependant regularisation needed when calculating cross-talk cancellation filters.

This thesis demonstrates an approach to composing with environmental sound that uses spatial concepts and techniques to bridge the aesthetic divide between reductive and relative compositional philosophies. It responds to the complex issues of contextual integration and separation through a detailed compositional approach, which uses the spatial medium of acousmatic music to translate aspects of acoustic ecology, biomimetics, soundscape and ecoacoustics into the domain of contemporary composition. This research makes claims to new knowledge by examining how compositional activities inform the development of novel techniques for environmental sound composition. It takes a practice-based approach to research, which uses the creative practices of field recording, fixed-media composition and software programming to highlight and respond to the issues implicit to the production of environmentally mediated spaces through sound. Furthermore, it provides new theoretical perspectives on the relations between musical form and the external environment. Central to this research is a body of creative work, which presents a portfolio of compositions and the custom software tools integral to its production as research outcomes. As the activities of practice and the insights gained through practice are as crucial to the practice-based research paradigm as its outcomes, this thesis uses a self-reflective approach to document how the knowledge generated during the composition process shaped the outcomes of the creative artefacts. An engagement with spatial concepts characterises the reflective discourse. The non-linear and iterative ways of shaping and applying these ideas in practice underpin the discussion of each composition and the algorithmic realisation of these concepts using the SuperCollider programming language.

Dans ce travail nous étudions la problématique des ingénieurs du son face au mixage d’un microphone principal HOA avec des microphones d’appoint, et notamment l’estimation des paramètres tels que le retard, la position et le gain des sources acoustiques associées aux microphones d’appoint. Nous proposons un algorithme fournissant les paramètres estimés (retard, position, gain) basé sur des équations d’encodage spatial au format HOA qui peuvent alors être utilisées pour traiter les signaux des microphones d’appoint durant le mixage. Cette extraction automatique des paramètres peut être vue comme une assistance pour les ingénieurs du son, leur permettant d’éviter un travail à faible valeur ajoutée (mesure de la distance et des angles entre microphones) afin de pouvoir se concentrer sur des problèmes artistiques comme l’ajustement des paramètres de niveau, d’égalisation ou de compression, voire l’ajustement fin des paramètres de retard, position, gain. La robustesse de l’algorithme est bien présentée pour les scènes sonores de différents niveaux de complexité (plusieurs sources acoustiques, réverbération, encodage réel du microphone…). Nous proposons des tests de performances pour les scènes sonores simulées et réels afin de montrer l’efficacité de l’algorithme ainsi que ces limites. La conclusion et les perspectives pour des futurs travaux complètent cette thèse à la fin du document
In this thesis we study the problematic of a sound engineer mixing HOA (Higher Order Ambisonics) and spot microphones, namely the estimation of parameters such as delay, position and gain of acoustic sources associated to spot microphones. We present a typical workflow in this context, and also propose an algorithm extracting parameters that could be applied to the spot microphone signals. This mixing assistance allows sound engineers to easily work with HOA 3D sound and to concentrate on artistic choices (fine adjustments of the parameters), by avoiding a low-added value work (coarse parameter estimation). The robustness of the estimators is evaluated on recorded and artificial sound scenes, with different degrees of complexity in terms of number of sources and acoustic conditions (reverberation, effect of real microphone encoding, …). We also provide performance evaluations, based on both sound scene simulations and real recordings, showing encouraging results along with actual limits, and conclude on perspectives

Pour ma thèse de doctorat, je propose d’explorer la piste de l’apprentissage supervisé, pour la tâche de localisation de sources acoustiques. Pour ce faire, j’ai développé une nouvelle architecture de réseau de neurones profonds. Mais, pour optimiser les millions de variables d’apprentissages de ce réseau, une base de données d’exemples conséquente est nécessaire. Ainsi, deux approches complémentaires sont proposées pour constituer ces exemples. La première est de réaliser des simulations numériques d’enregistrements microphoniques. La seconde, est de placer une antenne de microphones au centre d’une sphère de haut-parleurs qui permet de spatialiser les sons en 3D, et d’enregistrer directement sur l’antenne de microphones les signaux émis par ce simulateur expérimental d’ondes sonores 3D. Le réseau de neurones a ainsi pu être testé dans différentes conditions, et ses performances ont pu être comparées à celles des algorithmes conventionnels de localisation de sources acoustiques. Il en ressort que cette approche permet une localisation généralement plus précise, mais aussi beaucoup plus rapide que les algorithmes conventionnels de la littérature
For my PhD thesis, I propose to explore the path of supervised learning, for the task of locating acoustic sources. To do so, I have developed a new deep neural network architecture. But, to optimize the millions of learning variables of this network, a large database of examples is needed. Thus, two complementary approaches are proposed to constitute these examples. The first is to carry out numerical simulations of microphonic recordings. The second one is to place a microphone antenna in the center of a sphere of loudspeakers which allows to spatialize the sounds in 3D, and to record directly on the microphone antenna the signals emitted by this experimental 3D sound wave simulator. The neural network could thus be tested under different conditions, and its performances could be compared to those of conventional algorithms for locating acoustic sources. The results show that this approach allows a generally more precise localization, but also much faster than conventional algorithms in the literature

L'objectif de cette thèse est l'optimisation de la qualité perçue de la reproduction sonore par un système audio multicanal, dans un contexte de salle d'écoute domestique. Les travaux de recherche présentés s'articulent selon deux axes. Le premier concerne l'effet de salle, et plus particulièrement les aspects physiques et perceptifs liés aux premières réflexions d'une salle. Ces éléments sont décrits spécifiquement, et une expérience psychoacoustique a été menée afin d'étendre les données disponibles quant à leur perceptibilité, c'est à dire leur capacité à modifier la perception du son direct, que ce soit en termes de timbre ou de localisation. Les résultats mettent en évidence la dépendance du seuil en fonction du type de stimulus, ainsi que son évolution en fonction de la configuration spatiale de l'onde directe et de la réflexion. Pour une condition donnée, le seuil de perceptibilité est décrit comme une fonction de directivité dépendant de l'incidence de la réflexion.Le deuxième axe de travail concerne les méthodes de correction de l'effet de la salle de reproduction. Les méthodes numériques classiques sont d'abord étudiées. Leur principale lacune réside dans l'absence de prise en compte du rôle spécifique des propriétés temporelles et spatiales des première réflexions. Le travail de thèse se termine par la proposition d'une nouvelle méthode de correction utilisant un algorithme itératif de type FISTA modifié afin de prendre en compte la perceptibilité des réflexions. Le traitement est implémenté dans une représentation où l'information spatiale est analysée sur la base des harmoniques sphériques
The goal of this Ph. D. thesis is to optimize the perceived quality of multichannel sound reproduction systems, in the context of a domestic listening room. The presented research work have been pursued in two different directions.The first deals with room effet, and more particularly with physical and perceptual aspects of first reflections within a room. These reflections are specifically described, and a psychoacoustical experiment have been carried out in order to extend the available data on their perceptibility, i.e. their potency in altering the perception of the direct sound, whether in its timbral or spatial features. Results exhibit the variation of the threshold depending on the type of stimulus, as well as on the spatial configuration of the direct sound and the reflection. For a given condition, the perceptibility threshold is given as a directivity function depending on the direction of incidence of the reflection.The second topic deals with room correction methods. Firstly, state-of-the art digital methods are investigated. Their main drawback is that they don't consider the specific impact of the temporal and spatial attributes of first reflections. A new correction method is therefore proposed. It uses an iterative algorithm, derivated from the FISTA method, in order to take into account the perceptibility of the reflections. All the processing is carried out in a spatial sound representation, where the spatial properties of the sound are analysed thanks to spherical harmonics

Research investigating the behavioral and physiological responses of bats to echoes typically includes analysis of acoustic signals from microphones and/or microphone arrays, using time difference of arrival (TDOA) between array elements or the microphones to locate flying bats (azimuth and elevation). This has provided insight into transmission adaptations with respect to target distance, clutter, and interference. Microphones recording transmitted signals and echoes near a stationary bat provide sound pressure as a function of time but no directional information. This dissertation introduces spatial audio techniques to bat biosonar studies as a complementary method to the current TDOA based acoustical study methods. This work proposes a couple of feasible methods based on spatial audio techniques, that both track bats in flight and pinpoint the directions of echoes received by a bat. A spatial audio/soundfield microphone array is introduced to measure sounds in the sonar frequency range (20-80 kHz) of the big brown bat (Eptesicus fuscus). The custom-built ultrasonic tetrahedral soundfield microphone consists of four capacitive microphones that were calibrated to match magnitude and phase responses using a transfer function approach. Ambisonics, a signal processing technique used in three-dimensional (3D) audio applications, is used for the basic processing and reproduction of the signals measured by the soundfield microphone. Ambisonics provides syntheses and decompositions of a signal containing its directional properties, using the relationship between the spherical harmonics and the directional properties. As the first proposed method, a spatial audio decoding technique called HARPEx (High Angular Resolution Planewave Expansion) was used to build a system providing angle and elevation estimates. HARPEx can estimate the direction of arrivals (DOA) for up to two simultaneous sources since it decomposes a signal into two dominant planewaves. Experiments proved that the estimation system based on HARPEx provides accurate DOA estimates of static or moving sources. It also reconstructed a smooth flight-path of a bat by accurately estimating its direction at each snapshot of pulse measurements in time. The performance of the system was also assessed using statistical analyses of simulations. A signal model was built to generate microphone capsule responses to a virtual source emitting an LFM signal (3 ms, two harmonics: 40-22 kHz and 80-44 kHz) at an angle of 30° in the simulations. Medians and RMSEs (root-mean-square error) of 10,000 simulations for each case represent the accuracy and precision of the estimations, respectively. Results show lower d (distance between a capsule and the soundfield microphone center) or/and higher SNR (signal-to-noise ratio) are required to achieve higher estimator performance. The Cramer-Rao lower bounds (CRLB) of the estimator are also computed with various d and SNR conditions. The CRLB which is for TDOA based methods does not cover the effects of different incident angles to the capsules and signal delays between the capsules due to a non-zero d, on the estimation system. This shows the CRLB is not a proper tool to assess the estimator performance. For the second proposed method, the matched-filter technique is used instead of HARPEx to build another estimation system. The signal processing algorithm based on Ambisonics and the matched-filter approach reproduces a measured signal in various directions, and computes matched-filter responses of the reproduced signals in time-series. The matched-filter result points a target(s) by the highest filter response. This is a sonar-like estimation system that provides information of the target (range, direction, and velocity) using sonar fundamentals. Experiments using a loudspeaker (emitter) and an artificial or natural target (either stationary or moving) show the system provides accurate estimates of the target's direction and range. Simulations of imitating a situation where a bat emits a pulse and receives an echo from a target (30°) were also performed. The echo sound level is determined using the sonar equation. The system processed the virtual bat pulse and echo, and accurately estimated the direction, range, and velocity of the target. The simulation results also appear to recommend an echo level over -3 dB for accurate and precise estimations (below 15% RMSE for all parameters). This work proposes two methods to track bats in flight or/and pinpoint the directions of targets using spatial audio techniques. The suggested methods provide accurate estimates of the direction, range, or/and velocity of a bat based on its pulses or of a target based on echoes. This demonstrates these methods can be used as key tools to reconstruct bat biosonar. They would be also an independent tool or a complementary option to TDOA based methods, for bat echolocation studies. The developed methods are believed to be also useful in improving man-made sonar technology.
Doctor of Philosophy
While bats are one of the most intriguing creatures to the general population, they are also a popular subject of study in various disciplines. Their extraordinary ability to navigate and forage irrespective of clutter using echolocation has gotten attention from many scientists and engineers. Research investigating bats typically includes analysis of acoustic signals from microphones and/or microphone arrays. Using time difference of arrival (TDOA) between the array elements or the microphones is probably the most popular method to locate flying bats (azimuth and elevation). Microphone responses to transmitted signals and echoes near a bat provide sound pressure but no directional information. This dissertation proposes a complementary way to the current TDOA methods, that delivers directional information by introducing spatial audio techniques. This work shows a couple of feasible methods based on spatial audio techniques, that can both track bats in flight and pinpoint the directions of echoes received by a bat. An ultrasonic tetrahedral soundfield microphone is introduced as a measurement tool for sounds in the sonar frequency range (20-80 kHz) of the big brown bat (Eptesicus fuscus). Ambisonics, a signal processing technique used in three-dimensional (3D) audio applications, is used for the basic processing of the signals measured by the soundfield microphone. Ambisonics also reproduces a measured signal containing its directional properties. As the first method, a spatial audio decoding technique called HARPEx (High Angular Resolution Planewave Expansion) was used to build a system providing angle and elevation estimates. HARPEx can estimate the direction of arrivals (DOA) for up to two simultaneous sound sources. Experiments proved that the estimation system based on HARPEx provides accurate DOA estimates of static or moving sources. The performance of the system was also assessed using statistical analyses of simulations. Medians and RMSEs (root-mean-square error) of 10,000 simulations for each simulation case represent the accuracy and precision of the estimations, respectively. Results show shorter distance between a capsule and the soundfield microphone center, or/and higher SNR (signal-to-noise ratio) are required to achieve higher performance. For the second method, the matched-filter technique is used to build another estimation system. This is a sonar-like estimation system that provides information of the target (range, direction, and velocity) using matched-filter responses and sonar fundamentals. Experiments using a loudspeaker (emitter) and an artificial or natural target (either stationary or moving) show the system provides accurate estimates of the target's direction and range. Simulations imitating a situation where a bat emits a pulse and receives an echo from a target (30°) were also performed. The system processed the virtual bat pulse and echo, and accurately estimated the direction, range, and velocity of the target. The suggested methods provide accurate estimates of the direction, range, or/and velocity of a bat based on its pulses or of a target based on echoes. This demonstrates these methods can be used as key tools to reconstruct bat biosonar. They would be also an independent tool or a complementary option to TDOA based methods, for bat echolocation studies. The developed methods are also believed to be useful in improving sonar technology.

The increasing interest for immersive experiences in areas such as augmented and virtual reality makes high quality 3D sound more important than ever before. A technique for capturing and rendering 3D audio which has received more attention during the last twenty years are Higher Order Ambisonics (HOA). Higher Order Ambisonics is a scene based audio format which has a lot of advantages compared to other standard formats. Hovever, one problem with HOA is that it requires a lot of bandwidth. For example, sending an uncoded high quality HOA signal requires 49 channels to be transmitted at the same time which requires a bandwidth of about 40 Mbps. A lot of effort has been made in the last ten years on coding HOA signals. In this thesis, two different approaches are taken on coding HOA signals. In one approach, called Sound Field Rotation (SFR) in this thesis, the microphone that records the sound field is virtually rotated to see if it is possible to make some of the channels zero. The second approach, called Sound Field Decomposition (SFD) in this thesis, use Principal component analysis to decompose a sound field into a foreground and background component. The Sound Field Decomposition approach is inspired by the emerging MPEG-H 3D Audio standard for coding HOA signals. The result shows that the Sound Field Rotation method only works for very simple sound scenes. It has also been shown that a 49 channels HOA signal can be reduced to as little as 7 channels if the sound scene consists of a point source. The Sound Field Deomposition method worked for more complex sound scenes. It was shown that a MPEG similar system could be improved. Result from MUSHRA (Multiple stimuli with hidden reference and anchor) listening tests showed that an improved MPEG similar system reached a MUSHRA score about 78 while the MPEG similar system reached 55 at a bitrate of 256 kbps. Without coding each monochannels with the 3GPP EVS (Enhanced voice services) codec, the improved MPEG similar system reached the MUSHRA score 85. At 256 kbps, the improved MPEG similar system coded the HOA signal into six channels instead of 49 for the uncoded signal. From objective results, it was shown that the improved MPEG similar system had largest effect at low bitrates.

Penser les traitements du son spatialisés en ambisonie permet de mettre en valeur le potentiel musical de la décomposition du champ sonore en harmoniques sphériques, et amène à redéfinir la représentation de l’espace sonore. Cette thèse défend que les représentations abstraites et intermédiaires de l’espace sonore permettent d’élaborer de nouvelles approches originales de la mise en espace du son. Le raisonnement amenant à cette affirmation commence par l’appropriation musicale de l’approche ambisonique. La création de nouveaux traitements de l’espace et du son amène à utiliser de manière originale les signaux associés aux harmoniques sphériques, et à concevoir différemment les relations qui les régissent, ainsi que leur hiérarchisation. La particularité de ces approches expérimentales et les caractéristiques singulières des champs sonores générés, nécessitent de concevoir de nouveaux outils théoriques et pratiques pour leur analyse et leur restitution. Les changements opérés permettent alors de libérer cette approche des enjeux techniques et matériels initiaux en ambisonie. Mais ils permettent surtout de s’émanciper des modèles psychoacoustiques et acoustiques sur lesquels ces techniques reposent originellement. Dans ce contexte, les signaux associés aux harmoniques sphériques ne sont plus nécessairement une représentation rationnelle du champ sonore, mais deviennent une représentation abstraite de l’espace sonore possédant en soi, un potentiel musical. Cette thèse propose alors un nouveau modèle de spatialisation fondé sur une décomposition matricielle de l’espace sonore permettant de valider les hypothèses
The creation of sound effects in space with Ambisonics highlights the musical potential of sound field decomposition by spherical harmonics, and redefines the representation of the sound space. This thesis defends that the abstract and intermediate representations of the sound space make it possible to develop new original approaches to sound spatialization. The reasoning that leads to this affirmation begins with the musical appropriation of the ambisonic approach. The creation of new space and sound processing patterns in Ambisonics leads to an original way of using signals associated with spherical harmonics, and to a different conception of the relations between them, and their hierarchization. The specificities of these experimental approaches and the singular characteristics of the sound fields generated call for the design of new theoretical and practical tools, for their analysis and restitution. The performed changes make it possible to free this approach from the initial technical and material issues of Ambisonics. But above all, it emancipates this approach from the psychoacoustic and acoustic models on which ambisonic techniques are originally defined. In this context, the signals associated with spherical harmonics are no longer necessarily a rational representation of the sound field but become an abstract representation of the sound space possessing in itself a musical potential. To validate the hypotheses, this thesis then proposes a new spatialization model based on a matrix decomposition of the sound space

Ce travail se place en contexte de télécommunications, et concerne plus particulièrement la transmission de signaux audio multicanaux. Quatre expériences psychoacoustiques ont été menées de façon à étudier la résolution spatiale du système auditif - également appelée flou de localisation - en présence de sons distracteurs. Il en résulte que le flou de localisation augmente quand ces distracteurs sont présents, mettant en évidence ce que nous appellerons le phénomène de "floutage spatial" auditif. Ces expériences estiment l'effet de plusieurs variables sur le floutage spatial : la fréquence de la source sonore considérée ainsi que celles des sources distractrices, leur niveau sonore, leur position spatiale, et le nombre de sources distractrices. Exceptée la position des sources distractrices, toutes ces variables ont montré un effet significatif sur le floutage spatial. Cette thèse aborde également la modélisation de ce phénomène, de sorte que la résolution spatiale auditive puisse être prédite en fonction des caractéristiques de la scène sonore (nombre de sources présentes, leur fréquence, et leur niveau). Enfin, deux schémas de codage audio multicanaux exploitant ce modèle à des fins de réduction de l'information à transmettre sont proposés : l'un basé sur une représentation paramétrique (downmix + paramètres spatiaux) du signal multicanal, et l'autre sur la représentation Higher-Order Ambisonics (HOA). Ces schémas sont tous deux basés sur l'idée originale d'ajuster dynamiquement la précision de la représentation spatiale du signal multicanal de façon à maintenir les distorsions spatiales résultantes dans le flou de localisation, afin que celles-ci restent indétectables.

La llegada del sonido 3D está imponiendo cambios en varias etapas del flujo de trabajo, desde los sistemas de captación hasta las metodologías de postproducción y las configuraciones de altavoces. Esta tesis trata varios aspectos relacionados con el audio 3D: en la parte de captación, presentamos un estudio sobre las características de los micrófonos tetraédricos y una solución para obtener las componentes Ambisonics del segundo orden usando un pequeño número de transductores del primer orden; en la parte de producción, se presenta una aplicación para la mezcla automatizada de eventos deportivos, para reducir la complexidad del multicanal en tiempo real; para la restitución del audio independiente del sistema de altavoces, en el que los niveles de salida a los altavoces son una incógnita hasta la decodificación, se propone un detector de clipping independiente del layout. Finalmente, se presentan test psico-acústicos para validar aspectos perceptivos relacionados con el audio 3D.
The advent of 3D audio is dictating changes in several stages of the audio work-flow, from recording systems and microphone configurations, to post-production methodologies and loudspeaker configurations. This thesis tackles aspects related to 3D audio arising in the various stages of production. In the recording part, we present a study on the accuracy of tetrahedral microphones and a solution for obtaining second-order Ambisonics responses from first-order transducers using a small number of sensors; in the production stage, we introduce an application for automated assisted mixing of sport events, to reduce the complexity of managing multiple audio channels in real time; a clipping detector is proposed for the rendering of layout-independent audio content to generic playback systems, where the signal levels sent to the speakers are unknown until the decoding stage; finally, psychoacoustic experiments are presented for the validation of perceptual and aesthetic aspects related to 3D audio.

Nowadays, a growing interest in the recording and reproduction of spatial audio has been observed. With virtual and augmented reality technologies spreading fast thanks to entertainment and video game industries, also the professional opportunities in the field of engineering are evolving. However, despite many microphone arrays are reaching the market, most of them is not optimized for engineering or diagnostic use and remains mainly confined to voice and music recordings. In this thesis, the design of two new systems for recording and analysing the spatial distribution of sound energy, employing arrays of transducers and cameras, is discussed. Both acoustic and visual spatial information is recorded and combined together to produce static and dynamic colour maps, with a specially designed software and employing Ambisonics and Spatial PCM Sampling (SPS), two common spatial audio formats, for signals processing. The first solution consists in a microphone array made of 32 capsules and a circular array of eight cameras, optimized for low frequencies. The size of the array is designed accordingly to the frequency range of interest for automotive Noise, Vibration & Harshness (NVH) applications. The second system is an underwater probe with four hydrophones and a panoramic camera, with which it is possible to monitor the effects of underwater noise produced by human activities on marine species. Finite Elements Method (FEM) simulations have been used to calculate the array response, thus deriving the filtering matrix and performing theoretical evaluation of the spatial performance. Field tests of the proposed solutions are presented in comparison with the current state-of-the-art equipment. The faithful reproduction of the spatial sound field arouses equally interest. Hence, a method to playback panoramic video with spatial audio is presented, making use of Virtual Reality (VR) technology, spatial audio, individualized Head Related Transfer Functions (HRTFs) and personalized headphones equalization. The work in its entirety presents a complete methodology for recording, analysing and reproducing the spatial information of soundscapes.

Since the success of James Cameron’s Avatar (2009),1 the feature film industry has embraced 3-D feature film technology. With 3-D films now setting a new benchmark for contemporary cinemagoers, the primary focus is directed towards these new stunning visuals. Sound is often neglected until the final filmmaking process as the visuals are taking up much of the film budget. 3-D has changed the relationship between the imagery and the accompanying soundtrack, losing aspects of the cohesive union compared with 2-D film. Having designed sound effects on Australia’s first digital animated 3-D film, Legend of the Guardians: The Owls of Ga’Hoole (2010),2 and several internationally released 3-D films since, it became apparent to me that the visuals are evolving technologically and artistically at a rate far greater than the soundtrack. This is creating a dislocation between the image and the soundtrack. Although cinema sound technology companies are trialing and releasing new ‘immersive’ technologies, they are not necessarily addressing the spatial relationship between the images and soundtracks of 3-D digital films. Through first hand experience, I question many of the working methodologies currently employed within the production and creation of the soundtrack for 3-D films. There is limited documentation on sound design within the 3-D feature film context, and as such, there are no rules or standards associated with this new practice. Sound designers and film sound mixers are continuing to use previous 2-D work practices in cinema sound, with limited and cautious experimentation of spatial sound design for 3-D. Although emerging technologies are capable of providing a superior and ‘more immersive’ soundtrack than previous formats, this does not necessarily mean that they provide an ideal solution for 3-D film. Indeed the film industry and cinema managers are showing some resistance in adopting these technologies, despite the push from technology vendors. Through practice-led research, I propose to research and question the following:Does the contemporary soundtrack suit 3-D films? ; Has sound technology used in 2-D film changed with the introduction of 3-D film? If it has, is this technology an ideal solution, or are further technical developments needed to allow greater creativity and cohesiveness of 3-D film sound design? ; How might industry practices need to develop in order to accommodate the increased dimension and image depth of 3-D visuals? ; Does a language exist to describe spatial sound design in 3-D cinema? ; What is the audience awareness of emerging film technologies? And what does this mean for filmmakers and the cinema? ; Looking beyond contemporary cinema practices, is there an alternative approach to creating a soundtrack that better represents the accompanying 3-D imagery?

A presente dissertação de mestrado foca-se no processo de produção musical em ambientes de espacialização tridimensional utilizando as técnicas Ambisonics.Através da análise de conceitos e metodologias de produção musical tradicional, da observação do conteúdo musical produzido utilizando as técnicas Ambisonics e da pesquisa sobre as técnicas Ambisonics e as tecnologias derivadas da mesma, pretende-se produzir conteúdo musical em um sistema de espacialização. A partir deste processo de produção, planeja-se a recolha de um conjunto de boas práticas que possam qualificar o mesmo.
The present Master's dissertation focuses on the process of musical production in three-dimensional spatialisation environments using Ambisonics techniques.Through the analysis of concepts and methodologies of traditional musical production, the observation of the musical content produced using the Ambisonics techniques and the research on the Ambisonics techniques and the technologies derived from it, it is intended to produce musical content in a system of spatialization. From this production process, it is planned to collect a set of good practices that could qualify the same.

A presente dissertação de mestrado foca-se no processo de produção musical em ambientes de espacialização tridimensional utilizando as técnicas Ambisonics.Através da análise de conceitos e metodologias de produção musical tradicional, da observação do conteúdo musical produzido utilizando as técnicas Ambisonics e da pesquisa sobre as técnicas Ambisonics e as tecnologias derivadas da mesma, pretende-se produzir conteúdo musical em um sistema de espacialização. A partir deste processo de produção, planeja-se a recolha de um conjunto de boas práticas que possam qualificar o mesmo.
The present Master's dissertation focuses on the process of musical production in three-dimensional spatialisation environments using Ambisonics techniques.Through the analysis of concepts and methodologies of traditional musical production, the observation of the musical content produced using the Ambisonics techniques and the research on the Ambisonics techniques and the technologies derived from it, it is intended to produce musical content in a system of spatialization. From this production process, it is planned to collect a set of good practices that could qualify the same.

L’espace est un élément peu exploré en musique. Méconnu des compositeurs, il n’est généralement pas pensé comme paramètre musical « composable ». Pourtant si la musique peut être perçue comme une organisation et une succession d’éléments dans le temps, pourquoi ne pourrait-elle pas l’être aussi dans l’espace? Ce travail se veut en quelque sorte un pont entre la recherche et la pratique, qui se construit par la synthèse de l’information que j’ai pu trouver sur chacune des quatre méthodes de spatialisation abordées ici. Dans un premier temps, je traiterai de leur développement, leur fonctionnement et des possibilités d’intégration de ces méthodes dans le processus de composition musicale, notamment en discutant les outils disponibles. Dans un second temps, les pièces Minimale Sédation et Fondations, toutes deux composées en octophonie seront discutées. J’expliquerai leurs processus de composition à travers les intentions, les techniques d’écriture et les outils qui ont menés à leurs créations.
Space is a parameter of sound that is relatively unexplored in music. Misunderstood by composers, it is not generally thought of as "composable" musical parameter. Yet if music can be seen as an organization and a succession of elements in time, why could it not also be in space? This work is intended to somehow bridge the gap between research and practice, by synthesizing the information I could find on each of the four sound spatialization methods discussed here. As a first step, I will discuss their development, operation and integration capabilities in the process of musical composition, as well as the tools available. In a second step, the work Minimale Sédation and Foundations, both composed in eight channels will be discussed. I will explain their process of composition through intentions, writing techniques and tools that have led to their creations. Keywords

This thesis covers the area of soundfield representation, reconstruction and perception. The complexity and information content of a soundfield presents many mathematical and engineering challenges for accurate reconstruction. After an in-depth review of the field of mathematical soundfield representation, an analysis of the numerical and practical constraints for soundfield reconstruction is presented. A review of work in experimental psycho-acoustics higlights the variability of spatial sound perception. It is shown that the error and uncertainty in perception is of a comparable magnitude to the accuracy achievable by present soundfield systems. Therefore, the effects of hearing adaption, sensory bias, sensory conflict, and contextual memory cannot be ignored. If the listening environment is inappropriate or in conflict with the desired perceptual experience, little is gained from more complex soundfield representation or reconstruction. The imp! lications of this result to the delivery of spatial audio is discussed and some open problems for further exploration and experimentation are detailed.

La idea surge de una motivación propia como grupo, ya que en varias ocasiones tuvimos la oportunidad de vivir la experiencia de visualización de producciones de realidad virtual o cine en 360º. Habiendo transitado la carrera de cine y teniendo una fuerte afinidad con las innovadoras formas de producción audiovisual, aspiramos a profundizar en este nuevo horizonte de realización. Sintiéndonos capaces de crear una historia inmersiva que servirá como material de análisis, al igual que otros trabajos realizados en lo que hace a las producciones dentro de este formato. Pretendemos que estas nuevas puertas creativas y artísticas puedan ser utilizadas por estudiantes de la universidad. Las técnicas de video 360º se vienen desarrollando hace ya varias décadas, sin embargo solo ha sido en los últimos años que se hizo popular para el público. Es así como el término no puede entenderse sin su ancestro el Kinetoscopio, inventado por Edison, que fue el primer intento de proporcionar una experiencia sensorial individualizada e inmersiva al usuario.
Fil: Arros Merino, Lucia Malén. Universidad Nacional de Córdoba. Facultad de Artes. Departamento Académico de Cine y Televisión; Argentina.
Fil: Gross, Alec Sacha. Universidad Nacional de Córdoba. Facultad de Artes. Departamento Académico de Cine y Televisión; Argentina.