Dissertations / Theses on the topic 'Data-driven synthesis'

The currently dominating approach for rendering audio content in interactivemedia, such as video games and virtual reality, involves playback of static audiofiles. This approach is inflexible and requires management of large quantities of audio data. An alternative approach is procedural audio, where sound models are used to generate audio in real time from live inputs. While providing many advantages, procedural audio has yet to find widespread use in commercial productions, partly due to the audio produced by many of the proposed models not meeting industry standards. This thesis investigates how procedural audio can be performed using datadriven methods. We do this by specifically investigating how to generate the sound of car engines using neural audio synthesis. Building on a recently published method that integrates digital signal processing with deep learning, called Differentiable Digital Signal Processing (DDSP), our method obtains sound models by training deep neural networks to reconstruct recorded audio examples from interpretable latent features. We propose a method for incorporating engine cycle phase information, as well as a differentiable transient synthesizer. Our results illustrate that DDSP can be used for procedural engine sounds; however, further work is needed before our models can generate engine sounds without undesired artifacts and before they can be used in live real-time applications. We argue that our approach can be useful for procedural audio in more general contexts, and discuss how our method can be applied to other sound sources.
Det i dagsläget dominerande tillvägagångssättet för rendering av ljud i interaktivamedia, såsom datorspel och virtual reality, innefattar uppspelning av statiska ljudfiler. Detta tillvägagångssätt saknar flexibilitet och kräver hantering av stora mängder ljuddata. Ett alternativt tillvägagångssätt är procedurellt ljud, vari ljudmodeller styrs för att generera ljud i realtid. Trots sina många fördelar används procedurellt ljud ännu inte i någon vid utsträckning inom kommersiella produktioner, delvis på grund av att det genererade ljudet från många föreslagna modeller inte når upp till industrins standarder. Detta examensarbete undersöker hur procedurellt ljud kan utföras med datadrivna metoder. Vi gör detta genom att specifikt undersöka metoder för syntes av bilmotorljud baserade på neural ljudsyntes. Genom att bygga på en nyligen publicerad metod som integrerar digital signalbehandling med djupinlärning, kallad Differentiable Digital Signal Processing (DDSP), kan vår metod skapa ljudmodeller genom att träna djupa neurala nätverk att rekonstruera inspelade ljudexempel från tolkningsbara latenta prediktorer. Vi föreslår en metod för att använda fasinformation från motorers förbränningscykler, samt en differentierbar metod för syntes av transienter. Våra resultat visar att DDSP kan användas till procedurella motorljud, men mer arbete krävs innan våra modeller kan generera motorljud utan oönskade artefakter samt innan de kan användas i realtidsapplikationer. Vi diskuterar hur vårt tillvägagångssätt kan vara användbart inom procedurellt ljud i mer generella sammanhang, samt hur vår metod kan tillämpas på andra ljudkällor

This project investigates the use of artificial neural networks for visual speech synthesis. The objective was to produce a framework for animated chat bots in Swedish. A survey of the literature on the topic revealed that the state-of-the-art approach was using ANNs with either audio or phoneme sequences as input. Three subjective surveys were conducted, both in the context of the final product, and in a more neutral context with less post-processing. They compared the ground truth, captured using the deep-sensing camera of the iPhone X, against both the ANN model and a baseline model. The statistical analysis used mixed effects models to find any statistically significant differences. Also, the temporal dynamics and the error were analyzed. The results show that a relatively simple ANN was capable of learning a mapping from phoneme sequences to blend shape weight sequences with satisfactory results, except for the fact that certain consonant requirements were unfulfilled. The issues with certain consonants were also observed in the ground truth, to some extent. Post-processing with consonant-specific overlays made the ANN’s animations indistinguishable from the ground truth and the subjects perceived them as more realistic than the baseline model’s animations. The ANN model proved useful in learning the temporal dynamics and coarticulation effects for vowels, but may have needed more data to properly satisfy the requirements of certain consonants. For the purposes of the intended product, these requirements can be satisfied using consonant-specific overlays.
Detta projekt utreder hur artificiella neuronnät kan användas för visuell talsyntes. Ändamålet var att ta fram ett ramverk för animerade chatbotar på svenska. En översikt över litteraturen kom fram till att state-of-the-art-metoden var att använda artificiella neuronnät med antingen ljud eller fonemsekvenser som indata. Tre enkäter genomfördes, både i den slutgiltiga produktens kontext, samt i en mer neutral kontext med mindre bearbetning. De jämförde sanningsdatat, inspelat med iPhone X:s djupsensorkamera, med både neuronnätsmodellen och en grundläggande så kallad baselinemodell. Den statistiska analysen använde mixed effects-modeller för att hitta statistiskt signifikanta skillnader i resultaten. Även den temporala dynamiken analyserades. Resultaten visar att ett relativt enkelt neuronnät kunde lära sig att generera blendshapesekvenser utifrån fonemsekvenser med tillfredsställande resultat, förutom att krav såsom läppslutning för vissa konsonanter inte alltid uppfylldes. Problemen med konsonanter kunde också i viss mån ses i sanningsdatat. Detta kunde lösas med hjälp av konsonantspecifik bearbetning, vilket gjorde att neuronnätets animationer var oskiljbara från sanningsdatat och att de samtidigt upplevdes vara bättre än baselinemodellens animationer. Sammanfattningsvis så lärde sig neuronnätet vokaler väl, men hade antagligen behövt mer data för att på ett tillfredsställande sätt uppfylla kraven för vissa konsonanter. För den slutgiltiga produktens skull kan dessa krav ändå uppnås med hjälp av konsonantspecifik bearbetning.

Semi-synthetic β-lactam antibiotics are the most prescribed class of antibiotics in the world. Chemical coupling of a β-lactam moiety with an acyl side chain has dominated the industrial production of semi-synthetic β-lactam antibiotics since their discovery in the early 1960s. Enzymatic coupling of a β-lactam moiety with an acyl side chain can be accomplished in a process that is much more environmentally benign but also results in a much lower yield. The goal of the research presented in this dissertation is to improve the enzymatic synthesis of β-lactam antibiotics via reaction engineering, medium engineering and data-drive protein engineering. Reaction engineering was employed to demonstrate that the hydrolysis of penicillin G to produce the β-lactam nucleus 6-aminopenicillanic acid (6-APA), and the synthesis of ampicillin from 6-APA and (R)-phenylglycine methyl ester ((R)-PGME), can be combined in a cascade conversion. In this work, penicillin G acylase (PGA) was utilized to catalyze the hydrolysis step, and PGA and α-amino ester hydrolase (AEH) were both studied to catalyze the synthesis step. Two different reaction configurations and various relative enzyme loadings were studied. Both configurations present a promising alternative to the current two-pot set-up which requires intermittent isolation of the intermediate, 6-APA. Medium engineering is primarily of interest in β-lactam antibiotic synthesis as a means to suppress the undesired primary and secondary hydrolysis reactions. The synthesis of ampicillin from 6-APA and (R)-PGME in the presence of ethylene glycol was chosen for study after a review of the literature. It was discovered that the transesterification product of (R)-PGME and ethylene glycol, (R)-phenylglycine hydroxyethyl ester, is transiently formed during the synthesis reactions. This never reported side reaction has the ability to positively affect yield by re-directing a portion of the consumption of (R)-PGME to an intermediate that could be used to synthesize ampicillin, rather than to an unusable hydrolysis product. Protein engineering was utilized to alter the selectivity of wild-type PGA with respect to the substituent on the alpha carbon of its substrates. Four residues were identified that had altered selectivity toward the desired product, (R)-ampicillin. Furthermore, the (R)-selective variants improved the yield from pure (R)-PGME up to 2-fold and significantly decreased the amount of secondary hydrolysis present in the reactions. Overall, we have expanded the applicability of PGA and AEH for the synthesis of semi-synthetic β-lactam antibiotics. We have shown the two enzymes can be combined in a novel one-pot cascade, which has the potential to eliminate an isolation step in the current manufacturing process. Furthermore, we have shown that the previously reported ex-situ mixed donor synthesis of ampicillin for PGA can also occur in-situ in the presence of a suitable side chain acyl donor and co-solvent. Finally, we have made significant progress towards obtaining a selective PGA that is capable of synthesizing diastereomerically pure semi-synthetic β-lactam antibiotics from racemic substrates.

Cette thèse porte sur la capture, l'annotation, la synthèse et l'évaluation des mouvements des mains et des bras pour l'animation d'avatars communiquant en Langues des Signes (LS). Actuellement, la production et la diffusion de messages en LS dépendent souvent d'enregistrements vidéo qui manquent d'informations de profondeur et dont l’édition et l'analyse sont difficiles. Les avatars signeurs constituent une alternative prometteuse à la vidéo. Ils sont généralement animés soit à l'aide de techniques procédurales, soit par des techniques basées données. L'animation procédurale donne souvent lieu à des mouvements peu naturels, mais n'importe quel signe peut être produit avec précision. Avec l’animation basée données, les mouvements de l'avatar sont réalistes mais la variété des signes pouvant être synthétisés est limitée et/ou biaisée par la base de données initiale. Privilégiant l’acceptation de l’avatar, nous avons choisi l'approche basée sur les données mais, pour remédier à sa principale limitation, nous proposons d'utiliser les mouvements annotés présents dans une base de mouvements de LS capturés pour synthétiser de nouveaux signes et énoncés absents de cette base. Pour atteindre cet objectif, notre première contribution est la conception, l'enregistrement et l'évaluation perceptuelle d'une base de données de capture de mouvements en Langue des Signes Française (LSF) composée de signes et d'énoncés réalisés par des enseignants sourds de LSF. Notre deuxième contribution est le développement de techniques d'annotation automatique pour différentes pistes d’annotation basées sur l'analyse des propriétés cinématiques de certaines articulations et des algorithmes d'apprentissage automatique existants. Notre dernière contribution est la mise en œuvre de différentes techniques de synthèse de mouvements basées sur la récupération de mouvements par composant phonologique et sur la reconstruction modulaire de nouveaux contenus de LSF avec l'utilisation de techniques de génération de mouvements, comme la cinématique inverse, paramétrées pour se conformer aux propriétés des mouvements réels
This thesis deals with the capture, annotation, synthesis and evaluation of arm and hand motions for the animation of avatars communicating in Sign Languages (SL). Currently, the production and dissemination of SL messages often depend on video recordings which lack depth information and for which editing and analysis are complex issues. Signing avatars constitute a powerful alternative to video. They are generally animated using either procedural or data-driven techniques. Procedural animation often results in robotic and unrealistic motions, but any sign can be precisely produced. With data-driven animation, the avatar's motions are realistic but the variety of the signs that can be synthesized is limited and/or biased by the initial database. As we considered the acceptance of the avatar to be a prime issue, we selected the data-driven approach but, to address its main limitation, we propose to use annotated motions present in an SL Motion Capture database to synthesize novel SL signs and utterances absent from this initial database. To achieve this goal, our first contribution is the design, recording and perceptual evaluation of a French Sign Language (LSF) Motion Capture database composed of signs and utterances performed by deaf LSF teachers. Our second contribution is the development of automatic annotation techniques for different tracks based on the analysis of the kinematic properties of specific joints and existing machine learning algorithms. Our last contribution is the implementation of different motion synthesis techniques based on motion retrieval per phonological component and on the modular reconstruction of new SL content with the additional use of motion generation techniques such as inverse kinematics, parameterized to comply to the properties of real motions

This masters thesis deals with analysis of tree structure data. The aim of this thesis is to design and implement a tool for automated detection of relation among samples of read data considering their three structure and node values. Output of the tool is a prescription for automated synthesis of data for testing purposes. The tool is a part of Testos platform developed at FIT BUT.

Environmental texture sounds are an integral, though often overlooked, part of our daily life. They constitute those elements of our sounding environment that we tend to perceive subconsciously but which we miss when they are missing. Those sounds are also increasingly important for adding realism to virtual environments, from immersive artificial worlds through computer games to mobile augmented reality systems. This work spans the spectrum from data-driven stochastic sound synthesis methods to distributed virtual reality environments and their aesthetic and technological implications. We propose a framework for statistically modelling environmental texture sounds in different sparse signal representations. We explore three different instantiations of this framework, two of which constitute a novel way of representing texture sounds in a physically-inspired sparse statistical model and of estimating model parameters from recorded sound examples.
Los sonidos texturales ambientales son parte integral de nuestra vida diaria, a pesar de que muchas veces pasen desapercibidos. Constituyen esos elementos de nuestro entorno sonoro que solemos percibir de manera subconsciente pero que extrañamos cuando desaparecen. Esos sonidos son también cada vez más importantes para añadir realismo a los ambientes virtuales, desde mundos artificiales de inmersión hasta sistemas móviles de realidad aumentada, pasando por juegos de ordenador. Este trabajo abarca todo el espectro desde métodos de síntesis de sonido estocásticos basados en datos hasta entornos distribuidos de realidad virtual, así como sus implicaciones estéticas y tecnológicas. Proponemos un marco para modelar estadísticamente sonidos ambientales texturales en diferentes representaciones sparse de señales. Exploramos tres diferentes instanciaciones de este marco, dos de las cuales constituyen una nueva manera de representar sonidos texturales en un modelo estadístico inspirado físicamente así como de estimar parámetros de modelo a partir de ejemplos de sonido grabados.

ISAGENis a tool for the automatic generation of structurally complex test inputs that imitate real communication in the context of modern information systems . Complex, typically tree-structured data currently represents the standard means of transmitting information between nodes in distributed information systems. Automatic generator ISAGENis founded on the methodology of data-driven testing and uses concrete data from the production environment as the primary characteristic and specification that guides the generation of new similar data for test cases satisfying given combinatorial adequacy criteria. The main contribution of this thesis is a comprehensive proposal of automated data generation techniques together with an implementation, which demonstrates their usage. The created solution enables testers to create more relevant testing data, representing production-like communication in information systems.

Embedded computer systems are used as control systems in many products, such as VCRs, digital cameras, washing machines, automobiles, airplanes, etc. As the complexity of embedded applications grows and time-to-market of the products they are used in reduces, designing reliable systems satisfying multiple require-ments is a great challenge. Successful design, nowadays, cannot be performed without good design tools based on powerful design methodologies. These tools should explore different design alternatives to find the best one and do that at high abstraction levels to manage the complexity and reduce the design time.

A design is specified using models. Different models are used at different de-sign stages and abstraction levels. For example, the functionality of an application can be specified using hierarchical functional blocks. However, for such design tasks as mapping and scheduling, a lower-level flat model of interacting processes is needed. Deriving this model from a higher-level model of functional blocks is the main focus of this thesis. Our objective is to develop efficient strategies for such derivations, aiming at producing a process graph specification, which helps the synthesis tasks to find schedulable implementations. We proposed several strategies and evaluated them experimentally.

碩士
國立交通大學
多媒體工程研究所
102
We present a novel data-driven synthesis approach to design and animate the fires. Our system precomputes a set of basis fire simulations with different configurations and stores these basis fires as pathlines to construct a basis database. Our system also provides an interface for users to design the shape of fires easily by using the skeletons. Then, we synthesize the designed fire animation frames by using the pathlines in the database to fit the skeletons of designed shapes and apply the flow graph to synthesize a fire animation with the desired shapes. The fitting method is employed to select a set of pathlines of basis fires in order to fit the skeletons of the designed fire shapes. The flow graph is used to generate a long-lasting fire animation by using the finite animation frames and make the synthesized fire animation run at real-time rates. Our design process enables users to design the fire shapes without worrying about tuning the physical parameters.

碩士
國立臺灣大學
資訊工程學研究所
102
A person''s handwriting appears differently within a typical range of variations, and the shapes of handwriting characters also show complex interaction with their nearby neighbours. This makes automatic synthesis of handwriting characters and paragraphs very challenging. In this paper, we propose a method for synthesizing handwriting texts according to a writer''s handwriting style. The synthesis algorithm is composed by two phases. First, we create the shape models for different characters based on one writer''s data. Then, we compute the cursive probability to decide whether each pair of neighbouring characters are conjoined together or not. By jointly modelling the handwriting style and conjoined property through a novel trajectory optimization, final handwriting words can be synthesized from a set of collected samples. Furthermore, the paragraphs'' layouts are also automatically generated and adjusted according to the writer''s style obtained from the same dataset. We demonstrate that our method can successfully synthesize an entire paragraph that imitate a writer''s handwriting using his/her collected handwriting samples.

The planning of organic syntheses, a critical problem in chemistry, can be directly modeled as resource- constrained branching plans in a discrete, fully-observable state space. Despite this clear relationship, the full artillery of artificial intelligence has not been brought to bear on this problem due to its inherent complexity and multidisciplinary challenges. In this thesis, I describe a mapping between organic synthesis and heuristic search and build a planner that can solve such problems automatically at the undergraduate level. Along the way, I show the need for powerful heuristic search algorithms and build large databases of synthetic information, which I use to derive a qualitatively new kind of heuristic guidance.

Indiana University-Purdue University Indianapolis (IUPUI)
This thesis proposes an artificial neural network (NN) model to predict fuel consumption in heavy vehicles. The model uses predictors derived from vehicle speed, mass, and road grade. These variables are readily available from telematics devices that are becoming an integral part of connected vehicles. The model predictors are aggregated over a fixed distance traveled (i.e., window) instead of fixed time interval. It was found that 1km windows is most appropriate for the vocations studied in this thesis. Two vocations were studied, refuse and delivery trucks. The proposed NN model was compared to two traditional models. The first is a parametric model similar to one found in the literature. The second is a linear regression model that uses the same features developed for the NN model. The confidence level of the models using these three methods were calculated in order to evaluate the models variances. It was found that the NN models produce lower point-wise error. However, the stability of the models are not as high as regression models. In order to improve the variance of the NN models, an ensemble based on the average of 5-fold models was created. Finally, the confidence level of each model is analyzed in order to understand how much error is expected from each model. The mean training error was used to correct the ensemble predictions for five K-Fold models. The ensemble K-fold model predictions are more reliable than the single NN and has lower confidence interval than both the parametric and regression models.

碩士
國立臺灣大學
電信工程學研究所
105
Nine different strategies are proposed to compensate the cross-track motion errors in synthetic aperture radar (SAR) imaging, based on estimating the phase coefficients of the phase history. A spline interpolation method and a subaperture reconstruction method are used to derive the phase history over the whole aperture, based on the phase coefficients previously estimated. Four different scenarios are designed to compare the performance of these nine strategies.