To see the other types of publications on this topic, follow the link: Stochastic neural networks.
Dissertations / Theses on the topic 'Stochastic neural networks'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 dissertations / theses for your research on the topic 'Stochastic neural networks.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse dissertations / theses on a wide variety of disciplines and organise your bibliography correctly.
1
Pensuwon, Wanida. "Stochastic dynamic hierarchical neural networks." Thesis, University of Hertfordshire, 2001. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.366030.
Full text
2
Sutherland, Connie. "Spatio-temporal feedback in stochastic neural networks." Thesis, University of Ottawa (Canada), 2007. http://hdl.handle.net/10393/27559.
Full textAbstract:
The mechanisms by which groups of neurons interact is an important facet to understanding how the brain functions. Here we study stochastic neural networks with delayed feedback. The first part of our study looks at how feedback and noise affect the mean firing rate of the network. Secondly we look at how the spatial profile of the feedback affects the behavior of the network.
Our numerical and theoretical results show that negative (inhibitory) feedback linearizes the frequency vs input current (f-I) curve via the divisive gain effect it has on the network. The interaction of the inhibitory feedback and the input bias is what produces the divisive decrease in the slope (known as the gain) of the f-I curve. Our work predicts that an increase in noise is required along with increase in inhibitory feedback to attain a divisive and subtractive shift of the gain as seen in experiments [1].
Our results also show that, although the spatial profile of the feedback does not effect the mean activity of the network, it does influence the overall dynamics of the network. Local feedback generates a network oscillation, which is more robust against disruption by noise or uncorrelated input or network heterogeneity, than that for the global feedback (all-to-all coupling) case. For example uncorrelated input completely disrupts the network oscillation generated by global feedback, but only diminishes the network oscillation due to local feedback. This is characterized by 1st and 2nd order spike train statistics. Further, our theory agrees well with numerical simulations of network dynamics.
3
CAMPOS, LUCIANA CONCEICAO DIAS. "PERIODIC STOCHASTIC MODEL BASED ON NEURAL NETWORKS." PONTIFÍCIA UNIVERSIDADE CATÓLICA DO RIO DE JANEIRO, 2010. http://www.maxwell.vrac.puc-rio.br/Busca_etds.php?strSecao=resultado&nrSeq=17076@1.
Full textAbstract:
CONSELHO NACIONAL DE DESENVOLVIMENTO CIENTÍFICO E TECNOLÓGICO<br>Processo Estocástico é um ramo da teoria da probabilidade onde se define um conjunto de modelos que permitem o estudo de problemas com componentes aleatórias. Muitos problemas reais apresentam características complexas, tais como não-linearidade e comportamento caótico, que necessitam de modelos capazes de capturar as reais características do problema para obter um tratamento apropriado. Porém, os modelos existentes ou são lineares, cuja aplicabilidade a esses problemas pode ser inadequada, ou necessitam de uma formulação complexa, onde a aplicabilidade é limitada e específica ao problema, ou dependem de suposições a priori sobre o comportamento do problema para poderem ser aplicados. Isso motivou a elaboração de um novo modelo de processo estocástico genérico, intrinsecamente não-linear, que possa ser aplicado em uma gama de problemas de fenômenos não-lineares, de comportamento altamente estocástico, e até mesmo com características periódicas. Como as redes neurais artificiais são modelos paramétricos não-lineares, simples de entendimento e implementação, capazes de capturar comportamentos de variados tipos de problemas, decidiu-se então utilizá-las como base do novo modelo proposto nessa tese, que é denominado Processo Estocástico Neural. A não-linearidade, obtida através das redes neurais desse processo estocástico, permite que se capture adequadamente o comportamento da série histórica de problemas de fenômenos não-lineares, com características altamente estocásticas e até mesmo periódicas. O objetivo é usar esse modelo para gerar séries temporais sintéticas, igualmente prováveis à série histórica, na solução desses tipos de problemas, como por exemplo os problemas que envolvem fenômenos climatológicos, econômicos, entre outros. Escolheu-se, como estudo de caso dessa tese, aplicar o modelo proposto no tratamento de afluências mensais sob o contexto do planejamento da operação do sistema hidrotérmico brasileiro. Os resultados mostraram que o Processo Estocástico Neural consegue gerar séries sintéticas com características similares às séries históricas de afluências mensais.<br>Stochastic Process is a branch of probability theory which defines a set of
templates that allow the study of problems with random components. Many
real problems exhibit complex characteristics such as nonlinearity and chaotic
behavior, which require models capable of capture the real characteristics
of the problem for a appropriate treatment. However, existing models have
limited application to certain problems or because they are linear models
(whose application gets results inconsistent or inadequate) or because they
require a complex formulation or depend on a priori assumptions about the
behavior of the problem, which requires a knowledge the problem at a level
of detail that there is not always available. This motivated the development
of a model stochastic process based on neural networks, so that is generic
to be applied in a range of problems involving highly stochastic phenomena
of behavior and also can be applied to phenomena that have periodic characteristics.
As artificial neural networks are non-linear models, simple to
understand and implementation, able to capture behaviors of varied types
problems, then decided to use them as the basis of new model proposed
in this thesis, which is an intrinsically non-linear model, called the Neural
Stochastic Process. Through neural networks that stochastic process,
can adequately capture the behavior problems of the series of phenomena
with features highly stochastic and / or periodical. The goal is to use this
model to generate synthetic time series, equally likely to historical series,
in solution of various problems, eg problems phenomena involving climatology,
economic, among others. It was chosen as a case study of this thesis,
applying the model proposed in the treatment of monthly inflows in the
context of operation planning of the Brazilian hydrothermal system. The
Results showed that the process can Stochastic Neural generate synthetic
series of similar characteristics to the historical monthly inflow series.
4
Ling, Hong. "Implementation of Stochastic Neural Networks for Approximating Random Processes." Master's thesis, Lincoln University. Environment, Society and Design Division, 2007. http://theses.lincoln.ac.nz/public/adt-NZLIU20080108.124352/.
Full textAbstract:
Artificial Neural Networks (ANNs) can be viewed as a mathematical model to simulate natural and biological systems on the basis of mimicking the information processing methods in the human brain. The capability of current ANNs only focuses on approximating arbitrary deterministic input-output mappings. However, these ANNs do not adequately represent the variability which is observed in the systems natural settings as well as capture the complexity of the whole system behaviour. This thesis addresses the development of a new class of neural networks called Stochastic Neural Networks (SNNs) in order to simulate internal stochastic properties of systems. Developing a suitable mathematical model for SNNs is based on canonical representation of stochastic processes or systems by means of Karhunen-Loève Theorem. Some successful real examples, such as analysis of full displacement field of wood in compression, confirm the validity of the proposed neural networks. Furthermore, analysis of internal workings of SNNs provides an in-depth view on the operation of SNNs that help to gain a better understanding of the simulation of stochastic processes by SNNs.
5
Zhao, Jieyu. "Stochastic bit stream neural networks : theory, simulations and applications." Thesis, Royal Holloway, University of London, 1996. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.338916.
Full text
6
Hyland, P. "On the implementation of neural networks using stochastic arithmetic." Thesis, Bangor University, 1992. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.306224.
Full text
7
Todeschi, Tiziano. "Calibration of local-stochastic volatility models with neural networks." Master's thesis, Alma Mater Studiorum - Università di Bologna, 2021. http://amslaurea.unibo.it/23052/.
Full textAbstract:
During the last twenty years several models have been proposed to improve the classic Black-Scholes framework for equity derivatives pricing. Recently a new model has been proposed: Local-Stochastic Volatility Model (LSV). This model considers volatility as the product between a deterministic and a stochastic term. So far, the model choice was not only driven by the capacity of capturing empirically observed market features well, but also by the computational tractability of the calibration process. This is now undergoing a big change since machine learning technologies offer new perspectives on model calibration. In this thesis we consider the calibration problem to be the search for a model which generates given market prices and where additionally technology from generative adversarial networks can be used. This means parametrizing the model pool in a way which is accessible for machine learning techniques and interpreting the inverse problems a training task of a generative network, whose quality is assessed by an adversary. The calibration algorithm proposed for LSV models use as generative models so-called neural stochastic differential equations (SDE), which just means to parameterize the drift and volatility of an Ito-SDE by neural networks.
8
陳穎志 and Wing-chi Chan. "Modelling of nonlinear stochastic systems using neural and neurofuzzy networks." Thesis, The University of Hong Kong (Pokfulam, Hong Kong), 2001. http://hub.hku.hk/bib/B31241475.
Full text
9
Chan, Wing-chi. "Modelling of nonlinear stochastic systems using neural and neurofuzzy networks /." Hong Kong : University of Hong Kong, 2001. http://sunzi.lib.hku.hk/hkuto/record.jsp?B22925843.
Full text
10
Rising, Barry John Paul. "Hardware architectures for stochastic bit-stream neural networks : design and implementation." Thesis, Royal Holloway, University of London, 2000. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.326219.
Full text
11
Zhu, Huaiyu. "Neural networks and adaptive computers : theory and methods of stochastic adaptive computation." Thesis, University of Liverpool, 1993. http://eprints.aston.ac.uk/365/.
Full textAbstract:
This thesis studies the theory of stochastic adaptive computation based on neural networks. A mathematical theory of computation is developed in the framework of information geometry, which generalises Turing machine (TM) computation in three aspects - It can be continuous, stochastic and adaptive - and retains the TM computation as a subclass called "data processing". The concepts of Boltzmann distribution, Gibbs sampler and simulated annealing are formally defined and their interrelationships are studied. The concept of "trainable information processor" (TIP) - parameterised stochastic mapping with a rule to change the parameters - is introduced as an abstraction of neural network models. A mathematical theory of the class of homogeneous semilinear neural networks is developed, which includes most of the commonly studied NN models such as back propagation NN, Boltzmann machine and Hopfield net, and a general scheme is developed to classify the structures, dynamics and learning rules. All the previously known general learning rules are based on gradient following (GF), which are susceptible to local optima in weight space. Contrary to the widely held belief that this is rarely a problem in practice, numerical experiments show that for most non-trivial learning tasks GF learning never converges to a global optimum. To overcome the local optima, simulated annealing is introduced into the learning rule, so that the network retains adequate amount of "global search" in the learning process. Extensive numerical experiments confirm that the network always converges to a global optimum in the weight space. The resulting learning rule is also easier to be implemented and more biologically plausible than back propagation and Boltzmann machine learning rules: Only a scalar needs to be back-propagated for the whole network. Various connectionist models have been proposed in the literature for solving various instances of problems, without a general method by which their merits can be combined. Instead of proposing yet another model, we try to build a modular structure in which each module is basically a TIP. As an extension of simulated annealing to temporal problems, we generalise the theory of dynamic programming and Markov decision process to allow adaptive learning, resulting in a computational system called a "basic adaptive computer", which has the advantage over earlier reinforcement learning systems, such as Sutton's "Dyna", in that it can adapt in a combinatorial environment and still converge to a global optimum. The theories are developed with a universal normalisation scheme for all the learning parameters so that the learning system can be built without prior knowledge of the problems it is to solve.
12
Balkhair, Khaled Saeed. "Artificial neural networks and conditional stochastic simulations for characterization of aquifer heterogeneity." Diss., The University of Arizona, 1999. http://hdl.handle.net/10150/284451.
Full textAbstract:
Although it is one of the most difficult tasks in hydrology, delineation of aquifer heterogeneity is essential for accurate simulation of groundwater flow and transport. There are various approaches used to delineate aquifer heterogeneity from a limited data set, and each has its own difficulties and drawbacks. The inverse problem is usually used for estimating different hydraulic properties (e.g. transmissivity) from scattered measurements of these properties, as well as hydraulic head. Difficulties associated with this approach are issues of indentifiability, uniqueness, and stability. The Iterative Conditional Simulation (ICS) approach uses kriging (or cokriging), to provide estimates of the property at unsampled locations while retaining the measured values at the sampled locations. Although the relation between transmissivity (T) and head (h) in the governing flow equation is nonlinear, the cross covariance function and the covariance of h are derived from a first-order-linearized version of the equation. Even if the log transformation of T is adopted, the nonlinear nature between f (mean removed Ln[T]) and h still remains. The linearized relations then, based on small perturbation theory, are valid only if the unconditional variance of f is less than 1.0. Inconsistent transmissivity and head fields may occur as a result of using a linear relation between T and h. In this dissertation, Artificial Neural Networks (ANN) is investigated as a means for delineating aquifer heterogeneity. Unlike ICS, this new computational tool does not rely on a prescribed relation, but seeks its own. Neural Networks are able to learn arbitrary non-linear input-output mapping directly from training data and have the very advantageous property of generalization. For this study, a random field generator was used to generate transmissivity fields from known geostatistical parameters. The corresponding head fields were obtained using the governing flow equation. Both T and h at sampled locations were used as input vectors for two different back-propagation neural networks designed for this research. The corresponding values of transmissivities at unsampled location (unknown), constituting the output vector, were estimated by the neural networks. Results from the ANN were compared to those obtained from the (ICS) approach for different degrees of heterogeneity. The degree of heterogeneity was quantified using the variance of the transmissivity field, where values of 1.0, 2.0, and 5.0 were used. It was found that ANN overcomes the limitations of ICS at high variances. Thus, ANN was better able to accurately map the highly heterogeneous fields using limited sample points.
13
Carelli, Pedro Valadão. "Modelagem estocástica de neurônios e sua interação em tempo real com neurônios biológicos." Universidade de São Paulo, 2008. http://www.teses.usp.br/teses/disponiveis/43/43134/tde-04092008-154828/.
Full textAbstract:
Desenvolvemos um modelo estocástico da atividade elétrica de um neurônio motor do gânglio estomatogástrico de crustáceos, a partir de um modelo determinístico eletrofisiologicamente plausível. Com isso recuperamos características da dinâmica neural sempre observadas em neurônios isolados, tais como irregularidades nos padrões de disparos que não são reproduzidas pelo modelo determinístico original. Implementamos otimizações e simplificações no método numérico de simulação estocástica que permitiram rodar a simulação em tempo real para interagir modelos computacionais com neurônios biológicos, implementando sinapses artificiais entre eles. Por fim utilizamos o modelo e os métodos de simulação desenvolvidos para substituir neurônios do gânglio estomatogástrico e construir sistemas híbridos, que foram usados para verificar como ocorre a transmissão de informação entre neurônios biológicos e artificiais, quando a dinâmicas destes é estocástica ou determinística.<br>We developed a mathematical model of the electrical activity of a motor neuron from the stomatogastric ganglion of crustaceans. It was inspired on a previous existing deterministic model which is considered as electrophysiologically plausible in the recent literature. However, this deterministic model were not able to reproduce the irregular bursting behavior found in those biological neurons when isolated from the neural circuit. Our model, based on the microscopic stochastic behavior of the membrane ion channels, successfully reproduced the intrinsic irregular properties that were missing in the original deterministic model. To allow the real time performing of the stochastic model simulations we have to deal with some simplifications and to implement several optimizations that are also describe in detail. The real time version of our stochastic model was implemented in a dynamic clamp protocol to interface the computational model to real neurons. Finally, we applied the implemented versions of real time simulation and interfacing protocols to replace some biological bursting neurons of the stomatogastric ganglion. These hibrid neural networks were used to study how the information (diferent patterns of interspike intervals) is transmitted between biological and two types of artificial neurons: deterministic and stochastic.
14
Orr, Genevieve Beth. "Dynamics and algorithms for stochastic search /." Full text open access at:, 1995. http://content.ohsu.edu/u?/etd,197.
Full text
15
Feurstein, Markus, and Martin Natter. "Neural networks, stochastic dynamic programming and a heuristic for valuing flexible manufacturing systems." SFB Adaptive Information Systems and Modelling in Economics and Management Science, WU Vienna University of Economics and Business, 1998. http://epub.wu.ac.at/1106/1/document.pdf.
Full textAbstract:
We compare the use of stochastic dynamic programming (SDP), Neural Networks and a simple approximation rule for calculating the real option value of a flexible production system. While SDP yields the best solution to the problem, it is computationally prohibitive for larger settings. We test two approximations of the value function and show that the results are comparable to those obtained via SDP. These methods have the advantage of a high computational performance and of no restrictions on the type of process used. Our approach is not only useful for supporting large investment decisions, but it can also be applied in the case of routine decisions like the determination of the production program when stochastic profit margins occur. (author's abstract)<br>Series: Working Papers SFB "Adaptive Information Systems and Modelling in Economics and Management Science"
16
Haverinen, J. (Janne). "Adaptation through a Stochastic Evolutionary Neuron Migration Process." Doctoral thesis, University of Oulu, 2004. http://urn.fi/urn:isbn:9514273079.
Full textAbstract:
Abstract
Artificial Life is an interdisciplinary scientific and engineering enterprise investigating the fundamental properties of living systems through the simulation and synthesis of life-like processes in artificial media. One of the avenues of investigation is autonomous robots and agents.
Mimicking of the growth and adaptation of a biological neural circuit in an artificial medium is a challenging task owing to our limited knowledge of the complex process taking place in a living organism. By combining several developmental mechanisms, including the chemical, mechanical, genetic, and electrical, researchers have succeeded in developing networks with interesting topology, morphology, and function within Artificial Computational Chemistry. However, most of these approaches still fail to create neural circuits able to solve real problems in perception and robot control.
In this thesis a phenomenological developmental model called a Stochastic Evolutionary Neuron Migration Process (SENMP) is proposed. Employing a spatial encoding scheme with lateral interaction of neurons for artificial neural networks, which represent candidate solutions within a neural network ensemble, neurons of the ensemble form problem-specific spatial patterns with the desired dynamics as they migrate under the selective pressure.
The approach is applied to gain new insights into development, adaptation and plasticity in neural networks and to evolve purposeful behaviors for mobile robots. In addition, the approach is used to study the relationship of spatial patterns, composed of interacting entities, and their dynamics.
The feasibility and advantages of the approach are demonstrated by evolving neural controllers for solving a non-Markovian double pole balancing problem and by evolving controllers that exhibit navigation behavior for simulated and real mobile robots in complex environments. Preliminary results regarding the behavior of the adapting neural network ensemble are also shown and, particularly, a phenomenon exhibiting Hebbian-like dynamics.
This thesis is a step toward a long range goal that aims to create an intelligent robot that is capable of learning complex skills and adapts rapidly to environmental changes.
17
Tran-Canh, Dung. "Simulating the flow of some non-Newtonian fluids with neural-like networks and stochastic processes." University of Southern Queensland, Faculty of Engineering and Surveying, 2004. http://eprints.usq.edu.au/archive/00001518/.
Full textAbstract:
The thesis reports a contribution to the development of neural-like network- based element-free methods for the numerical simulation of some non-Newtonian fluid flow problems. The numerical approximation of functions and solution of the governing partial differential equations are mainly based on radial basis function networks. The resultant micro-macroscopic approaches do not require any element-based discretisation and only rely on a set of unstructured collocation points and hence are truly meshless or element-free. The development of the present methods begins with the use of the multi-layer perceptron networks (MLPNs) and radial basis function networks (RBFNs) to effectively eliminate the volume integrals in the integral formulation of fluid flow problems. An adaptive velocity gradient domain decomposition (AVGDD) scheme is incorporated into the computational algorithm. As a result, an improved feed forward neural network boundary-element-only method (FFNN- BEM) is created and verified. The present FFNN-BEM successfully simulates the flow of several Generalised Newtonian Fluids (GNFs), including the Carreau, Power-law and Cross models. To the best of the author's knowledge, the present FFNN-BEM is the first to achieve convergence for difficult flow situations when the power-law indices are very small (as small as 0.2). Although some elements are still used to discretise the governing equations, but only on the boundary of the analysis domain, the experience gained in the development of element-free approximation in the domain provides valuable skills for the progress towards an element-free approach. A least squares collocation RBFN-based mesh-free method is then developed for solving the governing PDEs. This method is coupled with the stochastic simulation technique (SST), forming the mesoscopic approach for analyzing viscoelastic flid flows. The velocity field is computed from the RBFN-based mesh-free method (macroscopic component) and the stress is determined by the SST (microscopic component). Thus the SST removes a limitation in traditional macroscopic approaches since closed form constitutive equations are not necessary in the SST. In this mesh-free method, each of the unknowns in the conservation equations is represented by a linear combination of weighted radial basis functions and hence the unknowns are converted from physical variables (e.g. velocity, stresses, etc) into network weights through the application of the general linear least squares principle and point collocation procedure. Depending on the type of RBFs used, a number of parameters will influence the performance of the method. These parameters include the centres in the case of thin plate spline RBFNs (TPS-RBFNs), and the centres and the widths in the case of multi-quadric RBFNs (MQ-RBFNs). A further improvement of the approach is achieved when the Eulerian SST is formulated via Brownian configuration fields (BCF) in place of the Lagrangian SST. The SST is made more efficient with the inclusion of the control variate variance reduction scheme, which allows for a reduction of the number of dumbbells used to model the fluid. A highly parallelised algorithm, at both macro and micro levels, incorporating a domain decomposition technique, is implemented to handle larger problems. The approach is verified and used to simulate the flow of several model dilute polymeric fluids (the Hookean, FENE and FENE-P models) in simple as well as non-trivial geometries, including shear flows (transient Couette, Poiseuille flows)), elongational flows (4:1 and 10:1 abrupt contraction flows) and lid-driven cavity flows.
18
Boscarino, Andrea. "Deep Learning Models with Stochastic Targets: an Application for Transprecision Computing." Master's thesis, Alma Mater Studiorum - Università di Bologna, 2020. http://amslaurea.unibo.it/20078/.
Full textAbstract:
Il presente elaborato di tesi è parte di un ampio progetto finanziato dall’Unione Europea, sotto il programma Horizon 2020 per la ricerca e l’innovazione, Open Transprecision Computing (OPRECOMP). Il progetto, della durata di 4 anni, punta a superare l’assunto conservativo secondo cui ogni calcolo compiuto da sistemi e applicazioni computazionali debba essere eseguito utilizzando la massima precisione numerica. Tale assunto è finora risultato sensato in vista di un’efficienza computazionale sempre migliore col passare del tempo, secondo la legge di Moore. Com’è noto, nell’era attuale tale legge ha iniziato a perdere di validità con l’approssimarsi dei limiti fisici che impediscono ulteriori miglioramenti di grande ordine previsti di anno in anno, dando piuttosto spazio a miglioramenti marginali.
L’approccio proposto dal progetto OPRECOMP (il cui sviluppo vuole beneficiare applicazioni che spaziano dai piccoli nodi computazionali per l’Internet-of-Things, fino ai centri computazionali di High Performance Computing) è basato sul paradigma del Transprecision Computing, che supera l’assunto della massima precisione in favore di calcoli approssimati; tramite tale paradigma si arriva ad un doppio vantaggio: computazioni più efficienti e brevi, e soprattutto, risparmio energetico.
Per fare ciò, OPRECOMP sfrutta il principio secondo cui quasi ogni applicazione computazionale utilizza nodi intermedi di calcolo, le cui precisioni possono essere tarate (in modo controllato) con conseguenze minime sull’affidabilità del risultato finale.
All'interno dell'elaborato vengono esplorate soluzioni e metodologie di machine learning (e in particolare modelli stocastici, ovvero distribuzioni probabilistiche caratterizzate da errore medio e varianza) con lo scopo di apprendere la relazione che incorre tra la scelta del numero di bit utilizzati per le variabili di alcuni benchmark matematici e il relativo errore rilevato rispetto alla stessa computazione eseguita a precisione massima.
19
Ahmadian, Mansooreh. "Hybrid Modeling and Simulation of Stochastic Effects on Biochemical Regulatory Networks." Diss., Virginia Tech, 2020. http://hdl.handle.net/10919/99481.
Full textAbstract:
A complex network of genes and proteins governs the robust progression through cell cycles in the presence of inevitable noise. Stochastic modeling is viewed as a key paradigm to study the effects of intrinsic and extrinsic noise on the dynamics of biochemical networks. A detailed quantitative description of such complex and multiscale networks via stochastic modeling poses several challenges. First, stochastic models generally require extensive computations, particularly when applied to large networks. Second, the accuracy of stochastic models is highly dependent on the quality of the parameter estimation based on experimental observations. The goal of this dissertation is to address these problems by developing new efficient methods for modeling and simulation of stochastic effects in biochemical systems. Particularly, a hybrid stochastic model is developed to represent a detailed molecular mechanism of cell cycle control in budding yeast cells. In a single multiscale model, the proposed hybrid approach combines the advantages of two regimes: 1) the computational efficiency of a deterministic approach, and 2) the accuracy of stochastic simulations. The results show that this hybrid stochastic model achieves high computational efficiency while generating simulation results that match very well with published experimental measurements. Furthermore, a new hierarchical deep classification (HDC) algorithm is developed to address the parameter estimation problem in a monomolecular system. The HDC algorithm adopts a neural network that, via multiple hierarchical search steps, finds reasonably accurate ranges for the model parameters. To train the neural network in the presence of experimental data scarcity, the proposed method leverages the domain knowledge from stochastic simulations to generate labeled training data. The results show that the proposed HDC algorithm yields accurate ranges for the model parameters and highlight the potentials of model-free learning for parameter estimation in stochastic modeling of complex biochemical networks.<br>Doctor of Philosophy<br>Cell cycle is a process in which a growing cell replicates its DNA and divides into two cells. Progression through the cell cycle is regulated by complex interactions between networks of genes, transcripts, and proteins. These interactions inside the confined volume of a cell are subject to inherent noise. To provide a quantitative description of the cell cycle, several deterministic and stochastic models have been developed. However, deterministic models cannot capture the intrinsic noise. In addition, stochastic modeling poses the following challenges.
First, stochastic models generally require extensive computations, particularly when applied to large networks. Second, the accuracy of stochastic models is highly dependent on the accuracy of the estimated model parameters. The goal of this dissertation is to address these challenges by developing new efficient methods for modeling and simulation of stochastic effects in biochemical networks. The results show that the proposed hybrid model that combines stochastic and deterministic modeling approaches can achieve high computational efficiency while generating accurate simulation results. Moreover, a new machine learning-based method is developed to address the parameter estimation problem in biochemical systems. The results show that the proposed method yields accurate ranges for the model parameters and highlight the potentials of model-free learning for parameter estimation in stochastic modeling of complex biochemical networks.
20
Myers, James William. "Stochastic algorithms for learning with incomplete data an application to Bayesian networks /." Full text available online (restricted access), 1999. http://images.lib.monash.edu.au/ts/theses/Myers.pdf.
Full text
21
Droh, Erik. "T-Distributed Stochastic Neighbor Embedding Data Preprocessing Impact on Image Classification using Deep Convolutional Neural Networks." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-237422.
Full textAbstract:
Image classification in Machine Learning encompasses the task of identification of objects in an image. The technique has applications in various areas such as e-commerce, social media and security surveillance. In this report the author explores the impact of using t-Distributed Stochastic Neighbor Embedding (t-SNE) on data as a preprocessing step when classifying multiple classes of clothing with a state-of-the-art Deep Convolutional Neural Network (DCNN). The t-SNE algorithm uses dimensionality reduction and groups similar objects close to each other in three-dimensional space. Extracting this information in the form of a positional coordinate gives us a new parameter which could help with the classification process since the features it uses can be different from that of the DCNN. Therefore, three slightly different DCNN models receives different input and are compared. The first benchmark model only receives pixel values, the second and third receive pixel values together with the positional coordinates from the t-SNE preprocessing for each data point, but with different hyperparameter values in the preprocessing step. The Fashion-MNIST dataset used contains 10 different clothing classes which are normalized and gray-scaled for easeof-use. The dataset contains 70.000 images in total. Results show minimum change in classification accuracy in the case of using a low-density map with higher learning rate as the data size increases, while a more dense map and lower learning rate performs a significant increase in accuracy of 4.4% when using a small data set. This is evidence for the fact that the method can be used to boost results when data is limited.<br>Bildklassificering i maskinlärning innefattar uppgiften att identifiera objekt i en bild. Tekniken har applikationer inom olika områden så som e-handel, sociala medier och säkerhetsövervakning. I denna rapport undersöker författaren effekten av att användat-Distributed Stochastic Neighbour Embedding (t-SNE) på data som ett förbehandlingssteg vid klassificering av flera klasser av kläder med ett state-of-the-art Deep Convolutio-nal Neural Network (DCNN). t-SNE-algoritmen använder dimensioneringsreduktion och grupperar liknande objekt nära varandra i tredimensionellt utrymme. Att extrahera denna information i form av en positionskoordinat ger oss en ny parameter som kan hjälpa till med klassificeringsprocessen eftersom funktionerna som den använder kan skilja sig från DCNN-modelen. Tre olika DCNN-modeller får olika in-data och jämförs därefter. Den första referensmodellen mottar endast pixelvärden, det andra och det tredje motar pixelvärden tillsammans med positionskoordinaterna från t-SNE-förbehandlingen för varje datapunkt men med olika hyperparametervärden i förbehandlingssteget. I studien används Fashion-MNIST datasetet som innehåller 10 olika klädklasser som är normaliserade och gråskalade för enkel användning. Datasetet innehåller totalt 70.000 bilder. Resultaten visar minst förändring i klassificeringsnoggrannheten vid användning av en låg densitets karta med högre inlärningsgrad allt eftersom datastorleken ökar, medan en mer tät karta och lägre inlärningsgrad uppnår en signifikant ökad noggrannhet på 4.4% när man använder en liten datamängd. Detta är bevis på att metoden kan användas för att öka klassificeringsresultaten när datamängden är begränsad.
22
Antrobus, Alexander Dennis. "Achieving baseline states in sparsely connected spiking-neural networks: stochastic and dynamic approaches in mathematical neuroscience." Master's thesis, University of Cape Town, 2015. http://hdl.handle.net/11427/19949.
Full textAbstract:
Networks of simple spiking neurons provide abstract models for studying the dynamics of biological neural tissue. At the expense of cellular-level complexity, they are a frame-work in which we can gain a clearer understanding of network-level dynamics. Substantial insight can be gained analytically, using methods from stochastic calculus and dynamical systems theory. This can be complemented by data generated from computational simulations of these models, most of which benefit easily from parallelisation. One cubic millimetre of mammalian cortical tissue can contain between fifty and one-hundred thousand neurons and display considerable homogeneity. Mammalian cortical tissue (or grey matter") also displays several distinct firing patterns which are widely and regularly observed in several species. One such state is the "input-free" state of low-rate, stochastic firing. A key objective over the past two decades of modelling spiking-neuron networks has been to replicate this background activity state using "biologically plausible" parameters. Several models have produced dynamically and statistically reasonable activity (to varying degrees) but almost all of these have relied on some driving component in the network, such as endogenous cells (i.e. cells which spontaneously fire) or wide-spread, randomised external input (put down to background noise from other brain regions). Perhaps it would be preferable to have a model where the system itself is capable of maintaining such a background state? This a functionally important question as it may help us understand how neural activity is generated internally and how memory works. There has also been some contention as to whether driven" models produce statistically realistic results. Recent numerical results show that there are connectivity regimes in which Self-Sustained, Asynchronous, Irregular (SSAI) firing activity can be achieved. In this thesis, I discuss the history and analysis of the key spiking-network models proposed in the progression toward addressing this problem. I also discuss the underlying constructions and mathematical theory from measure theory and the theory of Markov processes which are used in the analysis of these models. I then present a small adjustment to a well known model and provide some original work in analysing the resultant dynamics. I compare this analysis to data generated by simulations. I also discuss how this analysis can be improved and what the broader future is for this line of research.
23
Yang, Jidong. "Road crack condition performance modeling using recurrent Markov chains and artificial neural networks." [Tampa, Fla.] : University of South Florida, 2004. http://purl.fcla.edu/fcla/etd/SFE0000567.
Full text
24
Weller, Tobias [Verfasser], and Y. [Akademischer Betreuer] Sure-Vetter. "Learning Latent Features using Stochastic Neural Networks on Graph Structured Data / Tobias Weller ; Betreuer: Y. Sure-Vetter." Karlsruhe : KIT-Bibliothek, 2021. http://d-nb.info/1230475656/34.
Full text
25
Larsson, Sofia. "A Study of the Loss Landscape and Metastability in Graph Convolutional Neural Networks." Thesis, KTH, Matematisk statistik, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-273622.
Full textAbstract:
Many novel graph neural network models have reported an impressive performance on benchmark dataset, but the theory behind these networks is still being developed. In this thesis, we study the trajectory of Gradient descent (GD) and Stochastic gradient descent (SGD) in the loss landscape of Graph neural networks by replicating Xing et al. [1] study for feed-forward networks. Furthermore, we empirically examine if the training process could be accelerated by an optimization algorithm inspired from Stochastic gradient Langevin dynamics and what effect the topology of the graph has on the convergence of GD by perturbing its structure. We find that the loss landscape is relatively flat and that SGD does not encounter any significant obstacles during its propagation. The noise-induced gradient appears to aid SGD in finding a stationary point with desirable generalisation capabilities when the learning rate is poorly optimized. Additionally, we observe that the topological structure of the graph plays a part in the convergence of GD but further research is required to understand how.<br>Många nya grafneurala nätverk har visat imponerande resultat på existerande dataset, dock är teorin bakom dessa nätverk fortfarande under utveckling. I denna uppsats studerar vi banor av gradientmetoden (GD) och den stokastiska gradientmetoden (SGD) i lösningslandskapet till grafiska faltningsnätverk genom att replikera studien av feed-forward nätverk av Xing et al. [1]. Dessutom undersöker vi empiriskt om träningsprocessen kan accelereras genom en optimeringsalgoritm inspirerad av Stokastisk gradient Langevin dynamik, samt om grafens topologi har en inverkan på konvergensen av GD genom att ändra strukturen. Vi ser att lösningslandskapet är relativt plant och att bruset inducerat i gradienten verkar hjälpa SGD att finna stabila stationära punkter med önskvärda generaliseringsegenskaper när inlärningsparametern har blivit olämpligt optimerad. Dessutom observerar vi att den topologiska grafstrukturen påverkar konvergensen av GD, men det behövs mer forskning för att förstå hur.
26
Kromer, Justus Alfred. "Noise in adaptive excitable systems and small neural networks." Doctoral thesis, Humboldt-Universität zu Berlin, Mathematisch-Naturwissenschaftliche Fakultät, 2017. http://dx.doi.org/10.18452/17683.
Full textAbstract:
Neuronen sind erregbare Systeme. Ihre Antwort auf Anregungen oberhalb eines bestimmten Schwellwertes sind Pulse. Häufig wird die Pulserzeugung von verschiedenen Rückkopplungsmechanismen beeinflusst, die auf langsamen Zeitskalen agieren. Das kann zu Phänomenen wie Feuerraten-Adaptation, umgekehrter Feuerraten-Adaptation oder zum Feuern von Pulsen in Salven führen. Weiterhin sind Neuronen verschiedenen Rauschquellen ausgesetzt und wechselwirken mit anderen Neuronen, in neuronalen Netzen. Doch wie beeinflusst das Zusammenspiel von Rückkopplungsmechanismen, Rauschen und der Wechselwirkung mit anderen Neuronen die Pulserzeugung? Diese Arbeit untersucht, wie die Pulserzeugung in rauschgetriebenen erregbaren Systemen von langsamen Rückkopplungsmechanismen und der Wechselwirkung mit anderen erregbaren Systemen beeinflusst wird. Dabei wird die Pulserzeugung in drei Szenarien betrachtet: (i) in einem einzelnen erregbaren System, das um einen langsamen Rückkopplungsmechanismus erweitert wurde, (ii) in gekoppelten erregbaren Systemen und (iii) in stark gekoppelten salvenfeuernden Neuronen. In jedem dieser Szenarien wird die Pulsstatistik mit Hilfe von analytischen Methoden und Computersimulationen untersucht. Das wichtigste Resultat im ersten Szenario ist, dass das Zusammenspiel von einer stark anregenden Rückkopplung und Rauschen zu rauschkontrollierter Bistabilität führt. Das erlaubt es dem System zwischen verschiedenen Modi der Pulserzeugung zu wechseln. In (ii) wird die Pulserzeugung stark von der Wahl der Kopplungsstärken und der Anzahl der Verbindungen beeinflusst. Analytische Näherungen werden abgeleitet, die einen Zusammenhang zwischen der Anzahl der Verbindungen und der Pulsrate, sowie der Pulszugvariabilität herstellen. In (iii) wird festgestellt, dass eine hemmende Rückkopplung zu sehr unregelmäßigem Verhalten der isolierten Neuronen führt, wohingegen eine starke Kopplung mit dem Netzwerk ein regelmäßigeres Feuern von Salven hervorruft.<br>Neurons are excitable systems. Their responses to excitations above a certain threshold are spikes. Usually, spike generation is shaped by several feedback mechanisms that can act on slow time scales. These can lead to phenomena such as spike-frequency adaptation, reverse spike-frequency adaptation, or bursting. In addition to these, neurons are subject to several sources of noise and interact with other neurons, in the connected complexity of a neural network. Yet how does the interplay of feedback mechanisms, noise as well as interaction with other neurons affect spike generation? This thesis examines how spike generation in noise-driven excitable systems is influenced by slow feedback processes and coupling to other excitable systems. To this end, spike generation in three setups is considered: (i) in a single excitable system, which is complemented by a slow feedback mechanism, (ii) in a set of coupled excitable systems, and (iii) in a set of strongly-coupled bursting neurons. In each of these setups, the statistics of spiking is investigated by a combination of analytical methods and computer simulations. The main result of the first setup is that the interplay of strong positive (excitatory) feedback and noise leads to noise-controlled bistability. It enables excitable systems to switch between different modes of spike generation. In (ii), spike generation is strongly affected by the choice of the coupling strengths and the number of connections. Analytical approximations are derived that relate the number of connections to the firing rate and the spike train variability. In (iii), it is found that negative (inhibitory) feedback causes very irregular behavior of the isolated bursters, while strong coupling to the network regularizes the bursting.
27
Koroko, Abdoulaye. "Natural gradient-based optimization methods for deep neural networks." Electronic Thesis or Diss., université Paris-Saclay, 2023. http://www.theses.fr/2023UPASG068.
Full textAbstract:
La méthode du gradient stochastique est la technologie actuellement prédominante pour effectuer la phase d'entraînement des réseaux de neurones. Par rapport à une descente classique, le calcul du vrai gradient comme une moyenne sur les données est remplacé par un élément aléatoire de la somme. En présence de données massives, cette approximation audacieuse permet de diminuer le nombre d'évaluations de gradients élémentaires et d'alléger le coût de chaque itération. Le prix à payer est l'apparition d'oscillations et la lenteur de convergence souvent excessive en nombre d'itérations. L'objectif de cette thèse est de concevoir une approche à la fois : (i) plus robuste, en faisant appel aux méthodes fondamentales qui ont fait leur preuve en optimisation classique, i.e., en dehors du cadre de l'apprentissage ; et (ii) plus rapide, en termes de vitesse convergence. Nous nous intéressons en particulier aux méthodes de second ordre, connues pour leur stabilité et leur rapidité de convergence. Pour éviter le goulot d'étranglement de ces méthodes, qui est le coût exorbitant d'une itération où intervient un système linéaire à matrice pleine, nous tentons d'améliorer une approximation récemment introduite sous le nom de Kronecker-Factorized Approximation of Curvature (KFAC) pour la matrice de Fisher, laquelle remplace la matrice hessienne dans ce contexte. Plus précisément, nos axes de travail sont : (i) construire de nouvelles factorisations de Kronecker fondées sur une justification mathématique plus rigoureuse que KFAC ; (ii) prendre en compte l'information issue des blocs hors diagonaux de la matrice de Fisher, qui représentent l'interaction entre les différentes couches ; (iii) généraliser KFAC à une architecture de réseau autre que celles pour lesquelles elle a été initialement développée<br>The stochastic gradient method is currently the prevailing technology for training neural networks. Compared to a classical descent, the calculation of the true gradient as an average over the data is replaced by a random element of the sum. When dealing with massive data, this bold approximation enables one to decrease the number of elementary gradient evaluations and to alleviate the cost of each iteration. The price to be paid is the appearance of oscillations and the slowness of convergence, which is often excessive in terms of number of iterations. The aim of this thesis is to design an approach that is both: (i) more robust, using the fundamental methods that have been successfully proven in classical optimization, i.e., outside the learning framework; and (ii) faster in terms of convergence speed. We are especially interested in second-order methods, known for their stability and speed of convergence. To circumvent the bottleneck of these methods, which lies in the prohibitive cost of an iteration involving a linear system with a full matrix, we attempt to improve an approximation recently introduced as Kronecker-Factorized Approximation of Curvature (KFAC) for the Fisher matrix, which replaces the Hessian matrix in this context. More specifically, our work focuses on: (i) building new Kronecker factorizations based on a more rigorous mathematical justification than in KFAC; (ii) taking into account the information from the off-diagonal blocks of the Fisher matrix, which represent the interaction between the different layers; (iii) generalizing KFAC to a network architecture other than those for which it had been initially developed
28
Alisar, Ibrahim. "Stochastic Modelling Of Wind Energy Generation." Master's thesis, METU, 2012. http://etd.lib.metu.edu.tr/upload/12614930/index.pdf.
Full textAbstract:
In this thesis work, electricty generation modeling of the wind energy -one type of the
renewable energy sources- is studied. The wind energy characteristics and the distribution of
wind speed in a specific region is also examined. In addition, the power curves of the wind
turbines are introduced and the relationship between the wind speed and wind power is
explained. The generation characteristics of the wind turbines from various types of
producers are also investigated. In this study, the main wind power forecasting methods are
presented and the advantages and disadvantages of the methods are analyzed. The physical
approaches, statistical methods and the Artificial Neural Network (ANN) methods are
introduced. The parameters that affect the capacity factor, the total energy generation and the
payback period are examined. In addition, the wind turbine models and their effect on the
total energy generation with different wind data from various sites are explained. As a part
of this study, a MATLAB-based software about wind speed and energy modelling and
payback period calculation has been developed. In order to simplify the calculation process,
a Graphical User Interface (GUI) has been designed. In addition, a simple wind energy
persistence model for wind power plant operator in the intra-day market has been developed.
29
Mignacco, Francesca. "Statistical physics insights on the dynamics and generalisation of artificial neural networks." Thesis, université Paris-Saclay, 2022. http://www.theses.fr/2022UPASP074.
Full textAbstract:
L'apprentissage machine est une technologie désormais omniprésente dans notre quotidien. Toutefois, ce domaine reste encore largement empirique et ses enjeux scientifiques manquent d'une compréhension théorique profonde. Cette thèse se penche vers la découverte des mécanismes sous-tendant l'apprentissage dans les réseaux de neurones artificiels à travers le prisme de la physique statistique. Dans une première partie, nous nous intéressons aux propriétés statiques des problèmes d'apprentissage, que nous introduisons au chapitre 1.1. Dans le chapitre 1.2, nous considérons la classification d'un mélange binaire de nuages gaussiens et nous dérivons des expressions rigoureuses pour les erreurs en dimension infinie, que nous appliquons pour éclairer le rôle des différents paramètres du problème. Dans le chapitre 1.3, nous montrons comment étendre le modèle de perceptron enseignant-étudiant pour considérer la classification multi-classes, en dérivant des expressions asymptotiques pour la performance optimale et la performance de la minimisation du risque empirique régularisé. Dans la deuxième partie, nous nous concentrons sur la dynamique de l'apprentissage, que nous introduisons dans le chapitre 2.1. Dans le chapitre 2.2, nous montrons comment décrire analytiquement la dynamique de l'algorithme du gradient stochastique à échantillonage mini-lots (mini-batch SGD) dans la classification binaire de mélanges gaussiens, en utilisant la théorie dynamique du champ moyen. Le chapitre 2.3 présente une analyse du bruit effectif introduit par SGD. Dans le chapitre 2.4, nous considérons le problème de la récupération des signes comme exemple d'optimisation hautement non convexe et montrons que la stochasticité est cruciale pour la généralisation. La conclusion de la thèse est présentée dans la troisième partie<br>Machine learning technologies have become ubiquitous in our daily lives. However, this field still remains largely empirical and its scientific stakes lack a deep theoretical understanding.This thesis explores the mechanisms underlying learning in artificial neural networks through the prism of statistical physics. In the first part, we focus on the static properties of learning problems, that we introduce in Chapter 1.1. In Chapter 1.2, we consider the prototype classification of a binary mixture of Gaussian clusters and we derive rigorous closed-form expressions for the errors in the infinite-dimensional regime, that we apply to shed light on the role of different problem parameters. In Chapter 1.3, we show how to extend the teacher-student perceptron model to encompass multi-class classification deriving asymptotic expressions for the optimal performance and the performance of regularised empirical risk minimisation. In the second part, we turn our focus to the dynamics of learning, that we introduce in Chapter 2.1. In Chapter 2.2, we show how to track analytically the training dynamics of multi-pass stochastic gradient descent (SGD) via dynamical mean-field theory for generic non convex loss functions and Gaussian mixture data. Chapter 2.3 presents a late-time analysis of the effective noise introduced by SGD in the underparametrised and overparametrised regimes. In Chapter 2.4, we take the sign retrieval problem as a benchmark highly non-convex optimisation problem and show that stochasticity is crucial to achieve perfect generalisation. The third part of the thesis contains the conclusions and some future perspectives
30
Russo, Elena Tea. "Fluctuation properties in random walks on networks and simple integrate and fire models." Master's thesis, Alma Mater Studiorum - Università di Bologna, 2015. http://amslaurea.unibo.it/9565/.
Full textAbstract:
In questa tesi si è studiato l’insorgere di eventi critici in un semplice modello neurale del tipo Integrate and Fire, basato su processi dinamici stocastici markoviani definiti su una rete. Il segnale neurale elettrico è stato modellato da un flusso di particelle. Si è concentrata l’attenzione sulla fase transiente del sistema, cercando di identificare fenomeni simili alla sincronizzazione neurale, la quale può essere considerata un evento critico. Sono state studiate reti particolarmente semplici, trovando che il modello proposto ha la capacità di produrre effetti "a cascata" nell’attività neurale, dovuti a Self Organized Criticality (auto organizzazione del sistema in stati instabili); questi effetti non vengono invece osservati in Random Walks sulle stesse reti. Si è visto che un piccolo stimolo random è capace di generare nell’attività della rete delle fluttuazioni notevoli, in particolar modo se il sistema si trova in una fase al limite dell’equilibrio. I picchi di attività così rilevati sono stati interpretati come valanghe di segnale neurale, fenomeno riconducibile alla sincronizzazione.
31
Boumaaza, Bouharket. "3D seismic attributes analysis and inversions for prospect evaluation and characterization of Cherokee sandstone reservoir in the Wierman field, Ness County, Kansas." Thesis, Kansas State University, 2017. http://hdl.handle.net/2097/35510.
Full textAbstract:
Master of Science<br>Department of Geology<br>Abdelmoneam Raef<br>Matthew W. Totten<br>This work focuses on the use of advanced seismically driven technologies to estimate the distribution of key reservoir properties which mainly includes porosity and hydrocarbon reservoir pay. These reservoir properties were estimated by using a multitude of seismic attributes derived from post-stack high resolution inversions, spectral imaging and volumetric curvature.
A pay model of the reservoir in the Wierman field in Ness County, Kansas is proposed. The proposed geological model is validated based on comparison with findings of one blind well. The model will be useful in determining future drilling prospects, which should improve the drilling success over previous efforts, which resulted in only few of the 14 wells in the area being productive. The rock properties that were modeled were porosity and Gamma ray. Water saturation and permeability were considered, but the data needed were not available.
Sequential geological modeling approach uses multiple seismic attributes as a building block to estimate in a sequential manner dependent petrophysical properties such as gamma ray, and porosity. The sequential modelling first determines the reservoir property that has the ability to be the primary property controlling most of the other subsequent reservoir properties. In this study, the gamma ray was chosen as the primary reservoir property. Hence, the first geologic model built using neural networks was a volume of gamma ray constrained by all the available seismic attributes.
The geological modeling included post-stack seismic data and the five wells with available well logs. The post-stack seismic data was enhanced by spectral whitening to gain as much resolution as possible. Volumetric curvature was then calculated to determine where major faults were located. Several inversions for acoustic impedance were then applied to the post-stack seismic data to gain as much information as possible about the acoustic impedance. Spectral attributes were also extracted from the post-stack seismic data.
After the most appropriate gamma ray and porosity models were chosen, pay zone maps were constructed, which were based on the overlap of a certain range of gamma ray values with a certain range of porosity values. These pay zone maps coupled with the porosity and gamma ray models explain the performance of previously drilled wells.
32
Ngatchou, Patrick. "Intelligent techniques for optimization and estimation /." Thesis, Connect to this title online; UW restricted, 2006. http://hdl.handle.net/1773/5827.
Full text
33
Kozel, Tomáš. "Stochastické řízení zásobní funkce nádrže s pomocí metod umělé inteligence." Doctoral thesis, Vysoké učení technické v Brně. Fakulta stavební, 2018. http://www.nusl.cz/ntk/nusl-390282.
Full textAbstract:
The main advantage of stochastic forecasting is fan of possible value, which deterministic method of forecasting could not give us. Future development of random process is described better by stochastic then deterministic forecasting. We can categorize discharge in measurement profile as random process. Stochastic management is worked with dispersion of controlling discharge value. In thesis is described construction and evaluation of adaptive stochastic model base on fuzzy logic, neural networks and evolution algorithm, which are used stochastic forecast from forecasting models described in thesis. The learning fuzzy model and neural network is used as replacement of classic optimization algorithm (evolution algorithm). Model was tested and validated on made up large open water reservoir. Results were evaluated and were compared with model base on traditional algorithms, which was used for 100% forecast (forecasted values are real values). The management of the large open water reservoir with storage function, which was given by stochastic adaptive managing, was logical. The main advantage of fuzzy model and neural network model is computing speed. Classical optimization model is needed much more time for same calculation as fuzzy and neural network model, therefore classic model used clusters for stochastic calculation.
34
Simonetti, Roberta. "Generalização e Robustez: Aprendizagem em Redes Neurais na Presença de Ruído." Universidade de São Paulo, 1997. http://www.teses.usp.br/teses/disponiveis/43/43133/tde-17122013-145626/.
Full textAbstract:
Neste trabalho investigamos o aprendizado supervisionado on-line, com ênfase nas habilidades de generalização, de redes neurais feedforward. O estudo de algoritmos de aprendizagem ótimos, no sentido da generalização, é estendido para duas diferentes classes de arquiteturas: a máquina paridade com estrutura de árvore e K unidades escondidas, e o perceptron reversed wedge, uma máquina de uma camada com função de transferência não monotônica. O papel do ruído é de fundamental importância na teoria de aprendizagem. Neste trabalho estudamos os processos com ruído que podem ser parametrizados por uma única quantidade, o nível de ruído. No caso da máquina paridade analisamos o aprendizado na presença de ruído multiplicativo (na saída). O algoritmo ótimo é muito superior aos algoritmos de aprendizagem até então apresentados, como o algoritmo de mínima ação (LAA), como podemos ver, por exemplo, através do comportamento do erro de generalização que decai após a apresentação de p exemplos, com l/p ao invés de l/\'p POT. 1/3\' como no caso do LAA. Além deste fato, observa-se que não existe um nível de ruído crítico a partir do qual a rede não é capaz de generalizar, como ocorre no LAA. Além do ruído multiplicativo, no caso do perceptron reversed wedge consideramos também o ruído aditivo. Analisamos a função de modulação fornecida pelo algoritmo ótimo e as curvas de aprendizagem. A aprendizagem ótima requer o uso de parâmetros que usualmente não estão disponíveis. Neste caso estudamos a influência da utilização de uma estimativa do nível de ruído sobre as curvas de aprendizado. Estes resultados são apresentados na forma do que chamamos de diagrama de robustez, no espaço de nível de ruído real versus nível de ruído estimado. As linhas de transição deste diagrama definem regiões com comportamentos dinâmicos diferentes. Entre as propriedades mais interessantes encontradas, destacamos a universalidade do diagrama de robustez para ruído multiplicativo, uma vez que é exatamente o mesmo para a máquina paridade e comitê com estrutura de árvore, e para o perceptron reversed-wedge. Entretanto, esta universalidade não se estende para o caso de ruído aditivo, uma vez que, neste caso, os diagramas dependem da arquitetura em questão.<br>In this work online supervised learning is investigated with emphasis on the generalization abilities of feedforward neural networks. The study of optimal learning algorithms, in the sense of generalization, is extended to two different classes of architectures; the tree parity machine (PM) with K hidden units and the reverse wedge perceptron (RWP), a single layer machine with a non monotonic transfer function. The role of noise is of fundamental importance in learning theory, and we study noise processes which can be parametrized by a single quantity, the noise level. For the PM we analize learning in the presence of multiplicative or output noise. The optimal algorithm is far superior than previous learning algorithms, such as the Least Action Algorithm (LAA), since for example, the generalization error\'s decay is proportional to l /p instead of l/\'p POT. 1/3\' for the LAA, after p examples have been used for training. Furthermore there is no critical noise level, beyond which no generalization ability is attainable, as is the case for the LAA. For the RW perceptron in addition to multiplicative noise we also consider additive noise. The optimal algorithm modulation function and the learning curves are analized. Optimal learning requires using certain usually unavailable parameters. In this case, we study the influence that misevaluation of the noise levels has on the learning curves. The results are presented in terms of what we have called Robustness Phase Diagrams (RPD), in a space of real noise level against assumed noise level. The RPD boundary lines separate between different dynamical behaviours. Among the most interesting properties, we have found the universality of the RPD for multiplicative noise, since it is exactly the same for the PM, RWP and the tree committee machine. However this universality does not hold for the additive noise case, since RPD\'s are shown to be architecture dependent.
35
Bono, Guillaume. "Deep multi-agent reinforcement learning for dynamic and stochastic vehicle routing problems." Thesis, Lyon, 2020. http://www.theses.fr/2020LYSEI096.
Full textAbstract:
La planification de tournées de véhicules dans des environnements urbains denses est un problème difficile qui nécessite des solutions robustes et flexibles. Les approches existantes pour résoudre ces problèmes de planification de tournées dynamiques et stochastiques (DS-VRPs) sont souvent basés sur les mêmes heuristiques utilisées dans le cas statique et déterministe, en figeant le problème à chaque fois que la situation évolue. Au lieu de cela, nous proposons dans cette thèse d’étudier l’application de méthodes d’apprentissage par renforcement multi-agent (MARL) aux DS-VRPs en s’appuyant sur des réseaux de neurones profonds (DNNs). Plus précisément, nous avons d’abord contribuer à étendre les méthodes basées sur le gradient de la politique (PG) aux cadres des processus de décision de Markov (MDPs) partiellement observables et décentralisés (Dec-POMDPs). Nous avons ensuite proposé un nouveau modèle de décision séquentiel en relâchant la contrainte d’observabilité partielle que nous avons baptisé MDP multi-agent séquentiel (sMMDP). Ce modèle permet de décrire plus naturellement les DS-VRPs, dans lesquels les véhicules prennent la décision de servir leurs prochains clients à l’issu de leurs précédents services, sans avoir à attendre les autres. Pour représenter nos solutions, des politiques stochastiques fournissant aux véhicules des règles de décisions, nous avons développé une architecture de DNN basée sur des mécanismes d’attention (MARDAM). Nous avons évalué MARDAM sur un ensemble de bancs de test artificiels qui nous ont permis de valider la qualité des solutions obtenues, la robustesse et la flexibilité de notre approche dans un contexte dynamique et stochastique, ainsi que sa capacité à généraliser à toute une classe de problèmes sans avoir à être ré-entraînée. Nous avons également développé un banc de test plus réaliste à base d’une simulation micro-traffic, et présenté une preuve de concept de l’applicabilité de MARDAM face à une variété de situations différentes<br>Routing delivery vehicles in dynamic and uncertain environments like dense city centers is a challenging task, which requires robustness and flexibility. Such logistic problems are usually formalized as Dynamic and Stochastic Vehicle Routing Problems (DS-VRPs) with a variety of additional operational constraints, such as Capacitated vehicles or Time Windows (DS-CVRPTWs). Main heuristic approaches to dynamic and stochastic problems simply consist in restarting the optimization process on a frozen (static and deterministic) version of the problem given the new information. Instead, Reinforcement Learning (RL) offers models such as Markov Decision Processes (MDPs) which naturally describe the evolution of stochastic and dynamic systems. Their application to more complex problems has been facilitated by recent progresses in Deep Neural Networks, which can learn to represent a large class of functions in high dimensional spaces to approximate solutions with high performances. Finding a compact and sufficiently expressive state representation is the key challenge in applying RL to VRPs. Recent work exploring this novel approach demonstrated the capabilities of Attention Mechanisms to represent sets of customers and learn policies generalizing to different configurations of customers. However, all existing work using DNNs reframe the VRP as a single-vehicle problem and cannot provide online decision rules for a fleet of vehicles.In this thesis, we study how to apply Deep RL methods to rich DS-VRPs as multi-agent systems. We first explore the class of policy-based approaches in Multi-Agent RL and Actor-Critic methods for Decentralized, Partially Observable MDPs in the Centralized Training for Decentralized Control (CTDC) paradigm. To address DS-VRPs, we then introduce a new sequential multi-agent model we call sMMDP. This fully observable model is designed to capture the fact that consequences of decisions can be predicted in isolation. Afterwards, we use it to model a rich DS-VRP and propose a new modular policy network to represent the state of the customers and the vehicles in this new model, called MARDAM. It provides online decision rules adapted to the information contained in the state and takes advantage of the structural properties of the model. Finally, we develop a set of artificial benchmarks to evaluate the flexibility, the robustness and the generalization capabilities of MARDAM. We report promising results in the dynamic and stochastic case, which demonstrate the capacity of MARDAM to address varying scenarios with no re-optimization, adapting to new customers and unexpected delays caused by stochastic travel times. We also implement an additional benchmark based on micro-traffic simulation to better capture the dynamics of a real city and its road infrastructures. We report preliminary results as a proof of concept that MARDAM can learn to represent different scenarios, handle varying traffic conditions, and customers configurations
36
Malmgren, Henrik. "Revision of an artificial neural network enabling industrial sorting." Thesis, Uppsala universitet, Institutionen för teknikvetenskaper, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-392690.
Full textAbstract:
Convolutional artificial neural networks can be applied for image-based object classification to inform automated actions, such as handling of objects on a production line. The present thesis describes theoretical background for creating a classifier and explores the effects of introducing a set of relatively recent techniques to an existing ensemble of classifiers in use for an industrial sorting system.The findings indicate that it's important to use spatial variety dropout regularization for high resolution image inputs, and use an optimizer configuration with good convergence properties. The findings also demonstrate examples of ensemble classifiers being effectively consolidated into unified models using the distillation technique. An analogue arrangement with optimization against multiple output targets, incorporating additional information, showed accuracy gains comparable to ensembling. For use of the classifier on test data with statistics different than those of the dataset, results indicate that augmentation of the input data during classifier creation helps performance, but would, in the current case, likely need to be guided by information about the distribution shift to have sufficiently positive impact to enable a practical application. I suggest, for future development, updated architectures, automated hyperparameter search and leveraging the bountiful unlabeled data potentially available from production lines.
37
Dickson, Scott M. "Stochastic neural network dynamics : synchronisation and control." Thesis, Loughborough University, 2014. https://dspace.lboro.ac.uk/2134/16508.
Full textAbstract:
Biological brains exhibit many interesting and complex behaviours. Understanding of the mechanisms behind brain behaviours is critical for continuing advancement in fields of research such as artificial intelligence and medicine. In particular, synchronisation of neuronal firing is associated with both improvements to and degeneration of the brain's performance; increased synchronisation can lead to enhanced information-processing or neurological disorders such as epilepsy and Parkinson's disease. As a result, it is desirable to research under which conditions synchronisation arises in neural networks and the possibility of controlling its prevalence. Stochastic ensembles of FitzHugh-Nagumo elements are used to model neural networks for numerical simulations and bifurcation analysis. The FitzHugh-Nagumo model is employed because of its realistic representation of the flow of sodium and potassium ions in addition to its advantageous property of allowing phase plane dynamics to be observed. Network characteristics such as connectivity, configuration and size are explored to determine their influences on global synchronisation generation in their respective systems. Oscillations in the mean-field are used to detect the presence of synchronisation over a range of coupling strength values. To ensure simulation efficiency, coupling strengths between neurons that are identical and fixed with time are investigated initially. Such networks where the interaction strengths are fixed are referred to as homogeneously coupled. The capacity of controlling and altering behaviours produced by homogeneously coupled networks is assessed through the application of weak and strong delayed feedback independently with various time delays. To imitate learning, the coupling strengths later deviate from one another and evolve with time in networks that are referred to as heterogeneously coupled. The intensity of coupling strength fluctuations and the rate at which coupling strengths converge to a desired mean value are studied to determine their impact upon synchronisation performance. The stochastic delay differential equations governing the numerically simulated networks are then converted into a finite set of deterministic cumulant equations by virtue of the Gaussian approximation method. Cumulant equations for maximal and sub-maximal connectivity are used to generate two-parameter bifurcation diagrams on the noise intensity and coupling strength plane, which provides qualitative agreement with numerical simulations. Analysis of artificial brain networks, in respect to biological brain networks, are discussed in light of recent research in sleep theory.
38
Botha, Marthinus Ignatius. "Modelling and simulation framework incorporating redundancy and failure probabilities for evaluation of a modular automated main distribution frame." Diss., University of Pretoria, 2013. http://hdl.handle.net/2263/33345.
Full textAbstract:
Maintaining and operating manual main distribution frames is labour-intensive. As a result, Automated
Main Distribution Frames (AMDFs) have been developed to alleviate the task of maintaining
subscriber loops. Commercial AMDFs are currently employed in telephone exchanges in some parts
of the world. However, the most significant factors limiting their widespread adoption are costeffective
scalability and reliability. Therefore, an impelling incentive is provided to create a simulation
framework in order to explore typical implementations and scenarios. Such a framework will
allow the evaluation and optimisation of a design in terms of both internal and external redundancies.
One of the approaches to improve system performance, such as system reliability, is to allocate the
optimal redundancy to all or some components in a system. Redundancy at the system or component
levels can be implemented in one of two schemes: parallel redundancy or standby redundancy. It is
also possible to mix these schemes for various components. Moreover, the redundant elements may
or may not be of the same type. If all the redundant elements are of different types, the redundancy
optimisation model is implemented with component mixing. Conversely, if all the redundant components are identical, the model is implemented without component mixing.
The developed framework can be used both to develop new AMDF architectures and to evaluate
existing AMDF architectures in terms of expected lifetimes, reliability and service availability. Two
simulation models are presented. The first simulation model is concerned with optimising central
office equipment within a telephone exchange and entails an environment of clients utilising services.
Currently, such a model does not exist. The second model is a mathematical model incorporating
stochastic simulation and a hybrid intelligent evolutionary algorithm to solve redundancy allocation
problems.
For the first model, the optimal partitioning of the model is determined to speed up the simulation
run efficiently. For the second model, the hybrid intelligent algorithm is used to solve the redundancy
allocation problem under various constraints. Finally, a candidate concept design of an AMDF is
presented and evaluated with both simulation models.<br>Dissertation (MEng)--University of Pretoria, 2013.<br>gm2014<br>Electrical, Electronic and Computer Engineering<br>unrestricted
39
Hu, Kaitong. "Jeux différentiels stochastiques non-Markoviens etdynamiques de Langevin à champ-moyen." Thesis, Institut polytechnique de Paris, 2020. http://www.theses.fr/2020IPPAX005.
Full textAbstract:
Cette thèse se compose de deux parties indépendantes et la première regroupant deux problématiques distinctes. Dans la première partie, nous étudions d’abord le problème de Principal-Agent dans des systèmes dégénérés, qui apparaissent naturellement dans des environnements à l’observation partielle où l’Agent et le Principal n’observent qu’une partie du système. Nous présentons une approche se basant sur le principe du maximum stochastique, dont le but est d’étendre les travaux existants qui utilisent le principe de la programmation dynamique dans des systèmes non-dégénérés. D’abord nous résolvons le problème du Principal dans un ensembledes contrats élargi donné par la condition du premier ordre du problème de l’Agent sous forme d’une équation différentielle stochastique progressive-rétrograde (abrégée EDSPR) dépendante de la trajectoire. Ensuite nous utilisons la condition suffisante du problème de l’Agent pour vérifier que le contrat optimal obtenu est bien implémentable. Une étude parallèle est consacrée à l’existence et l’unicité de la solution d'EDSPRs dépendantes de la trajectoire dans le chapitre IV. Nous étendons la méthode de champ de découplage aux cas où les coefficients des équations peuvent dépendre de la trajectoire du processus forward. Nous démontrons également une propriété de stabilité pour ce genre d'EDSPRs. Enfin, nous étudions le problème de hasard moral avec plusieurs Principals. L’Agent ne peut travailler que pour un seul Principal à la fois et fait donc face à un problème de switching optimal. En utilisant la méthode de randomisation nous montrons que la fonction valeur de l’Agent et son effort optimal sont donnés par un processus d’Itô. Cette représentation nous aide à résoudre ensuite le problème du Principal lorsqu’il y a une infinité de Principals en équilibre selon un jeu à champ-moyen. Nous justifions la formulation à champ-moyen par un argument de propagation de chaos.La deuxième partie de cette thèse est constituée des chapitres V et VI. La motivation de ces travaux est de donner un fondement théorique rigoureux pour la convergence des algorithmes du type descente de gradient très souvent utilisés dans la résolution des problème non-convexes comme la calibration d’un réseau de neurones. Pour les problèmes non-convexes du type réseaux de neurones à une couche cachée, l’idée clé est de transformer le problème en un problème convexe en le relevant dans l’espace des mesures. Nous montrons que la fonction d’énergie correspondante admet un unique minimiseur qui peut être caractérisé par une condition du premier ordre utilisant la dérivation dans l’espace des mesures au sens de Lions. Nous présentons ensuite une analyse du comportement à long terme de la dynamique de Langevin à champ-moyen, qui possède une structure de flot de gradient dans la métrique de 2-Wasserstein. Nous montrons que le flot de la loi marginale induite par la dynamique de Langevin à champ-moyen converge vers une loi stationnaire en utilisant le principe d’invariance de La Salle, qui est le minimiseur de la fonction d’énergie.Dans le cas des réseaux de neurones profonds, nous les modélisons à l’aide d’un problème de contrôle optimal en temps continu. Nous donnons d’abord la conditiondu premier ordre à l’aide du principe de Pontryagin, qui nous aidera ensuiteà introduire le système d’équation de Langevin à champ-moyen, dont la mesure invariante correspond au minimiseur du problème de contrôle optimal. Enfin, avec la méthode de couplage par réflexion nous montrons que la loi marginale du système de Langevin à champ-moyen converge vers la mesure invariante avec une vitesse exponentielle<br>Two independent subjects are studied in this thesis, the first of which consists of two distinct problems.In the first part, we begin with the Principal-Agent problem in degenerate systems, which appear naturally in partially observed random environment in which the Agent and the Principal can only observe one part of the system. Our approach is based on the stochastic maximum principle, the goal of which is to extend the existing results using dynamic programming principle to the degenerate case. We first solve the Principal's problem in an enlarged set of contracts given by the first order condition of the Agent's problem in form of a path-dependent forward-backward stochastic differential equation (abbreviated FBSDE). Afterward, we use the sufficient condition of the Agent's problem to verify that the previously obtained optimal contract is indeed implementable. Meanwhile, a parallel study is devoted to the wellposedness of path-dependent FBSDEs in the chapter IV. We generalize the decoupling field method to the case where the coefficients of the equations can depend on the whole path of the forward process and show the stability property of this type of FBSDEs. Finally, we study the Principal-Agent problem with multiple Principals. The Agent can only work for one Principal at a time and therefore needs to solve an optimal switching problem. By using randomization, we show that the value function of the Agent's problem and his optimal control are given by an Itô process. This representation allows us to solve the Principal's problem in the mean-field case when there is an infinite number of Principals. We justify the mean-field formulation using an argument of backward propagation of chaos.The second part of the thesis consists of chapter V and VI. The motivation of this work is to give a rigorous theoretical underpinning for the convergence of gradient-descent type of algorithms frequently used in non-convex optimization problems like calibrating a deep neural network.For one-layer neural networks, the key insight is to convexify the problem by lifting it to the measure space. We show that the corresponding energy function has a unique minimiser which can be characterized by some first order condition using derivatives in measure space. We present a probabilistic analysis of the long-time behavior of the mean-field Langevin dynamics, which have a gradient flow structure in 2-Wasserstein metric. By using a generalization of LaSalle's invariance principle, we show that the flow of marginal laws induced by the mean-field Langevin dynamics converges to the stationary distribution, which is exactly the minimiser of the energy function.As for deep neural networks, we model them as some continuous-time optimal control problems. Firstly, we find the first order condition by using Pontryagin maximum principle, which later helps us find the associated mean-field Langevin system, the invariant measure of which is again the minimiser of the optimal control problem. As last, by using the reflection coupling, we show that the marginal distribution of the mean-field Langevin system converges to the unique invariant measure exponentially
40
Canals, Guinand Vicente José. "Implementación en hardware de sistemas de alta fiabilidad basados en metodologías estocásticas." Doctoral thesis, Universitat de les Illes Balears, 2012. http://hdl.handle.net/10803/84125.
Full textAbstract:
La sociedad actual demanda cada vez más aplicaciones computacionalmente exigentes y
que se implementen de forma energéticamente eficiente. Esto obliga a la industria del
semiconductor a mantener una continua progresión de la tecnología CMOS. No obstante,
los expertos vaticinan que el fin de la era de la progresión de la tecnología CMOS se
acerca, puesto que se prevé que alrededor del 2020 la tecnología CMOS llegue a su límite.
Cuando ésta llegue al punto conocido como “Red Brick Wall”, las limitaciones físicas,
tecnológicas y económicas no harán viable el proseguir por esta senda. Todo ello ha
motivado que a lo largo de la última década tanto instituciones públicas como privadas
apostasen por el desarrollo de soluciones tecnológicas alternativas como es el caso de la
nanotecnología (nanotubos, nanohilos, tecnologías basadas en el grafeno, etc.). En esta tesis
planteamos una solución alternativa para poder afrontar algunos de los problemas
computacionalmente exigentes. Esta solución hace uso de la tecnología CMOS actual
sustituyendo la forma de computación clásica desarrollada por Von Neumann por formas
de computación no convencionales. Éste es el caso de las computaciones basadas en lógicas
pulsantes y en especial la conocida como computación estocástica, la cual proporciona un
aumento de la fiabilidad y del paralelismo en los sistemas digitales.
En esta tesis se presenta el desarrollo y evaluación de todo un conjunto de bloques
computacionales estocásticos implementados mediante elementos digitales clásicos. A
partir de estos bloques se proponen diversas metodologías computacionalmente eficientes
que mediante su uso permiten afrontar algunos problemas de computación masiva de forma
mucho más eficiente. En especial se ha centrado el estudio en los problemas relacionados
con el campo del reconocimiento de patrones.<br>Today's society demands the use of applications with a high computational complexity that
must be executed in an energy-efficient way. Therefore the semiconductor industry is
forced to maintain the CMOS technology progression. However, experts predict that the
end of the age of CMOS technology progression is approaching. It is expected that at 2020
CMOS technology would reach the point known as "Red Brick Wall" at which the
physical, technological and economic limitations of CMOS technology will be unavoidable.
All of this has caused that over the last decade public and private institutions has bet by the
development of alternative technological solutions as is the case of nanotechnology
(nanotubes, nanowires, graphene, etc.). In this thesis we propose an alternative solution to
address some of the computationally exigent problems by using the current CMOS
technology but replacing the classical computing way developed by Von Neumann by other
forms of unconventional computing. This is the case of computing based on pulsed logic
and especially the stochastic computing that provide a significant increase of the
parallelism and the reliability of the systems. This thesis presents the development and
evaluation of different stochastic computing methodologies implemented by digital gates.
The different methods proposed are able to face some massive computing problems more
efficiently than classical digital electronics. This is the case of those fields related to pattern
recognition, which is the field we have focused the main part of the research work
developed in this thesis.
41
Georgieva, Antoniya. "Stochastic methods and genetic algorithms for neural network learning." Thesis, University of Portsmouth, 2008. https://researchportal.port.ac.uk/portal/en/theses/stochastic-methods-and-genetic-algorithms-for-neural-network-learning(67dae83c-ec3d-4db2-875c-6e7407a4144f).html.
Full textAbstract:
This thesis presents results from the developemnt, investigation, testing and evaluation of novel meta-heuristic techniques aiming to further improve the <i>state-of-the-art </i>of algorithms for local minima free Neural Network supervised learning. Several approaches for solving Global Optimisation problems that make use of novel meta-heuristic techniques, so called Low-discrepancy Sequences, and hybrid Evolutionary Algorithms are proposed here, investigated and critically discussed. Furthermore, the novel methods are tested on a number of multimodal mathematical function optimisation problems, as well as on a variety of Neural Network learning tasks, including real-world benchmark datasets. Comparison of the results from the investigated methods with such from standard Backpropagation, Evolutionary Algorithms, and other stochastic approaches (Simulated Annealing, Tabu Search, etc.) is conducted in order to demonstrate their competitiveness in terms of number of function evaluations, learning speed and Neural Network generalisation abilities. Finally, the investigated techniques are applied and tested on real-world problems for the intelligent recognition and classification of cork tiles. An Intelligent Computer Vision system is built. The system includes the following stages: image acquisition; image processing (feature extraction and statistical data processing); Neural Network architecture design; supervised learning utilising the proposed Global Optimisation techniques; and finally, extensive system evaluation. The presented examples and case studies demonstrate that the proposed techniques can be effectively applied for the optimisation of mathematical multimodal functions. The investigated methods are successful in local minima free Neural Network learning, and they can be used for solving real-world industrial problems.
42
Anisenia, Andrei. "Stochastic Search Genetic Algorithm Approximation of Input Signals in Native Neuronal Networks." Thèse, Université d'Ottawa / University of Ottawa, 2013. http://hdl.handle.net/10393/26220.
Full textAbstract:
The present work investigates the applicability of Genetic Algorithms (GA) to the problem of signal propagation in Native Neuronal Networks (NNNs). These networks are comprised of neurons, some of which receive input signals. The signals propagate though the network by transmission between neurons. The research focuses on the regeneration of the output signal of the network without knowing the original input signal. The computational complexity of the problem is prohibitive for the exact computation. We propose to use a heuristic approach called Genetic Algorithm. Three algorithms are developed, based on the GA technique. The developed algorithms are tested on two different networks with varying input signals. The results obtained from the testing indicate significantly better performance of the developed algorithms compared to the Uniform Random Search (URS) technique, which is used as a control group. The importance of the research is in the demonstration of the ability of GA-based algorithms to successfully solve the problem at hand.
43
Aguiar, Eliane Martins de. "Aplicação do Word2vec e do Gradiente descendente dstocástico em tradução automática." reponame:Repositório Institucional do FGV, 2016. http://hdl.handle.net/10438/16798.
Full textAbstract:
Submitted by Eliane Martins de Aguiar (elianemart@gmail.com) on 2016-08-01T21:03:09Z
No. of bitstreams: 1
dissertacao-ElianeMartins.pdf: 6062037 bytes, checksum: 14567c2feca25a81d6942be3b8bc8a65 (MD5)<br>Approved for entry into archive by Janete de Oliveira Feitosa (janete.feitosa@fgv.br) on 2016-08-03T20:29:34Z (GMT) No. of bitstreams: 1
dissertacao-ElianeMartins.pdf: 6062037 bytes, checksum: 14567c2feca25a81d6942be3b8bc8a65 (MD5)<br>Approved for entry into archive by Maria Almeida (maria.socorro@fgv.br) on 2016-08-23T20:12:35Z (GMT) No. of bitstreams: 1
dissertacao-ElianeMartins.pdf: 6062037 bytes, checksum: 14567c2feca25a81d6942be3b8bc8a65 (MD5)<br>Made available in DSpace on 2016-08-23T20:12:54Z (GMT). No. of bitstreams: 1
dissertacao-ElianeMartins.pdf: 6062037 bytes, checksum: 14567c2feca25a81d6942be3b8bc8a65 (MD5)
Previous issue date: 2016-05-30<br>O word2vec é um sistema baseado em redes neurais que processa textos e representa pa- lavras como vetores, utilizando uma representação distribuída. Uma propriedade notável são as relações semânticas encontradas nos modelos gerados. Este trabalho tem como objetivo treinar dois modelos utilizando o word2vec, um para o Português e outro para o Inglês, e utilizar o gradiente descendente estocástico para encontrar uma matriz de tradução entre esses dois espaços.
44
Martí, Ortega Daniel. "Neural stochastic dynamics of perceptual decision making." Doctoral thesis, Universitat Pompeu Fabra, 2008. http://hdl.handle.net/10803/7552.
Full textAbstract:
Models computacionals basats en xarxes a gran escala d'inspiració neurobiològica permeten descriure els correlats neurals de la decisió observats en certes àrees corticals com una transició entre atractors de la xarxa cortical. L'estimulació provoca un canvi en el paisatge d'atractors que afavoreix la transició entre l'atractor neutre inicial a un dels atractors associats a les eleccions categòriques. El soroll present en el sistema introdueix indeterminació en les transicions. En aquest treball mostrem l'existència de dos mecanismes de decisió qualitativament diferents, cadascun amb signatures psicofísiques diferenciades. El mecanisme que apareix a baixes intensitats, induït exclusivament pel soroll, dóna lloc a temps de decisió distribuïts asimètricament, amb una mitjana dictada per l'amplitud del soroll.A més, tant els temps de decisió com el rendiment psicofísic són funcions decreixents de l'estimulació externa. També proposem dos mètodes, un basat en l'aproximació macroscòpica i un altre en la teoria de la varietat central, que simplifiquen la descripció de sistemes estocàstics multistables.<br>Computational models based on large-scale, neurobiologically-inspired networks describe the decision-related activity observed in some cortical areas as a transition between attractors of the cortical network. Stimulation induces a change in the attractor configuration and drives the system out from its initial resting attractor to one of the existing attractors associated with the categorical choices. The noise present in the system renders transitions random. We show that there exist two qualitatively different mechanisms for decision, each with distinctive psychophysical signatures. The decision mechanism arising at low inputs, entirely driven by noise, leads to skewed distributions of decision times, with a mean governed by the amplitude of the noise. Moreover, both decision times and performances are monotonically decreasing functions of the overall external stimulation. We also propose two methods, one based on the macroscopic approximation and one based on center manifold theory, to simplify the description of multistable stochastic neural systems.
45
Vellmer, Sebastian. "Applications of the Fokker-Planck Equation in Computational and Cognitive Neuroscience." Doctoral thesis, Humboldt-Universität zu Berlin, 2020. http://dx.doi.org/10.18452/21597.
Full textAbstract:
In dieser Arbeit werden mithilfe der Fokker-Planck-Gleichung die Statistiken, vor allem die Leistungsspektren, von Punktprozessen berechnet, die von mehrdimensionalen Integratorneuronen [Engl. integrate-and-fire (IF) neuron],
Netzwerken von IF Neuronen und Entscheidungsfindungsmodellen erzeugt werden.
Im Gehirn werden Informationen durch Pulszüge von Aktionspotentialen kodiert.
IF Neurone mit radikal vereinfachter Erzeugung von Aktionspotentialen haben sich in Studien die auf Pulszeiten fokussiert sind als Standardmodelle etabliert. Eindimensionale IF Modelle können jedoch beobachtetes Pulsverhalten oft nicht beschreiben und müssen dazu erweitert werden. Im erste Teil dieser Arbeit wird eine Theorie zur Berechnung der Pulszugleistungsspektren von stochastischen, multidimensionalen IF Neuronen entwickelt. Ausgehend von der zugehörigen Fokker-Planck-Gleichung werden partiellen Differentialgleichung abgeleitet, deren Lösung sowohl die stationäre Wahrscheinlichkeitsverteilung und Feuerrate, als auch das Pulszugleistungsspektrum beschreibt.
Im zweiten Teil wird eine Theorie für große, spärlich verbundene und homogene Netzwerke aus IF Neuronen entwickelt, in der berücksichtigt wird, dass die zeitlichen Korrelationen von Pulszügen selbstkonsistent sind. Neuronale Eingangströme werden durch farbiges Gaußsches Rauschen modelliert, das von einem mehrdimensionalen Ornstein-Uhlenbeck Prozess (OUP) erzeugt wird. Die Koeffizienten des OUP sind vorerst unbekannt und sind als Lösung der Theorie definiert. Um heterogene Netzwerke zu untersuchen, wird eine iterative Methode erweitert.
Im dritten Teil wird die Fokker-Planck-Gleichung auf Binärentscheidungen von Diffusionsentscheidungsmodellen [Engl. diffusion-decision models (DDM)] angewendet. Explizite Gleichungen für die Entscheidungszugstatistiken werden für den einfachsten und analytisch lösbaren Fall von der Fokker-Planck-Gleichung hergeleitet. Für nichtliniear Modelle wird die Schwellwertintegrationsmethode erweitert.<br>This thesis is concerned with the calculation of statistics, in particular the power spectra, of point processes generated by stochastic multidimensional integrate-and-fire (IF) neurons, networks of IF neurons and decision-making models from the corresponding Fokker-Planck equations.
In the brain, information is encoded by sequences of action potentials. In studies that focus on spike timing, IF neurons that drastically simplify the spike generation have become the standard model. One-dimensional IF neurons do not suffice to accurately model neural dynamics, however, the extension towards multiple dimensions yields realistic behavior at the price of growing complexity. The first part of this work develops a theory of spike-train power spectra for stochastic, multidimensional IF neurons. From the Fokker-Planck equation, a set of partial differential equations is derived that describes the stationary probability density, the firing rate and the spike-train power spectrum.
In the second part of this work, a mean-field theory of large and sparsely connected homogeneous networks of spiking neurons is developed that takes into account the self-consistent temporal correlations of spike trains. Neural input is approximated by colored Gaussian noise generated by a multidimensional Ornstein-Uhlenbeck process of which the coefficients are initially unknown but determined by the self-consistency condition and define the solution of the theory. To explore heterogeneous networks, an iterative scheme is extended to determine the distribution of spectra.
In the third part, the Fokker-Planck equation is applied to calculate the statistics of sequences of binary decisions from diffusion-decision models (DDM). For the analytically tractable DDM, the statistics are calculated from the corresponding Fokker-Planck equation. To determine the statistics for nonlinear models, the threshold-integration method is generalized.
46
Kourkoulas-Chondrorizos, Alexandros. "Online optimisation of information transmission in stochastic spiking neural systems." Thesis, University of Edinburgh, 2012. http://hdl.handle.net/1842/5832.
Full textAbstract:
An Information Theoretic approach is used for studying the effect of noise on various spiking neural systems. Detailed statistical analyses of neural behaviour under the influence of stochasticity are carried out and their results related to other work and also biological neural networks. The neurocomputational capabilities of the neural systems under study are put on an absolute scale. This approach was also used in order to develop an optimisation framework. A proof-of-concept algorithm is designed, based on information theory and the coding fraction, which optimises noise through maximising information throughput. The algorithm is applied with success to a single neuron and then generalised to an entire neural population with various structural characteristics (feedforward, lateral, recurrent connections). It is shown that there are certain positive and persistent phenomena due to noise in spiking neural networks and that these phenomena can be observed even under simplified conditions and therefore exploited. The transition is made from detailed and computationally expensive tools to efficient approximations. These phenomena are shown to be persistent and exploitable under a variety of circumstances. The results of this work provide evidence that noise can be optimised online in both single neurons and neural populations of varying structures.
47
Ncube, Israel. "Stochastic approximation of artificial neural network-type learning algorithms, a dynamical systems approach." Thesis, National Library of Canada = Bibliothèque nationale du Canada, 2001. http://www.collectionscanada.ca/obj/s4/f2/dsk3/ftp04/NQ60559.pdf.
Full text
48
Curram, Stephen Paul. "Representing intelligent decision making in discrete event simulation : a stochastic neural network approach." Thesis, University of Warwick, 1997. http://wrap.warwick.ac.uk/59461/.
Full textAbstract:
The problem of representing decision making behaviour in discrete event simulation was investigated. Of particular interest was modelling variety in the decisions, where different people might make different decisions even where the same circumstances hold. An initial investigation of existing and alternative approaches for representing decision making was carried out. This led to the suggestion of using a neural network to represent the decision making behaviour in the form of a multi-criteria probability distribution based on data of observed decision making. The feasibility of the stochastic neural network approach was investigated. Models were fitted using artificial data from discrete and continuous distributions that included the shape parameters as inputs, and tested against known results from the distributions. Also a bank simulation was used to collect data from volunteers who controlled the queuing decisions of customers inside the bank. Models of their behaviour were created and implemented in the bank simulation to automate the decision making of customers. The investigation established the feasibility of the approach, although it indicated the need for substantial amounts of data showing examples of decision making. A hybrid model that combined the stochastic neural network approach with a rule-based approach allowed the development of more general models of decision making behaviour.
49
Glover, John Sigsworth. "The hardware implementation of an artificial neural network using stochastic pulse rate encoding principles." Thesis, Durham University, 1995. http://etheses.dur.ac.uk/5423/.
Full textAbstract:
In this thesis the development of a hardware artificial neuron device and artificial neural network using stochastic pulse rate encoding principles is considered. After a review of neural network architectures and algorithmic approaches suitable for hardware implementation, a critical review of hardware techniques which have been considered in analogue and digital systems is presented. New results are presented demonstrating the potential of two learning schemes which adapt by the use of a single reinforcement signal. The techniques for computation using stochastic pulse rate encoding are presented and extended with new novel circuits relevant to the hardware implementation of an artificial neural network. The generation of random numbers is the key to the encoding of data into the stochastic pulse rate domain. The formation of random numbers and multiple random bit sequences from a single PRBS generator have been investigated. Two techniques, Simulated Annealing and Genetic Algorithms, have been applied successfully to the problem of optimising the configuration of a PRBS random number generator for the formation of multiple random bit sequences and hence random numbers. A complete hardware design for an artificial neuron using stochastic pulse rate encoded signals has been described, designed, simulated, fabricated and tested before configuration of the device into a network to perform simple test problems. The implementation has shown that the processing elements of the artificial neuron are small and simple, but that there can be a significant overhead for the encoding of information into the stochastic pulse rate domain. The stochastic artificial neuron has the capability of on-line weight adaption. The implementation of reinforcement schemes using the stochastic neuron as a basic element are discussed.
50
Jansa, Jakub. "Stochastická předpověď průměrných měsíčních průtoku ve vybraném vodoměrném profilu." Master's thesis, Vysoké učení technické v Brně. Fakulta stavební, 2016. http://www.nusl.cz/ntk/nusl-240072.
Full textAbstract:
The diploma thesis is focused on the average monthly flows forecast in the selected hydrometric profile. Aim of this work will be evaluation of the calculated values and the interpretation of the results in understandable form. The next step will be find an appropriate connection between randomly-generated inputs in the form of random real flow series using the standard hydrological prediction models. This models are based on the principles of artificial intelligence and probability model. The result of the work will be verification of procedures and compilation of mean monthly flow stochastic forecast in selected hydrometric profile, which would be used for a reservoirs management, respectively for water systems management.