To see the other types of publications on this topic, follow the link: Data filtering.
Dissertations / Theses on the topic 'Data filtering'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 dissertations / theses for your research on the topic 'Data filtering.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse dissertations / theses on a wide variety of disciplines and organise your bibliography correctly.
1
Faber, Marc. "On-Board Data Processing and Filtering." International Foundation for Telemetering, 2015. http://hdl.handle.net/10150/596433.
Full textAbstract:
ITC/USA 2015 Conference Proceedings / The Fifty-First Annual International Telemetering Conference and Technical Exhibition / October 26-29, 2015 / Bally's Hotel & Convention Center, Las Vegas, NVOne of the requirements resulting from mounting pressure on flight test schedules is the reduction of time needed for data analysis, in pursuit of shorter test cycles. This requirement has ramifications such as the demand for record and processing of not just raw measurement data but also of data converted to engineering units in real time, as well as for an optimized use of the bandwidth available for telemetry downlink and ultimately for shortening the duration of procedures intended to disseminate pre-selected recorded data among different analysis groups on ground. A promising way to successfully address these needs consists in implementing more CPU-intelligence and processing power directly on the on-board flight test equipment. This provides the ability to process complex data in real time. For instance, data acquired at different hardware interfaces (which may be compliant with different standards) can be directly converted to more easy-to-handle engineering units. This leads to a faster extraction and analysis of the actual data contents of the on-board signals and busses. Another central goal is the efficient use of the available bandwidth for telemetry. Real-time data reduction via intelligent filtering is one approach to achieve this challenging objective. The data filtering process should be performed simultaneously on an all-data-capture recording and the user should be able to easily select the interesting data without building PCM formats on board nor to carry out decommutation on ground. This data selection should be as easy as possible for the user, and the on-board FTI devices should generate a seamless and transparent data transmission, making a quick data analysis viable. On-board data processing and filtering has the potential to become the future main path to handle the challenge of FTI data acquisition and analysis in a more comfortable and effective way.
2
Zhou, Yilun S. M. Massachusetts Institute of Technology. "Data-driven path filtering in ConceptNet." Thesis, Massachusetts Institute of Technology, 2019. https://hdl.handle.net/1721.1/122731.
Full textAbstract:
Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019Cataloged from PDF version of thesis.
Includes bibliographical references (pages 49-52).
In many applications, it is important to characterize the way in which two concepts are semantically related. Knowledge graphs such as ConceptNet provide a rich source of information for such characterizations by encoding relations between concepts as edges in a graph. When two concepts are not directly connected by an edge, their relationship can still be described in terms of the paths that connect them. Unfortunately, many of these paths are uninformative and noisy, meaning that the success of applications that use such path features crucially relies on their ability to select high-quality paths. In existing applications, this path selection process is based on relatively simple heuristics. In this thesis I instead propose to learn to predict path quality from crowdsourced human assessments. Since a generic task-independent notion of quality is concerned, human participants are asked to rank paths according to their subjective assessment of the paths' naturalness, without being given specific definitions or guidelines. Experiments show that a neural network model trained on these assessments is able to predict human judgments on unseen paths with near optimal performance. Most notably, the resulting path selection method is substantially better than the current heuristic approaches at identifying meaningful paths in various applications.
by Yilun Zhou.
S.M.
S.M. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science
3
Cirkic, Mirsad. "Modular General-Purpose Data Filtering for Tracking." Thesis, Linköping University, Department of Electrical Engineering, 2008. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-14917.
Full textAbstract:
In nearly allmodern tracking systems, signal processing is an important part with state estimation as the fundamental component. To evaluate and to reassess different tracking systems in an affordable way, simulations that are in accordance with reality are largely used. Simulation software that is composed of many different simulating modules, such as high level architecture (HLA) standardized software, is capable of simulating very realistic data and scenarios.
A modular and general-purpose state estimation functionality for filtering provides a profound basis for simulating most modern tracking systems, which in this thesis work is precisely what is created and implemented in an HLA-framework. Some of the most widely used estimators, the iterated Schmidt extended Kalman filter, the scaled unscented Kalman filter, and the particle filter, are chosen to form a toolbox of such functionality. An indeed expandable toolbox that offers both unique and general features of each respective filter is designed and implemented, which can be utilized in not only tracking applications but in any application that is in need of fundamental state estimation. In order to prepare the user to make full use of this toolbox, the filters’ methods are described thoroughly, some of which are modified with adjustments that have been discovered in the process.
Furthermore, to utilize these filters easily for the sake of user-friendliness, a linear algebraic shell is created, which has very straight-forward matrix handling and uses BOOST UBLAS as the underlying numerical library. It is used for the implementation of the filters in C++, which provides a very independent and portable code.
4
Torgrimsson, Jan. "Adaptive filtering of VLF data from space." Thesis, KTH, Rymd- och plasmafysik, 2009. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-91544.
Full text
5
Čirkić, Mirsad. "Modular General-Purpose Data Filtering for Tracking." Thesis, Linköpings universitet, Institutionen för systemteknik, 2008. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-14917.
Full textAbstract:
In nearly allmodern tracking systems, signal processing is an important part with state estimation as the fundamental component. To evaluate and to reassess different tracking systems in an affordable way, simulations that are in accordance with reality are largely used. Simulation software that is composed of many different simulating modules, such as high level architecture (HLA) standardized software, is capable of simulating very realistic data and scenarios. A modular and general-purpose state estimation functionality for filtering provides a profound basis for simulating most modern tracking systems, which in this thesis work is precisely what is created and implemented in an HLA-framework. Some of the most widely used estimators, the iterated Schmidt extended Kalman filter, the scaled unscented Kalman filter, and the particle filter, are chosen to form a toolbox of such functionality. An indeed expandable toolbox that offers both unique and general features of each respective filter is designed and implemented, which can be utilized in not only tracking applications but in any application that is in need of fundamental state estimation. In order to prepare the user to make full use of this toolbox, the filters’ methods are described thoroughly, some of which are modified with adjustments that have been discovered in the process. Furthermore, to utilize these filters easily for the sake of user-friendliness, a linear algebraic shell is created, which has very straight-forward matrix handling and uses BOOST UBLAS as the underlying numerical library. It is used for the implementation of the filters in C++, which provides a very independent and portable code.
6
Olsson, Jakob, and Viktor Yberg. "Log data filtering in embedded sensor devices." Thesis, KTH, Skolan för informations- och kommunikationsteknik (ICT), 2015. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-175367.
Full textAbstract:
Data filtering is the disposal of unnecessary data in a data set, to save resources such as server capacity and bandwidth. The method is used to reduce the amount of stored data and thereby prevent valuable resources from processing insignificant information.The purpose of this thesis is to find algorithms for data filtering and to find out which algorithm gives the best effect in embedded devices with resource limitations. This means that the algorithm needs to be resource efficient in terms of memory usage and performance, while saving enough data points to avoid modification or loss of information. After an algorithm has been found it will also be implemented to fit the Exqbe system.The study has been done by researching previously done studies in line simplification algorithms and their applications. A comparison between several well-known and studied algorithms has been done to find which suits this thesis problem best.The comparison between the different line simplification algorithms resulted in an implementation of an extended version of the Ramer-Douglas-Peucker algorithm. The algorithm has been optimized and a new filter has been implemented in addition to the algorithm.Datafiltrering är att ta bort onödig data i en datamängd, för att spara resurser såsom serverkapacitet och bandbredd. Metoden används för att minska mängden lagrad data och därmed förhindra att värdefulla resurser används för att bearbeta obetydlig information. Syftet med denna tes är att hitta algoritmer för datafiltrering och att undersöka vilken algoritm som ger bäst resultat i inbyggda system med resursbegränsningar. Det innebär att algoritmen bör vara resurseffektiv vad gäller minnesanvändning och prestanda, men spara tillräckligt många datapunkter för att inte modifiera eller förlora information. Efter att en algoritm har hittats kommer den även att implementeras för att passa Exqbe-systemet. Studien är genomförd genom att studera tidigare gjorda studier om datafiltreringsalgoritmer och dess applikationer. Jämförelser mellan flera välkända algoritmer har utförts för att hitta vilken som passar denna tes bäst. Jämförelsen mellan de olika filtreringsalgoritmerna resulterade i en implementation av en utökad version av Ramer-Douglas-Peucker-algoritmen. Algoritmen har optimerats och ett nytt filter har implementerats utöver algoritmen.
7
Chilo, José. "Filtering and extracting features from infrasound data /." Stockholm, 2006. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-3978.
Full text
8
Karasalo, Maja. "Data Filtering and Control Design for Mobile Robots." Doctoral thesis, KTH, Optimeringslära och systemteori, 2009. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-11011.
Full textAbstract:
In this thesis, we consider problems connected to navigation and tracking for autonomousrobots under the assumption of constraints on sensors and kinematics. We study formation controlas well as techniques for filtering and smoothing of noise contaminated input. The scientific contributions of the thesis comprise five papers.In Paper A, we propose three cascaded, stabilizing formation controls for multi-agent systems.We consider platforms with non-holonomic kinematic constraints and directional rangesensors. The resulting formation is a leader-follower system, where each follower agent tracksits leader agent at a specified angle and distance. No inter-agent communication is required toexecute the controls. A switching Kalman filter is introduced for active sensing, and robustnessis demonstrated in experiments and simulations with Khepera II robots.In Paper B, an optimization-based adaptive Kalman filteringmethod is proposed. The methodproduces an estimate of the process noise covariance matrix Q by solving an optimization problemover a short window of data. The algorithm recovers the observations h(x) from a system˙ x = f (x), y = h(x)+v without a priori knowledge of system dynamics. The algorithm is evaluatedin simulations and a tracking example is included, for a target with coupled and nonlinearkinematics. In Paper C, we consider the problem of estimating a closed curve in R2 based on noisecontaminated samples. A recursive control theoretic smoothing spline approach is proposed, thatyields an initial estimate of the curve and subsequently computes refinements of the estimateiteratively. Periodic splines are generated by minimizing a cost function subject to constraintsimposed by a linear control system. The optimal control problem is shown to be proper, andsufficient optimality conditions are derived for a special case of the problem using Hamilton-Jacobi-Bellman theory.Paper D continues the study of recursive control theoretic smoothing splines. A discretizationof the problem is derived, yielding an unconstrained quadratic programming problem. Aproof of convexity for the discretized problem is provided, and the recursive algorithm is evaluatedin simulations and experiments using a SICK laser scanner mounted on a PowerBot from ActivMedia Robotics. Finally, in Paper E we explore the issue of optimal smoothing for control theoretic smoothingsplines. The output of the control theoretic smoothing spline problem is essentially a tradeoff between faithfulness to measurement data and smoothness. This tradeoff is regulated by the socalled smoothing parameter. In Paper E, a method is developed for estimating the optimal valueof this smoothing parameter. The procedure is based on general cross validation and requires noa priori information about the underlying curve or level of noise in the measurements.QC 20100722
9
Walter, Patrick L. "FILTERING CONSIDERATIONS WHEN TELEMETERING SHOCK AND VIBRATION DATA." International Foundation for Telemetering, 2001. http://hdl.handle.net/10150/607681.
Full textAbstract:
International Telemetering Conference Proceedings / October 22-25, 2001 / Riviera Hotel and Convention Center, Las Vegas, NevadaThe accurate measurement of shock and vibration data via flight telemetry is necessary to validate structural models, indicate off-nominal system performance, and/or generate environmental qualification criteria for airborne systems. Digital telemetry systems require anti-aliasing filters designed into them. If not properly selected and located, these filters can distort recorded time histories and modify their spectral content. This paper provides filter design guidance to optimize the quality of recorded flight structural dynamics data. It is based on the anticipated end use of the data. Examples of filtered shock data are included.
10
Wunnava, Sashi Prabha. "Kalman Filtering Approach to Optimize OFDM Data Rate." Thesis, University of North Texas, 2011. https://digital.library.unt.edu/ark:/67531/metadc84303/.
Full textAbstract:
This study is based on applying a non-linear mapping method, here the unscented Kalman filter; to estimate and optimize data rate resulting from the arrival rate having a Poisson distribution in an orthogonal frequency division multiplexing (OFDM) transmission system. OFDM is an emerging multi-carrier modulation scheme. With the growing need for quality of service in wireless communications, it is highly necessary to optimize resources in such a way that the overall performance of the system models should rise while keeping in mind the objective to achieve high data rate and efficient spectral methods in the near future. In this study, the results from the OFDM-TDMA transmission system have been used to apply cross-layer optimization between layers so as to treat different resources between layers simultaneously. The main controller manages the transmission of data between layers using the multicarrier modulation techniques. The unscented Kalman filter is used here to perform nonlinear mapping by estimating and optimizing the data rate, which result from the arrival rate having a Poisson distribution.
11
Li, Yifu. "Data Filtering and Modeling for Smart Manufacturing Network." Diss., Virginia Tech, 2020. http://hdl.handle.net/10919/99713.
Full textAbstract:
A smart manufacturing network connects machines via sensing, communication, and actuation networks. The data generated from the networks are used in data-driven modeling and decision-making to improve quality, productivity, and flexibility while reducing the cost. This dissertation focuses on improving the data-driven modeling of the quality-process relationship in smart manufacturing networks. The quality-process variable relationships are important to understand for guiding the quality improvement by optimizing the process variables. However, several challenges emerge. First, the big data sets generated from the manufacturing network may be information-poor for modeling, which may lead to high data transmission and computational loads and redundant data storage. Second, the data generated from connected machines often contain inexplicit similarities due to similar product designs and manufacturing processes. Modeling such inexplicit similarities remains challenging. Third, it is unclear how to select representative data sets for modeling in a manufacturing network setting, considering inexplicit similarities. In this dissertation, a data filtering method is proposed to select a relatively small and informative data subset. Multi-task learning is combined with latent variable decomposition to model multiple connected manufacturing processes that are similar-but-non-identical. A data filtering and modeling framework is also proposed to filter the manufacturing data for manufacturing network modeling adaptively. The proposed methodologies have been validated through simulation and the applications to real manufacturing case studies.Doctor of Philosophy
The advancement of the Internet-of-Things (IoT) integrates manufacturing processes and equipment into a network. Practitioners analyze and apply the data generated from the network to model the manufacturing network to improve product quality. The data quality directly affects the modeling performance and decision effectiveness. However, the data quality is not well controlled in a manufacturing network setting. In this dissertation, we propose a data quality assurance method, referred to as data filtering. The proposed method selects a data subset from raw data collected from the manufacturing network. The proposed method reduces the complexity in modeling while supporting decision effectiveness. To model the data from multiple similar-but-non-identical manufacturing processes, we propose a latent variable decomposition-based multi-task learning model to study the relationships between the process variables and product quality variable. Lastly, to adaptively determine the appropriate data subset for modeling each process in the manufacturing network, we further proposed an integrated data filtering and modeling framework. The proposed integrated framework improved the modeling performance of data generated by babycare manufacturing and semiconductor manufacturing.
12
Devaraju, Balaji [Verfasser], and Nico [Akademischer Betreuer] Sneeuw. "Understanding filtering on the sphere : experiences from filtering GRACE data / Balaji Devaraju. Betreuer: Nico Sneeuw." Stuttgart : Universitätsbibliothek der Universität Stuttgart, 2016. http://d-nb.info/1081936096/34.
Full text
13
Liao, ChenHan. "Transaction-filtering data mining and a predictive model for intelligent data management." Thesis, Cranfield University, 2008. http://dspace.lib.cranfield.ac.uk/handle/1826/7027.
Full textAbstract:
This thesis, first of all, proposes a new data mining paradigm (transaction-filtering
association rule mining) addressing a time consumption issue caused by the repeated scans
of original transaction databases in conventional associate rule mining algorithms. An
in-memory transaction filter is designed to discard those infrequent items in the pruning
steps. This filter is a data structure to be updated at the end of each iteration. The results
based on an IBM benchmark show that an execution time reduction of 10% - 19% is
achieved compared with the base case.
Next, a data mining-based predictive model is then established contributing to intelligent
data management within the context of Centre for Grid Computing. The capability of
discovering unseen rules, patterns and correlations enables data mining techniques
favourable in areas where massive amounts of data are generated. The past behaviours of
two typical scenarios (network file systems and Data Grids) have been analyzed to build
the model. The future popularity of files can be forecasted with an accuracy of 90% by
deploying the above predictor based on the given real system traces. A further step towards
intelligent policy design is achieved by analyzing the prediction results of files’ future
popularity. The real system trace-based simulations have shown improvements of 2-4 times
in terms of data response time in network file system scenario and 24% mean job time
reduction in Data Grids compared with conventional cases.
14
ALMEIDA, Marcos Antonio Martins de. "Statistical analysis applied to data classification and image filtering." Universidade Federal de Pernambuco, 2016. https://repositorio.ufpe.br/handle/123456789/25506.
Full textAbstract:
Submitted by Fernanda Rodrigues de Lima (fernanda.rlima@ufpe.br) on 2018-08-03T20:52:13Z
No. of bitstreams: 2
license_rdf: 811 bytes, checksum: e39d27027a6cc9cb039ad269a5db8e34 (MD5)
TESE Marcos Antonio Martins de Almeida.pdf: 11555397 bytes, checksum: db589d39915a5dda1d8b9e763a9cf4c0 (MD5)Approved for entry into archive by Alice Araujo (alice.caraujo@ufpe.br) on 2018-08-09T20:49:00Z (GMT) No. of bitstreams: 2 license_rdf: 811 bytes, checksum: e39d27027a6cc9cb039ad269a5db8e34 (MD5) TESE Marcos Antonio Martins de Almeida.pdf: 11555397 bytes, checksum: db589d39915a5dda1d8b9e763a9cf4c0 (MD5)
Made available in DSpace on 2018-08-09T20:49:01Z (GMT). No. of bitstreams: 2 license_rdf: 811 bytes, checksum: e39d27027a6cc9cb039ad269a5db8e34 (MD5) TESE Marcos Antonio Martins de Almeida.pdf: 11555397 bytes, checksum: db589d39915a5dda1d8b9e763a9cf4c0 (MD5) Previous issue date: 2016-12-21
Statistical analysis is a tool of wide applicability in several areas of scientific knowledge. This thesis makes use of statistical analysis in two different applications: data classification and image processing targeted at document image binarization. In the first case, this thesis presents an analysis of several aspects of the consistency of the classification of the senior researchers in computer science of the Brazilian research council, CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico. The second application of statistical analysis developed in this thesis addresses filtering-out the back to front interference which appears whenever a document is written or typed on both sides of translucent paper. In this topic, an assessment of the most important algorithms found in the literature is made, taking into account a large quantity of parameters such as the strength of the back to front interference, the diffusion of the ink in the paper, and the texture and hue of the paper due to aging. A new binarization algorithm is proposed, which is capable of removing the back-to-front noise in a wide range of documents. Additionally, this thesis proposes a new concept of “intelligent” binarization for complex documents, which besides text encompass several graphical elements such as figures, photos, diagrams, etc.
Análise estatística é uma ferramenta de grande aplicabilidade em diversas áreas do conhecimento científico. Esta tese faz uso de análise estatística em duas aplicações distintas: classificação de dados e processamento de imagens de documentos visando a binarização. No primeiro caso, é aqui feita uma análise de diversos aspectos da consistência da classificação de pesquisadores sêniores do CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico, na área de Ciência da Computação. A segunda aplicação de análise estatística aqui desenvolvida trata da filtragem da interferência frente-verso que surge quando um documento é escrito ou impresso em ambos os lados da folha de um papel translúcido. Neste tópico é inicialmente feita uma análise da qualidade dos mais importantes algoritmos de binarização levando em consideração parâmetros tais como a intensidade da interferência frente-verso, a difusão da tinta no papel e a textura e escurecimento do papel pelo envelhecimento. Um novo algoritmo para a binarização eficiente de documentos com interferência frente-verso é aqui apresentado, tendo se mostrado capaz de remover tal ruído em uma grande gama de documentos. Adicionalmente, é aqui proposta a binarização “inteligente” de documentos complexos que envolvem diversos elementos gráficos (figuras, diagramas, etc).
15
Contro, Alessandro. "Multi-sensing Data Fusion: Target tracking via particle filtering." Master's thesis, Alma Mater Studiorum - Università di Bologna, 2018. http://amslaurea.unibo.it/16835/.
Full textAbstract:
In this Master's thesis, Multi-sensing Data Fusion is firstly introduced with a focus on perception and the concepts that are the base of this work, like the mathematical tools that make it possible. Particle filters are one class of these tools that allow a computer to perform fusion of numerical information that is perceived from real environment by sensors. For this reason they are described and state of the art mathematical formulas and algorithms for particle filtering are also presented. At the core of this project, a simple piece of software has been developed in order to test these tools in practice. More specifically, a Target Tracking Simulator software is presented where a virtual trackable object can freely move in a 2-dimensional simulated environment and distributed sensor agents, dispersed in the same environment, should be able to perceive the object through a state-dependent measurement affected by additive Gaussian noise. Each sensor employs particle filtering along with communication with other neighboring sensors in order to update the perceived state of the object and track it as it moves in the environment. The combination of Java and AgentSpeak languages is used as a platform for the development of this application.
16
Li, Lingjie Luo Zhi-Quan. "Data fusion and filtering for target tracking and identification /." *McMaster only, 2003.
Find full text
17
Gill, Mark Richard. "Filtering down : open data in smaller scaled Canadian municipalities." Thesis, University of British Columbia, 2016. http://hdl.handle.net/2429/57590.
Full textAbstract:
This thesis is a case study examination of open data initiatives in smaller scaled municipalities in Canada. My research questions explore how open data initiatives are being developed and deployed as well as how notions of scale affect these initiatives. I used three avenues of investigation: first, I performed an assessment of district and municipal open data websites in British Columbia; second, I interviewed government open data experts in Western Canada; and finally, I reviewed open data policies from participant governments. From this research, I found that there is a high level of variability in the benefits and challenges associated with offering open data at the municipal and district level. These challenges include: technological barriers to publishing and using data; the current culture around data management; a lack of understanding about who is using open data from a government perspective; and, the need for standardized procedures relating to open data. For governments, challenges associated with open data can create barriers to realizing the potential benefits of open data. In looking at the effect of scale on the development and deployment of open data, two scale effects emerge: the limitation of size and data jurisdiction. In the first, smaller scaled municipalities focus on the size of a municipality as a determining factor for the success of open data. In the second, data jurisdiction produces borders and boundaries for open data users in a way that reifies a traditional data management model. I conclude with recommendations to reduce barriers associated with open data initiatives, and present some theoretical considerations of scale in open data initiatives as groundwork for future research.Graduate Studies, College of (Okanagan)
Graduate
18
Guttman, Michael. "Sampled-data IIR filtering via time-mode signal processing." Thesis, McGill University, 2010. http://digitool.Library.McGill.CA:80/R/?func=dbin-jump-full&object_id=86770.
Full textAbstract:
In this work, the design of sampled-data infinite impulse response filters based on time-mode signal processing circuits is presented. Time-mode signal processing (TMSP), defined as the processing of sampled analog information using time-difference variables, has become one of the more popular emerging technologies in circuit design. As TMSP is still relatively new, there is still much development needed to extend the technology into a general signal-processing tool. In this work, a set of general building block will be introduced that perform the most basic mathematical operations in the time-mode. By arranging these basic structures, higher-order time-mode systems, specifically, time-mode filters, will be realized. Three second-order time-mode filters (low-pass, band-reject, high-pass) are modeled using MATLAB, and simulated in Spectre to verify the design methodology. Finally, a damped integrator and a second-order low-pass time-mode IIR filter are both implemented using discrete components.Dans ce mémoire, la conception de filtres de données-échantillonnées ayant une réponse impulsionnelle infinie basée sur le traitement de signal en mode temporel est présentée. Le traitement de signal dans le domaine temporel (TSDT), définie comme étant le traitement d'information analogique échantillonnée en utilisant des différences de temps comme variables, est devenu une des techniques émergentes de conception de circuits des plus populaires. Puisque le TSDT est toujours relativement récent, il y a encore beaucoup de développements requis pour étendre cette technologie comme un outil de traitement de signal général. Dans cette recherche, un ensemble de blocs d'assemblage capable de réaliser la plupart des opérations mathématiques dans le domaine temporel sera introduit. En arrangeant ces structures élémentaires, des systèmes en mode temporel d'ordre élevé, plus spécifiquement des filtres en mode temporel, seront réalisés. Trois filtres de deuxième ordre dans le domaine temporel (passe-bas, passe-bande et passe-haut) sont modélisés sur MATLAB et simulé sur Spectre afin de vérifier la méthodologie de conception. Finalement, un intégrateur amorti et un filtre passe-bas IIR de deuxième ordre en mode temporel sont implémentés avec des composantes discrètes.
19
Wang, Xiwei. "Data Privacy Preservation in Collaborative Filtering Based Recommender Systems." UKnowledge, 2015. http://uknowledge.uky.edu/cs_etds/35.
Full textAbstract:
This dissertation studies data privacy preservation in collaborative filtering based recommender systems and proposes several collaborative filtering models that aim at preserving user privacy from different perspectives.
The empirical study on multiple classical recommendation algorithms presents the basic idea of the models and explores their performance on real world datasets. The algorithms that are investigated in this study include a popularity based model, an item similarity based model, a singular value decomposition based model, and a bipartite graph model. Top-N recommendations are evaluated to examine the prediction accuracy.
It is apparent that with more customers' preference data, recommender systems can better profile customers' shopping patterns which in turn produces product recommendations with higher accuracy. The precautions should be taken to address the privacy issues that arise during data sharing between two vendors. Study shows that matrix factorization techniques are ideal choices for data privacy preservation by their nature. In this dissertation, singular value decomposition (SVD) and nonnegative matrix factorization (NMF) are adopted as the fundamental techniques for collaborative filtering to make privacy-preserving recommendations. The proposed SVD based model utilizes missing value imputation, randomization technique, and the truncated SVD to perturb the raw rating data. The NMF based models, namely iAux-NMF and iCluster-NMF, take into account the auxiliary information of users and items to help missing value imputation and privacy preservation. Additionally, these models support efficient incremental data update as well.
A good number of online vendors allow people to leave their feedback on products. It is considered as users' public preferences. However, due to the connections between users' public and private preferences, if a recommender system fails to distinguish real customers from attackers, the private preferences of real customers can be exposed. This dissertation addresses an attack model in which an attacker holds real customers' partial ratings and tries to obtain their private preferences by cheating recommender systems. To resolve this problem, trustworthiness information is incorporated into NMF based collaborative filtering techniques to detect the attackers and make reasonably different recommendations to the normal users and the attackers. By doing so, users' private preferences can be effectively protected.
20
Tong, Shan. "Dynamic physiological information recovery : a sampled-data filtering framework /." View abstract or full-text, 2008. http://library.ust.hk/cgi/db/thesis.pl?ECED%202008%20TONG.
Full text
21
Rosswog, James. "Improving classification of spatiotemporal data using adaptive history filtering." Diss., Online access via UMI:, 2007.
Find full text
22
Kim, Yoonsoo. "Addressing the data recency problem in collaborative filtering systems." Link to electronic thesis, 2004. http://www.wpi.edu/Pubs/ETD/Available/etd-09244-024515/.
Full textAbstract:
Thesis (M.S.)--Worcester Polytechnic Institute.Keywords: Data recency problem; Recommender system; Time-based forgetting function; Time-based forgetting strategy; Collaborative filtering system. Includes bibliographical references (p. 73-74).
23
Lui, Chiu-sing Gilbert. "Some statistical topics on sequential data assimilation." Click to view the E-thesis via HKUTO, 2008. http://sunzi.lib.hku.hk/hkuto/record/b40204005.
Full text
24
Lui, Chiu-sing Gilbert, and 雷照盛. "Some statistical topics on sequential data assimilation." Thesis, The University of Hong Kong (Pokfulam, Hong Kong), 2008. http://hub.hku.hk/bib/B40204005.
Full text
25
Parameswaran, Rupa. "A Robust Data Obfuscation Technique for Privacy Preserving Collaborative Filtering." Diss., Georgia Institute of Technology, 2006. http://hdl.handle.net/1853/11459.
Full textAbstract:
Privacy is defined as the freedom from unauthorized intrusion. The availability of personal information through online databases, such as government records, medical records, and voters and #146; lists, pose a threat to personal privacy. The concern over individual privacy has led to the development of legal codes for safeguarding privacy in several countries. However, the ignorance of individuals as well as loopholes in the systems, have led to information breaches even in the presence of such rules and regulations. Protection against data privacy requires modification of the data itself. The term {em data obfuscation} is used to refer to the class of algorithms that modify the values of the data items without distorting the usefulness of the data. The main goal of this thesis is the development of a data obfuscation technique that provides robust privacy protection with minimal loss in usability of the data.
Although medical and financial services are two of the major areas where information privacy is a concern, privacy breaches are not restricted to these domains. One of the areas where the concern over data privacy is of growing interest is collaborative filtering. Collaborative filtering systems are being widely used in E-commerce applications to provide recommendations to users regarding products that might be of interest to them. The prediction accuracy of these systems is dependent on the size and accuracy of the data provided by users. However, the lack of sufficient guidelines governing the use and distribution of user data raises concerns over individual privacy. Users often provide the minimal information that is required for accessing these E-commerce services. The lack of rules governing the use and distribution of data disallows sharing of data among different communities for collaborative filtering. The goals of this thesis are (a) the definition of a standard for classifying DO techniques, (b) the development of a robust cluster preserving data obfuscation algorithm, and (c) the design and implementation of a privacy-preserving shared collaborative filtering framework using the data obfuscation algorithm.
26
López, Martinez Carlos. "Multidimensional speckle noise. Modelling and filtering related to sar data." Doctoral thesis, Universitat Politècnica de Catalunya, 2003. http://hdl.handle.net/10803/6921.
Full textAbstract:
Los Radares de Apertura Sintética, o sistemas SAR, representan el mejorejemplo de sistemas activos de teledetección por microondas. Debido a su naturaleza coherente, un sistema SAR es capaz de adquirir información dedispersión electromagnética con una alta resolución espacial, pero por otro lado, esta naturaleza coherente provoca también la aparición de speckle.A pesar de que el speckle es una medida electromagnética, sólo puede ser analizada como una componente de ruido debido a la complejidad asociadacon el proceso de dispersión electromagnética.Para eliminar los efectos del ruido speckle adecuadamente, es necesario un modelo de ruido, capaz de identificar las fuentes de ruido y como éstasdegradan la información útil. Mientras que este modelo existe para sistemasSAR unidimensionales, conocido como modelo de ruido speckle multiplicativo,éste no existe en el caso de sistemas SAR multidimensionales.El trabajo presentado en esta tesis presenta la definición y completa validación de nuevos modelos de ruido speckle para sistemas SAR multidimensionales,junto con su aplicación para la reducción de ruido speckle y la extracción de información.En esta tesis, los datos SAR multidimensionales, se consideran bajo una formulación basada en la matriz de covarianza, ya que permite el análisisde datos sobre la base del producto complejo Hermítico de pares de imágenesSAR. Debido a que el mantenimiento de la resolución especial es un aspectoimportante del procesado de imágenes SAR, la reducción de ruido speckleestá basada, en este trabajo, en la teoría de análisis wavelet.
27
López, Martinez Carlos. "Multidimensional speckle noise, modelling and filtering related to SAR data /." Köln : DLR, Bibliotheks- und Informationswesen, 2004. http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&doc_number=015380575&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA.
Full text
28
Guan, Xin. "On reducing the data sparsity in collaborative filtering recommender systems." Thesis, University of Warwick, 2017. http://wrap.warwick.ac.uk/97978/.
Full textAbstract:
A recommender system is one of the most common software tools and techniques for generating personalized recommendations. Collaborative filtering, as an effective recommender system approach, predicts a user's preferences (ratings) on an item based on the previous preferences of other users. However, collaborative filtering suffers from the data sparsity problem, that is, the users' preference data on items are usually too few to understand the users’ true preferences, which makes the recommendation task difficult. This thesis focuses on approaches to reducing the data sparsity in collaborative filtering recommender systems. Active learning algorithms are effective in reducing the sparsity problem for recommender systems by requesting users to give ratings to some items when they come in. However, this process focuses on new users and is often based on the assumption that a user can provide ratings for any queried items, which is unrealistic and costly. Take movie recommendation for example, to rate a movie that is generated by an active learning strategy, a user has to watch it. On the other hand, the user may be frustrated when asked to rate a movie that he/she has not watched. This could lower the customer's confidence and expectation of the recommender system. Instead, an ESVD algorithm is proposed which combines classic matrix factorization algorithms with ratings completion inspired by active learning, allowing the system to 'add' ratings automatically through learning. This general framework can be incorporated with different SVD-based algorithms such as SVD++ by proposing the ESVD++ method. The proposed EVSD model is further explored by presenting the MESVD approach, which learns the model iteratively, to get more precise prediction results. Two variants of ESVD model: IESVD and UESVD are also proposed to handle the imbalanced datasets that contains more users than items or more items than users, respectively. These algorithms can be seen as pure collaborative filtering algorithms since they do not require human efforts to give ratings. Experimental results show the reduction of the prediction error when compared with collaborative filtering algorithms (matrix factorization). Secondly, traditional active learning methods only evaluate each user or items independently and only consider the benefits of the elicitations to new users or items, but pay less attention to the effects of the system. In this thesis, the traditional methods are extended by proposing a novel generalized system-driven active learning framework. Specifically, it focuses on the elicitations of the past users instead of the new users and considers a more general scenario where users repeatedly come back to the system instead of during the sign-up process. In the proposed framework the ratings are elicited by combining the user-focused active learning with item-focused active learning, for the purpose of improving the performance of the whole system. A variety of active learning strategies are evaluated on the proposed framework. Experimental results demonstrate its effectiveness on reducing the sparsity, and then enables improvements on the system performance. Thirdly, traditional recommender systems suggest items belonging to a single domain, therefore existing research on active learning only applies and evaluates elicitation strategies on a single-domain scenario. Cross-domain recommendation utilizes the knowledge derived from the auxiliary domain(s) with sufficient ratings to alleviate the data sparsity in the target domain. A special case of cross-domain recommendation is multi-domain recommendation that utilizes the shared knowledge across multiple domains to alleviate the data sparsity in all domains. A multi-domain active learning framework is proposed by combining active learning with the cross-domain collaborative filtering algorithm (RMGM) in the multi-domain scenarios, in which the sparsity problem can be further alleviated by sharing knowledge among multiple sources, along with the data acquired from users. The proposed algorithms are evaluated on real-world recommender system datasets and experimental results confirmed their effectiveness.
29
Mitra, Subhadeep. "Particle filtering with Lagrangian data in a point vortex model." Thesis, Massachusetts Institute of Technology, 2012. http://hdl.handle.net/1721.1/72873.
Full textAbstract:
Thesis (S.M.)--Massachusetts Institute of Technology, Computation for Design and Optimization Program, 2012.Cataloged from PDF version of thesis.
Includes bibliographical references (p. 131-138).
Particle filtering is a technique used for state estimation from noisy measurements. In fluid dynamics, a popular problem called Lagrangian data assimilation (LaDA) uses Lagrangian measurements in the form of tracer positions to learn about the changing flow field. Particle filtering can be applied to LaDA to track the flow field over a period of time. As opposed to techniques like Extended Kalman Filter (EKF) and Ensemble Kalman Filter (EnKF), particle filtering does not rely on linearization of the forward model and can provide very accurate estimates of the state, as it represents the true Bayesian posterior distribution using a large number of weighted particles. In this work, we study the performance of various particle filters for LaDA using a two-dimensional point vortex model; this is a simplified fluid dynamics model wherein the positions of vortex singularities (point vortices) define the state. We consider various parameters associated with algorithm and examine their effect on filtering performance under several vortex configurations. Further, we study the effect of different tracer release positions on filtering performance. Finally, we relate the problem of optimal tracer deployment to the Lagrangian coherent structures (LCS) of point vortex system.
by Subhadeep Mitra.
S.M.
30
Malconian, Daniel R. "Automating data aggregation for collaborative filtering in Ruby on Rails." Thesis, Massachusetts Institute of Technology, 2007. http://hdl.handle.net/1721.1/46016.
Full textAbstract:
Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2008.Includes bibliographical references (p. 59).
Collaborative filtering and information filtering are tried and proven methods of utilizing aggregated data about a website's users to provide catered content. Passive filters are one subset of such algorithms that utilize data about a user's interactions with a website in viewing content, purchasing items, etc. My work develops a set of extensions for Ruby on Rails that, when inserted into an existing application, will comprehensively log information associated with different types of user interactions to provide a sound base for many passive filter implementations. The extensions will log how users interact with the application server (content accessed, forms submitted, etc) as well as how users interact with that content on their own browser (scrolling, AJAX requests, JavaScript calls, etc). Given existing open-source collaborative filtering algorithms, the ability to automatically aggregate user-interaction data in any arbitrary Rails application significantly decreases the barrier to implementing passive filtering in an already efficient agile web development framework. Further, my work utilizes the logged data to implement a web interface to view analytic information about the components of an application.
by Daniel R. Malconian.
M.Eng.
31
Malvar, Henrique Sarmento. "Optimal pre- and post-filtering in noisy sampled-data systems." Thesis, Massachusetts Institute of Technology, 1986. http://hdl.handle.net/1721.1/14895.
Full textAbstract:
Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1986.MICROFICHE COPY AVAILABLE IN ARCHIVES AND ENGINEERING.
Includes bibliographies.
by Henrique Sarmento Malvar.
Ph.D.
32
Dourbal, Paul, Ivan Goranov, and Chris Dehmelt. "Multichannel Telemetry Data Acquisition Using a Synchronous DSP Filtering Approach." International Foundation for Telemetering, 2011. http://hdl.handle.net/10150/595755.
Full textAbstract:
ITC/USA 2011 Conference Proceedings / The Forty-Seventh Annual International Telemetering Conference and Technical Exhibition / October 24-27, 2011 / Bally's Las Vegas, Las Vegas, NevadaToday's telemetry data acquisition systems demand an increased number of highly configurable acquisition channels delivering synchronized data samples in a specific format [1]. At the same time, there are additional requirements that these systems be compact and cost effective. In order to design such systems, novel approaches in digital signal processing are required. In this paper, we compare the typical analog signal sampling approach used in current systems with a flexible system architecture that is based on digital signal processing, allowing for precise synchronization and simultaneous sampling. An appropriate DSP filter structure is discussed, and a Xilinx FPGA based implementation example of this multi-channel filter that utilizes a minimal number of key signal processing components while easing the analog component requirements is presented.
33
Bektas, Oguz. "An adaptive data filtering model for remaining useful life estimation." Thesis, University of Warwick, 2018. http://wrap.warwick.ac.uk/106052/.
Full textAbstract:
The field of Prognostics and Health Management is becoming ever more important in the modern maintenance era, with advanced techniques of automation and mechanisation becoming increasingly prevalent. Prognostic technology has promising abilities to forecast remaining useful life and likely future circumstances of complex systems. However, the evolution of data processing and its critical impact on remaining useful life predictions continually demand increasing development so as to meet higher performance levels. There is often a gap between the adequacy of prognostic pre-processing and the prediction methods. One way to reduce this gap would be to design an adaptive data processing method that can filter multidimensional condition monitoring data by incorporating useful information from multiple data sources. Due to the incomplete understanding on the multi-dimensional failure mechanisms and the collaboration between data sources, current prognostic methods lack the ability to deal effectively with complicated interdependency, multidimensional condition monitoring information and noisy data. Further conventional methods are unable to deal with these efficiently. The methodology proposed in this research handles these deficiencies by introducing a prognostic framework that allows the effective use of monitoring data from different resources to predict the lifetime of systems. The methodology presents a feed-forward neural network filtering approach for trajectory similarity based remaining useful life predictions. The extraction of health indicators is applied as a type of dynamic filtering, in which the time series having full operational conditions are used to train a neural network mapping between raw training inputs and a health indicator output. This trained network function is evaluated by repeating condition monitoring information from multiple data subsets. After the network filtering, the training trajectories are used as baselines to predict the future behaviours of test trajectories. The similarity between these data subsets compares the relationship between the historical performance deterioration of a system's prior operating period with a similar system's degradation behaviour. The proposed prognostic technique, together with dynamic data filtering and remaining useful estimation, holds the promise of increased prediction performance levels. The presented methodology was tested using the PHM08 data challenge provided by the Prognostics Centre of Excellence at NASA Ames Research Centre, and it achieved the overall leading score in the published literature.
34
Moon, Taesup. "Learning from noisy data with applications to filtering and denoising /." May be available electronically:, 2008. http://proquest.umi.com/login?COPT=REJTPTU1MTUmSU5UPTAmVkVSPTI=&clientId=12498.
Full text
35
Andersson, Morgan. "Personal news video recommendations based on implicit feedback : An evaluation of different recommender systems with sparse data." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-234137.
Full textAbstract:
The amount of video content online will nearly triple in quantity by 2021 compared to 2016. The implementation of sophisticated filters is of paramount importance to manage this information flow. The research question of this thesis asks to what extent it is possible to generate personal recommendations, based on the data that news videos implies. The objective is to evaluate how different recommender systems compare to complete random, each other and how they are received by users in a test environment. This study was performed during the spring of 2018, and explore four different algorithms. These recommender systems include a content-based, a collaborative-filter, a hybrid model and a popularity model as a baseline. The dataset originates from a news media startup called Newstag, who provide video news on a global scale. The data is sparse and includes implicit feedback only. Three offline experiments and a user test were performed. The metric that guided the algorithms offline performance was their recall at 5 and 10, due to the fact that the top list of recommended items are of most interest. A comparison was done on different amounts of meta-data included during training. Another test explored respective algorithms performance as the density of the data increased. In the user test, a mean opinion score was calculated based on the quality of recommendations that each of the algorithms generated for the test subjects. The user test also included randomly sampled news videos to compare with as a baseline. The results indicate that for this specific setting and data set, the content-based recommender system performed best in both the recall at five and ten, as well as in the user test. All of the algorithms outperformed the random baseline.Mängden video som finns tillgänglig på internet förväntas att tredubblas år 2021 jämfört med 2016. Detta innebär ett behov av sofistikerade filter för att kunna hantera detta informationsflöde. Detta examensarbete ämnar att svara på till vilken grad det går att generera personliga rekommendationer baserat på det data som nyhetsvideo innebär. Syftet är att utvärdera och jämföra olika rekommendationssystem och hur de står sig i ett användartest. Studien utfördes under våren 2018 och utvärderar fyra olika algoritmer. Dessa olika rekommendationssystem innefattar tekniker som content-based, collaborative-filter, hybrid och en popularitetsmodell används som basvärde. Det dataset som används är glest och har endast implicita attribut. Tre experiment utförs samt ett användartest. Mätpunkten för algoritmernas prestanda utgjordes av recall at 5 och recall at 10, dvs. att man mäter hur väl algoritmerna lyckas generera värdefulla rekommendationer i en topp-fem respektive topp-10-lista av videoklipp. Detta då det är av intresse att ha de mest relevanta videorna högst upp i sin lista av resultat. En jämförelse gjordes mellan olika mängd metadata som inkluderades vid träning. Ett annat test gick ut på att utforska hur algoritmerna presterar då datasetet blir mindre glest. I användartestet användes en utvärderingsmetod kallad mean-opinion-score och denna räknades ut per algoritm genom att testanvändare gav betyg på respektive rekommendation, baserat på hur intressant videon var för dem. Användartestet inkluderade även slumpmässigt generade videos för att kunna jämföras i form av basvärde. Resultaten indikerar, för detta dataset, att algoritmen content-based presterar bäst både med hänsyn till recall at 5 & 10 samt den totala poängen i användartestet. Alla algoritmer presterade bättre än slumpen.
36
Ramljak, Dusan. "Data Driven High Performance Data Access." Diss., Temple University Libraries, 2018. http://cdm16002.contentdm.oclc.org/cdm/ref/collection/p245801coll10/id/530207.
Full textAbstract:
Computer and Information SciencePh.D.
Low-latency, high throughput mechanisms to retrieve data become increasingly crucial as the cyber and cyber-physical systems pour out increasing amounts of data that often must be analyzed in an online manner. Generally, as the data volume increases, the marginal utility of an ``average'' data item tends to decline, which requires greater effort in identifying the most valuable data items and making them available with minimal overhead. We believe that data analytics driven mechanisms have a big role to play in solving this needle-in-the-haystack problem. We rely on the claim that efficient pattern discovery and description, coupled with the observed predictability of complex patterns within many applications offers significant potential to enable many I/O optimizations. Our research covers exploitation of storage hierarchy for data driven caching and tiering, reduction of distance between data and computations, removing redundancy in data, using sparse representations of data, the impact of data access mechanisms on resilience, energy consumption, storage usage, and the enablement of new classes of data driven applications. For caching and prefetching, we offer a powerful model that separates the process of access prediction from the data retrieval mechanism. Predictions are made on a data entity basis and used the notions of ``context'' and its aspects such as ``belief'' to uncover and leverage future data needs. This approach allows truly opportunistic utilization of predictive information. We elaborate on which aspects of the context we are using in areas other than caching and prefetching different situations and why it is appropriate in the specified situation. We present in more details the methods we have developed, BeliefCache for data driven caching and prefetching and AVSC for pattern mining based compression of data. In BeliefCache, using a belief, an aspect of context representing an estimate of the probability that the storage element will be needed, we developed modular framework BeliefCache, to make unified informed decisions about that element or a group. For the workloads we examined we were able to capture complex non-sequential access patterns better than a state-of-the-art framework for optimizing cloud storage gateways. Moreover, our framework is also able to adjust to variations in the workload faster. It also does not require a static workload to be effective since modular framework allows for discovering and adapting to the changes in the workload. In AVSC, using an aspect of context to gauge the similarity of the events, we perform our compression by keeping relevant events intact and approximating other events. We do that in two stages. We first generate a summarization of the data, then approximately match the remaining events with the existing patterns if possible, or add the patterns to the summary otherwise. We show gains over the plain lossless compression for a specified amount of accuracy for purposes of identifying the state of the system and a clear tradeoff in between the compressibility and fidelity. In other mentioned research areas we present challenges and opportunities with the hope that will spur researchers to further examine those issues in the space of rapidly emerging data intensive applications. We also discuss the ideas how our research in other domains could be applied in our attempts to provide high performance data access.
Temple University--Theses
37
Almosallam, Ibrahim Ahmad Shang Yi. "A new adaptive framework for collaborative filtering prediction." Diss., Columbia, Mo. : University of Missouri-Columbia, 2008. http://hdl.handle.net/10355/5630.
Full textAbstract:
Thesis (M.S.)--University of Missouri-Columbia, 2008.The entire dissertation/thesis text is included in the research.pdf file; the official abstract appears in the short.pdf file (which also appears in the research.pdf); a non-technical general description, or public abstract, appears in the public.pdf file. Title from title screen of research.pdf file (viewed on August 22, 2008) Includes bibliographical references.
38
Bakhtiar, Qutub A. "Mitigating Inconsistencies by Coupling Data Cleaning, Filtering, and Contextual Data Validation in Wireless Sensor Networks." FIU Digital Commons, 2009. http://digitalcommons.fiu.edu/etd/99.
Full textAbstract:
With the advent of peer to peer networks, and more importantly sensor networks, the desire to extract useful information from continuous and unbounded streams of data has become more prominent. For example, in tele-health applications, sensor based data streaming systems are used to continuously and accurately monitor Alzheimer's patients and their surrounding environment. Typically, the requirements of such applications necessitate the cleaning and filtering of continuous, corrupted and incomplete data streams gathered wirelessly in dynamically varying conditions. Yet, existing data stream cleaning and filtering schemes are incapable of capturing the dynamics of the environment while simultaneously suppressing the losses and corruption introduced by uncertain environmental, hardware, and network conditions. Consequently, existing data cleaning and filtering paradigms are being challenged. This dissertation develops novel schemes for cleaning data streams received from a wireless sensor network operating under non-linear and dynamically varying conditions. The study establishes a paradigm for validating spatio-temporal associations among data sources to enhance data cleaning. To simplify the complexity of the validation process, the developed solution maps the requirements of the application on a geometrical space and identifies the potential sensor nodes of interest. Additionally, this dissertation models a wireless sensor network data reduction system by ascertaining that segregating data adaptation and prediction processes will augment the data reduction rates. The schemes presented in this study are evaluated using simulation and information theory concepts. The results demonstrate that dynamic conditions of the environment are better managed when validation is used for data cleaning. They also show that when a fast convergent adaptation process is deployed, data reduction rates are significantly improved. Targeted applications of the developed methodology include machine health monitoring, tele-health, environment and habitat monitoring, intermodal transportation and homeland security.
39
Huang, Xiaodi, and xhuang@turing une edu au. "Filtering, clustering and dynamic layout for graph visualization." Swinburne University of Technology, 2004. http://adt.lib.swin.edu.au./public/adt-VSWT20050428.111554.
Full textAbstract:
Graph visualization plays an increasingly important role in software engineering and
information systems. Examples include UML, E-R diagrams, database structures,
visual programming, web visualization, network protocols, molecular structures,
genome diagrams, and social structures.
Many classical algorithms for graph visualization have already been developed
over the past decades. However, these algorithms face difficulties in practice, such as
the overlapping nodes, large graph layout, and dynamic graph layout. In order to solve
these problems, this research aims to systematically address both algorithmic and
approach issues related to a novel framework that describes the process of graph
visualization applications. At the same time, all the proposed algorithms and
approaches can be applied to other situations as well.
First of all, a framework for graph visualization is described, along with a generic
approach to the graphical representation of a relational information source. As the
important parts of this framework, two main approaches, Filtering and Clustering, are
then particularly investigated to deal with large graph layouts effectively.
In order to filter 'noise' or less important nodes in a given graph, two new
methods are proposed to compute importance scores of nodes called NodeRank, and
then to control the appearances of nodes in a layout by ranking them.
Two novel algorithms for clustering graphs, KNN and SKM, are developed to
reduce visual complexity. Identifying seed nodes as initial members of clusters, both
algorithms make use of either the k-nearest neighbour search or a novel node
similarity matrix to seek groups of nodes with most affinities or similarities among
them. Such groups of relatively highly connected nodes are then replaced with
abstract nodes to form a coarse graph with reduced dimensions.
An approach called MMD to the layout of clustered graphs is provided using a
multiple-window�multiple-level display.
As for the dynamic graph layout, a new approach to removing overlapping nodes
called Force-Transfer algorithm is developed to greatly improve the classical Force-
Scan algorithm.
Demonstrating the performance of the proposed algorithms and approaches, the
framework has been implemented in a prototype called PGD. A number of
experiments as well as a case study have been carried out.
40
Zhu, Zhaochen. "Computational methods in air quality data." HKBU Institutional Repository, 2017. https://repository.hkbu.edu.hk/etd_oa/402.
Full textAbstract:
In this thesis, we have investigated several computational methods on data assimilation for air quality prediction, especially on the characteristic of sparse matrix and the underlying information of gradient in the concentration of pollutant species. In the first part, we have studied the ensemble Kalman filter (EnKF) for chemical species simulation in air quality forecast data assimilation. The main contribution of this paper is to study the sparse data observations and make use of the matrix structure of the Kalman filter updated equations to design an algorithm to compute the analysis of chemical species in the air quality forecast system efficiently. The proposed method can also handle the combined observations from multiple species together. We have applied the proposed method and tested its performance for real air quality data assimilation. Numerical examples have demonstrated the efficiency of the proposed computational method for Kalman filter update, and the effectiveness of the proposed method for NO2, NO, CO, SO2, O3, PM2.5 and PM10 in air quality data assimilation. Within the third part, we have set up an automatic workflow to connect the management system of the chemical transport model - CMAQ with our proposed data assimilation methods. The setup has successfully integrated the data assimilation into the management system and shown that the accuracy of the prediction has risen to a new level. This technique has transformed the system into a real-time and high-precision system. When the new observations are available, the predictions can then be estimated almost instantaneously. Then the agencies are able to make the decisions and respond to the situations immediately. In this way, citizens are able to protect themselves effectively. Meanwhile, it allows the mathematical algorithm to be industrialized implying that the improvements on data assimilation have directly positive effects on the developments of the environment, the human health and the society. Therefore, this has become an inspiring indication to encourage us to study, achieve and even devote more research into this promising method.
41
Landström, Anders. "Adaptive tensor-based morphological filtering and analysis of 3D profile data." Licentiate thesis, Luleå tekniska universitet, Signaler och system, 2012. http://urn.kb.se/resolve?urn=urn:nbn:se:ltu:diva-26510.
Full textAbstract:
Image analysis methods for processing 3D profile data have been investigated and developed. These methods include; Image reconstruction by prioritized incremental normalized convolution, morphology-based crack detection for steel slabs, and adaptive morphology based on the local structure tensor. The methods have been applied to a number of industrial applications.An issue with 3D profile data captured by laser triangulation is occlusion, which occurs when the line-of-sight between the projected laser light and the camera sensor is obstructed. To overcome this problem, interpolation of missing surface in rock piles has been investigated and a novel interpolation method for filling in missing pixel values iteratively from the edges of the reliable data, using normalized convolution, has been developed.3D profile data of the steel surface has been used to detect longitudinal cracks in casted steel slabs. Segmentation of the data is done using mathematical morphology, and the resulting connected regions are assigned a crack probability estimate based on a statistic logistic regression model. More specifically, the morphological filtering locates trenches in the data, excludes scale regions for further analysis, and finally links crack segments together in order to obtain a segmented region which receives a crack probability based on its depth and length.Also suggested is a novel method for adaptive mathematical morphology intended to improve crack segment linking, i.e. for bridging gaps in the crack signature in order to increase the length of potential crack segments. Standard morphology operations rely on a predefined structuring element which is repeatedly used for each pixel in the image. The outline of a crack, however, can range from a straight line to a zig-zag pattern. A more adaptive method for linking regions with a large enough estimated crack depth would therefore be beneficial. More advanced morphological approaches, such as morphological amoebas and path openings, adapt better to curvature in the image. For our purpose, however, we investigate how the local structure tensor can be used to adaptively assign to each pixel an elliptical structuring element based on the local orientation within the image. The information from the local structure tensor directly defines the shape of the elliptical structuring element, and the resulting morphological filtering successfully enhances crack signatures in the data.Godkänd; 2012; 20121017 (andlan); LICENTIATSEMINARIUM Ämne: Signalbehandling/Signal Processing Examinator: Universitetslektor Matthew Thurley, Institutionen för system- och rymdteknik, Luleå tekniska universitet Diskutant: Associate Professor Cris Luengo, Centre for Image Analysis, Uppsala Tid: Onsdag den 21 november 2012 kl 12.30 Plats: A1545, Luleå tekniska universitet
42
Adzemovic, Haris, and Alexander Sandor. "Comparison of user and item-based collaborative filtering on sparse data." Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-209445.
Full textAbstract:
Recommender systems are used extensively today in many areas to help users and consumers with making decisions. Amazon recommends books based on what you have previously viewed and purchased, Netflix presents you with shows and movies you might enjoy based on your interactions with the platform and Facebook serves personalized ads to every user based on gathered browsing information. These systems are based on shared similarities and there are several ways to develop and model them. This study compares two methods, user and item-based filtering in k nearest neighbours systems.The methods are compared on how much they deviate from the true answer when predicting user ratings of movies based on sparse data. The study showed that none of the methods could be considered objectively better than the other and that the choice of system should be based on the data set.Idag används rekommendationssystem extensivt inom flera områden för att hjälpa användare och konsumenter i deras val. Amazon rekommenderar böcker baserat på vad du tittat på och köpt, Netflix presenterar serier och filmer du antagligen kommer gilla baserat på interaktioner med plattformen och Facebook visar personaliserad, riktad reklam för varje enskild användare baserat på tidigare surfvanor. Dessa system är baserade på delade likheter och det finns flera sätt att utveckla och modellera dessa på. I denna rapport jämförs två metoder, användar- och objektbaserad filtrering i k nearest neighbours system. Metoderna jämförs på hur mycket de avviker från det sanna svaret när de försöker förutse användarbetyg på filmer baserat på gles data. Studien visade att man ej kan peka ut någon metod som objektivt bättre utan att val av metod bör baseras på datasetet.
43
Berdugo, Albert, and Louis Natale. "IRIG-106 CHAPTER 10 RECORDER WITH BUILT-IN DATA FILTERING MECHANISM." International Foundation for Telemetering, 2007. http://hdl.handle.net/10150/604523.
Full textAbstract:
ITC/USA 2007 Conference Proceedings / The Forty-Third Annual International Telemetering Conference and Technical Exhibition / October 22-25, 2007 / Riviera Hotel & Convention Center, Las Vegas, NevadaSixteen years ago, RCC added Chapter 8 to the IRIG-106 standard for the acquisition of 100% MIL-STD-1553 data from up to eight buses for recording and/or transmission. In the past 5 years, the RCC recording committee added Chapter 10 to the IRIG-106 standard for acquisition of 100% data from PCM, MIL-STD-1553 busses, Video, ARINC-429, Ethernet, IEEE-1394, and others. IRIG-106 Chapter 10 recorder suppliers have further developed customer-specific interfaces to meet additional customer needs. These needs have included unique radar and avionic bus interfaces such as F-16 Fibre Channel, F-35 Fibre Channel, F-22 FOTR, and others. IRIG-106 Chapter 8 and Chapter 10 have provided major challenges to the user community when the acquired avionics bus data included data that must be filtered and never leave the test platform via TM or recording media. The preferred method of filtering data to ensure that it is never recorded or transmitted is to do so at the interface level with the avionic busses. This paper describes the data filtering used on the F-22 Program for the MIL-STD-1553 buses and the FOTR bus as part of the IRIG-106 Chapter 10 Multiplexer/Recorder System. This filtering method blocks selected data at the interface level prior to being transferred over the system bus to the media(s). Additionally, the paper describes the configuration method for defining the data to be blocked and the report generated in order to allow for a second party to verify proper programming of the system.
44
Slocumb, Benjamin J. "Adaptive data association methods for pulse train analysis and deinterleaving." Diss., Georgia Institute of Technology, 1998. http://hdl.handle.net/1853/13392.
Full text
45
Damtie, B. (Baylie). "New incoherent scatter radar measurement techniques and data analysis methods." Doctoral thesis, Oulun yliopisto, 2004. http://urn.fi/urn:isbn:9514273125.
Full textAbstract:
Abstract This dissertation presents new incoherent scatter radar measurement techniques and data analysis methods. The measurements used in the study were collected by connecting a computer-based receiver to the EISCAT (European Incoherent SCATter) radar on Svalbard. This hardware consists of a spectrum analyzer, a PCI-bus-based programmable digital I/O card and a desktop computer with a large-capacity hard disk. It takes in the 70-MHz signal from the ESR (Eiscat Svalbard Radar) signal path and carries out down-conversion, AD conversion, quadrature detection, and finally stores the output samples effective sampling rate is 1 MHz, large enough to span all the frequency channels used in the experiment. Hence the total multichannel signal was stored instead of separate lagged products for each frequency channel, which is the procedure in the standard hardware. This solution has some benefits including elimination of ground clutter with only a small loss in statistical accuracy. The capability of our hardware in storing the incoherent scatter radar signals directly allows us to use very flexible and versatile signal processing methods, which include clutter suppression, filtering, decoding, lag prole calculation, inversion and optimal height integration. The performance of these incoherent scatter radar measurement techniques and data analysis methods are demonstrated by employing an incoherent scatter experiment that applies a new binary phase code. Each bit of this code has been further coded by a 5-bit Barker code. In the analysis, stochastic inversion has been used for the first time in decoding Barker-coded incoherent scatter measurements, and this method takes care of the ambiguity problems associated with the measurements. Finally, we present new binary phase codes with corresponding sidelobe-free decoding filters that maximize the signal-to-noise ratio (SNR) and at the same time eliminate unwanted sidelobes completelyOriginal papers The original papers are not included in the electronic version of the dissertation. Lehtinen, M., Markkanen, J., Väänänen, A., Huuskonen, A., Damtie, B., Nygrén, T., & Rahkola, J. (2002). A new incoherent scatter technique in the EISCAT Svalbard Radar. Radio Science, 37(4), 3-1-3–14. https://doi.org/10.1029/2001rs002518 Damtie, B., Nygrén, T., Lehtinen, M. S., & Huuskonen, A. (2002). High resolution observations of sporadic-E layers within the polar cap ionosphere using a new incoherent scatter radar experiment. Annales Geophysicae, 20(9), 1429–1438. https://doi.org/10.5194/angeo-20-1429-2002 Damtie, B., Lehtinen, M. S., & Nygrén, T. (2004). Decoding of Barker-coded incoherent scatter measurements by means of mathematical inversion. Annales Geophysicae, 22(1), 3–13. https://doi.org/10.5194/angeo-22-3-2004 Lehtinen, M. S., Damtie, B., & Nygrén, T. (2004). Optimal binary phase codes and sidelobe-free decoding filters with application to incoherent scatter radar. Annales Geophysicae, 22(5), 1623–1632. https://doi.org/10.5194/angeo-22-1623-2004
46
Wickramarathne, Thanuka Lakmal. "A Belief Theoretic Approach for Automated Collaborative Filtering." Scholarly Repository, 2008. http://scholarlyrepository.miami.edu/oa_theses/182.
Full textAbstract:
WICKRAMARATHNE, T. L. (M.S., Electrical and Computer Engineering) A Belief Theoretic Approach for Automated Collaborative Filtering (May 2008) Abstract of a thesis at the University of Miami. Thesis supervised by Professor Kamal Premaratne. No. of pages in text. (84) Automated Collaborative Filtering (ACF) is one of the most successful strategies available for recommender systems. Application of ACF in more sensitive and critical applications however has been hampered by the absence of better mechanisms to accommodate imperfections (ambiguities and uncertainties in ratings, missing ratings, etc.) that are inherent in user preference ratings and propagate such imperfections throughout the decision making process. Thus one is compelled to make various "assumptions" regarding the user preferences giving rise to predictions that lack sufficient integrity. With its Dempster-Shafer belief theoretic basis, CoFiDS, the automated Collaborative Filtering algorithm proposed in this thesis, can (a) represent a wide variety of data imperfections; (b) propagate the partial knowledge that such data imperfections generate throughout the decision-making process; and (c) conveniently incorporate contextual information from multiple sources. The "soft" predictions that CoFiDS generates provide substantial exibility to the domain expert. Depending on the associated DS theoretic belief-plausibility measures, the domain expert can either render a "hard" decision or narrow down the possible set of predictions to as smaller set as necessary. With its capability to accommodate data imperfections, CoFiDS widens the applicability of ACF, from the more popular domains, such as movie and book recommendations, to more sensitive and critical problem domains, such as medical expert support systems, homeland security and surveillance, etc. We use a benchmark movie dataset and a synthetic dataset to validate CoFiDS and compare it to several existing ACF systems.
47
Karaman, Hilal. "A Content Based Movie Recommendation System Empowered By Collaborative Missing Data Prediction." Master's thesis, METU, 2010. http://etd.lib.metu.edu.tr/upload/12612037/index.pdf.
Full textAbstract:
The evolution of the Internet has brought us into a world that represents a huge amount of information items such as music, movies, books, web pages, etc. with varying quality. As a result of this huge universe of items, people get confused and the question &ldquoWhich one should I choose?&rdquo
arises in their minds. Recommendation Systems address the problem of getting confused about items to choose, and filter a specific type of information with a specific information filtering technique that attempts to present information items that are likely of interest to the user. A variety of information filtering techniques have been proposed for performing recommendations, including content-based and collaborative techniques which are the most commonly used approaches in recommendation systems. This thesis work introduces ReMovender, a content-based movie recommendation system which is empowered by collaborative missing data prediction. The distinctive point of this study lies in the methodology used to correlate the users in the system with one another and the usage of the content information of movies. ReMovender makes it possible for the users to rate movies in a scale from one to five. By using these ratings, it finds similarities among the users in a collaborative manner to predict the missing ratings data. As for the content-based part, a set of movie features are used in order to correlate the movies and produce recommendations for the users.
48
Thorne, Richard L. "Asychronous [i.e. asynchronous] data fusion for AUV navigation using extended Kalman filtering." Thesis, Monterey, Calif. : Springfield, Va. : Naval Postgraduate School ; Available from National Technical Information Service, 1997. http://handle.dtic.mil/100.2/ADA331863.
Full textAbstract:
Thesis (M.S. in Mechanical Engineering) Naval Postgraduate School, March 1997.Thesis advisor(s): Healey, Anthony J. "March 1997." Includes bibliographical references (p. 151). Also available online.
49
Guevara, Saul Ernesto. "Analysis and filtering of near-surface effects in land multicomponent seismic data." Thesis, National Library of Canada = Bibliothèque nationale du Canada, 2001. http://www.collectionscanada.ca/obj/s4/f2/dsk3/ftp05/MQ64956.pdf.
Full text
50
Li, Ting. "Contributions to Mean Shift filtering and segmentation : Application to MRI ischemic data." Phd thesis, INSA de Lyon, 2012. http://tel.archives-ouvertes.fr/tel-00768315.
Full textAbstract:
Medical studies increasingly use multi-modality imaging, producing multidimensional data that bring additional information that are also challenging to process and interpret. As an example, for predicting salvageable tissue, ischemic studies in which combinations of different multiple MRI imaging modalities (DWI, PWI) are used produced more conclusive results than studies made using a single modality. However, the multi-modality approach necessitates the use of more advanced algorithms to perform otherwise regular image processing tasks such as filtering, segmentation and clustering. A robust method for addressing the problems associated with processing data obtained from multi-modality imaging is Mean Shift which is based on feature space analysis and on non-parametric kernel density estimation and can be used for multi-dimensional filtering, segmentation and clustering. In this thesis, we sought to optimize the mean shift process by analyzing the factors that influence it and optimizing its parameters. We examine the effect of noise in processing the feature space and how Mean Shift can be tuned for optimal de-noising and also to reduce blurring. The large success of Mean Shift is mainly due to the intuitive tuning of bandwidth parameters which describe the scale at which features are analyzed. Based on univariate Plug-In (PI) bandwidth selectors of kernel density estimation, we propose the bandwidth matrix estimation method based on multi-variate PI for Mean Shift filtering. We study the interest of using diagonal and full bandwidth matrix with experiment on synthesized and natural images. We propose a new and automatic volume-based segmentation framework which combines Mean Shift filtering and Region Growing segmentation as well as Probability Map optimization. The framework is developed using synthesized MRI images as test data and yielded a perfect segmentation with DICE similarity measurement values reaching the highest value of 1. Testing is then extended to real MRI data obtained from animals and patients with the aim of predicting the evolution of the ischemic penumbra several days following the onset of ischemia using only information obtained from the very first scan. The results obtained are an average DICE of 0.8 for the animal MRI image scans and 0.53 for the patients MRI image scans; the reference images for both cases are manually segmented by a team of expert medical staff. In addition, the most relevant combination of parameters for the MRI modalities is determined.