Relevant bibliographies by topics / Streaming Data Analysis / Journal articles

To see the other types of publications on this topic, follow the link: Streaming Data Analysis.

Journal articles on the topic 'Streaming Data Analysis'

Author: Grafiati

Published: 10 December 2022

Last updated: 28 January 2023

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Streaming Data Analysis.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

Chowdhury, Sugnik Roy. "Twitter Data Analysis by Live Streaming." Journal of Computational and Theoretical Nanoscience 16, no. 8 (2019): 3178–82. http://dx.doi.org/10.1166/jctn.2019.8156.

Full text

Abstract:

Streaming now a days have been of great use when comes to Social Media. Streaming of data have made it easy for Companies to understand the pros and cons of their product. Streaming acts as a survey now a days which a few years ago were done by a team of individual using pen and papers. In order to collect and process the streaming data from various streaming sites to produce an analytical report that helps to get a clear pictorial representation of events, the assets of streaming process generates a huge volume of real time data mainly referred to as “Big Data.” In order to aggregate, store and analyses the streaming data that are being generated Day-By-Day we get into the concept of Hadoop and Flume Technologies, API that helps to collect data from Twitter/other streaming sites by using “#” tag/Keywords. Tweets by the News channel and retweets by the public are being collected.

APA, Harvard, Vancouver, ISO, and other styles

Ibrahim, Omar A., Yiqing Wang, and James M. Keller. "Analysis of Incremental Cluster Validity for Big Data Applications." International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 26, Suppl. 2 (2018): 47–62. http://dx.doi.org/10.1142/s0218488518400111.

Full text

Abstract:

Online clustering has attracted attention due to the explosion of ubiquitous continuous sensing. Streaming clustering algorithms need to look for new structures and adapt as the data evolves, such that outliers are detected, and that new emerging clusters are automatically formed. The performance of a streaming clustering algorithm needs to be monitored over time to understand the behavior of the streaming data in terms of new emerging clusters and number of outlier data points. Small datasets with 2 or 3 dimensions can be monitored by plotting the clustering results as data evolves. However, as the size and dimensions of streaming data increase, plotting the clustering result becomes unfeasible. Therefore, incremental internal Validity Indices (iCVIs) could be applied for monitoring the performance of an online clustering algorithm. In this paper, we study the internal incremental Davies-Bouldin (iDB) cluster validity index in the context of big streaming data analysis. Also, we study the effect of large number of samples on the values of the iCVI (iDB). Finally, we propose a way to project streaming data into a lower space for cases where the distance measure does not perform as expected in the high dimensional space.

APA, Harvard, Vancouver, ISO, and other styles

Kim, Kyeongjoo. "Real-time Streaming Data Analysis using Spark." International Journal of Emerging Trends in Engineering Research 6, no. 1 (2018): 1–5. http://dx.doi.org/10.30534/ijeter/2018/01612018.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Guo, Shu-Hui, and Xin Lu. "Live streaming: Data mining and behavior analysis." Acta Physica Sinica 69, no. 8 (2020): 088908. http://dx.doi.org/10.7498/aps.69.20191776.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Bagui, Sikha, and Katie Jin. "A Survey of Challenges Facing Streaming Data." Transactions on Machine Learning and Artificial Intelligence 8, no. 4 (2020): 63–73. http://dx.doi.org/10.14738/tmlai.84.8579.

Full text

Abstract:

This survey performs a thorough enumeration and analysis of existing methods for data stream processing. It is a survey of the challenges facing streaming data. The challenges addressed are preprocessing of streaming data, detection and dealing with concept drifts in streaming data, data reduction in the face of data streams, approximate queries and blocking operations in streaming data.

APA, Harvard, Vancouver, ISO, and other styles

López-Lagunas, Abelardo, and Sek Chai. "Streaming Data Movement for Real-Time Image Analysis." Journal of Signal Processing Systems 62, no. 1 (2009): 29–42. http://dx.doi.org/10.1007/s11265-008-0336-x.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Matteussi, Kassiano J., Julio C. S. dos Anjos, Valderi R. Q. Leithardt, and Claudio F. R. Geyer. "Performance Evaluation Analysis of Spark Streaming Backpressure for Data-Intensive Pipelines." Sensors 22, no. 13 (2022): 4756. http://dx.doi.org/10.3390/s22134756.

Full text

Abstract:

A significant rise in the adoption of streaming applications has changed the decision-making processes in the last decade. This movement has led to the emergence of several Big Data technologies for in-memory processing, such as the systems Apache Storm, Spark, Heron, Samza, Flink, and others. Spark Streaming, a widespread open-source implementation, processes data-intensive applications that often require large amounts of memory. However, Spark Unified Memory Manager cannot properly manage sudden or intensive data surges and their related in-memory caching needs, resulting in performance and throughput degradation, high latency, a large number of garbage collection operations, out-of-memory issues, and data loss. This work presents a comprehensive performance evaluation of Spark Streaming backpressure to investigate the hypothesis that it could support data-intensive pipelines under specific pressure requirements. The results reveal that backpressure is suitable only for small and medium pipelines for stateless and stateful applications. Furthermore, it points out the Spark Streaming limitations that lead to in-memory-based issues for data-intensive pipelines and stateful applications. In addition, the work indicates potential solutions.

APA, Harvard, Vancouver, ISO, and other styles

Okan, Emmanuel Tettey. "Forensic Analysis on Streaming Multimedia." Advances in Multidisciplinary and scientific Research Journal Publication 1, no. 1 (2022): 221–26. http://dx.doi.org/10.22624/aims/crp-bk3-p36.

Full text

Abstract:

Since the advent of technology and digitalization of multimedia, there has been a massive increase in cybercrime. During streaming, with the availability of a network or internet source, multimedia; audio and visual can easily be accessed whiles being aired live. This technology dates as far back as 1990s. Similar to still videos and images, the user is able to download, pause, reverse or forward the show. The ability to stream multimedia has made it easier for users to partake or retrieve multimedia from the comfort of their homes, offices or personal spaces without necessarily being present. However, there are several challenges that affect the functionality of this technology, slow network connection and cybercrime. The issue of slow network may easily be handled by network providers, but cybercrimes has become rampant over the years. These attackers, also known as cyber criminals, use various activities to attack data. Some of their activities include phishing, data breach, identity theft and harassment. The paper has been written to assess forensic analysis of streaming multimedia. While exploring existing studies, it was realized that despite the rich availability of digital image forensics, video forensics hasn’t been explored much. This is because of the difficulty involved in analyzing the video data. Video data is always presented in a compressed form, unlike still images that are obtained in their original state. The compressed data often cancels or totally compromises the existing fingerprints, hence making it difficult to monitor or recover data. It was also revealed that, much has not been done so far as the research area is concerned. Keywords: Mobile Forensics, Cybersecurity, Streaming, Media, Video, Networks BOOK Chapter ǀ Research Nexus in IT, Law, Cyber Security & Forensics. Open Access. Distributed Free Citation: Emmanuel Tettey Okan (2022): Forensic Analysis On Streaming Multimedia Book Chapter Series on Research Nexus in IT, Law, Cyber Security & Forensics. Pp 221-226 www.isteams.net/ITlawbookchapter2022. dx.doi.org/10.22624/AIMS/CRP-BK3-P36

APA, Harvard, Vancouver, ISO, and other styles

Adewole, Kayode S., Taofeekat T. Salau-Ibrahim, Agbotiname Lucky Imoize, et al. "Empirical Analysis of Data Streaming and Batch Learning Models for Network Intrusion Detection." Electronics 11, no. 19 (2022): 3109. http://dx.doi.org/10.3390/electronics11193109.

Full text

Abstract:

Network intrusion, such as denial of service, probing attacks, and phishing, comprises some of the complex threats that have put the online community at risk. The increase in the number of these attacks has given rise to a serious interest in the research community to curb the menace. One of the research efforts is to have an intrusion detection mechanism in place. Batch learning and data streaming are approaches used for processing the huge amount of data required for proper intrusion detection. Batch learning, despite its advantages, has been faulted for poor scalability due to the constant re-training of new training instances. Hence, this paper seeks to conduct a comparative study using selected batch learning and data streaming algorithms. The batch learning and data streaming algorithms considered are J48, projective adaptive resonance theory (PART), Hoeffding tree (HT) and OzaBagAdwin (OBA). Furthermore, binary and multiclass classification problems are considered for the tested algorithms. Experimental results show that data streaming algorithms achieved considerably higher performance in binary classification problems when compared with batch learning algorithms. Specifically, binary classification produced J48 (94.73), PART (92.83), HT (98.38), and OBA (99.67), and multiclass classification produced J48 (87.66), PART (87.05), HT (71.98), OBA (82.80) based on accuracy. Hence, the use of data streaming algorithms to solve the scalability issue and allow real-time detection of network intrusion is highly recommended.

APA, Harvard, Vancouver, ISO, and other styles

Lee, Heung Ki, Jaehee Jung, Kyung Jin Ahn, Hwa-Young Jeong, and Gangman Yi. "Numeric Analysis for Relationship-Aware Scalable Streaming Scheme." Journal of Applied Mathematics 2014 (2014): 1–12. http://dx.doi.org/10.1155/2014/195781.

Full text

Abstract:

Frequent packet loss of media data is a critical problem that degrades the quality of streaming services over mobile networks. Packet loss invalidates frames containing lost packets and other related frames at the same time. Indirect loss caused by losing packets decreases the quality of streaming. A scalable streaming service can decrease the amount of dropped multimedia resulting from a single packet loss. Content providers typically divide one large media stream into several layers through a scalable streaming service and then provide each scalable layer to the user depending on the mobile network. Also, a scalable streaming service makes it possible to decode partial multimedia data depending on the relationship between frames and layers. Therefore, a scalable streaming service provides a way to decrease the wasted multimedia data when one packet is lost. However, the hierarchical structure between frames and layers of scalable streams determines the service quality of the scalable streaming service. Even if whole packets of layers are transmitted successfully, they cannot be decoded as a result of the absence of reference frames and layers. Therefore, the complicated relationship between frames and layers in a scalable stream increases the volume of abandoned layers. For providing a high-quality scalable streaming service, we choose a proper relationship between scalable layers as well as the amount of transmitted multimedia data depending on the network situation. We prove that a simple scalable scheme outperforms a complicated scheme in an error-prone network. We suggest an adaptive set-top box (AdaptiveSTB) to lower the dependency between scalable layers in a scalable stream. Also, we provide a numerical model to obtain the indirect loss of multimedia data and apply it to various multimedia streams. Our AdaptiveSTB enhances the quality of a scalable streaming service by removing indirect loss.

APA, Harvard, Vancouver, ISO, and other styles

Kube, R., R. M. Churchill, C. S. Chang, et al. "Near real-time streaming analysis of big fusion data." Plasma Physics and Controlled Fusion 64, no. 3 (2022): 035015. http://dx.doi.org/10.1088/1361-6587/ac3f42.

Full text

Abstract:

Abstract Experiments on fusion plasmas produce high-dimensional data time series with ever-increasing magnitude and velocity, but turn-around times for analysis of this data have not kept up. For example, many data analysis tasks are often performed in a manual, ad-hoc manner some time after an experiment. In this article, we introduce the Delta framework that facilitates near real-time streaming analysis of big and fast fusion data. By streaming measurement data from fusion experiments to a high-performance compute center, Delta allows computationally expensive data analysis tasks to be performed in between plasma pulses. This article describes the modular and expandable software architecture of Delta and presents performance benchmarks of individual components as well as of an example workflow. Focusing on a streaming analysis workflow where electron cyclotron emission imaging (ECEi) data is measured at KSTAR on the National Energy Research Scientific Computing Center’s (NERSC’s) supercomputer we routinely observe data transfer rates of about 4 Gigabit per second. In NERSC, a demanding turbulence analysis workflow effectively utilizes multiple nodes and graphical processing units and executes them in under 5 min. We further discuss how Delta uses modern database systems and container orchestration services to provide web-based real-time data visualization. For the case of ECEi data we demonstrate how data visualizations can be augmented with outputs from machine learning models. By providing session leaders and physics operators, results of higher-order data analysis using live visualizations may make more informed decisions on how to configure the machine for the next shot.

APA, Harvard, Vancouver, ISO, and other styles

Issa, Shadi A., Romeo Kienzler, Mohamed El-Kalioby, et al. "Streaming Support for Data Intensive Cloud-Based Sequence Analysis." BioMed Research International 2013 (2013): 1–16. http://dx.doi.org/10.1155/2013/791051.

Full text

Abstract:

Cloud computing provides a promising solution to the genomics data deluge problem resulting from the advent of next-generation sequencing (NGS) technology. Based on the concepts of “resources-on-demand” and “pay-as-you-go”, scientists with no or limited infrastructure can have access to scalable and cost-effective computational resources. However, the large size of NGS data causes a significant data transfer latency from the client’s site to the cloud, which presents a bottleneck for using cloud computing services. In this paper, we provide a streaming-based scheme to overcome this problem, where the NGS data is processed while being transferred to the cloud. Our scheme targets the wide class of NGS data analysis tasks, where the NGS sequences can be processed independently from one another. We also provide theelastreampackage that supports the use of this scheme with individual analysis programs or with workflow systems. Experiments presented in this paper show that our solution mitigates the effect of data transfer latency and saves both time and cost of computation.

APA, Harvard, Vancouver, ISO, and other styles

Zeng, Xue-Qiang, and Guo-Zheng Li. "Incremental partial least squares analysis of big streaming data." Pattern Recognition 47, no. 11 (2014): 3726–35. http://dx.doi.org/10.1016/j.patcog.2014.05.022.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Law, Jonathan, and Darren J. Wilkinson. "Composable models for online Bayesian analysis of streaming data." Statistics and Computing 28, no. 6 (2017): 1119–37. http://dx.doi.org/10.1007/s11222-017-9783-1.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Thomas, Mathew, Kerstin Kleese-van Dam, Matthew J. Marshall, et al. "Towards Adaptive, Streaming Analysis of X-ray Tomography Data." Synchrotron Radiation News 28, no. 2 (2015): 10–14. http://dx.doi.org/10.1080/08940886.2015.1013414.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Fejfar, Jiří, Jiří Šťastný, Martin Pokorný, Jiří Balej, and Petr Zach. "Analysis of sound data streamed over the network." Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis 61, no. 7 (2013): 2105–10. http://dx.doi.org/10.11118/actaun201361072105.

Full text

Abstract:

In this paper we inspect a difference between original sound recording and signal captured after streaming this original recording over a network loaded with a heavy traffic. There are several kinds of failures occurring in the captured recording caused by network congestion. We try to find a method how to evaluate correctness of streamed audio. Usually there are metrics based on a human perception of a signal such as “signal is clear, without audible failures”, “signal is having some failures but it is understandable”, or “signal is inarticulate”. These approaches need to be statistically evaluated on a broad set of respondents, which is time and resource consuming. We try to propose some metrics based on signal properties allowing us to compare the original and captured recording. We use algorithm called Dynamic Time Warping (Müller, 2007) commonly used for time series comparison in this paper. Some other time series exploration approaches can be found in (Fejfar, 2011) and (Fejfar, 2012). The data was acquired in our network laboratory simulating network traffic by downloading files, streaming audio and video simultaneously. Our former experiment inspected Quality of Service (QoS) and its impact on failures of received audio data stream. This experiment is focused on the comparison of sound recordings rather than network mechanism.We focus, in this paper, on a real time audio stream such as a telephone call, where it is not possible to stream audio in advance to a “pool”. Instead it is necessary to achieve as small delay as possible (between speaker voice recording and listener voice replay). We are using RTP protocol for streaming audio.

APA, Harvard, Vancouver, ISO, and other styles

Xiao, Wen, and Juan Hu. "SWEclat: a frequent itemset mining algorithm over streaming data using Spark Streaming." Journal of Supercomputing 76, no. 10 (2020): 7619–34. http://dx.doi.org/10.1007/s11227-020-03190-5.

Full text

Abstract:

Abstract Finding frequent itemsets in a continuous streaming data is an important data mining task which is widely used in network monitoring, Internet of Things data analysis and so on. In the era of big data, it is necessary to develop a distributed frequent itemset mining algorithm to meet the needs of massive streaming data processing. Apache Spark is a unified analytic engine for massive data processing which has been successfully used in many data mining fields. In this paper, we propose a distributed algorithm for mining frequent itemsets over massive streaming data named SWEclat. The algorithm uses sliding window to process streaming data and uses vertical data structure to store the dataset in the sliding window. This algorithm is implemented by Apache Spark and uses Spark RDD to store streaming data and dataset in vertical data format, so as to divide these RDDs into partitions for distributed processing. Experimental results show that SWEclat algorithm has good acceleration, parallel scalability and load balancing.

APA, Harvard, Vancouver, ISO, and other styles

Tu, Doan Quang, A. S. M. Kayes, Wenny Rahayu, and Kinh Nguyen. "IoT streaming data integration from multiple sources." Computing 102, no. 10 (2020): 2299–329. http://dx.doi.org/10.1007/s00607-020-00830-9.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Syamsuddin, Irfan, Rini Nur, Hafsah Nirwana, Ibrahim Abduh, and David Al-Dabass. "Decision Making Analysis of Video Streaming Algorithm for Private Cloud Computing Infrastructure." International Journal of Electrical and Computer Engineering (IJECE) 7, no. 6 (2017): 3529. http://dx.doi.org/10.11591/ijece.v7i6.pp3529-3535.

Full text

Abstract:

The issue on how to effectively deliver video streaming contents over cloud computing infrastructures is tackled in this study. Basically, quality of service of video streaming is strongly influenced by bandwidth, jitter and data loss problems. A number of intelligent video streaming algorithms are proposed by using different techniques to deal with such issues. This study aims to propose and demonstrate a novel decision making analysis which combines ISO 9126 (international standard for software engineering) and Analytic Hierarchy Process to help experts selecting the best video streaming algorithm for the case of private cloud computing infrastructure. The given case study concluded that Scalable Streaming algorithm is the best algorithm to be implemented for delivering high quality of service of video streaming over the private cloud computing infrastructure.

APA, Harvard, Vancouver, ISO, and other styles

Li, Ping, Jiashi Feng, Xiaojie Jin, Luming Zhang, Xianghua Xu, and Shuicheng Yan. "Online Robust Low-Rank Tensor Modeling for Streaming Data Analysis." IEEE Transactions on Neural Networks and Learning Systems 30, no. 4 (2019): 1061–75. http://dx.doi.org/10.1109/tnnls.2018.2860964.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Alwaisi, Shaimaa Safaa Ahmed, Maan Nawaf Abbood, Luma Fayeq Jalil, et al. "A Review on Big Data Stream Processing Applications: Contributions, Benefits, and Limitations." JOIV : International Journal on Informatics Visualization 5, no. 4 (2021): 456. http://dx.doi.org/10.30630/joiv.5.4.737.

Full text

Abstract:

The amount of data in our world has been rapidly keep growing from time to time. In the era of big data, the efficient processing and analysis of big data using machine learning algorithm is highly required, especially when the data comes in form of streams. There is no doubt that big data has become an important source of information and knowledge in making decision process. Nevertheless, dealing with this kind of data comes with great difficulties; thus, several techniques have been used in analyzing the data in the form of streams. Many techniques have been proposed and studied to handle big data and give decisions based on off-line batch analysis. Today, we need to make a constructive decision based on online streaming data analysis. Many researchers in recent years proposed some different kind of frameworks for processing the big data streaming. In this work, we explore and present in detail some of the recent achievements in big data streaming in term of contributions, benefits, and limitations. As well as some of recent platforms suitable to be used for big data streaming analytics. Moreover, we also highlight several issues that will be faced in big data stream processing. In conclusion, it is hoped that this study will assist the researchers in choosing the best and suitable framework for big data streaming projects.

APA, Harvard, Vancouver, ISO, and other styles

Montenegro-Burke, J. Rafael, Aries E. Aisporna, H. Paul Benton, et al. "Data Streaming for Metabolomics: Accelerating Data Processing and Analysis from Days to Minutes." Analytical Chemistry 89, no. 2 (2017): 1254–59. http://dx.doi.org/10.1021/acs.analchem.6b03890.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Akilandeswari, P., R. Harshita, and Sumanth KO.M. "Sentiment Analysis using Machine Learning through Twitter Streaming API." International Journal of Engineering & Technology 7, no. 3.12 (2018): 1168. http://dx.doi.org/10.14419/ijet.v7i3.12.17781.

Full text

Abstract:

Social media allows to share the experiences with many best suggestions and provides opportunities to share the ideas about any topics at any time. In the current trending, twitter is used to gather different kinds of information as user need and it is a social network service which enables the user for better communication and gaining of knowledge. Accurate representation of the user interactions can be done based on the facts sematic content. The pre-processed tweets which are stored in database are been identified and classified whether it relates to the user keywords related posts. The best suggestion using polarity can be predicted using the user keywords. For the interactive automatic system which predicts the tweets posted by the user this system deals with the challenges that appears during the sentimental analysis. It deals with effective study prior to the subjective information. The basic task in this is to identify the polarity of a given tweet in the sentence whether it is positive, negative or neutral. However the polarity of the tweets has been identified, it was difficult for us to check with the meaningless data. To address this challenge the extracted tweets are been pre-processed by replacing the full form instead of short term words. The better performance can be achieved using more training data. However the analysis was frequently done using the previously stored data, it was a challenging task to do it using the streaming data. There are very few works related to the sentiment analysis using online streaming data. In this paper, we propose that the sentiment analysis can be improved using the online streaming data. For online streaming data all the data related to the given topic will be collected according to the current data in the twitter. For better up-to-date analysis, the streaming data is used and can achieve better results. In contrast by conducting the continuous learning from the streaming data, this approach provides better results than the traditional way of using the training data and it achieves the overall performance and computational efficiency.

APA, Harvard, Vancouver, ISO, and other styles

Foster, Anita K., and Gene R. Springs. "Running up the hill – long-term streaming video pilots: process, analysis and outcomes." Collection and Curation 41, no. 2 (2022): 37–46. http://dx.doi.org/10.1108/cc-12-2020-0046.

Full text

Abstract:

Purpose Academic libraries are struggling to support the growing demand for streaming video. The purpose of this paper is to detail the experience of running three long-term pilots with different streaming video platforms, including processes involved, lessons learned and next steps. Design/methodology/approach This paper uses a mixed methods approach, combining analysis of usage data with case study observations. Findings The length of the pilots allowed for deep understanding of the needs of this academic library’s community’s engagement with streaming video in the classroom, and confirmed anecdotal information that availability of multiple platforms supports diverse needs which led to continuing access to all platforms, operationalized to be managed within existing processes. Using usage data and feedback from a task force led to decisions to continue with all three platforms that were piloted. Research limitations/implications While this research describes the experience at one academic library, the information may be generalizable enough that other libraries may use it for their streaming video collection development decisions. Originality/value Long-term pilot studies for streaming video platforms can be challenging for many libraries to undertake. With a modest initial financial commitment, the library was able to explore how the community might use streaming video. Through analysis of usage data, the library was able to see when, where and what was being used and could make better informed decisions about where to concentrate future funds for streaming video support.

APA, Harvard, Vancouver, ISO, and other styles

Lin, Qiang, and Xilin Zhang. "Key Technologies of Media Big Data in-Depth Analysis System Based on 5G Platform." Journal of Physics: Conference Series 2294, no. 1 (2022): 012007. http://dx.doi.org/10.1088/1742-6596/2294/1/012007.

Full text

Abstract:

Abstract To meet the needs of large-scale users for personalized streaming media services with high speed, low delay, and high quality in a 5G mobile network environment, this paper studies the resource allocation mechanism of streaming media based on a 5G network from the perspective of user demand prediction, which can alleviate the pressure of mobile network, improve the utilization rate of streaming media resources and the quality of user service experience. The augmented reality visualization of large-scale social media data must rely on the computing power of distributed clusters. This paper constructs a distributed parallel processing framework in a high-performance cluster environment, which adopts a loosely coupled organizational structure. Each module can be combined, called, and expanded arbitrarily under the condition of following a unified interface. In this paper, the algebraic method of parallel computing algorithm is innovatively proposed to describe parallel processing tasks and organize and call large-scale data-parallel processing operators, which effectively supports the business requirements of large-scale parallel processing of large-scale spatial social media data and solves the bottleneck of large-scale spatial social media data-parallel processing.

APA, Harvard, Vancouver, ISO, and other styles

Hu, Zhigang, Hui Kang, and Meiguang Zheng. "Stream Data Load Prediction for Resource Scaling Using Online Support Vector Regression." Algorithms 12, no. 2 (2019): 37. http://dx.doi.org/10.3390/a12020037.

Full text

Abstract:

A distributed data stream processing system handles real-time, changeable and sudden streaming data load. Its elastic resource allocation has become a fundamental and challenging problem with a fixed strategy that will result in waste of resources or a reduction in QoS (quality of service). Spark Streaming as an emerging system has been developed to process real time stream data analytics by using micro-batch approach. In this paper, first, we propose an improved SVR (support vector regression) based stream data load prediction scheme. Then, we design a spark-based maximum sustainable throughput of time window (MSTW) performance model to find the optimized number of virtual machines. Finally, we present a resource scaling algorithm TWRES (time window resource elasticity scaling algorithm) with MSTW constraint and streaming data load prediction. The evaluation results show that TWRES could improve resource utilization and mitigate SLA (service level agreement) violation.

APA, Harvard, Vancouver, ISO, and other styles

Alkwai, Lamia, Abdelfettah Belghith, Achraf Gazdar, and Saad Al-Ahmadi. "Comparative Analysis of Producer Mobility Management Approaches in Named Data Networking." Applied Sciences 12, no. 24 (2022): 12581. http://dx.doi.org/10.3390/app122412581.

Full text

Abstract:

Seamless management of producer mobility in named data networks (NDNs) has become an inherent requirement to satisfy the ever-increasing number of mobile user devices and the streaming of widespread real-time multimedia content. In this paper, we first classify the various producer mobility management (MM) schemes into four different approaches. Then, we select a representative scheme from each approach and conduct a comparative analysis between them to suggest the most suitable producer MM approach for a broad class of latency sensitive applications, such as video and audio streaming and broadcasting over NDNs. To assess and compare the efficiency and effectiveness of the representative schemes, we implemented them in the NDN defacto NdnSIM simulator and used the same network scenarios and mobility settings. The results show the superiority of the producer MM scheme that follows the data plane-based approach, which yielded lower data loss rates, lower data delivery delays and lower signaling overheads.

APA, Harvard, Vancouver, ISO, and other styles

Lv, Jia-Ke, Yang Li, and Xuan Wang. "Log Data Real Time Analysis Using Big Data Analytic Framework with Storm and Hadoop." MATEC Web of Conferences 246 (2018): 03009. http://dx.doi.org/10.1051/matecconf/201824603009.

Full text

Abstract:

The log data real-time processing platform which is built using Storm On YARN integrated MapReduce and Storm that use MapReduce to complete large-scale off-line data global knowledge extraction, sudden knowledge extraction of small-scale data in Kafka buffers through Storm, and continuous real-time calculation of streaming data in combination with global knowledge. We tested our technique with the well-known KDD99 CUP data set. The experimentation results prove the system to be effective and efficient.

APA, Harvard, Vancouver, ISO, and other styles

Kompalli, Prasanna Lakshmi, and Ramesh Kumar Cherku. "Efficient Mining of Data Streams Using Associative Classification Approach." International Journal of Software Engineering and Knowledge Engineering 25, no. 03 (2015): 605–31. http://dx.doi.org/10.1142/s0218194015500059.

Full text

Abstract:

Data stream associative classification poses many challenges to the data mining community. In this paper, we address four major challenges posed, namely, infinite length, extraction of knowledge with single scan, processing time, and accuracy. Since data streams are infinite in length, it is impractical to store and use all the historical data for training. Mining such streaming data for knowledge acquisition is a unique opportunity and even a tough task. A streaming algorithm must scan data once and extract knowledge. While mining data streams, processing time, and accuracy have become two important aspects. In this paper, we propose PSTMiner which considers the nature of data streams and provides an efficient classifier for predicting the class label of real data streams. It has greater potential when compared with many existing classification techniques. Additionally, we propose a compact novel tree structure called PSTree (Prefix Streaming Tree) for storing data. Extensive experiments conducted on 24 real datasets from UCI repository and synthetic datasets from MOA (Massive Online Analysis) show that PSTMiner is consistent. Empirical results show that performance of PSTMiner is highly competitive in terms of accuracy and performance time when compared with other approaches under windowed streaming model.

APA, Harvard, Vancouver, ISO, and other styles

Lishchytovych, Andriy, Volodymyr Pavlenko, Alexander Shmatok, and Yuriy Finenko. "COMPARATIVE ANALYSIS OF SYSTEM LOGS AND STREAMING DATA ANOMALY DETECTION ALGORITHMS." Information systems and technologies security, no. 1 (2) (2020): 5–7. http://dx.doi.org/10.17721/ists.2020.1.50-59.

Full text

Abstract:

This paper provides with the description, comparative analysis of multiple commonly used approaches of the analysis of system logs, and streaming data massively generated by company IT infrastructure with an unattended anomaly detection feature. An importance of the anomaly detection is dictated by the growing costs of system downtime due to the events that would have been predicted based on the log entries with the abnormal data reported. Anomaly detection systems are built using standard workflow of the data collection, parsing, information extraction and detection steps. Most of the document is related to the anomaly detection step and algorithms like regression, decision tree, SVM, clustering, principal components analysis, invariants mining and hierarchical temporal memory model. Model-based anomaly algorithms and hierarchical temporary memory algorithms were used to process HDFS, BGL and NAB datasets with ~16m log messages and 365k data points of the streaming data. The data was manually labeled to enable the training of the models and accuracy calculation. According to the results, supervised anomaly detection systems achieve high precision but require significant training effort, while HTM-based algorithm shows the highest detection precision with zero training. Detection of the abnormal system behavior plays an important role in large-scale incident management systems. Timely detection allows IT administrators to quickly identify issues and resolve them immediately. This approach reduces the system downtime dramatically.Most of the IT systems generate logs with the detailed information of the operations. Therefore, the logs become an ideal data source of the anomaly detection solutions. The volume of the logs makes it impossible to analyze them manually and requires automated approaches.

APA, Harvard, Vancouver, ISO, and other styles

Baig, Mirza Uzair, Lei Yu, Zixiang Xiong, Anders Host-Madsen, Houqiang Li, and Weiping Li. "On the Energy-Delay Tradeoff in Streaming Data: Finite Blocklength Analysis." IEEE Transactions on Information Theory 66, no. 3 (2020): 1861–81. http://dx.doi.org/10.1109/tit.2019.2954347.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Nair, Lekha R., Sujala D. Shetty, and Siddhant Deepak Shetty. "Streaming Big Data Analysis for Real-Time Sentiment based Targeted Advertising." International Journal of Electrical and Computer Engineering (IJECE) 7, no. 1 (2017): 402. http://dx.doi.org/10.11591/ijece.v7i1.pp402-407.

Full text

Abstract:

Big Data constituting from the information shared in the various social network sites have great relevance for research to be applied in diverse fields like marketing, politics, health or disaster management. Social network sites like Facebook and Twitter are now extensively used for conducting business, marketing products and services and collecting opinions and feedbacks regarding the same. Since data gathered from these sites regarding a product/brand are up-to-date and are mostly supplied voluntarily, it tends to be more realistic, massive and reflects the general public opinion. Its analysis on real time can lead to accurate insights and responding to the results sooner is undoubtedly advantageous than responding later. In this paper, a cloud based system for real time targeted advertising based on tweet sentiment analysis is designed and implemented using the big data processing engine Apache Spark, utilizing its streaming library. Application is meant to promote cross selling and provide better customer support.

APA, Harvard, Vancouver, ISO, and other styles

Das, Sushree, Ranjan Kumar Behera, Mukesh kumar, and Santanu Kumar Rath. "Real-Time Sentiment Analysis of Twitter Streaming data for Stock Prediction." Procedia Computer Science 132 (2018): 956–64. http://dx.doi.org/10.1016/j.procs.2018.05.111.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Hollocombe, Jonathan, Shaun de Witt, Ivan Lupelli, David Muir, and Rob Akers. "Soft real-time analysis of ITER magnetics streaming data using SPECTRE." Fusion Engineering and Design 123 (November 2017): 869–72. http://dx.doi.org/10.1016/j.fusengdes.2017.03.113.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Nagarajan, Senthil Murugan, and Usha Devi Gandhi. "Classifying streaming of Twitter data based on sentiment analysis using hybridization." Neural Computing and Applications 31, no. 5 (2018): 1425–33. http://dx.doi.org/10.1007/s00521-018-3476-3.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Virani, Nurali, Devesh K. Jha, Asok Ray, and Shashi Phoha. "Sequential hypothesis tests for streaming data via symbolic time-series analysis." Engineering Applications of Artificial Intelligence 81 (May 2019): 234–46. http://dx.doi.org/10.1016/j.engappai.2019.02.015.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Agarwal, Sachin, Jatinder Pal Singh, and Shruti Dube. "Analysis and Implementation of Gossip-Based P2P Streaming with Distributed Incentive Mechanisms for Peer Cooperation." Advances in Multimedia 2007 (2007): 1–12. http://dx.doi.org/10.1155/2007/84150.

Full text

Abstract:

Peer-to-peer (P2P) systems are becoming a popular means of streaming audio and video content but they are prone to bandwidth starvation if selfish peers do not contribute bandwidth to other peers. We prove that an incentive mechanism can be created for a live streaming P2P protocol while preserving the asymptotic properties of randomized gossip-based streaming. In order to show the utility of our result, we adapt a distributed incentive scheme from P2P file storage literature to the live streaming scenario. We provide simulation results that confirm the ability to achieve a constant download rate (in time, per peer) that is needed for streaming applications on peers. The incentive scheme fairly differentiates peers' download rates according to the amount of useful bandwidth they contribute back to the P2P system, thus creating a powerful quality-of-service incentive for peers to contribute bandwidth to other peers. We propose a functional architecture and protocol format for a gossip-based streaming system with incentive mechanisms, and present evaluation data from a real implementation of a P2P streaming application.

APA, Harvard, Vancouver, ISO, and other styles

Cormode, Graham, Zohar Karnin, Edo Liberty, Justin Thaler, and Pavel Vesely. "Relative Error Streaming Quantiles." ACM SIGMOD Record 51, no. 1 (2022): 69–76. http://dx.doi.org/10.1145/3542700.3542717.

Full text

Abstract:

Estimating ranks, quantiles, and distributions over streaming data is a central task in data analysis and monitoring. Given a stream of n items from a data universe equipped with a total order, the task is to compute a sketch (data structure) of size polylogarithmic in n. Given the sketch and a query item y, one should be able to approximate its rank in the stream, i.e., the number of stream elements smaller than or equal to y.

APA, Harvard, Vancouver, ISO, and other styles

Dixit, Y., Maria Casado, R. Cama, P. J. Cullen, and Carl Sullivan. "Near Infrared Data Analysis Using R: Live Streaming Graph Generation and Processed Data Visualisation." NIR news 26, no. 5 (2015): 15–17. http://dx.doi.org/10.1255/nirn.1544.

Full text

APA, Harvard, Vancouver, ISO, and other styles

da Silva, Daniel V. C., Antonio A. de A. Rocha, and Pedro B. Velloso. "Mobile vs. Non-Mobile Live-Streaming: A Comparative Analysis of Users Engagement and Interruption Using Big Data from a Large CDN Perspective." Sensors 21, no. 16 (2021): 5616. http://dx.doi.org/10.3390/s21165616.

Full text

Abstract:

Video streaming on the Internet is constantly changing and growing. New devices and new video delivery mechanisms generate huge gaps in the understanding of how video application works. From exploratory research of one among the largest streaming services in Brazil, this work presents a comparison between mobile and non-mobile users, in large-scale lives. This work focuses on metrics such as engagement, interruption, churn, and payload. This work also presents a report-overview of mobile-users, considering the operating system, geolocation, network access, interruption, and engagement. These results might offer potential information for streaming improvement, in addition to serving as a historical mark.

APA, Harvard, Vancouver, ISO, and other styles

Zaki, Nashwan Dheyaa, Nada Yousif Hashim, Yasmin Makki Mohialden, Mostafa Abdulghafoor Mohammed, Tole Sutikno, and Ahmed Hussein Ali. "A real-time big data sentiment analysis for iraqi tweets using spark streaming." Bulletin of Electrical Engineering and Informatics 9, no. 4 (2020): 1411–19. http://dx.doi.org/10.11591/eei.v9i4.1897.

Full text

Abstract:

The scale of data streaming in social networks, such as Twitter, is increasing exponentially. Twitter is one of the most important and suitable big data sources for machine learning research in terms of analysis, prediction, extract knowledge, and opinions. People use Twitter platform daily to express their opinion which is a fundamental fact that influence their behaviors. In recent years, the flow of Iraqi dialect has been increased, especially on the Twitter platform. Sentiment analysis for different dialects and opinion mining has become a hot topic in data science researches. In this paper, we will attempt to develop a real-time analytic model for sentiment analysis and opinion mining to Iraqi tweets using spark streaming, also create a dataset for researcher in this field. The Twitter handle Bassam AlRawi is the case study here. The new method is more suitable in the current day machine learning applications and fast online prediction.

APA, Harvard, Vancouver, ISO, and other styles

Yu, Kangqing, Wei Shi, and Nicola Santoro. "Designing a Streaming Algorithm for Outlier Detection in Data Mining—An Incrementa Approach." Sensors 20, no. 5 (2020): 1261. http://dx.doi.org/10.3390/s20051261.

Full text

Abstract:

To design an algorithm for detecting outliers over streaming data has become an important task in many common applications, arising in areas such as fraud detections, network analysis, environment monitoring and so forth. Due to the fact that real-time data may arrive in the form of streams rather than batches, properties such as concept drift, temporal context, transiency, and uncertainty need to be considered. In addition, data processing needs to be incremental with limited memory resource, and scalable. These facts create big challenges for existing outlier detection algorithms in terms of their accuracies when they are implemented in an incremental fashion, especially in the streaming environment. To address these problems, we first propose C_KDE_WR, which uses sliding window and kernel function to process the streaming data online, and reports its results demonstrating high throughput on handling real-time streaming data, implemented in a CUDA framework on Graphics Processing Unit (GPU). We also present another algorithm, C_LOF, based on a very popular and effective outlier detection algorithm called Local Outlier Factor (LOF) which unfortunately works only on batched data. Using a novel incremental approach that compensates the drawback of high complexity in LOF, we show how to implement it in a streaming context and to obtain results in a timely manner. Like C_KDE_WR, C_LOF also employs sliding-window and statistical-summary to help making decision based on the data in the current window. It also addresses all those challenges of streaming data as addressed in C_KDE_WR. In addition, we report the comparative evaluation on the accuracy of C_KDE_WR with the state-of-the-art SOD_GPU using Precision, Recall and F-score metrics. Furthermore, a t-test is also performed to demonstrate the significance of the improvement. We further report the testing results of C_LOF on different parameter settings and drew ROC and PR curve with their area under the curve (AUC) and Average Precision (AP) values calculated respectively. Experimental results show that C_LOF can overcome the masquerading problem, which often exists in outlier detection on streaming data. We provide complexity analysis and report experiment results on the accuracy of both C_KDE_WR and C_LOF algorithms in order to evaluate their effectiveness as well as their efficiencies.

APA, Harvard, Vancouver, ISO, and other styles

Minartz, Koen, Jens E. d'Hondt, and Odysseas Papapetrou. "Multivariate correlations discovery in static and streaming data." Proceedings of the VLDB Endowment 15, no. 6 (2022): 1266–78. http://dx.doi.org/10.14778/3514061.3514072.

Full text

Abstract:

Correlation analysis is an invaluable tool in many domains, for better understanding data and extracting salient insights. Most works to date focus on detecting high pairwise correlations. A generalization of this problem with known applications but no known efficient solutions involves the discovery of strong multivariate correlations, i.e., finding vectors (typically in the order of 3 to 5 vectors) that exhibit a strong dependence when considered altogether. In this work we propose algorithms for detecting multivariate correlations in static and streaming data. Our algorithms, which rely on novel theoretical results, support two different correlation measures, and allow for additional constraints. Our extensive experimental evaluation examines the properties of our solution and demonstrates that our algorithms outperform the state-of-the-art, typically by an order of magnitude.

APA, Harvard, Vancouver, ISO, and other styles

Gong, Xiaoshi. "Analysis and Visualization of Streaming Media Platforms Based on the R Language——Take Netflix as An Example." Journal of Education, Humanities and Social Sciences 4 (November 17, 2022): 199–202. http://dx.doi.org/10.54097/ehss.v4i.2766.

Full text

Abstract:

[Objective] This paper aims at finding the business content provided by Netflix through data analysis and discovering the conditions of the industry. [Methods] Netflix is selected as representatives of video platforms. R language is the core tool used for data analyzing to show various characteristics in content provision. [Results] Today's streaming media technology continues to develop rapidly, and many streaming media platforms focus on the richness of content. [Conclusion] Mastering the content of resources helps to achieve a more accurate recommendation, so as to retain users, and it can also be a reference for the development of streaming media in China.

APA, Harvard, Vancouver, ISO, and other styles

Hiriyannaiah, Srinidhi, G. M. Siddesh, and K. G. Srinivasa. "Real-Time Streaming Data Analysis Using a Three-Way Classification Method for Sentimental Analysis." International Journal of Information Technology and Web Engineering 13, no. 3 (2018): 99–111. http://dx.doi.org/10.4018/ijitwe.2018070107.

Full text

Abstract:

This article describes how recent advances in computing have led to an increase in the generation of data in fields such as social media, medical, power and others. With the rapid increase in internet users, social media has given power for sentiment analysis or opinion mining. It is a highly challenging task for storing, querying and analyzing such types of data. This article aims at providing a solution to store, query and analyze streaming data using Apache Kafka as the platform and twitter data as an example for analysis. A three-way classification method is proposed for sentimental analysis of twitter data that combines both the approaches for knowledge-based and machine-learning using three stages namely emotion classification, word classification and sentiment classification. The hybrid three-way classification approach was evaluated using a sample of five query strings on twitter and compared with existing emotion classifier, polarity classifier and Naïve Bayes classifier for sentimental analysis. The accuracy of the results of the proposed approach is superior when compared to existing approaches.

APA, Harvard, Vancouver, ISO, and other styles

Zhang, Xiongwei, Hager Saleh, Eman M. G. Younis, Radhya Sahal, and Abdelmgeid A. Ali. "Predicting Coronavirus Pandemic in Real-Time Using Machine Learning and Big Data Streaming System." Complexity 2020 (December 19, 2020): 1–10. http://dx.doi.org/10.1155/2020/6688912.

Full text

Abstract:

Twitter is a virtual social network where people share their posts and opinions about the current situation, such as the coronavirus pandemic. It is considered the most significant streaming data source for machine learning research in terms of analysis, prediction, knowledge extraction, and opinions. Sentiment analysis is a text analysis method that has gained further significance due to social networks’ emergence. Therefore, this paper introduces a real-time system for sentiment prediction on Twitter streaming data for tweets about the coronavirus pandemic. The proposed system aims to find the optimal machine learning model that obtains the best performance for coronavirus sentiment analysis prediction and then uses it in real-time. The proposed system has been developed into two components: developing an offline sentiment analysis and modeling an online prediction pipeline. The system has two components: the offline and the online components. For the offline component of the system, the historical tweets’ dataset was collected in duration 23/01/2020 and 01/06/2020 and filtered by #COVID-19 and #Coronavirus hashtags. Two feature extraction methods of textual data analysis were used, n-gram and TF-ID, to extract the dataset’s essential features, collected using coronavirus hashtags. Then, five regular machine learning algorithms were performed and compared: decision tree, logistic regression, k-nearest neighbors, random forest, and support vector machine to select the best model for the online prediction component. The online prediction pipeline was developed using Twitter Streaming API, Apache Kafka, and Apache Spark. The experimental results indicate that the RF model using the unigram feature extraction method has achieved the best performance, and it is used for sentiment prediction on Twitter streaming data for coronavirus.

APA, Harvard, Vancouver, ISO, and other styles

Friedrich, Johannes, Andrea Giovannucci, and Eftychios A. Pnevmatikakis. "Online analysis of microendoscopic 1-photon calcium imaging data streams." PLOS Computational Biology 17, no. 1 (2021): e1008565. http://dx.doi.org/10.1371/journal.pcbi.1008565.

Full text

Abstract:

In vivo calcium imaging through microendoscopic lenses enables imaging of neuronal populations deep within the brains of freely moving animals. Previously, a constrained matrix factorization approach (CNMF-E) has been suggested to extract single-neuronal activity from microendoscopic data. However, this approach relies on offline batch processing of the entire video data and is demanding both in terms of computing and memory requirements. These drawbacks prevent its applicability to the analysis of large datasets and closed-loop experimental settings. Here we address both issues by introducing two different online algorithms for extracting neuronal activity from streaming microendoscopic data. Our first algorithm, OnACID-E, presents an online adaptation of the CNMF-E algorithm, which dramatically reduces its memory and computation requirements. Our second algorithm proposes a convolution-based background model for microendoscopic data that enables even faster (real time) processing. Our approach is modular and can be combined with existing online motion artifact correction and activity deconvolution methods to provide a highly scalable pipeline for microendoscopic data analysis. We apply our algorithms on four previously published typical experimental datasets and show that they yield similar high-quality results as the popular offline approach, but outperform it with regard to computing time and memory requirements. They can be used instead of CNMF-E to process pre-recorded data with boosted speeds and dramatically reduced memory requirements. Further, they newly enable online analysis of live-streaming data even on a laptop.

APA, Harvard, Vancouver, ISO, and other styles

Tiurin, V. O., А. Yu Doroshenko, and E. V. Savchuk. "Analytical store for streaming data with huge volume." PROBLEMS IN PROGRAMMING, no. 1 (March 2022): 067–74. http://dx.doi.org/10.15407/pp2022.01.067.

Full text

Abstract:

A concept for organizing an analytical data warehouse has been developed, which includes a method of interaction between data producers and a repository, a method of data circuit control, a method of data streaming, a method of storing initial data, a method of data processing and a method of providing secure data access. Other concepts on the market are discussed, namely: SDLF as the leading standard recommended by AWS, IronSource DL using Upsolver, SimilarWeb DL using Upsolver. A comparative analysis was conducted (mostly with SDLF, as its implementation is open, and the implementation by private companies is hidden). The advantages of the proposed concept over the existing ones are examined in detail. Recommendations on how to integrate the concept with data schema control applications are given. A service for streaming data using Apache Beam in Java has been developed. A repository architecture for analytics was designed and developed. A data schema management model was developed as well as a data schema management model and a model for secure access to data. The research that has been conducted can be improved by the experience of implementing the concept in business, as well as by collecting and systematizing knowledge about other standards that will be created.

APA, Harvard, Vancouver, ISO, and other styles

Su, Yuan, Yanni Yu, and Ning Zhang. "Carbon emissions and environmental management based on Big Data and Streaming Data: A bibliometric analysis." Science of The Total Environment 733 (September 2020): 138984. http://dx.doi.org/10.1016/j.scitotenv.2020.138984.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Kılınç, Deniz. "A spark‐based big data analysis framework for real‐time sentiment prediction on streaming data." Software: Practice and Experience 49, no. 9 (2019): 1352–64. http://dx.doi.org/10.1002/spe.2724.

Full text

APA, Harvard, Vancouver, ISO, and other styles

We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!