To see the other types of publications on this topic, follow the link: Addresses (data processing).
Journal articles on the topic 'Addresses (data processing)'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Addresses (data processing).'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Johnson, Sarah L. "Quantum Machine Learning Algorithms for Big Data Processing." International Journal of Innovative Computer Science and IT Research 1, no. 02 (2025): 1–11. https://doi.org/10.63665/ijicsitr.v1i02.04.
Full textAbstract:
Quantum Machine Learning (QML) is a new discipline that unites artificial intelligence and quantum computing and can address computational problems of big data analysis. Traditional machine learning algorithms may be pushed to their limits in dealing with the increased complexity and scale of today's data sets and thus are unable to find useful insights within a reasonable time frame. Quantum computing, capable of tapping quantum mechanical processes like superposition and entanglement, is capable of turning this field upside down. In this paper, the concepts behind quantum computing are discussed and how machine learning could be used using the assistance of quantum algorithms in order to better deal with big data. It explains the most optimal quantum algorithms like Quantum Support Vector Machines (QSVM), Quantum Principal Component Analysis (QPCA), and Quantum k-Means Clustering, and why they are better and faster compared to their classical counterparts. It also explores actual applications in medicine, finance, and artificial intelligence. It also addresses the limits and disadvantages of existing quantum technology like hardware limitations, noise, and complexity of algorithms. Last but not least, it also considers the future direction of trends within the field, with emphasis placed on hybrid quantum-classical systems and quantum machine learning application within the construction of big data analysis.
2
Chanda, Deepak. "Automated ETL Pipelines for Modern Data Warehousing: Architectures, Challenges, and Emerging Solutions." Eastasouth Journal of Information System and Computer Science 1, no. 03 (2024): 209–12. https://doi.org/10.58812/esiscs.v1i03.523.
Full textAbstract:
The paper addresses the evolution of automated Extract, Transform, Load (ETL) pipelines in contemporary data warehousing environments, highlighting their essential role in enabling timely analytics and business intelligence. Recent architectural approaches like cloud-native ETL, stream processing architectures, and metadata-driven automation are addressed in the context of increasing data volume and variety. The article addresses typical challenges like schema evolution management, data quality assurance, and cross-platform integration in the context of discussing novel solutions based on leveraging artificial intelligence for pipeline optimization. Through a survey of current implementations and future perspectives, this research provides an in-depth view of how automated ETL workflows are transforming data warehouse environments and enabling more agile, scalable business intelligence solutions.
3
Researcher. "REAL-TIME DATA PROCESSING IN MICROSERVICES ARCHITECTURES." International Journal of Computer Engineering and Technology (IJCET) 15, no. 6 (2024): 760–73. https://doi.org/10.5281/zenodo.14228620.
Full textAbstract:
Real-time data processing in modern distributed systems has evolved significantly, transforming how organizations across various sectors handle operational demands. This comprehensive article explores the fundamental aspects of real-time processing in microservices architectures, examining key technological advancements, implementation strategies, and architectural patterns. The article investigates the impact of event-driven architectures, message brokers, and stream processing technologies while detailing best practices for maintaining data consistency and system performance. Examining cloud integration patterns and serverless computing models, the article provides insights into scaling strategies and resource optimization techniques. The article also addresses common challenges in distributed systems. It presents proven solutions for maintaining system reliability and performance at scale, offering a thorough understanding of modern real-time processing architectures and their practical implementations.
4
Romanchuk, Vitaliy. "Mathematical support and software for data processing in robotic neurocomputer systems." MATEC Web of Conferences 161 (2018): 03004. http://dx.doi.org/10.1051/matecconf/201816103004.
Full textAbstract:
The paper addresses classification and formal definition of neurocomputer systems for robotic complexes, based on the types of associations among their elements. We suggest analytical expressions for performance evaluation in neural computer information processing, aimed at development of methods, algorithms and software that optimize such systems.
5
Hahanov, V. I., V. H. Abdullayev, S. V. Chumachenko, E. I. Lytvynova, and I. V. Hahanova. "IN-MEMORY INTELLIGENT COMPUTING." Radio Electronics, Computer Science, Control, no. 1 (April 2, 2024): 161. http://dx.doi.org/10.15588/1607-3274-2024-1-15.
Full textAbstract:
Context. Processed big data has social significance for the development of society and industry. Intelligent processing of big data is a condition for creating a collective mind of a social group, company, state and the planet as a whole. At the same time, the economy of big data (Data Economy) takes first place in the evaluation of processing mechanisms, since two parameters are very important: speed of data processing and energy consumption. Therefore, mechanisms focused on parallel processing of large data within the data storage center will always be in demand on the IT market.
 Objective. The goal of the investigation is to increase the economy of big data (Data Economy) thanks to the analysis of data as truth table addresses for the identification of patterns of production functionalities based on the similarity-difference metric.
 Method. Intelligent computing architectures are proposed for managing cyber-social processes based on monitoring and analysis of big data. It is proposed to process big data as truth table addresses to solve the problems of identification, clustering, and classification of patterns of social and production processes. A family of automata is offered for the analysis of big data, such as addresses. The truth table is considered as a reasonable form of explicit data structures that have a useful constant – a standard address routing order. The goal of processing big data is to make it structured using a truth table for further identification before making actuator decisions. The truth table is considered as a mechanism for parallel structuring and packing of large data in its column to determine their similarity-difference and to equate data at the same addresses. Representation of data as addresses is associated with unitary encoding of patterns by binary vectors on the found universe of primitive data. The mechanism is focused on processorless data processing based on read-write transactions using in-memory computing technology with significant time and energy savings. The metric of truth table big data processing is parallelism, technological simplicity, and linear computational complexity. The price for such advantages is the exponential memory costs of storing explicit structured data.
 Results. Parallel algorithms of in-memory computing are proposed for economic mechanisms of transformation of large unstructured data, such as addresses, into useful structured data. An in-memory computing architecture with global feedback and an algorithm for matrix parallel processing of large data such as addresses are proposed. It includes a framework for matrix analysis of big data to determine the similarity between vectors that are input to the matrix sequencer. Vector data analysis is transformed into matrix computing for big data processing. The speed of the parallel algorithm for the analysis of big data on the MDV matrix of deductive vectors is linearly dependent on the number of bits of the input vectors or the power of the universe of primitives. A method of identifying patterns using key words has been developed. It is characterized by the use of unitary coded data components for the synthesis of the truth table of the business process. This allows you to use read-write transactions for parallel processing of large data such as addresses.
 Conclusions. The scientific novelty consists in the development of the following innovative solutions: 1) a new vector-matrix technology for parallel processing of large data, such as addresses, is proposed, characterized by the use of read-write transactions on matrix memory without the use of processor logic; 2) an in-memory computing architecture with global feedback and an algorithm for matrix parallel processing of large data such as addresses are proposed; 3) a method of identifying patterns using keywords is proposed, which is characterized by the use of unitary coded data components for the synthesis of the truth table of the business process, which makes it possible to use the read-write transaction for parallel processing of large data such as addresses. The practical significance of the study is that any task of artificial intelligence (similarity-difference, classification-clustering and recognition, pattern identification) can be solved technologically simply and efficiently with the help of a truth table (or its derivatives) and unitarily coded big data . Research prospects are related to the implementation of this digital modeling technology devices on the EDA market. KEYWORDS: Intelligent
6
Gururaj T. and Siddesh G. M. "Hybrid Approach for Enhancing Performance of Genomic Data for Stream Matching." International Journal of Cognitive Informatics and Natural Intelligence 15, no. 4 (2021): 1–18. http://dx.doi.org/10.4018/ijcini.20211001.oa38.
Full textAbstract:
In gene expression analysis, the expression levels of thousands of genes are analyzed, such as separate stages of treatments or diseases. Identifying particular gene sequence pattern is a challenging task with respect to performance issues. The proposed solution addresses the performance issues in genomic stream matching by involving assembly and sequencing. Counting the k-mer based on k-input value and while performing DNA sequencing tasks, the researches need to concentrate on sequence matching. The proposed solution addresses performance issue metrics such as processing time for k-mer counting, number of operations for matching similarity, memory utilization while performing similarity search, and processing time for stream matching. By suggesting an improved algorithm, Revised Rabin Karp(RRK) for basic operation and also to achieve more efficiency, the proposed solution suggests a novel framework based on Hadoop MapReduce blended with Pig & Apache Tez. The measure of memory utilization and processing time proposed model proves its efficiency when compared to existing approaches.
7
Sun, Xihuang, Peng Liu, Yan Ma, Dingsheng Liu, and Yechao Sun. "Streaming Remote Sensing Data Processing for the Future Smart Cities." International Journal of Distributed Systems and Technologies 7, no. 1 (2016): 1–14. http://dx.doi.org/10.4018/ijdst.2016010101.
Full textAbstract:
The explosion of data and the increase in processing complexity, together with the increasing needs of real-time processing and concurrent data access, make remote sensing data streaming processing a wide research area to study. This paper introduces current situation of remote sensing data processing and how timely remote sensing data processing can help build future smart cities. Current research on remote sensing data streaming is also introduced where the three typical and open-source stream processing frameworks are introduced. This paper also discusses some design concerns for remote sensing data streaming processing systems, such as data model and transmission, system model, programming interfaces, storage management, availability, etc. Finally, this research specifically addresses some of the challenges of remote sensing data streaming processing, such as scalability, fault tolerance, consistency, load balancing and throughput.
8
Nguyen, Minh Duc. "A Scientific Workflow System for Satellite Data Processing with Real-Time Monitoring." EPJ Web of Conferences 173 (2018): 05012. http://dx.doi.org/10.1051/epjconf/201817305012.
Full textAbstract:
This paper provides a case study on satellite data processing, storage, and distribution in the space weather domain by introducing the Satellite Data Downloading System (SDDS). The approach proposed in this paper was evaluated through real-world scenarios and addresses the challenges related to the specific field. Although SDDS is used for satellite data processing, it can potentially be adapted to a wide range of data processing scenarios in other fields of physics.
9
Prabagar, S., Vinay K. Nassa, Senthil V. M, Shilpa Abhang, Pravin P. Adivarekar, and Sridevi R. "Python-based social science applications’ profiling and optimization on HPC systems using task and data parallelism." Scientific Temper 14, no. 03 (2023): 870–76. http://dx.doi.org/10.58414/scientifictemper.2023.14.3.48.
Full textAbstract:
This research addresses the pressing need to optimize Python-based social science applications for high-performance computing (HPC)systems, emphasizing the combined use of task and data parallelism techniques. The paper delves into a substantial body of research,recognizing Python’s interpreted nature as a challenge for efficient social science data processing. The paper introduces a Pythonprogram that exemplifies the proposed methodology. This program uses task parallelism with multi-processing and data parallelismwith dask to optimize data processing workflows. It showcases how researchers can effectively manage large datasets and intricatecomputations on HPC systems. The research offers a comprehensive framework for optimizing Python-based social science applicationson HPC systems. It addresses the challenges of Python’s performance limitations, data-intensive processing, and memory efficiency.Incorporating insights from a rich literature survey, it equips researchers with valuable tools and strategies for enhancing the efficiencyof their social science applications in HPC environments.
10
Krishnamurthi, Rajalakshmi, Adarsh Kumar, Dhanalekshmi Gopinathan, Anand Nayyar, and Basit Qureshi. "An Overview of IoT Sensor Data Processing, Fusion, and Analysis Techniques." Sensors 20, no. 21 (2020): 6076. http://dx.doi.org/10.3390/s20216076.
Full textAbstract:
In the recent era of the Internet of Things, the dominant role of sensors and the Internet provides a solution to a wide variety of real-life problems. Such applications include smart city, smart healthcare systems, smart building, smart transport and smart environment. However, the real-time IoT sensor data include several challenges, such as a deluge of unclean sensor data and a high resource-consumption cost. As such, this paper addresses how to process IoT sensor data, fusion with other data sources, and analyses to produce knowledgeable insight into hidden data patterns for rapid decision-making. This paper addresses the data processing techniques such as data denoising, data outlier detection, missing data imputation and data aggregation. Further, it elaborates on the necessity of data fusion and various data fusion methods such as direct fusion, associated feature extraction, and identity declaration data fusion. This paper also aims to address data analysis integration with emerging technologies, such as cloud computing, fog computing and edge computing, towards various challenges in IoT sensor network and sensor data analysis. In summary, this paper is the first of its kind to present a complete overview of IoT sensor data processing, fusion and analysis techniques.
11
Chatzakis, Manos, Panagiota Fatourou, Eleftherios Kosmas, Themis Palpanas, and Botao Peng. "Odyssey: A Journey in the Land of Distributed Data Series Similarity Search." Proceedings of the VLDB Endowment 16, no. 5 (2023): 1140–53. http://dx.doi.org/10.14778/3579075.3579087.
Full textAbstract:
This paper presents Odyssey, a novel distributed data-series processing framework that efficiently addresses the critical challenges of exhibiting good speedup and ensuring high scalability in data series processing by taking advantage of the full computational capacity of modern distributed systems comprised of multi-core servers. Odyssey addresses a number of challenges in designing efficient and highly-scalable distributed data series index, including efficient scheduling, and load-balancing without paying the prohibitive cost of moving data around. It also supports a flexible partial replication scheme, which enables Odyssey to navigate through a fundamental trade-off between data scalability and good performance during query answering. Through a wide range of configurations and using several real and synthetic datasets, our experimental analysis demonstrates that Odyssey achieves its challenging goals.
12
Šprem, Šimun, Nikola Tomažin, Jelena Matečić, and Marko Horvat. "Building Advanced Web Applications Using Data Ingestion and Data Processing Tools." Electronics 13, no. 4 (2024): 709. http://dx.doi.org/10.3390/electronics13040709.
Full textAbstract:
Today, advanced websites serve as robust data repositories that constantly collect various user-centered information and prepare it for subsequent processing. The data collected can include a wide range of important information from email addresses, usernames, and passwords to demographic information such as age, gender, and geographic location. User behavior metrics are also collected, including browsing history, click patterns, and time spent on pages, as well as different preferences like product selection, language preferences, and individual settings. Interactions, device information, transaction history, authentication data, communication logs, and various analytics and metrics contribute to the comprehensive range of user-centric information collected by websites. A method to systematically ingest and transfer such differently structured information to a central message broker is thoroughly described. In this context, a novel tool—Dataphos Publisher—for the creation of ready-to-digest data packages is presented. Data acquired from the message broker are employed for data quality analysis, storage, conversion, and downstream processing. A brief overview of the commonly used and freely available tools for data ingestion and processing is also provided.
13
Sikarwar, Tarika Singh, Abhijeet Singh Chauhan, Nidhi Jain, and Harshita Mathur. "Application of Predictive Analytics in IOT Data Processing." Indian Journal of Information Sources and Services 15, no. 2 (2025): 340–48. https://doi.org/10.51983/ijiss-2025.ijiss.15.2.42.
Full textAbstract:
Working with predictive models mean hugeopportunities for the business-using real-time data, while therapid growth of the Internet of Things-(IoT)-provides uniqueopportunities and hurdles to business. Predictive analytics hasemerged as a cutting-edge approach in the analysis of vast andintricate IoT datasets, using statistics, machine learningalgorithms, and artificial intelligence. In other words, thischapter elaborates on how predictive analytics could fit into IoTdata management as an enabler for proactive decision-makingand outlines its use in forecasting trends, behaviours, andoutcomes.Use in manufacturing, transportation, health, and agriculture,where predictive models reduce risk, improves efficiency, andoptimize operating processes spanning a wide array of potentialsectors in following. The chapter, too, addresses privacy,scalability, and quality of data concerns and proposes variousdevelopments, such as edge computing, Explainable AI, andsustainable analytics, as areas for further development. Prettymuch, predictive analytics impinges heavily into creatingsmarter and data-driven solutions and promoting innovationacross diverse sectors.
14
Datyev, I. O., A. М. Fedorov, and A. A. Reviakin. "Focused collection and processing of open social media data." Ontology of designing 14, no. 4 (2024): 569–81. http://dx.doi.org/10.18287/2223-9537-2024-14-4-569-581.
Full textAbstract:
The article addresses the development of data collection technologies and the complexities that challenge this process. Methods for focusing at various levels are discussed, ranging from managing scanning boundaries to leveraging diverse properties of web pages. Here, the term "focusing" is used to accurately reflect the specific characteristics of targeted data collection and processing of open social media data. This process is multi-stage, employing adaptive control mechanisms that adjust dynamically toward the specified objective. During control, these defined constraints are either narrowed or broadened to align with the target goal. The article also presents insights from the design of an information system’s architecture and software, enabling automated, focused collection and processing of open social media data.
15
Jain, Pritesh. "QUANTUM-ENHANCED EDGE COMPUTING FOR REAL-TIME DATA PROCESSING IN AUTONOMOUS SYSTEMS." COMPUSOFT: An International Journal of Advanced Computer Technology 10 (September 21, 2021): 3984–86. https://doi.org/10.5281/zenodo.15026322.
Full textAbstract:
The proliferation of Internet of Things (IoT) devices and autonomous systems has necessitated advancements in realtime data processing capabilities. Edge computing addresses latency and bandwidth issues by processing data closer to the source. However, traditional edge computing approaches struggle with the computational demands of complex algorithms, especially in autonomous systems. This paper introduces a novel approach that leverages quantum computing principles to enhance edge computing frameworks. We propose a hybrid architecture combining quantum-enhanced processing units with classical edge nodes to optimize real-time data processing. Our results demonstrate significant improvements in processing speed and efficiency, making this approach viable for deployment in autonomous vehicles and smart city infrastructures.
16
Ridwan Kolapo, Fatima Mohammed Kawu, Aminu Dalhatu Abdulmalik, Ubong Anietie Edem, Maureen Atisi Young, and Emmanuel Chukwudinma Mordi. "Edge computing: Revolutionizing data processing for IoT applications." International Journal of Science and Research Archive 13, no. 2 (2024): 023–29. http://dx.doi.org/10.30574/ijsra.2024.13.2.2082.
Full textAbstract:
Edge computing is emerging as a transformative solution for managing the vast amounts of data generated by the Internet of Things (IoT). By decentralizing data processing and bringing computation closer to the data source, edge computing addresses critical limitations of traditional cloud computing, including latency, bandwidth constraints, and security vulnerabilities. This review explores the key benefits of edge computing, such as reduced latency, bandwidth optimization, improved reliability, enhanced data privacy, and scalability. It discusses the architecture and components of edge computing, highlighting the roles of edge devices, edge nodes, and fog computing. The review also examines various use cases across sectors, including autonomous vehicles, smart cities, healthcare, industrial IoT, and retail. Finally, the review considers the challenges facing edge computing, including hardware limitations, network security, interoperability, and cost considerations. The future outlook suggests that advancements in 5G technology and artificial intelligence will further enhance the potential of edge computing in driving IoT innovations.
17
Researcher. "ARTIFICIAL INTELLIGENCE IN DYNAMIC DATA TRANSFORMATION: A FRAMEWORK FOR ENTERPRISE INTEGRATION AND OPTIMIZATION." International Journal of Computer Engineering and Technology (IJCET) 15, no. 6 (2024): 1255–69. https://doi.org/10.5281/zenodo.14370114.
Full textAbstract:
The exponential growth in data volume and complexity has created an urgent need for more sophisticated approaches to data transformation in enterprise environments. This article presents a comprehensive framework for implementing artificial intelligence (AI) in dynamic data transformation processes, addressing key challenges in data quality, schema evolution, and real-time processing. Through multiple case studies across different industries, we examine the implementation of machine learning algorithms, natural language processing, and predictive analytics in automating and optimizing data transformation workflows. The article demonstrates how AI-driven approaches significantly improve operational efficiency, reduce manual intervention, and enhance data quality while maintaining system scalability. The findings indicate that organizations implementing AI-based transformation strategies achieve substantial improvements in processing speed, accuracy, and adaptability to changing data patterns. The article also addresses critical integration considerations, including architecture design, security implications, and change management strategies. This article contributes to both theoretical understanding and practical implementation of AI in data transformation, providing a structured approach for organizations seeking to modernize their data processing capabilities. The article concludes with recommendations for practitioners and identifies emerging trends that will shape the future of AI-driven data transformation.
18
Venkata Nagendra Kumar Kundavaram. "Event-Driven Data Pipelines : A Cloud-Based Approach to Real-Time Data Processing." International Journal of Scientific Research in Computer Science, Engineering and Information Technology 10, no. 6 (2024): 364–69. http://dx.doi.org/10.32628/cseit24106183.
Full textAbstract:
This article comprehensively analyzes event-driven data pipelines in cloud computing environments, examining their architecture, implementation considerations, and real-world applications. The article explores the fundamental components of event-driven systems, from event sources and message brokers to processing layers, while evaluating their performance characteristics and reliability mechanisms. Through detailed analysis of system architectures, we investigate the integration of various cloud services and their role in enabling scalable, real-time data processing. The article demonstrates how modern event-driven architectures achieve sub-millisecond processing times and handle millions of events per second while maintaining system resilience. Our findings reveal significant improvements in operational efficiency across various industries, including financial services, marketing automation, and IoT monitoring solutions. The article also addresses critical implementation aspects, including security frameworks, integration protocols, and best practices for system optimization. These insights provide valuable guidance for organizations leveraging event-driven architectures in their digital transformation initiatives. The article concludes that event-driven data pipelines represent a crucial advancement in data processing technology, offering unprecedented capabilities in handling real-time data streams while maintaining scalability, reliability, and cost-effectiveness.
19
Researcher. "THE IMPACT OF 5G ON CLOUD DATA ARCHITECTURES." International Journal of Research In Computer Applications and Information Technology (IJRCAIT) 7, no. 2 (2024): 879–89. https://doi.org/10.5281/zenodo.14046052.
Full textAbstract:
This article explores the transformative impact of 5G technology on cloud data architectures. It examines how 5G's high-speed, low-latency capabilities are driving significant changes in data processing paradigms, database designs, and real-time applications. The article focuses on key areas including edge computing acceleration, real-time data processing, and the evolution of database architectures. It also addresses the challenges of data security, privacy, and energy efficiency in 5G networks. The article concludes by discussing future directions and the potential for innovation in cloud computing as 5G technology matures and becomes more widespread.
20
Vijaya, Dr V. Krishna. "INVOICE DATA EXTRACTION USING OCR TECHNIQUE." INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT 08, no. 04 (2024): 1–5. http://dx.doi.org/10.55041/ijsrem29981.
Full textAbstract:
Traditional invoice processing involves manual entry of data, leading to human errors, delays,and increased operational costs. The lack of automation results in inefficiencies, hindering organizations from promptly accessing critical financial information. This research addresses the pressing need for a reliable OCR-based solution to automate invoice data extraction, ultimately improving accuracy, reducing processing time, and enhancing overall business productivity. The project aims to automate invoice data extraction through Optical Character Recognition (OCR) techniques. Leveraging advanced image processing and machine learning, the system will analyze scanned or photographed invoices, extracting relevant information such as vendor details, itemized costs, and dates.This automation streamlines manual data entry processes, enhancing accuracy and efficiency in managing financial records. OCR invoicing is the process of training a template-based OCR model for specific invoice layouts, setting up input paths for these invoices, extracting data, and integrating the extracted data with a structured database. Keywords: Invoice, OCR, YOLO algorithm, Data Extraction, Image Processing, Database Integration.
21
Avinash Dulam. "Enhancing data processing with Apache spark: A technical deep dive." World Journal of Advanced Engineering Technology and Sciences 15, no. 3 (2025): 1279–84. https://doi.org/10.30574/wjaets.2025.15.3.0910.
Full textAbstract:
Apache Spark has revolutionized big data processing by introducing a unified computing framework that addresses the challenges of distributed data processing, real-time analytics, and machine learning at scale. The framework's architecture, built on Resilient Distributed Datasets (RDDs), enables fault-tolerant parallel operations while providing sophisticated optimization techniques for enhanced performance. Through advanced features like Structured Streaming, DataFrame abstractions, and MLlib integration, Spark offers comprehensive solutions for modern data processing needs, from batch processing to real-time analytics, effectively supporting organizations in managing exponentially growing data volumes while maintaining processing efficiency and scalability. The platform's innovative approach to data abstraction, combined with its robust optimization capabilities and integration with modern computing paradigms, establishes it as a cornerstone technology for enterprises seeking to harness the power of big data while minimizing operational complexity and maximizing resource utilization across diverse processing environments.
22
Li, Wenqi, Pengyi Zhang, and Jun Wang. "Humanities Scholars' Understanding of Data and the Implications for Humanities Data Curation." Proceedings of the Association for Information Science and Technology 60, no. 1 (2023): 1034–36. http://dx.doi.org/10.1002/pra2.936.
Full textAbstract:
ABSTRACTThis study addresses the need for a shared understanding of humanities data to enhance data curation. Through interviews with 27 scholars, it identifies two ways scholars conceptualize data ‐ by format or role in research. It highlights three unique aspects: diverse requirements of materiality and processing levels, significance of authorship and perspective, and the dual role of tertiary sources. The study suggests prioritizing provenance, facilitating data documentation, curating tertiary sources for wider use, and establishing scholarly communication mechanisms for effective data curation.
23
Gopinath Govindarajan. "Building a strong foundation in data engineering: a comprehensive guide for aspiring data analysts." World Journal of Advanced Research and Reviews 26, no. 1 (2025): 3901–7. https://doi.org/10.30574/wjarr.2025.26.1.1508.
Full textAbstract:
This comprehensive article explores the fundamental aspects of building a strong foundation in data engineering, focusing on the transformation of data processing and management in modern organizations. The article examines the evolution of data engineering practices, highlighting the integration of artificial intelligence, cloud technologies, and automated workflows in contemporary data architectures. It investigates core technical foundations, including database management, SQL optimization, and Python programming, while analyzing the impact of cloud-native services and distributed computing on data processing capabilities. The article also delves into automation and orchestration practices, examining how modern tools and frameworks have revolutionized data pipeline management. Additionally, the article addresses critical aspects of data security and governance, providing insights into emerging best practices and regulatory compliance frameworks in the data engineering landscape.
24
Krupal Gangapatnam. "Automated data anonymization tools to comply with GDPR regulations, processing billions of data points stored across multiple cloud environments." International Journal of Science and Research Archive 14, no. 1 (2025): 787–96. https://doi.org/10.30574/ijsra.2025.14.1.0116.
Full textAbstract:
The rapid evolution of data processing demands has led to innovative approaches in enterprise-scale data anonymization and protection. This comprehensive examination explores the implementation of Delphix across diverse cloud environments, focusing on its technical architecture, performance metrics, and compliance features. The platform demonstrates exceptional capabilities in handling sensitive data through advanced machine learning algorithms and sophisticated processing pipelines. The architecture incorporates robust security mechanisms, parallel processing capabilities, and intelligent resource optimization across multiple geographical regions. Integration with major cloud providers enables seamless scalability while maintaining strict data protection standards. The implementation showcases significant improvements in processing efficiency, reduced data breach risks, and enhanced compliance adherence through automated controls. Best practices and deployment guidelines ensure optimal performance through carefully calibrated infrastructure requirements and monitoring systems. The solution addresses the critical challenges of data privacy and security while maintaining high throughput rates and system availability across distributed environments.
25
BRIA, Vasile, and Marius BODOR. "Automated Processing of Mechanical Test Data for Various Composite Materials Using MATLAB." Annals of “Dunarea de Jos” University of Galati. Fascicle IX, Metallurgy and Materials Science 48, no. 1 (2025): 28–38. https://doi.org/10.35219/mms.2025.1.05.
Full textAbstract:
The behaviour of composite materials during the mechanical testing process might exhibit, in some situations, very different patterns compared to those of conventional materials. This is why the eventual automation of data processing might require additional steps towards obtaining realistic results from mechanical testing. The present work addresses this issue by proposing an algorithm written using MATLAB software and applying it in processing data from mechanical testing of selected composite materials with various compositions and behaviours.
26
Moustakidis, Serafeim, Athanasios Anagnostis, Apostolos Chondronasios, Patrik Karlsson, and Kostas Hrissagis. "Excitation-invariant pre-processing of thermographic data." Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability 232, no. 4 (2018): 435–46. http://dx.doi.org/10.1177/1748006x18770888.
Full textAbstract:
There is a large number of industries that make extensive use of composite materials in their respective sectors. This rise in composites’ use has necessitated the development of new non-destructive inspection techniques that focus on manufacturing quality assurance, as well as in-service damage testing. Active infrared thermography is now a popular nondestructive testing method for detecting defects in composite structures. Non-uniform emissivity, uneven heating of the test surface, and variation in thermal properties of the test material are some of the crucial factors in experimental thermography. These unwanted thermal effects are typically coped with the application of a number of well-established thermographic techniques including pulse phase thermography and thermographic signal reconstruction. This article addresses this problem of the induced uneven heating at the pre-processing phase prior to the application of the thermographic processing techniques. To accomplish this, a number of excitation invariant pre-processing techniques were developed and tested in this article addressing the unwanted effect of non-uniform excitation in the collected thermographic data. Various fitting approaches were validated in light of modeling the non-uniform heating effect, and new normalization approaches were proposed following a time-dependent framework. The proposed pre-processing techniques were validated on a testing composite sample with pre-determined defects. The results demonstrated the effectiveness of the proposed processing algorithms in terms of removing the unwanted heat distribution effect along with the signal-to-noise ratio of the produced infrared images.
27
Venkata, Gummadi. "Designing a Scalable Architecture for Customer Data Engineering Platform on Cloud Infrastructure: A Comprehensive Framework." Journal of Scientific and Engineering Research 10, no. 12 (2023): 243–51. https://doi.org/10.5281/zenodo.14012383.
Full textAbstract:
The exponential growth of customer data in modern enterprises has created unprecedented challenges in data engineering, necessitating architectures capable of handling petabyte-scale processing while maintaining real-time analytics capabilities. This paper presents a comprehensive architectural framework for designing and implementing scalable customer data engineering platforms utilizing cloud infrastructure. The proposed architecture addresses critical challenges including real-time data processing, horizontal scalability, data governance, and security considerations. Through rigorous experimental validation and performance analysis conducted over a six-month period across three geographic regions, we demonstrate that the proposed architecture achieves a 40% reduction in data processing latency, 99.99% system availability, and 65% improvement in resource utilization compared to traditional architectures. The framework introduces novel approaches to data partitioning, processing optimization, and automated scaling, while maintaining cost-effectiveness. Our results indicate significant improvements in key performance indicators, including a 75% reduction in data retrieval time and 99.999% data durability. This research contributes to the field by providing a validated framework for building enterprise-scale data platforms that can adapt to evolving business requirements while maintaining operational efficiency.
28
Suchit, Kumar Vyas, and Satish Kumar Dr. "Critical issues of data security and privacy of mobile cloud computing." International Journal of Trends in Emerging Research and Development 2, no. 2 (2024): 165–70. https://doi.org/10.5281/zenodo.13118086.
Full textAbstract:
Cloud computing is a rapidly evolving technology that provides shared processing resources and data to computers and other devices on demand. Mobile computing, on the other hand, facilitates the transmission of data, voice, and video. The convergence of these technologies has given rise to Mobile Cloud Computing (MCC), a concept that not only addresses the limitations of mobile computing but also integrates cloud computing into mobile environments to tackle challenges related to performance, security, and resource constraints. MCC is gaining momentum as the number of mobile users continues to grow. Despite its advantages, data privacy and security remain significant concerns. MCC can deliver infrastructure, computational power, software, and platform services to even basic smartphones. However, several security issues need to be addressed, including network security, web application security, data access, authentication, authorization, data confidentiality, and data breaches. Given the limited storage and processing power of mobile devices, data capacity is constrained. To establish a secure MCC environment, it is essential to thoroughly study and analyze security threats. In this paper, we propose an algorithm designed to enhance the security of mobile cloud computing, ensuring data integrity and confidentiality. Additionally, we discuss the security threats present in MCC environments and propose solutions to address these challenges.
29
Mohammed-Javed Padinhakara. "Vehicular data management at scale: Architectural frameworks for cars as mobile data centers." World Journal of Advanced Engineering Technology and Sciences 15, no. 3 (2025): 752–58. https://doi.org/10.30574/wjaets.2025.15.3.0940.
Full textAbstract:
The emerging paradigm of modern vehicles as sophisticated mobile data centers generates unprecedented volumes of telemetry, sensor, and interaction data that require novel management approaches. The architectural framework addresses dual requirements of edge processing for latency-sensitive applications and cloud infrastructure for deeper analytics and model development. Vehicle-to-everything communication protocols integrate with software-defined networks and distributed ledger technologies to ensure secure, efficient data exchange across the ecosystem. Technical challenges including bandwidth constraints, data redundancy, and privacy regulations are primary motivators for solutions based on federated learning, optimized compression algorithms, and context-aware processing. Resilient vehicular data management necessitates a multi-layered approach balancing computational requirements across the edge-cloud continuum while maintaining robust security postures. These foundations enable scaling next-generation intelligent transportation systems were vehicles function as key nodes in broader smart city infrastructures.
30
Jiang, Wei, Jian-Hua Zhang, Xiao-Feng Cao, Bo Yang, and Wen-Tao Wang. "Fast data packet sorting method based on FPGA on-chip RAM." Journal of Instrumentation 20, no. 02 (2025): P02023. https://doi.org/10.1088/1748-0221/20/02/p02023.
Full textAbstract:
Abstract The study of fast sorting algorithms has long been an enduring research focus. Traditional sorting algorithms often suffer from high time complexity, typically staying at (n 2) or O(n × log2 n). Given the parallel processing advantages of field programmable gate array (FPGA), it has become a popular platform for algorithm acceleration. However, existing hardware sorting acceleration methods remain rooted in classical software algorithm models, merely leveraging hardware for parallel execution, without fully exploring the unique architecture and resources of FPGAs. In response, this paper proposes a fast sorting method that leverages on-chip random-access memory (RAM), uniquely tailored to FPGA characteristics. First, a mapping is established between the key fields of data packets and the on-chip RAM addresses. Then, the data packets are written into RAM based on this mapping, which also inevitably result in some RAM addresses being left empty. Next, a indicator register is maintained to dynamically track which RAM addresses are empty. Finally, the data packets are sequentially read from RAM addresses, and the indicator register helps skip empty addresses to enhance readout efficiency. Due to the inherent ordering of RAM addresses, the data packets become naturally ordered after read. Simulation results confirm that this method can reduce the time complexity to O(n), providing a novel solution for fast sorting in real-time data streams.
31
Crosetto, M., N. Devanthéry, M. Cuevas-González, O. Monserrat, and B. Crippa. "Exploitation of the full potential of Persistent Scatterer Interferometry data." ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XL-7 (September 19, 2014): 75–78. http://dx.doi.org/10.5194/isprsarchives-xl-7-75-2014.
Full textAbstract:
The potential of Persistent Scatterer Interferometry (PSI) for deformation monitoring has been increasing in the last years and it will continue to do so in the short future, especially with the advent of the Sentinel-1 mission. The full exploitation of this potential requires two important components. The first one is the improvement of the PSI processing tools, to achieve massive and systematic data processing capabilities. The second one is the need to increase the capabilities to correctly analyze and interpret the PSI results. The paper addresses both components. The key features of the PSI processing chain implemented by the authors, which is named PSIG chain, are described. This is followed by a brief discussion of the key elements needed to analyse and interpret the results of a given PSI processing. The paper concludes with a description of the results obtained by processing a full frame of very high resolution TerraSAR-X data that covers the metropolitan area of Barcelona (Spain).
32
Leja, Laura, Vitālijs Purlans, Rihards Novickis, Andrejs Cvetkovs, and Kaspars Ozols. "Mathematical Model and Synthetic Data Generation for Infra-Red Sensors." Sensors 22, no. 23 (2022): 9458. http://dx.doi.org/10.3390/s22239458.
Full textAbstract:
A key challenge in further improving infrared (IR) sensor capabilities is the development of efficient data pre-processing algorithms. This paper addresses this challenge by providing a mathematical model and synthetic data generation framework for an uncooled IR sensor. The developed model is capable of generating synthetic data for the design of data pre-processing algorithms of uncooled IR sensors. The mathematical model accounts for the physical characteristics of the focal plane array, bolometer readout, optics and the environment. The framework permits the sensor simulation with a range of sensor configurations, pixel defectiveness, non-uniformity and noise parameters.
33
Shkirdov, D. A., E. S. Sagatov, and P. S. Dmitrenko. "Trap method in ensuring data security." Information Technology and Nanotechnology, no. 2416 (2019): 189–98. http://dx.doi.org/10.18287/1613-0073-2019-2416-189-198.
Full textAbstract:
This paper presents the results of data analysis from a geographically distributed honeypot network. Such honeypot servers were deployed in Samara, Rostov on Don, Crimea and the USA two years ago. Methods for processing statistics are discussed in detail for secure remote access SSH. Lists of attacking addresses are highlighted, and their geographical affiliation is determined. Rank distributions were used as the basis for statistical analysis. The intensity of requests to each of the 10 installed services was then calculated.
34
Sarah, L. Johnson. "Quantum Machine Learning Algorithms for Big Data Processing." International Journal of Innovative Computer Science and IT Research 01, no. 02 (2025): 31–41. https://doi.org/10.5281/zenodo.15147384.
Full textAbstract:
Quantum Machine Learning (QML) is a new discipline that unites artificial intelligence and quantum computing and can address computational problems of big data analysis. Traditional machine learning algorithms may be pushed to their limits in dealing with the increased complexity and scale of today's data sets and thus are unable to find useful insights within a reasonable time frame. Quantum computing, capable of tapping quantum mechanical processes like superposition and entanglement, is capable of turning this field upside down. In this paper, the concepts behind quantum computing are discussed and how machine learning could be used using the assistance of quantum algorithms in order to better deal with big data. It explains the most optimal quantum algorithms like Quantum Support Vector Machines (QSVM), Quantum Principal Component Analysis (QPCA), and Quantum k-Means Clustering, and why they are better and faster compared to their classical counterparts. It also explores actual applications in medicine, finance, and artificial intelligence. It also addresses the limits and disadvantages of existing quantum technology like hardware limitations, noise, and complexity of algorithms. Last but not least, it also considers the future direction of trends within the field, with emphasis placed on hybrid quantum-classical systems and quantum machine learning application within the construction of big data analysis.
35
Ravi, Shankar Koppula. "Real-world Use Cases of Databricks in Big Data Projects." Journal of Scientific and Engineering Research 8, no. 12 (2021): 253–63. https://doi.org/10.5281/zenodo.12798308.
Full textAbstract:
As data becomes a critical asset for businesses, the need for efficient and scalable data processing frameworks has never been greater. Apache Spark, a powerful big data technology, addresses these demands with its user-friendly interface and rapid performance. Databricks, a managed service built on Apache Spark, further enhances this capability by offering a cloud-based platform that streamlines the development and deployment of data products. This paper explores several real-world use cases of Databricks in large-scale data projects, highlighting its role in data ingestion, transformation, exploration, analysis, and machine learning. Databricks accelerates AI development and promotes secure innovation through collaborative data science and engineering workflows. This unified data analytics platform supports businesses in leveraging big data for predictive modeling, reporting, and real-time analytics, ultimately facilitating scalable and efficient data management.
36
Researcher. "DATA ENGINEERING FOR REAL-TIME ANALYTICS: TURNING DATA INTO INSTANT INSIGHTS." International Journal of Computer Engineering and Technology (IJCET) 15, no. 5 (2024): 752–63. https://doi.org/10.5281/zenodo.13889767.
Full textAbstract:
This article explores the transformative impact of real-time analytics across industries driven by advances in data engineering. It examines the key components of real-time analytics systems, including data ingestion, stream processing, in-memory storage, and visualization tools. The article demonstrates how real-time analytics enables personalized user experiences, optimized operations, and data-driven decision-making through case studies of American subscription video streaming services and American multinational retail corporations. The benefits of rapid insights, improved customer experiences, operational efficiencies, and competitive advantages are discussed. While highlighting the tremendous potential, the article also addresses challenges in implementing real-time analytics, such as managing data volume and velocity, ensuring data quality, system scalability, security and privacy concerns, skill gaps, and integration complexities.
37
Suárez-Paniagua, Víctor, Arlene Casey, Charis A. Marwick, et al. "Care home resident identification: A comparison of address matching methods with Natural Language Processing." PLOS ONE 19, no. 12 (2024): e0309341. https://doi.org/10.1371/journal.pone.0309341.
Full textAbstract:
Background Care home residents are a highly vulnerable group, but identifying care home residents in routine data is challenging. This study aimed to develop and validate Natural Language Processing (NLP) methods to identify care home residents from primary care address records. Methods The proposed system applies an NLP sequential filtering and preprocessing of text, then the calculation of similarity scores between general practice (GP) addresses and care home registered addresses. Performance was evaluated in a diagnostic test study comparing NLP prediction to independent, gold-standard manual identification of care home addresses. The analysis used population data for 771,588 uniquely written addresses for 819,911 people in two NHS Scotland health board regions. The source code is publicly available at https://github.com/vsuarezpaniagua/NLPcarehome. Results Care home resident identification by NLP methods overall was better in Fife than in Tayside, and better in the over-65s than in the whole population. Methods with the best performance were Correlation (sensitivity 90.2%, PPV 92.0%) for Fife data and Cosine (sensitivity 90.4%, PPV 93.7%) for Tayside. For people aged ≥65 years, the best methods were Jensen-Shannon (sensitivity 91.5%, PPV 98.7%) for Fife and City Block (sensitivity 94.4%, PPV 98.3%) for Tayside. These results show the feasibility of applying NLP methods to real data concluding that computing address similarities outperforms previous works. Conclusions Address-matching techniques using NLP methods can determine with reasonable accuracy if individuals live in a care home based on their GP-registered addresses. The performance of the system exceeds previously reported results such as Postcode matching, Markov score or Phonics score.
38
Sambu, Patach Arrojula. "Efficient Event Grouping Algorithm for Mobile Analytics: Reducing Data Footprint." European Journal of Advances in Engineering and Technology 10, no. 1 (2023): 124–29. https://doi.org/10.5281/zenodo.13919609.
Full textAbstract:
The rapid growth in mobile applications has led to an exponential increase in the number of events generated and transmitted for analytics purposes. This surge in data volume significantly impacts data costs and performance for clients, while also demanding more resources for backend processing. In this paper, we propose an innovative approach to mitigate these challenges by reducing the footprint of events through efficient grouping and redundancy elimination. Our method involves aggregating related events into predefined buckets and optimizing storage by removing redundant data, retaining only a single copy of common elements within each bucket. This strategy not only alleviates the data load on client uploads but also reduces the computational and storage resources required for backend processing. Initial evaluations show that our approach reduces data transmission size by an average of 26.7%, demonstrating its effectiveness in enhancing overall system performance. This scalable solution addresses the growing demands of mobile analytics by significantly reducing data transmission costs and resource utilization.
39
Gururaj Thite. "Building Expertise in Data Engineering for AI Applications: A Comprehensive Guide." Journal of Computer Science and Technology Studies 7, no. 3 (2025): 01–07. https://doi.org/10.32996/jcsts.2025.7.3.1.
Full textAbstract:
Data engineering has evolved significantly with the integration of artificial intelligence in the financial sector, demanding robust infrastructures and sophisticated practices. This comprehensive guide explores the essential competencies, tools, and best practices required for modern data engineers to excel in AI-driven financial systems. It details the transformation from traditional batch processing to real-time streaming architectures, examining distributed computing solutions, cloud-native implementations, and quality assurance frameworks. The guide addresses critical aspects of system architecture, security protocols, and compliance requirements while highlighting emerging trends in stream processing, edge computing, and automation technologies that shape the future of data engineering.
40
Deepak Chanda. "Optimizing AI and Robotics-driven Automation Systems: The Synergy of Data Engineering and Data Science in Scalable Intelligent Automation." Journal of Electrical Systems 21, no. 1s (2025): 126–31. https://doi.org/10.52783/jes.8360.
Full textAbstract:
The intersection of data engineering and artificial intelligence (AI) has revolutionized modern industries using scalable, efficient, and intelligent automation. AI applications rely on robust data engineering frameworks for data ingestion, processing, and storage to feed high-quality inputs to machine learning algorithms. This paper explores the symbiosis between AI and data engineering in terms of automation, robotics, scalability, and real-time analytics. Data integration, governance, and performance optimization issues are considered, along with AI-driven solutions that streamline data workflows. The paper also addresses emerging technologies like edge computing and quantum processing, and their impact on data engineering. As AI continues to expand, optimization of data-driven architectures will be key for organizations seeking a competitive advantage in a rapidly digitizing world.
41
Raghavendra Kurva. "Real-Time Data Integrity Validation Using Blockchain for Autonomous Vehicles." International Journal of Scientific Research in Computer Science, Engineering and Information Technology 11, no. 2 (2025): 1275–82. https://doi.org/10.32628/cseit25112455.
Full textAbstract:
This article presents a comprehensive framework for implementing blockchain-based data integrity validation in autonomous vehicles. The proposed system addresses critical challenges in securing real-time sensor data through a hybrid architecture combining Hyperledger Fabric with Apache Kafka. By integrating distributed ledger technology with optimized data processing mechanisms, the system achieves both security and performance requirements essential for autonomous vehicle operations. The architecture incorporates smart contracts for data validation, multi-layered security protocols, and efficient data streaming capabilities. Results demonstrate that the proposed solution successfully balances the competing demands of data security and real-time processing, making it suitable for deployment in production autonomous vehicle environments.
42
Jaydeep Taralkar. "Designing scalable financial data pipelines with cloudera." Global Journal of Engineering and Technology Advances 23, no. 1 (2025): 420–26. https://doi.org/10.30574/gjeta.2025.23.1.0102.
Full textAbstract:
This technical article explores the design and implementation of scalable financial data pipelines using Cloudera's ecosystem. It examines the unique challenges facing financial institutions in managing massive volumes of diverse data types for applications including high-frequency trading, risk assessment, and regulatory compliance. The article details how Cloudera's integrated platform of open-source technologies—including Hadoop, Spark, Kafka, and specialized components—addresses these challenges through a comprehensive architectural paradigm. The article presents evidence-based performance metrics from financial institutions across four key areas: data ingestion and processing capabilities, performance optimization strategies, scalability methods, and security frameworks with AI integration. Real-world implementation examples demonstrate how financial organizations have achieved significant improvements in processing efficiency, latency reduction, and cost savings while maintaining regulatory compliance and enabling advanced analytics capabilities.
43
J.J.Jayakanth. "Dynamic Object Detection in Surveillance Videos using Temporal Convolutional Networks and Federated Learning in Edge Computing Environments." Journal of Electrical Systems 20, no. 5s (2024): 2009–15. http://dx.doi.org/10.52783/jes.2537.
Full textAbstract:
This research addresses the importance of advancing dynamic object detection in surveillance videos by introducing a novel framework that integrates Temporal Convolutional Networks (TCNs) and Federated Learning (FL) within edge computing environments. This research is motivated by the critical need for real-time threat response, enhanced security measures, and privacy preservation in dynamic surveillance scenarios. Leveraging TCNs, the system captures temporal dependencies, providing a comprehensive understanding of object movements. FL ensures decentralized model training, mitigating privacy concerns associated with centralized approaches. Current challenges in real-time processing, privacy preservation, and adaptability to dynamic environments are addressed through innovative solutions. Model optimization techniques optimize TCN efficiency, ensuring real-time processing. Advanced privacy-preserving mechanisms secure FL, addressing privacy concerns. Transfer learning and data augmentation enhance adaptability to dynamic scenarios. The proposed system not only addresses existing challenges but also contributes to the evolution of surveillance technology. Comprehensive metrics, including accuracy, sensitivity, specificity, and real-time processing metrics, provide a thorough evaluation of the system's performance. This research introduces an approach to dynamic object detection, combining TCN and FL in edge computing environments. Results show accuracy exceeding 97%, sensitivity and specificity at 97% and 98%, and F1 score reaching 96%.
44
Martyniuk, T. B., and D. O. Katashynskyi. "Peculiarities of associative data processing in intelligent systems." Optoelectronic Information-Power Technologies 49, no. 1 (2025): 44–52. https://doi.org/10.31649/1681-7893-2025-49-1-44-52.
Full textAbstract:
Associative operations are computational massively parallel procedures over large data sets. This explains their widespread use in such application areas as database management systems (DBMS), searching and sorting IP addresses in computer networks, and ranking data, for example, in decision-making subsystems as part of intelligent systems, in particular, for medical diagnostics. This is due, not least, to the fact that associative operations include selection by foreign key, searching for data by analogy, sorting and ranking of elements of a data set. This paper presents the results of an analysis of the features of the application of associative data processing methods for solving problems in intelligent systems. The definition of intelligent memory is considered as one that is expanded due to the functional capabilities of associative memory, i.e. memory with content-addressing. In this case, associative data processing includes not only a search by association, that is, by a foreign key, but also a search for an extreme (maximum/minimum) element in a numerical array. Another example of the application of associative data processing are varieties of neural networks that perform the functions of auto- and heteroassociative memory. The use of neural networks in intelligent control systems of mobile robots is especially relevant today, since their structure is provided by associative processing levels. Another popular approach is the use of a classifier with extended functional capabilities as part of decision support subsystems for expert systems for various purposes. These examples indicate a specific connection between associative data processing methods and the implementation of neurotechnologies in the creation of intelligent systems for various purposes.
45
Bhumeka, Narra Dheeraj Varun Kumar Reddy Buddula Hari Hara Sudheer Patchipulusu Achuthananda Reddy Polu Navya Vattikonda and Anuj Kumar Gupta. "Advanced Edge Computing Frameworks for Optimizing Data Processing and Latency in IoT Networks." JOETSR-Journal of Emerging Trends in Scientific Research 01, no. 01 (2023): 1–10. https://doi.org/10.5281/zenodo.15335462.
Full textAbstract:
The advent of edge computing has been revolutionary in improving data processing efficiency andreducing latency in IoT networks. By decentralizing computational tasks, edge computing enables real-timeanalytics and scalability while reducing reliance on centralized cloud infrastructures. This paper exploresadvanced frameworks, including hierarchical and decentralized architectures, that integrate AI and machinelearning to enhance predictive optimization and resource management. It also addresses challenges such asprotocol compatibility, energy efficiency, and operational complexities. The review provides insights into currentadvancements and identifies future opportunities for improving IoT network performance and supportinglatency-sensitive applications like smart cities and industrial automation.
46
Moore, Philip T., and Hai V. Pham. "Personalization and rule strategies in data-intensive intelligent context-aware systems." Knowledge Engineering Review 30, no. 2 (2015): 140–56. http://dx.doi.org/10.1017/s0269888914000265.
Full textAbstract:
AbstractThe concept of personalization in its many forms has gained traction driven by the demands of computer-mediated interactions generally implemented in large-scale distributed systems and ad hoc wireless networks. Personalization requires the identification and selection of entities based on a defined profile (a context); an entity has been defined as a person, place, or physical or computational object. Context employs contextual information that combines to describe an entities current state. Historically, the range of contextual information utilized (in context-aware systems) has been limited to identity, location, and proximate data; there has, however, been advances in the range of data and information addressed. As such, context can be highly dynamic with inherent complexity. In addition, context-aware systems must accommodate constraint satisfaction and preference compliance.This article addresses personalization and context with consideration of the domains and systems to which context has been applied and the nature of the contextual data. The developments in computing and service provision are addressed with consideration of the relationship between the evolving computing landscape and context. There is a discussion around rule strategies and conditional relationships in decision support. Logic systems are addressed with an overview of the open world assumption versus the closed world assumption and the relationship with the Semantic Web. The event-driven rule-based approach, which forms the basis upon which intelligent context processing can be realized, is presented with an evaluation and proof-of-concept. The issues and challenges identified in the research are considered with potential solutions and research directions; alternative approaches to context processing are discussed. The article closes with conclusions and open research questions.
47
Fasihuddin, Mirza. "Architecting scalable and reliable data ingestion pipelines to efficiently ingest large volumes of data into Hadoop clusters." European Journal of Advances in Engineering and Technology 9, no. 3 (2022): 153–58. https://doi.org/10.5281/zenodo.11213915.
Full textAbstract:
The rapid growth in data generation, fueled by trends like IoT proliferation and digital transformation, highlights the urgent need for scalable and reliable data ingestion pipelines to manage vast datasets within Hadoop clusters. This paper addresses the challenges of designing such pipelines, focusing on architecture, scalability, and reliability. It explores strategies for implementing resilient pipelines, considering fault tolerance, data consistency, and adaptability to evolving data sources and business needs. By comprehensively addressing these challenges, organizations can optimize data processing workflows and maximize the value derived from big data analytics initiatives in today's dynamic data landscape.
48
More, Arjit Amol. "Natural Language Processing - Based Structured Data Extraction from Unstructured Clinical Notes." Journal of Contemporary Medical Practice 6, no. 8 (2024): 327–30. http://dx.doi.org/10.53469/jcmp.2024.06(08).67.
Full textAbstract:
Electronic Health Records (EHRs) are pivotal in modern healthcare, housing a treasure trove of patient information. They are real-time, patient-centered records that make information available instantly and securely to authorized users. However, a substantial portion of this data resides in unstructured clinical notes, presenting significant challenges for data extraction and utilization. This research paper investigates the issues posed by unstructured clinical notes application of Natural Language Processing (NLP) techniques in the healthcare sector to extract structured patient data from unstructured clinical notes. By utilizing NLP algorithms, healthcare institutions can unlock invaluable insights within EHRs, leading to improved patient care, clinical research, and administrative efficiency. This paper addresses various NLP approaches, the implementation of pre-trained SpaCy and Med7Modelfor extracting structural data, and the potential for future advancements in this critical area of healthcare informatics.
49
Comandè, Giovanni, and Giulia Schneider. "Regulatory Challenges of Data Mining Practices: The Case of the Never-ending Lifecycles of ‘Health Data’." European Journal of Health Law 25, no. 3 (2018): 284–307. http://dx.doi.org/10.1163/15718093-12520368.
Full textAbstract:
Abstract Health data are the most special of the ‘special categories’ of data under Art. 9 of the General Data Protection Regulation (GDPR). The same Art. 9 GDPR prohibits, with broad exceptions, the processing of ‘data concerning health’. Our thesis is that, through data mining technologies, health data have progressively undergone a process of distancing from the healthcare sphere as far as the generation, the processing and the uses are concerned. The case study aims thus to test the endurance of the ‘special category’ of health data in the face of data mining technologies and the never-ending lifecycles of health data they feed. At a more general level of analysis, the case of health data shows that data mining techniques challenge core data protection notions, such as the distinction between sensitive and non-sensitive personal data, requiring a shift in terms of systemic perspectives that the GDPR only partly addresses.
50
Adhwaryu, Himanshu. "Real-Time Data Ecosystems in Insurance: A Comprehensive Analysis of Claims Processing and Policy Management Transformation." International Journal of Advances in Engineering and Management 7, no. 4 (2025): 286–93. https://doi.org/10.35629/5252-0704286293.
Full textAbstract:
Real-time data ecosystems are transforming the property and casualty (P&C) insurance industry through enhanced claims processing, policy management, and customer service capabilities. This transformation encompasses advanced analytics, artificial intelligence, and cloud-native architectures that enable insurers to process and analyze vast amounts of data instantaneously. The implementation of these systems addresses traditional challenges, such as data silos and batch processing limitations, while enabling sophisticated fraud detection and dynamic risk assessment. Through Apache Kafka and Flink architectures, insurers can achieve improved operational efficiency and deliver superior customer experiences. Integrating AI-driven models and machine learning capabilities has revolutionized fraud detection and risk assessment, significantly improving accuracy and processing times.