Academic literature on the topic 'Internet of Multimedia Things (IoMT)'

L'objectif de ma thèse est d'analyser les caractéristiques des signaux inertiels et physiologiques qui générés par les mouvements inhabituels des patients lorsque la crise survient et de développer un algorithme pour détecter la crise. Notre approche dans le chapitre III commence par dériver la moyenne quadratique pour l'ACM et le Gyro, suivie de la normalisation de signaux entiers dans la même plage puis de l'agrégation en un seul signal. Le contrôle du graphique avec ses limites supérieure et inférieure est défini lors de la phase au repos et utilisé pour détecter les crises anormales et pour déclencher une alarme. La procédure dans le chapitre IV de détection s'exécute dans un dispositif de collecte de données portable et déclenche une alarme. Cet algorithme est basé sur la dérivation des mesures instantanées dans une plage de données glissante contenant des mesures inertielles de 3D (ACM), 3D Gyro et de EMG. La différence entre la puissance estimée et la puissance mesurée est utilisé comme entrée pour l'algorithme de détection basé sur la carte de contrôle de Shewhart. Lorsque la différence entre la puissance prévue et la puissance dérivée dépasse les limites [limite inférieure/supérieure] pour plusieurs créneaux consécutifs, une alarme est déclenchée. L'approche que nous proposons permet une bonne détection avec un FAR de 4\% et une sensibilité de 97\%. Notre modèle dans Le chapitre V commence par réduire la dimension des données collectées grâce à l'utilisation de la moyenne quadratique pour dériver un signal de 3D ACM et un signal du 3D Gyro. Avec les 3 signaux dérivés (ACM, Gyro et EMG), nous appliquons le TVP pour dériver un signal utilisé comme entrée pour le mécanisme de détection d'anomalie. La version robuste du z-score est appliquée sur le signal résultant produit pour détecter les déviations associées aux crises avant de déclencher une alarme. Nos résultats expérimentaux montrent que notre approche proposée est robuste contre les mouvements nocturnes et atteint un haut niveau de précision de détection avec un faible FAR. Ensuite, nous comparons les performances de notre approche avec la méthode des passages à zéro calculées à partir de sEMG. Notre approche montre que la précision de détection à l'aide du VTP surpasse le nombre de passages à zéro sur une plage glissante de chevauchement de 1 seconde. Dans le chapitre VI, Les appareils IoMT sont utilisés pour acquérir ACM, Gyro et EMG et pour transmettre les mesures à LPU pour traitement. Lorsque le LPU détecte des changements anormaux dans les mesures, il déclenche une alarme. Notre approche proposée utilise SVM avec option de rejet pour distinguer les crises des activités normales de la vie quotidienne. Les caractéristiques présentant des changements physiologiques de l'activité musculaire et les données inertielles ont été extraites dans LPU et sont utilisées comme entrée pour l'algorithme de détection. L'option de rejet dans SVM est utilisée pour améliorer la fiabilité du système de surveillance et pour réduire les fausses alarmes, où l'utilisateur est averti et a la possibilité de supprimer l'alarme dans son smartphone en l'absence de saisie. Les expériences menées ont prouvé que notre approche proposée peut atteindre une bonne précision pour distinguer les crises des activités normales avec seulement 4% de taux de FAR. Dans chapitre VII, nous proposons un cadre pour empêcher une MitM de perturber les opérations et interdire le déclenchement d'alarmes par le système de surveillance à distance des soins de santé. Pour réduire la consommation d'énergie lours de la transmission normale des données et préserver la confidentialité des données de santé, notre système transmet une signature de plus petite taille dérivée des données acquises avec un code d'authentification de message, où la clé est dérivée de RSSI. Nos résultats expérimentaux montrent que notre approche peut atteindre une précision de détection élevée avec un faible FAR de 3%<br>The goal of my PhD is to investigate the characteristics of inertial and physiological signals via IoMT systems generated by epileptic seizure and to develop an algorithm to detect the seizure. The focus of the algorithms lies in nocturnal seizures where the risk of SUDEP is high because the patients are unsupervised while sleeping. In chapter III analysis we propose an IoMT platform for seizure detection. The proposed framework approach starts by deriving the RMS for ACM and Gyro, followed by the normalization of whole signals (ACM, Gyro and EMG) in the same range, and aggregate all into one signal. The chart’s control with its upper and lower limits are derived in the training phase and used to detect abnormal seizures and to raise an alarm. In chapter IV Our proposed algorithm is based on deriving instantaneous power measurements in a sliding window containing 3D ACM or 3D Gyro or EMG. The residual between forecasted and measured power is used as input for the detection algorithm based on Shewhart Control Chart (SCC). When the difference between forecasted and derived power exceeds chart limits [lower, upper] for several consecutive slots, an alarm is raised. Our proposed approach provides low FAR (4%) and sensitivity of 97%. In Chapter V our proposed method starts by reducing the dimension of collected data using RMS to derive one signal from 3D ACM and one signal from 3D Gyro. With the derived 3 collected signals (ACM, Gyro and EMG), we apply VTP to derive one signal used as input for anomaly detection mechanism. The robust version of z-score is applied on the resulting product signal to detect deviations associated with seizures before raising an alarm. Our experimental results show that our proposed approach is robust against nocturnal movements and achieves a high level of detection accuracy with low false alarm rate. Afterward, we compare the performance of our approach with the zero-crossings method calculated from sEMG. Our approach shows that the detection accuracy using VTP outperforms zero-crossing count over an overlapping sliding window of 1 second. In chapter VI, we propose an approach using the IoMT devices to acquire EMG, ACM and Gyro data and to transmit the measurements to a LPU for processing. When the LPU detects abnormal changes in the measurements, it raises an alarm for assistant. Our proposed approach uses SVM with reject option to distinguish seizures from normal daily life activity. Features presenting physiological changes of muscular activity and inertial data were extracted in LPU and are used as input for the detection algorithm. The reject option in SVM is used to enhance the reliability of the monitoring system and to reduce FAR, where the user is notified and can discard the alarm in his smartphone in the absence of seizure. The conducted experiments proved that our proposed approach could achieve a good accuracy with only 4% of false alarm rate. Finally, since we are using IoMT sensors, which are susceptible to data security issues. We proposed a solution to prevent Man in the Middle (MitM) attack, which can identify healthcare emergencies of monitored patients and replay normal physiological data to prevent the system from raising an alarm. In this chapter, we propose a framework to prevent a MitM from disrupting the operations and prohibiting the remote healthcare monitoring system. To reduce energy consumption for normal data transmission, and preserve the privacy of health data, our framework transmits a smaller size signature derived from acquired data with message authentication code, where the key is derived from Received Signal Strength Indication (RSSI). Our experimental results for emergency detection show that our approach can achieve a high detection accuracy with a low false alarm rate of 3%

Abstract Multimedia Internet of Things (MIoT) refers to IP-enabled Wireless Multimedia Sensor Networks (WMSN) which are used to retrieve, not only scalar data, but also video and audio streams, and still images from the physical environment. Despite the prominent growth in demand of MIoT, several technical challenges still arise when dealing with practical deployments of WMSN. Most technical challenges in MIoT-and IoT in general-are, one way or another, related to the constrained nature of devices. This thesis provides novel contributions towards optimizing the most precious resource of wireless multimedia sensor nodeꟷthe energy. First, the dissertation proposes sleepyCAM power management model, which uses hierarchy in sensor-node architecture to minimize the idle power consumption of a camera node. Second, a prototype is developed to realize the energy saving potential of sleepyCAM in an event driven live video streaming application. Third, a heterogeneous multi-tier WMSN is developed to improve idle power consumption of camera nodes in large scale deployment. It applies hierarchy in sensor-network design, where low-power sensor nodes wake up more energy-consuming multimedia sensor nodes only when needed. A simple power consumption model is also formulated and applied to estimate the battery-life of MIoT devices. Finally, this thesis offers solutions to enhance manageability and service orchestration of WMSN software using container based virtualization, and study their energy implications. The measurement results show that both hierarchy in sensor-node and multi-tier network architecture significantly reduce the idle power consumption of WMSNs. Moreover, the empirical results also indicate that containers add fixed overhead during the boot-up and shutdown phase of the cameras, but nevertheless, have negligible impact during the video streaming session<br>Tiivistelmä Multimediakyvykkäällä esineiden internetillä (Multimedia IoT, MIoT) viitataan IP-pohjaisiin langattomiin sensoriverkkoihin, jotka kykenevät perinteisen skalaarisen sensoridatan lisäksi tallentamaan ympäristöstään myös video- ja ääni- ja kuvadataa. Vaikka multimediakyvykkään esineiden internetin tarve kasvaa jatkuvasti useilla alueilla, teknologian kannattavan hyödyntämisen tiellä on vielä useita haasteita. Suurin osa näistä haasteista liittyy tavalla tai toisella esineiden internetin laitteiden rajoitettuun laitteisto- ja energiakapasiteettiin. Tämä väitöskirja esittelee uusia tapoja multimediakyvykkään esineiden internetin energiatehokkuuden parantamiseen, sillä esineiden internetin laitteiden käytettävissä oleva energiakapasiteetti on tyypillisesti erittäin rajallinen. Työn ensimmäisessä vaiheessa kehitettiin hierarkkinen sensorilaitearkkitehtuuri, sleepyCAM, joka tähtää kameralaitteen valmiustilan energiankulutuksen minimointiin herättämällä laitteen enemmän energiaa kuluttavat multimediasensoritoiminnot vain tarvittaessa. Työn seuraavassa vaiheessa sleepyCAM-mallista kehitettiin prototyyppi, jolla tutkittiin mallin energiansäästöpotentiaalia todellisen maailman videovalvontasovelluksessa. Kolmannessa vaiheessa kehitettiin hierarkkinen sensoriverkkoarkkitehtuuri, jossa matalamman energiatason sensorilaitteet herättävät enemmän energiaa kuluttavia multimediasensorilaitteita vain tarvittaessa, mikä parantaa valmiustilan energiatehokkuutta laajemmissa multimediasensoriverkoissa. Työssä kehitettiin myös yksinkertainen energiankulutusmalli multimediakyvykkäiden esineiden internetin laitteiden akunkeston arviointiin. Lopuksi väitöskirjassa tutkittiin multimediasensoriverkon palveluiden hallittavuuden parantamista konttipohjaisella orkestroidulla virtualisoinnilla sekä tutkittiin ratkaisun vaikutuksia energiankulutukseen. Prototyypeillä tehdyt todellisen maailman mittaukset osoittavat, että sekä sleepyCAM että hierarkkinen verkkoarkkitehtuuri vähentävät huomattavasti multimediasensorijärjestelmän kokonaisenergiankulutusta. Tulokset osoittavat myös, että virtualisoinnin käyttö lisää energiankulutusta videosensorilaitteen käynnistyksen ja sammutuksen yhteydessä, mutta videonsiirron aikana konttipohjaisen virtualisoinnin vaikutus energiankulutukseen on olematon

The prevalent visions of ambient intelligence leverage natural interactions between users and available services in a smart space. In recent years, we have seen a huge interest from industry and academia in using handheld devices to interact with things, places and people in the real world. To facilitate such interactions, things are usually annotated with RFID tags or visual markers. These tags or markers are read by a handheld device equipped with an integrated RFID reader or a camera, in order to fetch related information and initiate further actions. Interacting with the Internet of Things (IoT) in a real environment has become increasingly desirable and feasible. This thesis contributes to the domain of physical interactions with IoT; however, we use a spatial-geometric approach instead of RFID or marker based solutions. Using this approach, for example, a user can point his/her handheld device to an annotated thing, from a distance, for the purpose of interaction. The pointing direction and location is determined based on the fusion of the mobile position and of the accelerometer data of the handheld device. To annotate things, their geometric coordinates are specified and related information or services are associated to them. In this thesis, we present a comprehensive and extensible framework to integrate various physical interactions with IoT into multimedia applications. The framework supports the implementations of pointMe, touchMe, and context-aware based interactions with geometrically annotated IoT. We define specific methods and practices that can be incorporated in order to build the interactions. We realize smart home, atlas learning, presentation interaction, smart haptic interaction, and learning based video interaction game prototypes in order to perform experiments and demonstrate the applicability and potential of the proposed geometric based annotation approach. In the analysis of the interaction techniques of the prototypes, we present the advantages and disadvantages of the geometric based annotation of IoT as seen by potential users, in comparison to RFID tags or visual markers based approaches.

Epileptic seizure presents a formidable threat to the life of its sufferers, leaving them unconscious within seconds of its onset. Having a mortality rate that is at least twice that of the general population, it is a true cause for concern which has gained ample attention from various research communities. About 800 million people in the world will have at least one seizure experience in their lifespan. Injuries sustained during a seizure crisis are one of the leading causes of death in epilepsy. These can be prevented by an early detection of seizure accompanied by a timely intervention mechanism. The research presented in this dissertation explores Kriging methods to exploit spatial correlations of electroencephalogram (EEG) Signals from the brain, for fast and accurate seizure detection in the Internet of Medical Things (IoMT) using edge computing paradigms, by modeling the brain as a three-dimensional spatial object, similar to a geographical panorama. This dissertation proposes basic, hierarchical and distributed Kriging models, with a deep neural network (DNN) wrapper in some instances. Experimental results from the models are highly promising for real-time seizure detection, with excellent performance in seizure detection latency and training time, as well as accuracy, sensitivity and specificity which compare well with other notable seizure detection research projects.

With the rapid growth of online devices, a new concept of Internet of Things (IoT) is emerging in which everyday devices will be connected to the Internet. As the number of devices in IoT is increasing, so is the complexity of the interactions between user and devices. There is a need to design intelligent user interfaces that could assist users in interactions. Many studies have been conducted on different interaction techniques such as proxemic and gesture interaction in order to propose an intuitive and intelligent system for controlling multimedia devices over the IoT, though most could not propose a universal solution. The present study proposes a proximity-based and gesture-enabled user interface for multimedia devices over IoT. The proposed method employs a cloud-based decision engine to support user to choose and interact with the most appropriate device, reliving the user from the burden of enumerating available devices manually. The decision engine observes the multimedia content and device properties, learns user preferences adaptively, and automatically recommends the most appropriate device to interact. In addition to that, the proposed system uses proximity information to find the user among people and provides her/him gesture control services. Furthermore, a new hand gesture vocabulary is proposed for controlling multimedia devices through conducting a multiphase elicitation study. The main advantage of this vocabulary is that it can be used for all multimedia devices. Both device recommendation system and gesture vocabulary are evaluated. The device recommendation system evaluation shows that the users agree with the proposed interaction 70% of the times. Moreover, the average agreement score of the proposed gesture vocabulary (0.56) exceeds the score of similar studies. An external user evaluation study shows that the average score of being a good-match is 4.08 out of 5 and the average of ease-of-performance equals to 4.21 out of 5. The memory test reveals that the proposed vocabulary is easy to remember since participants could remember and perform gestures in 3.13 seconds on average. In addition to that, the average accuracy of remembering gestures equals to 91.54%.

Ci troviamo in un mondo in costante cambiamento ed evoluzione, in particolar modo nel settore informatico. È proprio in questo continuo processo di evoluzione che spicca l’IoT ovvero Interent Of Things (Internet delle cose). Oggetti, anche di tutti i giorni, collegati ad Internet capaci di scambiarsi messaggi e di comunicare tra loro. Kevin Ashton, pioniere dell’IoT, dice che questa sarà la prossima rivoluzione tecnologica come lo è stato a suo tempo Internet. Tuttavia, come sarà discusso in questa tesi, alcuni ambiti di applicazione dell’IoT come il sistema medico ed ospedaliero non sono ancora del tutto uniformati ma anzi, presentano vari ostacoli nell’ottica di un sistema interoperabile ed intelligente. Per quanto riguarda quest’ultimo caso si parla di Internet of Medical Things(IoMT), Internet of Healthcare Things (IoHT) o ancora Medicina 4.0 (QuartaRivoluzione Industraile in ambito medico). Possiamo pensare ad un ospedale con vari dispositivi capaci di condividere dati ed informazioni tra loro e di mettere adisposizione questi dati in tempo reale al personale sanitario e al paziente anche a distanza.

The increasing need to safeguard patient data in Internet of Medical Things (IoMT) devices highlights the critical importance of reducing vulnerabilities within these systems. The widespread adoption of IoMT has transformed healthcare by enabling continuous remote patient monitoring (RPM), which enhances patient outcomes and optimizes healthcare delivery. However, the integration of IoMT devices into healthcare systems presents significant security challenges, particularly in protecting sensitive patient data and ensuring the reliability of medical devices. The diversity of data formats used by various vendors in RPM complicates data aggregation and fusion, thereby hindering overall cybersecurity efforts. This thesis proposes a novel semantic framework for vulnerability detection in RPM settings within the IoMT system. The framework addresses interoperability, heterogeneity, and integration challenges through meaningful data aggregation. The core of this framework is a domain ontology that captures the semantics of concepts and properties related to the primary security aspects of IoT medical devices. This ontology is supported by a comprehensive ruleset and complex queries over aggregated knowledge. Additionally, the implementation integrates medical device data with the National Vulnerability Database (NVD) via an API, enabling real-time detection of vulnerabilities and improving the security of RPM systems. By capturing the semantics of medical devices and network components, the proposed semantic model facilitates partial automation in detecting network anomalies and vulnerabilities. A logic-based ruleset enhances the system’s robustness and efficiency, while its reasoning capabilities enable the identification of potential vulnerabilities and anomalies in IoMT systems, thereby improving security measures in remote monitoring settings. The semantic framework also supports knowledge graph visualization and efficient querying through SPARQL. The knowledge graph provides a structured representation of interconnected data and stores Cyber Threat Intelligence (CTI) to enhance data integration, visualization, and semantic enrichment. The query mechanism enables healthcare providers to extract valuable insights from IoMT data, notifying them about new system vulnerabilities or vulnerable medical devices. This demonstrates the impact of vulnerabilities on cybersecurity requirements (Confidentiality, Integrity, and Availability) and facilitates countermeasures based on severity. Consequently, the framework promotes timely decision-making, enhancing the overall efficiency and effectiveness of IoMT systems. The semantic framework is validated through various use cases and existing frameworks, demonstrating its effectiveness and robustness in vulnerability detection within the domain of IoMT security.

Les réseaux de capteurs visuels sans fil basés sur les réseaux maillés IEEE 802.11 sont des solutions efficaces et adaptées aux systèmes de vidéosurveillance pour surveiller les intrusions dans des zones sélectionnées. Les réseaux de capteurs visuels basés sur IEEE 802.11 offrent des transmissions vidéo à haut débit mais souffrent de problèmes d'inefficacité énergétique. De plus, la transmission vidéo dans les réseaux de capteurs visuels nécessite une qualité de service (QoS) stricte en termes de bande passante et de délai. En outre, il est difficile de réduire la consommation énergétique globale du réseau tout en garantissant une qualité de service garantie en termes de bande passante et de délai dans les réseaux de capteurs visuels sans fil à énergie limitée. La principale contribution de cette thèse est de fournir un réseau de vidéosurveillance économe en énergie sans compromettre l'exigence de qualité de service de la transmission vidéo. Premièrement, nous proposons une nouvelle architecture de réseau hybride IoT pour un système de vidéosurveillance qui détecte et suit un intrus dans la zone de surveillance. Le réseau IoT hybride intègre les réseaux de capteurs visuels multi-sauts basés sur IEEE 802.11 et le réseau LoRa pour fournir un système de vidéosurveillance autonome, économe en énergie et à haut débit. Tirant parti des caractéristiques du réseau LoRa, le réseau LoRa est utilisé comme un réseau toujours actif pour la détection et le suivi préliminaires des mouvements. De plus, le réseau LoRa décide également quels nœuds de capteurs visuels réveiller en fonction des informations de suivi. Le filtre de Kalman est étudié pour suivre la trajectoire de l'intrus à partir des mesures de bruit des capteurs de mouvement à faible puissance afin d'activer uniquement les nœuds de capteurs visuels le long de la trajectoire de l'intrus pour fournir une surveillance vidéo efficace. Nous avons montré par simulation que le filtre de Kalman estime et prédit la trajectoire de l'intrus avec une précision raisonnable. De plus, l'approche de réseau hybride IoT proposée réduit considérablement la consommation d'énergie par rapport à un réseau de capteurs visuels à un seul niveau de surveillance continue traditionnelle et toujours active. Ensuite, la contribution de cette thèse se concentre sur un mécanisme de routage sensible à l'énergie et QoS pour le réseau de capteurs visuels multi-sauts basé sur IEEE 802.11 du réseau hybride IoT. Nous proposons un algorithme de routage qui route un ensemble de flux vidéo vers la passerelle avec une QoS garantie en termes de bande passante et de délai tout en minimisant le nombre de nœuds capteurs visuels impliqués dans le routage. Cela maximise le nombre de nœuds pouvant être complètement désactivés pour optimiser la consommation énergétique globale du réseau sans compromettre les performances QoS. Le problème de routage proposé est formulé comme un programme linéaire entier (ILP) et résolu à l'aide d'un algorithme branch-and-bound. Grâce à la simulation informatique, les performances de l'approche proposée sont comparées aux algorithmes de routage de pointe existants dans la littérature. Les résultats montrent clairement que le mécanisme proposé permet d'économiser une quantité significative de la consommation d'énergie globale tout en garantissant la QoS en termes de bande passante et de délai<br>Wireless visual sensor networks based on IEEE 802.11 mesh networks are effective and suitable solutions for video surveillance systems in monitoring intrusions in selected areas. The IEEE 802.11-based visual sensor networks offer high bit rate video transmissions but suffer from energy inefficiency issues. Moreover, the video transmission in the visual sensor networks requires strict quality of service (QoS) in terms of bandwidth and delay. Also, it is challenging to decrease the overall energy consumption of the network while assuring guaranteed QoS in terms of bandwidth and delay in energy-constrained wireless visual sensor networks. The main contribution of this dissertation is to provide an energy-efficient video surveillance network without compromising the QoS requirement of video transmission. First, we propose a new hybrid IoT network architecture for a video surveillance system that detects and tracks an intruder in the monitoring area. The hybrid IoT network integrates the IEEE 802.11-based multi-hop visual Sensor Networks and LoRa network to provide an autonomous, energy-efficient, high-bitrate video surveillance system. Leveraging the LoRa network characteristics, the LoRa network is utilized as an always-active network for preliminary motion detection and tracking. Moreover, the LoRa network also decides which visual sensor nodes to wake up depending on the tracking information. The Kalman filter is investigated to track the intruder's trajectory from noise measurements of low-power motion sensors to activate only the visual sensor nodes along the intruder's trajectory to provide effective video vigilance. We showed through simulation that Kalman filter estimates and predicts intruder trajectory with reasonable accuracy. Moreover, the proposed hybrid IoT network approach reduces energy consumption significantly compared with a traditional, always active continuous monitoring single-tier visual sensor network. Next, the contribution of this dissertation focuses on an energy-aware and QoS routing mechanism for the IEEE 802.11-based multi-hop visual sensor network of the hybrid IoT network. We propose a routing algorithm that routes a set of video streams to the gateway with guaranteed QoS in terms of bandwidth and delay while minimizing the number of visual sensor nodes that are involved in routing. This maximizes the number of nodes that can be turned off completely to optimize the overall energy consumption of the network without compromising QoS performance. The proposed routing problem is formulated as an Integer Linear Program (ILP) and solved using the branch-and-bound algorithm. Through computer simulation, the performance of the proposed approach is compared with the existing state-of-the-art routing algorithms from the literature. The results clearly show that the proposed mechanism saves a significant amount of the overall energy consumption while guaranteeing QoS in terms of bandwidth and delay

yes<br>Welcome to this special issue of the Future Generation Computer Systems (FGCS) journal. The special issue compiles seven technical contributions that significantly advance the state-of-the-art in exploration of Internet of Things (IoT) generated big data using semantic web techniques and technologies.