To see the other types of publications on this topic, follow the link: Multi-camera tracking data.
Journal articles on the topic 'Multi-camera tracking data'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Multi-camera tracking data.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Mahalakshmi, N., and S. R. Saranya. "Robust Visual Tracking for Multiple Targets with Data Association and Track Management." International Journal of Advance Research and Innovation 3, no. 2 (2015): 68–71. http://dx.doi.org/10.51976/ijari.321516.
Full textAbstract:
Multi-object tracking is still a challenging task in computer vision. A robust approach is proposed to realize multi-object tracking using camera networks. Detection algorithms are utilized to detect object regions with confidence scores for initialization of individual particle filters. Since data association is the key issue in Tracking-by-Detection mechanism, an efficient HOG algorithm and SVM classifier algorithm are used for tracking multiple objects. Furthermore, tracking in single cameras is realized by a greedy matching method. Afterwards, 3D geometry positions are obtained from the rectangular relationship between objects. Corresponding objects are tracked in cameras to take the advantages of camera based tracking. The proposed algorithm performs online and does not need any information about the scene, any restrictions of enter-and-exit zones, no assumption of areas where objects are moving on and can be extended to any class of object tracking. Experimental results show the benefits of using camera by the higher accuracy and detect the objects.
2
Bamrungthai, Pongsakon, and Viboon Sangveraphunsiri. "CU-Track: A Multi-Camera Framework for Real-Time Multi-Object Tracking." Applied Mechanics and Materials 415 (September 2013): 325–32. http://dx.doi.org/10.4028/www.scientific.net/amm.415.325.
Full textAbstract:
This paper presents CU-Track, a multi-camera framework for real-time multi-object tracking. The developed framework includes a processing unit, the target object, and the multi-object tracking algorithm. A PC-cluster has been developed as the processing unit of the framework to process data in real-time. To setup the PC-cluster, two PCs are connected by using PCI interface cards that memory can be shared between the two PCs to ensure high speed data transfer and low latency. A novel mechanism for PC-to-PC communication is proposed. It is realized by a dedicated software processing module called the Cluster Module. Six processing modules have been implemented to realize system operations such as camera calibration, camera synchronization and 3D reconstruction of each target. Multiple spherical objects with the same size are used as the targets to be tracked. Two configurations of them, active and passive, can be used for tracking by the system. The algorithm based on Kalman filter and nearest neighbor searching is developed for multi-object tracking. Two applications have been implemented on the system, which confirm the validity and effectiveness of the developed framework.
3
Sharma, Anil, Saket Anand, and Sanjit K. Kaul. "Reinforcement Learning Based Querying in Camera Networks for Efficient Target Tracking." Proceedings of the International Conference on Automated Planning and Scheduling 29 (May 25, 2021): 555–63. http://dx.doi.org/10.1609/icaps.v29i1.3522.
Full textAbstract:
Surveillance camera networks are a useful monitoring infrastructure that can be used for various visual analytics applications, where high-level inferences and predictions could be made based on target tracking across the network. Most multi-camera tracking works focus on re-identification problems and trajectory association problems. However, as camera networks grow in size, the volume of data generated is humongous, and scalable processing of this data is imperative for deploying practical solutions. In this paper, we address the largely overlooked problem of scheduling cameras for processing by selecting one where the target is most likely to appear next. The inter-camera handover can then be performed on the selected cameras via re-identification or another target association technique. We model this scheduling problem using reinforcement learning and learn the camera selection policy using Q-learning. We do not assume the knowledge of the camera network topology but we observe that the resulting policy implicitly learns it. We evaluate our approach using NLPR MCT dataset, which is a real multi-camera multi-target tracking benchmark and show that the proposed policy substantially reduces the number of frames required to be processed at the cost of a small reduction in recall.
4
Cant, Olivia, Stephanie Kovalchik, Rod Cross, and Machar Reid. "Validation of ball spin estimates in tennis from multi-camera tracking data." Journal of Sports Sciences 38, no. 3 (November 29, 2019): 296–303. http://dx.doi.org/10.1080/02640414.2019.1697189.
Full text
5
Nikodem, Maciej, Mariusz Słabicki, Tomasz Surmacz, Paweł Mrówka, and Cezary Dołęga. "Multi-Camera Vehicle Tracking Using Edge Computing and Low-Power Communication." Sensors 20, no. 11 (June 11, 2020): 3334. http://dx.doi.org/10.3390/s20113334.
Full textAbstract:
Typical approaches to visual vehicle tracking across large area require several cameras and complex algorithms to detect, identify and track the vehicle route. Due to memory requirements, computational complexity and hardware constrains, the video images are transmitted to a dedicated workstation equipped with powerful graphic processing units. However, this requires large volumes of data to be transmitted and may raise privacy issues. This paper presents a dedicated deep learning detection and tracking algorithms that can be run directly on the camera’s embedded system. This method significantly reduces the stream of data from the cameras, reduces the required communication bandwidth and expands the range of communication technologies to use. Consequently, it allows to use short-range radio communication to transmit vehicle-related information directly between the cameras, and implement the multi-camera tracking directly in the cameras. The proposed solution includes detection and tracking algorithms, and a dedicated low-power short-range communication for multi-target multi-camera tracking systems that can be applied in parking and intersection scenarios. System components were evaluated in various scenarios including different environmental and weather conditions.
6
Lyu, Pengfei, Minxiang Wei, and Yuwei Wu. "Multi-Vehicle Tracking Based on Monocular Camera in Driver View." Applied Sciences 12, no. 23 (November 30, 2022): 12244. http://dx.doi.org/10.3390/app122312244.
Full textAbstract:
Multi-vehicle tracking is used in advanced driver assistance systems to track obstacles, which is fundamental for high-level tasks. It requires real-time performance while dealing with object illumination variations and deformations. To this end, we propose a novel multi-vehicle tracking algorithm based on a monocular camera in driver view. It follows the tracking-by-detection paradigm and integrates detection and appearance descriptors into a single network. The one-stage detection approach consists of a backbone, a modified BiFPN as a neck layer, and three prediction heads. The data association consists of a two-step matching strategy together with a Kalman filter. Experimental results demonstrate that the proposed approach outperforms state-of-the-art algorithms. It is also able to solve the tracking problem in driving scenarios while maintaining 16 FPS on the test dataset.
7
Straw, Andrew D., Kristin Branson, Titus R. Neumann, and Michael H. Dickinson. "Multi-camera real-time three-dimensional tracking of multiple flying animals." Journal of The Royal Society Interface 8, no. 56 (July 14, 2010): 395–409. http://dx.doi.org/10.1098/rsif.2010.0230.
Full textAbstract:
Automated tracking of animal movement allows analyses that would not otherwise be possible by providing great quantities of data. The additional capability of tracking in real time—with minimal latency—opens up the experimental possibility of manipulating sensory feedback, thus allowing detailed explorations of the neural basis for control of behaviour. Here, we describe a system capable of tracking the three-dimensional position and body orientation of animals such as flies and birds. The system operates with less than 40 ms latency and can track multiple animals simultaneously. To achieve these results, a multi-target tracking algorithm was developed based on the extended Kalman filter and the nearest neighbour standard filter data association algorithm. In one implementation, an 11-camera system is capable of tracking three flies simultaneously at 60 frames per second using a gigabit network of nine standard Intel Pentium 4 and Core 2 Duo computers. This manuscript presents the rationale and details of the algorithms employed and shows three implementations of the system. An experiment was performed using the tracking system to measure the effect of visual contrast on the flight speed of Drosophila melanogaster . At low contrasts, speed is more variable and faster on average than at high contrasts. Thus, the system is already a useful tool to study the neurobiology and behaviour of freely flying animals. If combined with other techniques, such as ‘virtual reality’-type computer graphics or genetic manipulation, the tracking system would offer a powerful new way to investigate the biology of flying animals.
8
Yi, Chunlei, Kunfan Zhang, and Nengling Peng. "A multi-sensor fusion and object tracking algorithm for self-driving vehicles." Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering 233, no. 9 (August 2019): 2293–300. http://dx.doi.org/10.1177/0954407019867492.
Full textAbstract:
Vehicles need to detect threats on the road, anticipate emerging dangerous driving situations and take proactive actions for collision avoidance. Therefore, the study on methods of target detection and recognition are of practical value to a self-driving system. However, single sensor has its weakness, such as poor weather adaptability with lidar and camera. In this article, we propose a novel spatial calibration method based on multi-sensor systems, and the approach utilizes rotation and translation of the coordinate system. The validity of the proposed spatial calibration method is tested through comparisons with the data calibrated. In addition, a multi-sensor fusion and object tracking algorithm based on target level to detect and recognize targets is tested. Sensors contain lidar, radar and camera. The multi-sensor fusion and object tracking algorithm takes advantages of various sensors such as target location from lidar, target velocity from radar and target type from camera. Besides, multi-sensor fusion and object tracking algorithm can achieve information redundancy and increase environmental adaptability. Compared with the results of single sensor, this new approach is verified to have the accuracy of location, velocity and recognition by real data.
9
Wu, Zhihong, Fuxiang Li, Yuan Zhu, Ke Lu, and Mingzhi Wu. "Design of a Robust System Architecture for Tracking Vehicle on Highway Based on Monocular Camera." Sensors 22, no. 9 (April 27, 2022): 3359. http://dx.doi.org/10.3390/s22093359.
Full textAbstract:
Multi-Target tracking is a central aspect of modeling the environment of autonomous vehicles. A mono camera is a necessary component in the autonomous driving system. One of the biggest advantages of the mono camera is it can give out the type of vehicle and cameras are the only sensors able to interpret 2D information such as road signs or lane markings. Besides this, it has the advantage of estimating the lateral velocity of the moving object. The mono camera is now being used by companies all over the world to build autonomous vehicles. In the expressway scenario, the forward-looking camera can generate a raw picture to extract information from and finally achieve tracking multiple vehicles at the same time. A multi-object tracking system, which is composed of a convolution neural network module, depth estimation module, kinematic state estimation module, data association module, and track management module, is needed. This paper applies the YOLO detection algorithm combined with the depth estimation algorithm, Extend Kalman Filter, and Nearest Neighbor algorithm with a gating trick to build the tracking system. Finally, the tracking system is tested on the vehicle equipped with a forward mono camera, and the results show that the lateral and longitudinal position and velocity can satisfy the need for Adaptive Cruise Control (ACC), Navigation On Pilot (NOP), Auto Emergency Braking (AEB), and other applications.
10
Gai, Wei, Meng Qi, Mingcong Ma, Lu Wang, Chenglei Yang, Juan Liu, Yulong Bian, Gerard de Melo, Shijun Liu, and Xiangxu Meng. "Employing Shadows for Multi-Person Tracking Based on a Single RGB-D Camera." Sensors 20, no. 4 (February 15, 2020): 1056. http://dx.doi.org/10.3390/s20041056.
Full textAbstract:
Although there are many algorithms to track people that are walking, existing methods mostly fail to cope with occluded bodies in the setting of multi-person tracking with one camera. In this paper, we propose a method to use people’s shadows as a clue to track them instead of treating shadows as mere noise. We introduce a novel method to track multiple people by fusing shadow data from the RGB image with skeleton data, both of which are captured by a single RGB Depth (RGB-D) camera. Skeletal tracking provides the positions of people that can be captured directly, while their shadows are used to track them when they are no longer visible. Our experiments confirm that this method can efficiently handle full occlusions. It thus has substantial value in resolving the occlusion problem in multi-person tracking, even with other kinds of cameras.
11
Khule, Shruti, Supriya Jaybhay, Pranjal Metkari, and Prof Balasaheb Balkhande. "Smart Surveillance System Real-Time Multi-Person Multi-Camera Tracking at the Edge." International Journal for Research in Applied Science and Engineering Technology 10, no. 4 (April 30, 2022): 1196–98. http://dx.doi.org/10.22214/ijraset.2022.40954.
Full textAbstract:
Abstract: Nowadays, new Artificial Intelligence (AI) and Deep Learning based processing methods are replacing traditional computer vision algorithms. On the other hand, the rise of the Internet of Things (IoT) and edge computing, has led to many research works that propose distributed video-surveillance systems based on this notion. Usually, the advanced systems process massive volumes of data in different computing facilities. Instead, this paper presents a system that incorporates AI algorithms into low-power embedded devices. The computer vision technique, which is commonly used in surveillance applications, is designed to identify, count, and monitor people's movements in the area. A distributed camera system is required for this application. The proposed AI system detects people in the monitored area using a MobileNet-SSD architecture. This algorithm can keep track of people in the surveillance providing the number of people present in the frame. The proposed framework is both privacy-aware and scalable supporting a processing pipeline on the edge consisting of person detection, tracking and robust person re-identification. The expected results show the usefulness of deploying this smart camera node throughout a distributed surveillance system. Keywords: Edge Analytics, Person detection, Person re-identification, Deep learning, embedded systems, artificial intelligence
12
Dinc, Semih, Farbod Fahimi, and Ramazan Aygun. "Mirage: an O(n) time analytical solution to 3D camera pose estimation with multi-camera support." Robotica 35, no. 12 (February 16, 2017): 2278–96. http://dx.doi.org/10.1017/s0263574716000874.
Full textAbstract:
SUMMARYMirage is a camera pose estimation method that analytically solves pose parameters in linear time for multi-camera systems. It utilizes a reference camera pose to calculate the pose by minimizing the 2D projection error between reference and actual pixel coordinates. Previously, Mirage has been successfully applied to trajectory tracking (visual servoing) problem. In this study, a comprehensive evaluation of Mirage is performed by particularly focusing on the area of camera pose estimation. Experiments have been performed using simulated and real data on noisy and noise-free environments. The results are compared with the state-of-the-art techniques. Mirage outperforms other methods by generating fast and accurate results in all tested environments.
13
Iaboni, Craig, Deepan Lobo, Ji-Won Choi, and Pramod Abichandani. "Event-Based Motion Capture System for Online Multi-Quadrotor Localization and Tracking." Sensors 22, no. 9 (April 23, 2022): 3240. http://dx.doi.org/10.3390/s22093240.
Full textAbstract:
Motion capture systems are crucial in developing multi-quadrotor systems due to their ability to provide fast and accurate ground truth measurements for tracking and control. This paper presents the implementation details and experimental validation of a relatively low-cost motion-capture system for multi-quadrotor motion planning using an event camera. The real-time, multi-quadrotor detection and tracking tasks are performed using a deep learning network You-Only-Look-Once (YOLOv5) and a k-dimensional (k-d) tree, respectively. An optimization-based decentralized motion planning algorithm is implemented to demonstrate the effectiveness of this motion capture system. Extensive experimental evaluations were performed to (1) compare the performance of four deep-learning algorithms for high-speed multi-quadrotor detection on event-based data, (2) study precision, recall, and F1 scores as functions of lighting conditions and camera motion, and (3) investigate the scalability of this system as a function of the number of quadrotors flying in the arena. Comparative analysis of the deep learning algorithms on a consumer-grade GPU demonstrates a 4.8× to 12× sampling/inference rate advantage that YOLOv5 provides over representative one- and two-stage detectors and a 1.14× advantage over YOLOv4. In terms of precision and recall, YOLOv5 performed 15% to 18% and 27% to 41% better than representative state-of-the-art deep learning networks. Graceful detection and tracking performance degradation was observed in the face of progressively darker ambient light conditions. Despite severe camera motion, YOLOv5 precision and recall values of 94% and 98% were achieved, respectively. Finally, experiments involving up to six indoor quadrotors demonstrated the scalability of this approach. This paper also presents the first open-source event camera dataset in the literature, featuring over 10,000 fully annotated images of multiple quadrotors operating in indoor and outdoor environments.
14
Sheu, Ruey-Kai, Mayuresh Pardeshi, Lun-Chi Chen, and Shyan-Ming Yuan. "STAM-CCF: Suspicious Tracking Across Multiple Camera Based on Correlation Filters." Sensors 19, no. 13 (July 9, 2019): 3016. http://dx.doi.org/10.3390/s19133016.
Full textAbstract:
There is strong demand for real-time suspicious tracking across multiple cameras in intelligent video surveillance for public areas, such as universities, airports and factories. Most criminal events show that the nature of suspicious behavior are carried out by un-known people who try to hide themselves as much as possible. Previous learning-based studies collected a large volume data set to train a learning model to detect humans across multiple cameras but failed to recognize newcomers. There are also several feature-based studies aimed to identify humans within-camera tracking. It would be very difficult for those methods to get necessary feature information in multi-camera scenarios and scenes. It is the purpose of this study to design and implement a suspicious tracking mechanism across multiple cameras based on correlation filters, called suspicious tracking across multiple cameras based on correlation filters (STAM-CCF). By leveraging the geographical information of cameras and YOLO object detection framework, STAM-CCF adjusts human identification and prevents errors caused by information loss in case of object occlusion and overlapping for within-camera tracking cases. STAM-CCF also introduces a camera correlation model and a two-stage gait recognition strategy to deal with problems of re-identification across multiple cameras. Experimental results show that the proposed method performs well with highly acceptable accuracy. The evidences also show that the proposed STAM-CCF method can continuously recognize suspicious behavior within-camera tracking and re-identify it successfully across multiple cameras.
15
Zhu, Jianguo, Yanyan Dong, Xiaobin Xu, Jiewen Wang, and Jianqing Wu. "A spatiotemporal registration method between roadside lidar and camera." Journal of Physics: Conference Series 2370, no. 1 (November 1, 2022): 012003. http://dx.doi.org/10.1088/1742-6596/2370/1/012003.
Full textAbstract:
With the rapid development of intelligent transportation, it is of great significance to improve road traffic safety and traffic efficiency by tracking road users in real time. In this paper, a roadside multi-source data acquisition platform is designed based on two main sensors, lidar and camera, to perform time synchronization between lidar data and camera data. The point cloud data and image data of the same frame are the research objects. Based on Zhang Zhengyou’s calibration method and the plane target model theory, the camera internal parameters and the joint external parameters of the lidar and camera are respectively calibrated to obtain the internal and external parameter matrix. The error is 0.09 pixels, and the external parameter calibration error is 0.22 pixels. The registration test of lidar and camera under different roadside layout conditions is designed to verify the adaptability of the registration method. When the camera is arranged and installed, the distance between the horizontal and vertical directions should not exceed 90 cm.
16
Trieu, Hang, Per Bergström, Mikael Sjödahl, J. Gunnar I. Hellström, Patrik Andreasson, and Henrik Lycksam. "Photogrammetry for Free Surface Flow Velocity Measurement: From Laboratory to Field Measurements." Water 13, no. 12 (June 17, 2021): 1675. http://dx.doi.org/10.3390/w13121675.
Full textAbstract:
This study describes a multi-camera photogrammetric approach to measure the 3D velocity of free surface flow. The properties of the camera system and particle tracking velocimetry (PTV) algorithm were first investigated in a measurement of a laboratory open channel flow to prepare for field measurements. The in situ camera calibration methods corresponding to the two measurement situations were applied to mitigate the instability of the camera mechanism and camera geometry. There are two photogrammetry-based PTV algorithms presented in this study regarding different types of surface particles employed on the water flow. While the first algorithm uses the particle tracking method applied for individual particles, the second algorithm is based on correlation-based particle clustering tracking applied for clusters of small size particles. In the laboratory, reference data are provided by particle image velocimetry (PIV) and laser Doppler velocimetry (LDV). The differences in velocities measured by photogrammetry and PIV, photogrammetry and LDV are 0.1% and 3.6%, respectively. At a natural river, the change of discharges between two measurement times is found to be 15%, and the corresponding value reported regarding mass flow through a nearby hydropower plant is 20%. The outcomes reveal that the method can provide a reliable estimation of 3D surface velocity with sufficient accuracy.
17
Guo, Xiaoxiao, Yuansheng Liu, Qixue Zhong, and Mengna Chai. "Research on Moving Target Tracking Algorithm Based on Lidar and Visual Fusion." Journal of Advanced Computational Intelligence and Intelligent Informatics 22, no. 5 (September 20, 2018): 593–601. http://dx.doi.org/10.20965/jaciii.2018.p0593.
Full textAbstract:
Multi-sensor fusion and target tracking are two key technologies for the environmental awareness system of autonomous vehicles. In this paper, a moving target tracking method based on the fusion of Lidar and binocular camera is proposed. Firstly, the position information obtained by the two types of sensors is fused at decision level by using adaptive weighting algorithm, and then the Joint Probability Data Association (JPDA) algorithm is correlated with the result of fusion to achieve multi-target tracking. Tested at a curve in the campus and compared with the Extended Kalman Filter (EKF) algorithm, the experimental results show that this algorithm can effectively overcome the limitation of a single sensor and track more accurately.
18
Svanström, Fredrik, Fernando Alonso-Fernandez, and Cristofer Englund. "Drone Detection and Tracking in Real-Time by Fusion of Different Sensing Modalities." Drones 6, no. 11 (October 26, 2022): 317. http://dx.doi.org/10.3390/drones6110317.
Full textAbstract:
Automatic detection of flying drones is a key issue where its presence, especially if unauthorized, can create risky situations or compromise security. Here, we design and evaluate a multi-sensor drone detection system. In conjunction with standard video cameras and microphone sensors, we explore the use of thermal infrared cameras, pointed out as a feasible and promising solution that is scarcely addressed in the related literature. Our solution integrates a fish-eye camera as well to monitor a wider part of the sky and steer the other cameras towards objects of interest. The sensing solutions are complemented with an ADS-B receiver, a GPS receiver, and a radar module. However, our final deployment has not included the latter due to its limited detection range. The thermal camera is shown to be a feasible solution as good as the video camera, even if the camera employed here has a lower resolution. Two other novelties of our work are the creation of a new public dataset of multi-sensor annotated data that expands the number of classes compared to existing ones, as well as the study of the detector performance as a function of the sensor-to-target distance. Sensor fusion is also explored, showing that the system can be made more robust in this way, mitigating false detections of the individual sensors.
19
Chen, Ting, Andrea Pennisi, Zhi Li, Yanning Zhang, and Hichem Sahli. "A Hierarchical Association Framework for Multi-Object Tracking in Airborne Videos." Remote Sensing 10, no. 9 (August 23, 2018): 1347. http://dx.doi.org/10.3390/rs10091347.
Full textAbstract:
Multi-Object Tracking (MOT) in airborne videos is a challenging problem due to the uncertain airborne vehicle motion, vibrations of the mounted camera, unreliable detections, changes of size, appearance and motion of the moving objects and occlusions caused by the interaction between moving and static objects in the scene. To deal with these problems, this work proposes a four-stage hierarchical association framework for multiple object tracking in airborne video. The proposed framework combines Data Association-based Tracking (DAT) methods and target tracking using a compressive tracking approach, to robustly track objects in complex airborne surveillance scenes. In each association stage, different sets of tracklets and detections are associated to efficiently handle local tracklet generation, local trajectory construction, global drifting tracklet correction and global fragmented tracklet linking. Experiments with challenging airborne videos show significant tracking improvement compared to existing state-of-the-art methods.
20
Korkalo, Otto, and Tapio Takala. "Measurement Noise Model for Depth Camera-Based People Tracking." Sensors 21, no. 13 (June 30, 2021): 4488. http://dx.doi.org/10.3390/s21134488.
Full textAbstract:
Depth cameras are widely used in people tracking applications. They typically suffer from significant range measurement noise, which causes uncertainty in the detections made of the people. The data fusion, state estimation and data association tasks require that the measurement uncertainty is modelled, especially in multi-sensor systems. Measurement noise models for different kinds of depth sensors have been proposed, however, the existing approaches require manual calibration procedures which can be impractical to conduct in real-life scenarios. In this paper, we present a new measurement noise model for depth camera-based people tracking. In our tracking solution, we utilise the so-called plan-view approach, where the 3D measurements are transformed to the floor plane, and the tracking problem is solved in 2D. We directly model the measurement noise in the plan-view domain, and the errors that originate from the imaging process and the geometric transformations of the 3D data are combined. We also present a method for directly defining the noise models from the observations. Together with our depth sensor network self-calibration routine, the approach allows fast and practical deployment of depth-based people tracking systems.
21
Zhang, Guowei, Jiyao Yin, Peng Deng, Yanlong Sun, Lin Zhou, and Kuiyuan Zhang. "Achieving Adaptive Visual Multi-Object Tracking with Unscented Kalman Filter." Sensors 22, no. 23 (November 23, 2022): 9106. http://dx.doi.org/10.3390/s22239106.
Full textAbstract:
As an essential part of intelligent monitoring, behavior recognition, automatic driving, and others, the challenge of multi-object tracking is still to ensure tracking accuracy and robustness, especially in complex occlusion environments. Aiming at the issues of the occlusion, background noise, and motion state violent change for multi-object in a complex scene, an improved DeepSORT algorithm based on YOLOv5 is proposed for multi-object tracking to enhance the speed and accuracy of tracking. Firstly, a general object motion model is devised, which is similar to the variable acceleration motion model, and a multi-object tracking framework with the general motion model is established. Then, the latest YOLOv5 algorithm, which has satisfactory detection accuracy, is utilized to obtain the object information as the input of multi-object tracking. An unscented Kalman filter (UKF) is proposed to estimate the motion state of multi-object to solve nonlinear errors. In addition, the adaptive factor is introduced to evaluate observation noise and detect abnormal observations so as to adaptively adjust the innovation covariance matrix. Finally, an improved DeepSORT algorithm for multi-object tracking is formed to promote robustness and accuracy. Extensive experiments are carried out on the MOT16 data set, and we compare the proposed algorithm with the DeepSORT algorithm. The results indicate that the speed and precision of the improved DeepSORT are increased by 4.75% and 2.30%, respectively. Especially in the MOT16 of the dynamic camera, the improved DeepSORT shows better performance.
22
Halimi, Oshri, Tuur Stuyck, Donglai Xiang, Timur Bagautdinov, He Wen, Ron Kimmel, Takaaki Shiratori, Chenglei Wu, Yaser Sheikh, and Fabian Prada. "Pattern-Based Cloth Registration and Sparse-View Animation." ACM Transactions on Graphics 41, no. 6 (November 30, 2022): 1–17. http://dx.doi.org/10.1145/3550454.3555448.
Full textAbstract:
We propose a novel multi-view camera pipeline for the reconstruction and registration of dynamic clothing. Our proposed method relies on a specifically designed pattern that allows for precise video tracking in each camera view. We triangulate the tracked points and register the cloth surface in a fine-grained geometric resolution and low localization error. Compared to state-of-the-art methods, our registration exhibits stable correspondence, tracking the same points on the deforming cloth surface along the temporal sequence. As an application, we demonstrate how the use of our registration pipeline greatly improves state-of-the-art pose-based drivable cloth models. Furthermore, we propose a novel model, Garment Avatar , for driving cloth from a dense tracking signal which is obtained from two opposing camera views. The method produces realistic reconstructions which are faithful to the actual geometry of the deforming cloth. In this setting, the user wears a garment with our custom pattern which enables our driving model to reconstruct the geometry. Our code and data are available at https://github.com/HalimiOshri/Pattern-Based-Cloth-Registration-and-Sparse-View-Animation. The released data includes our pattern and registered mesh sequences containing four different subjects and 15k frames in total.
23
Guidolin, Mattia, Emanuele Menegatti, and Monica Reggiani. "UNIPD-BPE: Synchronized RGB-D and Inertial Data for Multimodal Body Pose Estimation and Tracking." Data 7, no. 6 (June 9, 2022): 79. http://dx.doi.org/10.3390/data7060079.
Full textAbstract:
The ability to estimate human motion without requiring any external on-body sensor or marker is of paramount importance in a variety of fields, ranging from human–robot interaction, Industry 4.0, surveillance, and telerehabilitation. The recent development of portable, low-cost RGB-D cameras pushed forward the accuracy of markerless motion capture systems. However, despite the widespread use of such sensors, a dataset including complex scenes with multiple interacting people, recorded with a calibrated network of RGB-D cameras and an external system for assessing the pose estimation accuracy, is still missing. This paper presents the University of Padova Body Pose Estimation dataset (UNIPD-BPE), an extensive dataset for multi-sensor body pose estimation containing both single-person and multi-person sequences with up to 4 interacting people. A network with 5 Microsoft Azure Kinect RGB-D cameras is exploited to record synchronized high-definition RGB and depth data of the scene from multiple viewpoints, as well as to estimate the subjects’ poses using the Azure Kinect Body Tracking SDK. Simultaneously, full-body Xsens MVN Awinda inertial suits allow obtaining accurate poses and anatomical joint angles, while also providing raw data from the 17 IMUs required by each suit. This dataset aims to push forward the development and validation of multi-camera markerless body pose estimation and tracking algorithms, as well as multimodal approaches focused on merging visual and inertial data.
24
Albert, Justin Amadeus, Victor Owolabi, Arnd Gebel, Clemens Markus Brahms, Urs Granacher, and Bert Arnrich. "Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect v2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study." Sensors 20, no. 18 (September 8, 2020): 5104. http://dx.doi.org/10.3390/s20185104.
Full textAbstract:
Gait analysis is an important tool for the early detection of neurological diseases and for the assessment of risk of falling in elderly people. The availability of low-cost camera hardware on the market today and recent advances in Machine Learning enable a wide range of clinical and health-related applications, such as patient monitoring or exercise recognition at home. In this study, we evaluated the motion tracking performance of the latest generation of the Microsoft Kinect camera, Azure Kinect, compared to its predecessor Kinect v2 in terms of treadmill walking using a gold standard Vicon multi-camera motion capturing system and the 39 marker Plug-in Gait model. Five young and healthy subjects walked on a treadmill at three different velocities while data were recorded simultaneously with all three camera systems. An easy-to-administer camera calibration method developed here was used to spatially align the 3D skeleton data from both Kinect cameras and the Vicon system. With this calibration, the spatial agreement of joint positions between the two Kinect cameras and the reference system was evaluated. In addition, we compared the accuracy of certain spatio-temporal gait parameters, i.e., step length, step time, step width, and stride time calculated from the Kinect data, with the gold standard system. Our results showed that the improved hardware and the motion tracking algorithm of the Azure Kinect camera led to a significantly higher accuracy of the spatial gait parameters than the predecessor Kinect v2, while no significant differences were found between the temporal parameters. Furthermore, we explain in detail how this experimental setup could be used to continuously monitor the progress during gait rehabilitation in older people.
25
Zhang, Wenxiao, Sikun Lin, Farshid Hassani Bijarbooneh, Hao-Fei Cheng, Tristan Braud, Pengyuan Zhou, Lik-Hang Lee, and Pan Hui. "EdgeXAR: A 6-DoF Camera Multi-target Interaction Framework for MAR with User-friendly Latency Compensation." Proceedings of the ACM on Human-Computer Interaction 6, EICS (June 14, 2022): 1–24. http://dx.doi.org/10.1145/3532202.
Full textAbstract:
The computational capabilities of recent mobile devices enable the processing of natural features for Augmented Reality (AR), but the scalability is still limited by the devices' computation power and available resources. In this paper, we propose EdgeXAR, a mobile AR framework that utilizes the advantages of edge computing through task offloading to support flexible camera-based AR interaction. We propose a hybrid tracking system for mobile devices that provides lightweight tracking with 6 Degrees of Freedom and hides the offloading latency from users' perception. A practical, reliable and unreliable communication mechanism is used to achieve fast response and consistency of crucial information. We also propose a multi-object image retrieval pipeline that executes fast and accurate image recognition tasks on the cloud and edge servers. Extensive experiments are carried out to evaluate the performance of EdgeXAR by building mobile AR apps upon it. Regarding the Quality of Experience (QoE), the mobile AR apps powered by EdgeXAR framework run on average at the speed of 30 frames per second with precise tracking of only 1-2 pixel errors and accurate image recognition of at least 97% accuracy. As compared to Vuforia, one of the leading commercial AR frameworks, EdgeXAR transmits 87% less data while providing a stable 30FPS performance and reducing the offloading latency by 50 to 70% depending on the transmission medium. Our work facilitates the large-scale deployment of AR as the next generation of ubiquitous interfaces.
26
Bruecker, Christoph, David Hess, and Bo Watz. "Volumetric Calibration Refinement of a Multi-Camera System Based on Tomographic Reconstruction of Particle Images." Optics 1, no. 1 (March 3, 2020): 114–35. http://dx.doi.org/10.3390/opt1010009.
Full textAbstract:
The calibration of a multi-camera system for volumetric measurements is a basic requirement of reliable 3D measurements and object tracking. In order to refine the precision of the mapping functions, a new, tomographic reconstruction-based approach is presented. The method is suitable for Volumetric Particle Image Velocimetry (PIV), where small particles, drops or bubbles are illuminated and precise 3D position tracking or velocimetry is applied. The technique is based on the 2D cross-correlation of original images of particles with regions from a back projection of a tomographic reconstruction of the particles. The off-set of the peaks in the correlation maps represent disparities, which are used to correct the mapping functions for each sensor plane in an iterative procedure. For validation and practical applicability of the method, a sensitivity analysis has been performed using a synthetic data set followed by the application of the technique on Tomo-PIV measurements of a jet-flow. The results show that initial large disparities could be corrected to an average of below 0.1 pixels during the refinement steps, which drastically improves reconstruction quality and improves measurement accuracy and reliability.
27
Pearce, Andre, J. Andrew Zhang, and Richard Xu. "A Combined mmWave Tracking and Classification Framework Using a Camera for Labeling and Supervised Learning." Sensors 22, no. 22 (November 16, 2022): 8859. http://dx.doi.org/10.3390/s22228859.
Full textAbstract:
Millimeter wave (mmWave) radar poses prosperous opportunities surrounding multiple-object tracking and sensing as a unified system. One of the most challenging aspects of exploiting sensing opportunities with mmWave radar is the labeling of mmWave data so that, in turn, a respective model can be designed to achieve the desired tracking and sensing goals. The labeling of mmWave datasets usually involves a domain expert manually associating radar frames with key events of interest. This is a laborious means of labeling mmWave data. This paper presents a framework for training a mmWave radar with a camera as a means of labeling the data and supervising the radar model. The methodology presented in this paper is compared and assessed against existing frameworks that aim to achieve a similar goal. The practicality of the proposed framework is demonstrated through experimentation in varying environmental conditions. The proposed framework is applied to design a mmWave multi-object tracking system that is additionally capable of classifying individual human motion patterns, such as running, walking, and falling. The experimental findings demonstrate a reliably trained radar model that uses a camera for labeling and supervision that can consistently produce high classification accuracy across environments beyond those in which the model was trained against. The research presented in this paper provides a foundation for future research in unified tracking and sensing systems by alleviating the labeling and training challenges associated with designing a mmWave classification model.
28
Borgmann, B., V. Schatz, H. Kieritz, C. Scherer-Klöckling, M. Hebel, and M. Arens. "DATA PROCESSING AND RECORDING USING A VERSATILE MULTI-SENSOR VEHICLE." ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences IV-1 (September 26, 2018): 21–28. http://dx.doi.org/10.5194/isprs-annals-iv-1-21-2018.
Full textAbstract:
<p><strong>Abstract.</strong> In this paper we present a versatile multi-sensor vehicle which is used in several research projects. The vehicle is equipped with various sensors in order to cover the needs of different research projects in the area of object detection and tracking, mobile mapping and change detection. We show an example for the capabilities of this vehicle by presenting camera- and LiDAR-based pedestrian detection methods. Besides this specific use case, we provide a more general in-depth description of the vehicle’s hard- and software design and its data-processing capabilities. The vehicle can be used as a sensor carrier for mobile mapping, but it also offers hardware and software components to allow for an adaptable onboard processing. This enables the development and testing of methods related to real-time applications or high-level driver assistance functions. The vehicle’s hardware and software layout result from several years of experience, and our lessons learned can help other researchers set up their own experimental platform.</p>
29
Patino, Luis, Michael Hubner, Rachel King, Martin Litzenberger, Laure Roupioz, Kacper Michon, Łukasz Szklarski, Julian Pegoraro, Nikolai Stoianov, and James Ferryman. "Fusion of Heterogenous Sensor Data in Border Surveillance." Sensors 22, no. 19 (September 28, 2022): 7351. http://dx.doi.org/10.3390/s22197351.
Full textAbstract:
Wide area surveillance has become of critical importance, particularly for border control between countries where vast forested land border areas are to be monitored. In this paper, we address the problem of the automatic detection of activity in forbidden areas, namely forested land border areas. In order to avoid false detections, often triggered in dense vegetation with single sensors such as radar, we present a multi sensor fusion and tracking system using passive infrared detectors in combination with automatic person detection from thermal and visual video camera images. The approach combines weighted maps with a rule engine that associates data from multiple weighted maps. The proposed approach is tested on real data collected by the EU FOLDOUT project in a location representative of a range of forested EU borders. The results show that the proposed approach can eliminate single sensor false detections and enhance accuracy by up to 50%.
30
Wang, Shuhang, Jianfeng Li, Pengshuai Yang, Tianxiao Gao, Alex R. Bowers, and Gang Luo. "Towards Wide Range Tracking of Head Scanning Movement in Driving." International Journal of Pattern Recognition and Artificial Intelligence 34, no. 13 (April 20, 2020): 2050033. http://dx.doi.org/10.1142/s0218001420500330.
Full textAbstract:
Gaining environmental awareness through lateral head scanning (yaw rotations) is important for driving safety, especially when approaching intersections. Therefore, head scanning movements could be an important behavioral metric for driving safety research and driving risk mitigation systems. Tracking head scanning movements with a single in-car camera is preferred hardware-wise, but it is very challenging to track the head over almost a [Formula: see text] range. In this paper, we investigate two state-of-the-art methods, a multi-loss deep residual learning method with 50 layers (multi-loss ResNet-50) and an ORB feature-based simultaneous localization and mapping method (ORB-SLAM). While deep learning methods have been extensively studied for head pose detection, this is the first study in which SLAM has been employed to innovatively track head scanning over a very wide range. Our laboratory experimental results showed that ORB-SLAM was more accurate than multi-loss ResNet-50, which often failed when many facial features were not in the view. On the contrary, ORB-SLAM was able to continue tracking as it does not rely on particular facial features. Testing with real driving videos demonstrated the feasibility of using ORB-SLAM for tracking large lateral head scans in naturalistic video data.
31
Martignac, François, Jean-Luc Baglinière, Dominique Ombredane, and Jean Guillard. "Efficiency of automatic analyses of fish passages detected by an acoustic camera using Sonar5-Pro." Aquatic Living Resources 34 (2021): 22. http://dx.doi.org/10.1051/alr/2021020.
Full textAbstract:
The acoustic camera is a non-intrusive method increasingly used to monitor fish populations. Acoustic camera data are video-like, providing information on fish behaviour and morphology helpful to discriminate fish species. However, acoustic cameras used in long-term monitoring studies generate a large amount of data, making one of the technical limitations the time spent analysing data, especially for multi-species fish communities. The specific analysis software provided for DIDSON acoustic cameras is problematic to use for large datasets. Sonar5-Pro, a popular software in freshwater studies offers several advantages due to its automatic tracking tool that follows targets moving into the detection beam and distinguishes fish from other targets. This study aims to assess the effectiveness of Sonar5-Pro for detecting and describing fish passages in a high fish diversity river in low flow conditions. The tool's accuracy was assessed by comparing Sonar5-Pro outputs with a complete manual analysis using morphological and behavioural descriptors. Ninety-eight percent of the fish moving into the detection beam were successfully detected by the software. The fish swimming direction estimation was 90% efficient. Sonar5-Pro and its automatic tracking tool have great potential as a database pre-filtering process and decrease the overall time spent on data analysis but some limits were also identified. Multi-counting issues almost doubled the true fish abundance, requiring manual operator validation. Furthermore, fish length of each tracked fish needed to be manually measured with another software (SMC). In conclusion, a combination of Sonar5-Pro and SMC software can provide reliable results with a significant reduction of manpower needed for the analysis of a long-term monitoring DIDSON dataset.
32
Liu, Fei, Jiayao Shan, Binyu Xiong, and Zheng Fang. "A Real-Time and Multi-Sensor-Based Landing Area Recognition System for UAVs." Drones 6, no. 5 (May 7, 2022): 118. http://dx.doi.org/10.3390/drones6050118.
Full textAbstract:
This paper presents a real-time and multi-sensor-based landing area recognition system for UAVs, which aims to enable UAVs to land safely on open and flat terrain and is suitable for comprehensive unmanned autonomous operation. The landing area recognition system for UAVs is built on the combination of a camera and a 3D LiDAR. The problem is how to fuse the image and point cloud information and realize the landing area recognition to guide the UAV landing autonomously and safely. To solve this problem, firstly, we use a deep learning method to realize the landing area recognition and tracking from images. After that, we project 3D LiDAR point cloud data into camera coordinates to obtain the semantic label of each point. Finally, we use the 3D LiDAR point cloud data with the semantic label to build the 3D environment map and calculate the most suitable area for UAV landing. Experiments show that the proposed method can achieve accurate and robust recognition of landing area for UAVs.
33
Rajkumar, Dr S., Aklilu Teklemariam, and Addisalem Mekonnen. "Hybrid Multi-Sensor Integration for Static or Dynamic Obstacle Detection, Tracking and Classification for Autonomous Vehicle." Journal of University of Shanghai for Science and Technology 23, no. 06 (June 22, 2021): 1288–93. http://dx.doi.org/10.51201/jusst/21/06424.
Full textAbstract:
Autonomous Vehicles (AV) reduces human intervention by perceiving the vehicle’s location with respect to the environment. In this regard, utilization of multiple sensors corresponding to various features of environment perception yields not only detection but also enables tracking and classification of the object leading to high security and reliability. Therefore, we propose to deploy hybrid multi-sensors such as Radar, LiDAR, and camera sensors. However, the data acquired with these hybrid sensors overlaps with the wide viewing angles of the individual sensors, and hence convolutional neural network and Kalman Filter (KF) based data fusion framework was implemented with a goal to facilitate a robust object detection system to avoid collisions inroads. The complete system tested over 1000 road scenarios for real-time environment perception showed that our hardware and software configurations outperformed numerous other conventional systems. Hence, this system could potentially find its application in object detection, tracking, and classification in a real-time environment.
34
Al Mdfaa, Mohamad Al, Geesara Kulathunga, and Alexandr Klimchik. "3D-SiamMask: Vision-Based Multi-Rotor Aerial-Vehicle Tracking for a Moving Object." Remote Sensing 14, no. 22 (November 14, 2022): 5756. http://dx.doi.org/10.3390/rs14225756.
Full textAbstract:
This paper aims to develop a multi-rotor-based visual tracker for a specified moving object. Visual object-tracking algorithms for multi-rotors are challenging due to multiple issues such as occlusion, quick camera motion, and out-of-view scenarios. Hence, algorithmic changes are required for dealing with images or video sequences obtained by multi-rotors. Therefore, we propose two approaches: a generic object tracker and a class-specific tracker. Both tracking settings require the object bounding box to be selected in the first frame. As part of the later steps, the object tracker uses the updated template set and the calibrated RGBD sensor data as inputs to track the target object using a Siamese network and a machine-learning model for depth estimation. The class-specific tracker is quite similar to the generic object tracker but has an additional auxiliary object classifier. The experimental study and validation were carried out in a robot simulation environment. The simulation environment was designed to serve multiple case scenarios using Gazebo. According to the experiment results, the class-specific object tracker performed better than the generic object tracker in terms of stability and accuracy. Experiments show that the proposed generic tracker achieves promising results on three challenging datasets. Our tracker runs at approximately 36 fps on GPU.
35
Baclig, Maria Martine, Noah Ergezinger, Qipei Mei, Mustafa Gül, Samer Adeeb, and Lindsey Westover. "A Deep Learning and Computer Vision Based Multi-Player Tracker for Squash." Applied Sciences 10, no. 24 (December 9, 2020): 8793. http://dx.doi.org/10.3390/app10248793.
Full textAbstract:
Sports pose a unique challenge for high-speed, unobtrusive, uninterrupted motion tracking due to speed of movement and player occlusion, especially in the fast and competitive sport of squash. The objective of this study is to use video tracking techniques to quantify kinematics in elite-level squash. With the increasing availability and quality of elite tournament matches filmed for entertainment purposes, a new methodology of multi-player tracking for squash that only requires broadcast video as an input is proposed. This paper introduces and evaluates a markerless motion capture technique using an autonomous deep learning based human pose estimation algorithm and computer vision to detect and identify players. Inverse perspective mapping is utilized to convert pixel coordinates to court coordinates and distance traveled, court position, ‘T’ dominance, and average speeds of elite players in squash is determined. The method was validated using results from a previous study using manual tracking where the proposed method (filtered coordinates) displayed an average absolute percent error to the manual approach of 3.73% in total distance traveled, 3.52% and 1.26% in average speeds <9 m/s with and without speeds <1 m/s, respectively. The method has proven to be the most effective in collecting kinematic data of elite players in squash in a timely manner with no special camera setup and limited manual intervention.
36
Wu, C., Q. Zhu, Y. T. Zhang, Z. Q. Du, Y. Zhou, X. Xie, and F. He. "AN ADAPTIVE ORGANIZATION METHOD OF GEOVIDEO DATA FOR SPATIO-TEMPORAL ASSOCIATION ANALYSIS." ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences II-4/W2 (July 10, 2015): 29–34. http://dx.doi.org/10.5194/isprsannals-ii-4-w2-29-2015.
Full textAbstract:
Public security incidents have been increasingly challenging to address with their new features, including large-scale mobility, multi-stage dynamic evolution, spatio-temporal concurrency and uncertainty in the complex urban environment, which require spatio-temporal association analysis among multiple regional video data for global cognition. However, the existing video data organizational methods that view video as a property of the spatial object or position in space dissever the spatio-temporal relationship of scattered video shots captured from multiple video channels, limit the query functions on interactive retrieval between a camera and its video clips and hinder the comprehensive management of event-related scattered video shots. GeoVideo, which maps video frames onto a geographic space, is a new approach to represent the geographic world, promote security monitoring in a spatial perspective and provide a highly feasible solution to this problem. This paper analyzes the large-scale personnel mobility in public safety events and proposes a multi-level, event-related organization method with massive GeoVideo data by spatio-temporal trajectory. This paper designs a unified object identify(ID) structure to implicitly store the spatio-temporal relationship of scattered video clips and support the distributed storage management of massive cases. Finally, the validity and feasibility of this method are demonstrated through suspect tracking experiments.
37
Giordano, Jacopo, Margherita Lazzaretto, Giulia Michieletto, and Angelo Cenedese. "Visual Sensor Networks for Indoor Real-Time Surveillance and Tracking of Multiple Targets." Sensors 22, no. 7 (March 30, 2022): 2661. http://dx.doi.org/10.3390/s22072661.
Full textAbstract:
The recent trend toward the development of IoT architectures has entailed the transformation of the standard camera networks into smart multi-device systems capable of acquiring, elaborating, and exchanging data and, often, dynamically adapting to the environment. Along this line, this work proposes a novel distributed solution that guarantees the real-time monitoring of 3D indoor structured areas and also the tracking of multiple targets, by employing a heterogeneous visual sensor network composed of both fixed and Pan-Tilt-Zoom (PTZ) cameras. The fulfillment of the twofold mentioned goal was ensured through the implementation of a distributed game-theory-based algorithm, aiming at optimizing the controllable parameters of the PTZ devices. The proposed solution is able to deal with the possible conflicting requirements of high tracking precision and maximum coverage of the surveilled area. Extensive numerical simulations in realistic scenarios validated the effectiveness of the outlined strategy.
38
Quintana, Marcos, Sezer Karaoglu, Federico Alvarez, Jose Menendez, and Theo Gevers. "Three-D Wide Faces (3DWF): Facial Landmark Detection and 3D Reconstruction over a New RGB–D Multi-Camera Dataset." Sensors 19, no. 5 (March 4, 2019): 1103. http://dx.doi.org/10.3390/s19051103.
Full textAbstract:
Latest advances of deep learning paradigm and 3D imaging systems have raised the necessity for more complete datasets that allow exploitation of facial features such as pose, gender or age. In our work, we propose a new facial dataset collected with an innovative RGB–D multi-camera setup whose optimization is presented and validated. 3DWF includes 3D raw and registered data collection for 92 persons from low-cost RGB–D sensing devices to commercial scanners with great accuracy. 3DWF provides a complete dataset with relevant and accurate visual information for different tasks related to facial properties such as face tracking or 3D face reconstruction by means of annotated density normalized 2K clouds and RGB–D streams. In addition, we validate the reliability of our proposal by an original data augmentation method from a massive set of face meshes for facial landmark detection in 2D domain, and by head pose classification through common Machine Learning techniques directed towards proving alignment of collected data.
39
Li, Hao, Gen Liu, Haoqi Wang, Xiaoyu Wen, Guizhong Xie, Guofu Luo, Shuai Zhang, and Miying Yang. "Mechanical movement data acquisition method based on the multilayer neural networks and machine vision in a digital twin environment." Digital Twin 1 (October 14, 2021): 6. http://dx.doi.org/10.12688/digitaltwin.17441.1.
Full textAbstract:
Background: Digital twin requires virtual reality mapping and optimization iteration between physical devices and virtual models. The mechanical movement data collection of physical equipment is essential for the implementation of accurate virtual and physical synchronization in a digital twin environment. However, the traditional approach relying on PLC (programmable logic control) fails to collect various mechanical motion state data. Additionally, few investigations have used machine visions for the virtual and physical synchronization of equipment. Thus, this paper presents a mechanical movement data acquisition method based on multilayer neural networks and machine vision. Methods: Firstly, various visual marks with different colors and shapes are designed for marking physical devices. Secondly, a recognition method based on the Hough transform and histogram feature is proposed to realize the recognition of shape and color features respectively. Then, the multilayer neural network model is introduced in the visual mark location. The neural network is trained by the dropout algorithm to realize the tracking and location of the visual mark. To test the proposed method, 1000 samples were selected. Results: The experiment results shows that when the size of the visual mark is larger than 6mm, the recognition success rate of the recognition algorithm can reach more than 95%. In the actual operation environment with multiple cameras, the identification points can be located more accurately. Moreover, the camera calibration process of binocular and multi-eye vision can be simplified by the multilayer neural networks. Conclusions: This study proposes an effective method in the collection of mechanical motion data of physical equipment in a digital twin environment. Further studies are needed to perceive posture and shape data of physical entities under the multi-camera redundant shooting.
40
Wang, Zihao, Sen Yang, Mengji Shi, and Kaiyu Qin. "An Image Augmentation Method Based on Limited Samples for Object Tracking Based on Mobile Platform." Sensors 22, no. 5 (March 2, 2022): 1967. http://dx.doi.org/10.3390/s22051967.
Full textAbstract:
This paper proposes an image augmentation model of limited samples on the mobile platform for object tracking. The augmentation method mainly aims at the detection failure caused by the small number of effective samples, jitter of tracking platform, and relative rotation between camera and object in the tracking process. Aiming at the object tracking problem, we first propose to use geometric projection transformation, multi-directional overlay blurring, and random background filling to improve the generalization ability of samples. Then, selecting suitable traditional augmentation methods as the supplements, an image augmentation model with an adjustable probability factor is provided to simulate various kinds of samples to help the detection model carry out more reliable training. Finally, combined with a spatial localization algorithm based on geometric constraints proposed by the author’s previous work, a framework for object tracking with an image augmentation method is proposed. SSD, YOLOv3, YOLOv4, and YOLOx are adopted in the experiment of this paper as the detection models. And a large number of object recognition and object tracking experiments are carried out by combining with common data sets OTB50 and OTB100 as well as the OTMP data set proposed by us for mobile platform. The augmented module proposed in this paper is conducive for the detection model to improve the detection accuracy by at least 10%. Especially for objects with planar characteristics, the affine and projection transformation used in this paper can greatly improve the detection accuracy of the model. Based on the object tracking framework of our augmented model, the RMSE is estimated to be less than 4.21 cm in terms of the actual tracking of indoor objects.
41
Li, Hongmin, Yuntong Dai, Hongxing Qiu, and Xiaoyuan He. "Application of multi-camera digital image correlation in the stability study of the long timber column with the circular cross-section under axial compression." BioResources 17, no. 1 (January 24, 2022): 1717–28. http://dx.doi.org/10.15376/biores.17.1.1717-1728.
Full textAbstract:
Displacement meter synthesis (DMS), as a traditional method, is widely used to capture the local deformation of a specimen. However, for axis-symmetrical circular columns, the axial compression instability cannot be accurately evaluated using the DMS method due to the uncertain bending direction. To address this problem, a 360° full-surface digital image correlation (DIC) system composed of eight cameras is proposed to capture the full-field deformation information of the column surfaces under axial compression. The objective of this study was to experimentally validate the efficiency of the proposed novel measurement in tracking the full-filled compressive deformation of a long wooden column with the circular cross-section. The test results showed that the columns experience instable failure. The multi-camera DIC system can completely reconstruct the three-dimensional shape of the circular column and obtain the whole process deformation state at any position on the surface of the timber column. The DIC method also can capture the uncertain lateral deformation direction and obtain the data of lateral deflection at the mid-span of timber columns. The multi-camera DIC provides an intuitive and comprehensive new test method for the test and analysis of the stability of axisymmetric long timber columns.
42
Hong, Sungjin, and Yejin Kim. "Dynamic Pose Estimation Using Multiple RGB-D Cameras." Sensors 18, no. 11 (November 10, 2018): 3865. http://dx.doi.org/10.3390/s18113865.
Full textAbstract:
Human poses are difficult to estimate due to the complicated body structure and the self-occlusion problem. In this paper, we introduce a marker-less system for human pose estimation by detecting and tracking key body parts, namely the head, hands, and feet. Given color and depth images captured by multiple red, green, blue, and depth (RGB-D) cameras, our system constructs a graph model with segmented regions from each camera and detects the key body parts as a set of extreme points based on accumulative geodesic distances in the graph. During the search process, local detection using a supervised learning model is utilized to match local body features. A final set of extreme points is selected with a voting scheme and tracked with physical constraints from the unified data received from the multiple cameras. During the tracking process, a Kalman filter-based method is introduced to reduce positional noises and to recover from a failure of tracking extremes. Our system shows an average of 87% accuracy against the commercial system, which outperforms the previous multi-Kinects system, and can be applied to recognize a human action or to synthesize a motion sequence from a few key poses using a small set of extremes as input data.
43
Chen, Taicong, and Zhou Zhou. "An Improved Vision Method for Robust Monitoring of Multi-Point Dynamic Displacements with Smartphones in an Interference Environment." Sensors 20, no. 20 (October 20, 2020): 5929. http://dx.doi.org/10.3390/s20205929.
Full textAbstract:
Current research on dynamic displacement measurement based on computer vision mostly requires professional high-speed cameras and an ideal shooting environment to ensure the performance and accuracy of the analysis. However, the high cost of the camera and strict requirements of sharp image contrast and stable environment during the shooting process limit the broad application of the technology. This paper proposes an improved vision method to implement multi-point dynamic displacement measurements with smartphones in an interference environment. A motion-enhanced spatio-temporal context (MSTC) algorithm is developed and applied together with the optical flow (OF) algorithm to realize a simultaneous tracking and dynamic displacement extraction of multiple points on a vibrating structure in the interference environment. Finally, a sine-sweep vibration experiment on a cantilever sphere model is presented to validate the feasibility of the proposed method in a wide-band frequency range. In the test, a smartphone was used to shoot the vibration process of the sine-sweep-excited sphere, and illumination change, fog interference, and camera jitter were artificially simulated to represent the interference environment. The results of the proposed method are compared to conventional displacement sensor data and current vision method results. It is demonstrated that, in an interference environment, (1) the OF method is prone to mismatch the feature points and leads to data deviated or lost; (2) the conventional STC method is sensitive to target selection and can effectively track those targets having a large proportion of pixels in the context with motion tendency similar to the target center; (3) the proposed MSTC method, however, can ease the sensitivity to target selection through in-depth processing of the information in the context and finally enhance the robustness of the target tracking. In addition, the MSTC method takes less than one second to track each target between adjacent frame images, implying a potential for online measurement.
44
Dong, Chuan-Zhi, Ozan Celik, and F. Necati Catbas. "Marker-free monitoring of the grandstand structures and modal identification using computer vision methods." Structural Health Monitoring 18, no. 5-6 (November 1, 2018): 1491–509. http://dx.doi.org/10.1177/1475921718806895.
Full textAbstract:
In this study, a vision-based multi-point structural dynamic monitoring framework is proposed. This framework aims to solve issues in current vision-based structural health monitoring. Limitations are due to manual markers, single-point monitoring, and synchronization between a multiple-camera setup and a sensor network. The proposed method addresses the first issue using virtual markers—features extracted from an image—instead of physical manual markers. The virtual markers can be selected according to each scenario, which makes them versatile. The framework also overcomes the issue of single-point monitoring by utilizing an advanced visual tracking algorithm based on optical flow, allowing multi-point displacement measurements. Besides, a synchronization mechanism between a multiple-camera setup and a sensor network is built. The proposed method is first tested on a grandstand simulator located in the laboratory. The experiment is to verify the performance of displacement measurement of the proposed method and conduct structural identification of the grandstand through multi-point displacement records. The results from the proposed method are then compared to the data gathered by traditional displacement sensors and accelerometers. A second experiment is conducted at a stadium during a football game to validate the feasibility of field application and the operational modal identification of the stadium under human crowd jumping through the measured displacement records. From these experiments, it is concluded that the proposed method can be employed to identify modal parameters for structural health monitoring.
45
Afifi, Ahmed, Chisato Takada, Yuichiro Yoshimura, and Toshiya Nakaguchi. "Real-Time Expanded Field-of-View for Minimally Invasive Surgery Using Multi-Camera Visual Simultaneous Localization and Mapping." Sensors 21, no. 6 (March 17, 2021): 2106. http://dx.doi.org/10.3390/s21062106.
Full textAbstract:
Minimally invasive surgery is widely used because of its tremendous benefits to the patient. However, there are some challenges that surgeons face in this type of surgery, the most important of which is the narrow field of view. Therefore, we propose an approach to expand the field of view for minimally invasive surgery to enhance surgeons’ experience. It combines multiple views in real-time to produce a dynamic expanded view. The proposed approach extends the monocular Oriented features from an accelerated segment test and Rotated Binary robust independent elementary features—Simultaneous Localization And Mapping (ORB-SLAM) to work with a multi-camera setup. The ORB-SLAM’s three parallel threads, namely tracking, mapping and loop closing, are performed for each camera and new threads are added to calculate the relative cameras’ pose and to construct the expanded view. A new algorithm for estimating the optimal inter-camera correspondence matrix from a set of corresponding 3D map points is presented. This optimal transformation is then used to produce the final view. The proposed approach was evaluated using both human models and in vivo data. The evaluation results of the proposed correspondence matrix estimation algorithm prove its ability to reduce the error and to produce an accurate transformation. The results also show that when other approaches fail, the proposed approach can produce an expanded view. In this work, a real-time dynamic field-of-view expansion approach that can work in all situations regardless of images’ overlap is proposed. It outperforms the previous approaches and can also work at 21 fps.
46
van Essen, Rick, Angelo Mencarelli, Aloysius van Helmond, Linh Nguyen, Jurgen Batsleer, Jan-Jaap Poos, and Gert Kootstra. "Automatic discard registration in cluttered environments using deep learning and object tracking: class imbalance, occlusion, and a comparison to human review." ICES Journal of Marine Science 78, no. 10 (November 27, 2021): 3834–46. http://dx.doi.org/10.1093/icesjms/fsab233.
Full textAbstract:
Abstract This paper presents and evaluates a method for detecting and counting demersal fish species in complex, cluttered, and occluded environments that can be installed on the conveyor belts of fishing vessels. Fishes on the conveyor belt were recorded using a colour camera and were detected using a deep neural network. To improve the detection, synthetic data were generated for rare fish species. The fishes were tracked over the consecutive images using a multi-object tracking algorithm, and based on multiple observations, the fish species was determined. The effect of the synthetic data, the amount of occlusion, and the observed dorsal or ventral fish side were investigated and a comparison with human electronic monitoring (EM) review was made. Using the presented method, a weighted counting error of 20% was achieved, compared to a counting error of 7% for human EM review on the same recordings.
47
Kalake, Lesole, Yanqiu Dong, Wanggen Wan, and Li Hou. "Enhancing Detection Quality Rate with a Combined HOG and CNN for Real-Time Multiple Object Tracking across Non-Overlapping Multiple Cameras." Sensors 22, no. 6 (March 9, 2022): 2123. http://dx.doi.org/10.3390/s22062123.
Full textAbstract:
Multi-object tracking in video surveillance is subjected to illumination variation, blurring, motion, and similarity variations during the identification process in real-world practice. The previously proposed applications have difficulties in learning the appearances and differentiating the objects from sundry detections. They mostly rely heavily on local features and tend to lose vital global structured features such as contour features. This contributes to their inability to accurately detect, classify or distinguish the fooling images. In this paper, we propose a paradigm aimed at eliminating these tracking difficulties by enhancing the detection quality rate through the combination of a convolutional neural network (CNN) and a histogram of oriented gradient (HOG) descriptor. We trained the algorithm with an input of 120 × 32 images size and cleaned and converted them into binary for reducing the numbers of false positives. In testing, we eliminated the background on frames size and applied morphological operations and Laplacian of Gaussian model (LOG) mixture after blobs. The images further underwent feature extraction and computation with the HOG descriptor to simplify the structural information of the objects in the captured video images. We stored the appearance features in an array and passed them into the network (CNN) for further processing. We have applied and evaluated our algorithm for real-time multiple object tracking on various city streets using EPFL multi-camera pedestrian datasets. The experimental results illustrate that our proposed technique improves the detection rate and data associations. Our algorithm outperformed the online state-of-the-art approach by recording the highest in precisions and specificity rates.
48
Zhang, R., S. Shi, X. Yi, and M. Jing. "APPLICATION OF RGB-D SLAM IN 3D TUNNEL RECONSTRUCTION BASED ON SUPERPIXEL AIDED FEATURE TRACKING." International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLIII-B2-2022 (May 30, 2022): 559–64. http://dx.doi.org/10.5194/isprs-archives-xliii-b2-2022-559-2022.
Full textAbstract:
Abstract. In large-scale projects such as hydropower and transportation, the real-time acquisition and generation of the 3D tunnel model can provide an important basis for the analysis and evaluation of the tunnel stability. The Simultaneous Localization And Mapping (SLAM) technology has the advantages of low cost and strong real-time, which can greatly improve the data acquisition efficiency during tunnel excavation. Feature tracking and matching are critical processes of traditional 3D reconstruction technologies such as Structure from Motion (SfM) and SLAM. However, the complicated rock mass structures on the tunnel surface and the limited lighting environment make feature tracking and matching difficult. Manhattan SLAM is a technology integrating superpixels and Manhattan world assumptions, in which both line features and planar features can be better extracted. Rock mass discontinuities including traces and structural planes are distributed on the inner surface of tunnels, which can be extracted for feature tracking and matching. Therefore, this paper proposes a 3D reconstruction pipeline for tunnels, in which, the Manhattan SLAM algorithm is applied for camera pose parameters estimation and the sparse point cloud generation, and the Patch-based Multi-View Stereo (PMVS) is adopted for dense reconstruction. In this paper, the Azure Kinect DK sensor is used for data acquisition. Experiments are proceeded and the results show that the proposed pipeline based on Manhattan SLAM and PMVS performs good robustness and feasibility for tunnels 3D reconstruction.
49
Sayour, Malak H., Sharbel E. Kozhaya, and Samer S. Saab. "Autonomous Robotic Manipulation: Real-Time, Deep-Learning Approach for Grasping of Unknown Objects." Journal of Robotics 2022 (June 30, 2022): 1–14. http://dx.doi.org/10.1155/2022/2585656.
Full textAbstract:
Recent advancement in vision-based robotics and deep-learning techniques has enabled the use of intelligent systems in a wider range of applications requiring object manipulation. Finding a robust solution for object grasping and autonomous manipulation became the focus of many engineers and is still one of the most demanding problems in modern robotics. This paper presents a full grasping pipeline proposing a real-time data-driven deep-learning approach for robotic grasping of unknown objects using MATLAB and convolutional neural networks. The proposed approach employs RGB-D image data acquired from an eye-in-hand camera centering the object of interest in the field of view using visual servoing. Our approach aims at reducing propagation errors and eliminating the need for complex hand tracking algorithm, image segmentation, or 3D reconstruction. The proposed approach is able to efficiently generate reliable multi-view object grasps regardless of the geometric complexity and physical properties of the object in question. The proposed system architecture enables simple and effective path generation and a real-time tracking control. In addition, our system is modular, reliable, and accurate in both end effector path generation and control. We experimentally justify the efficacy and effectiveness of our overall system on the Barrett Whole Arm Manipulator.
50
Zhang, Hui, Xieyuanli Chen, Huimin Lu, and Junhao Xiao. "Distributed and collaborative monocular simultaneous localization and mapping for multi-robot systems in large-scale environments." International Journal of Advanced Robotic Systems 15, no. 3 (May 1, 2018): 172988141878017. http://dx.doi.org/10.1177/1729881418780178.
Full textAbstract:
In this article, we propose a distributed and collaborative monocular simultaneous localization and mapping system for the multi-robot system in large-scale environments, where monocular vision is the only exteroceptive sensor. Each robot estimates its pose and reconstructs the environment simultaneously using the same monocular simultaneous localization and mapping algorithm. Meanwhile, they share the results of their incremental maps by streaming keyframes through the robot operating system messages and the wireless network. Subsequently, each robot in the group can obtain the global map with high efficiency. To build the collaborative simultaneous localization and mapping architecture, two novel approaches are proposed. One is a robust relocalization method based on active loop closure, and the other is a vision-based multi-robot relative pose estimating and map merging method. The former is used to solve the problem of tracking failures when robots carry out long-term monocular simultaneous localization and mapping in large-scale environments, while the latter uses the appearance-based place recognition method to determine multi-robot relative poses and build the large-scale global map by merging each robot’s local map. Both KITTI data set and our own data set acquired by a handheld camera are used to evaluate the proposed system. Experimental results show that the proposed distributed multi-robot collaborative monocular simultaneous localization and mapping system can be used in both indoor small-scale and outdoor large-scale environments.