To see the other types of publications on this topic, follow the link: Gesture recognition module.
Journal articles on the topic 'Gesture recognition module'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Gesture recognition module.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Chen, Shang-Liang, and Li-Wu Huang. "Using Deep Learning Technology to Realize the Automatic Control Program of Robot Arm Based on Hand Gesture Recognition." International Journal of Engineering and Technology Innovation 11, no. 4 (2021): 241–50. http://dx.doi.org/10.46604/ijeti.2021.7342.
Full textAbstract:
In this study, the robot arm control, computer vision, and deep learning technologies are combined to realize an automatic control program. There are three functional modules in this program, i.e., the hand gesture recognition module, the robot arm control module, and the communication module. The hand gesture recognition module records the user’s hand gesture images to recognize the gestures’ features using the YOLOv4 algorithm. The recognition results are transmitted to the robot arm control module by the communication module. Finally, the received hand gesture commands are analyzed and executed by the robot arm control module. With the proposed program, engineers can interact with the robot arm through hand gestures, teach the robot arm to record the trajectory by simple hand movements, and call different scripts to satisfy robot motion requirements in the actual production environment.
2
Nguyen, Ngoc-Hoang, Tran-Dac-Thinh Phan, Soo-Hyung Kim, Hyung-Jeong Yang, and Guee-Sang Lee. "3D Skeletal Joints-Based Hand Gesture Spotting and Classification." Applied Sciences 11, no. 10 (2021): 4689. http://dx.doi.org/10.3390/app11104689.
Full textAbstract:
This paper presents a novel approach to continuous dynamic hand gesture recognition. Our approach contains two main modules: gesture spotting and gesture classification. Firstly, the gesture spotting module pre-segments the video sequence with continuous gestures into isolated gestures. Secondly, the gesture classification module identifies the segmented gestures. In the gesture spotting module, the motion of the hand palm and fingers are fed into the Bidirectional Long Short-Term Memory (Bi-LSTM) network for gesture spotting. In the gesture classification module, three residual 3D Convolution Neural Networks based on ResNet architectures (3D_ResNet) and one Long Short-Term Memory (LSTM) network are combined to efficiently utilize the multiple data channels such as RGB, Optical Flow, Depth, and 3D positions of key joints. The promising performance of our approach is obtained through experiments conducted on three public datasets—Chalearn LAP ConGD dataset, 20BN-Jester, and NVIDIA Dynamic Hand gesture Dataset. Our approach outperforms the state-of-the-art methods on the Chalearn LAP ConGD dataset.
3
Min, Huasong, Ziming Chen, Bin Fang, et al. "Cross-Individual Gesture Recognition Based on Long Short-Term Memory Networks." Scientific Programming 2021 (July 6, 2021): 1–11. http://dx.doi.org/10.1155/2021/6680417.
Full textAbstract:
Gestures recognition based on surface electromyography (sEMG) has been widely used for human-computer interaction. However, there are few research studies on overcoming the influence of physiological factors among different individuals. In this paper, a cross-individual gesture recognition method based on long short-term memory (LSTM) networks is proposed, named cross-individual LSTM (CI-LSTM). CI-LSTM has a dual-network structure, including a gesture recognition module and an individual recognition module. By designing the loss function, the individual information recognition module assists the gesture recognition module to train, which tends to orthogonalize the gesture features and individual features to minimize the impact of individual information differences on gesture recognition. Through cross-individual gesture recognition experiments, it is verified that compared with other selected algorithm models, the recognition accuracy obtained by using the CI-LSTM model can be improved by an average of 9.15%. Compared with other models, CI-LSTM can overcome the influence of individual characteristics and complete the task of cross-individual hand gestures recognition. Based on the proposed model, online control of the prosthetic hand is realized.
4
Bhargavi, Mrs Jangam, Chitikala Sairam, and Donga Hemanth. "Real time interface for deaf-hearing communication." International Scientific Journal of Engineering and Management 04, no. 03 (2025): 1–7. https://doi.org/10.55041/isjem02356.
Full textAbstract:
Bridging the communication gap between the deaf and hearing communities using AI is achieved by integrating two key modules: Speech-to-Sign Language Translation and Sign Gesture Detection in Real Time. The first module translates English spoken language into American Sign Language (ASL) animations. It consists of three sub-modules: speech-to-text conversion using the speech recognition module in Python, English text to ASL gloss translation using an NLP model, and ASL gloss to animated video generation, where DWpose Pose Estimation, and an avatar is used for visual representation. The second module focuses on real-time sign gesture detection, where a dataset is created from the WLASL and MS-ASL datasets. Hand gestures are labeled using Labeling, and a YOLO-based model is trained for hand pose detection to enable real-time recognition. The system aims to enhance accessibility and interaction between deaf and hearing users through an efficient, automated translation and recognition pipeline. Keywords: Speech-to-sign translation, real-time sign language recognition, ASL gloss, YOLO hand pose detection, AI for accessibility, deep learning for sign language, gesture recognition, DWpose Pose Estimation, NLP, dataset labeling, real-time gesture recognition.
5
Tao, Chongben, and Guodong Liu. "A Multilayer Hidden Markov Models-Based Method for Human-Robot Interaction." Mathematical Problems in Engineering 2013 (2013): 1–12. http://dx.doi.org/10.1155/2013/384865.
Full textAbstract:
To achieve Human-Robot Interaction (HRI) by using gestures, a continuous gesture recognition approach based on Multilayer Hidden Markov Models (MHMMs) is proposed, which consists of two parts. One part is gesture spotting and segment module, the other part is continuous gesture recognition module. Firstly, a Kinect sensor is used to capture 3D acceleration and 3D angular velocity data of hand gestures. And then, a Feed-forward Neural Networks (FNNs) and a threshold criterion are used for gesture spotting and segment, respectively. Afterwards, the segmented gesture signals are respectively preprocessed and vector symbolized by a sliding window and a K-means clustering method. Finally, symbolized data are sent into Lower Hidden Markov Models (LHMMs) to identify individual gestures, and then, a Bayesian filter with sequential constraints among gestures in Upper Hidden Markov Models (UHMMs) is used to correct recognition errors created in LHMMs. Five predefined gestures are used to interact with a Kinect mobile robot in experiments. The experimental results show that the proposed method not only has good effectiveness and accuracy, but also has favorable real-time performance.
6
R N, Pushpa. "Sign Language Detection Using Deep Learning." International Journal for Research in Applied Science and Engineering Technology 12, no. 12 (2024): 565–71. https://doi.org/10.22214/ijraset.2024.65817.
Full textAbstract:
Sign language recognition is an essential tool for bridging communication gaps between individuals with hearing or speech impairments and the broader community. This study introduces an advanced sign language recognition system leveraging computer vision and machine learning techniques. The system utilizes real-time hand tracking and gesture recognition to identify and classify hand gestures associated with common phrases such as "Hello," "I love you," and "Thank you." A two-step approach is implemented: first, a data collection module captures hand images using a robust preprocessing pipeline, ensuring uniformity in image size and quality; second, a classification module uses a trained deep learning model to accurately predict gestures in real-time. The framework integrates OpenCV for image processing, CVZone modules for hand detection, and TensorFlow for gesture classification. Extensive testing demonstrates the system's capability to process live video input, classify gestures accurately, and display corresponding labels seamlessly. This solution addresses challenges in gesture recognition, such as variable hand shapes and dynamic backgrounds, through efficient preprocessing and model training. By offering a scalable and efficient design, this work has the potential to contribute significantly to assistive technologies and accessible communication systems, paving the way for further advancements in human-computer interaction and inclusive technology.
7
Chen, Zengzhao, Wenkai Huang, Hai Liu, Zhuo Wang, Yuqun Wen, and Shengming Wang. "ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition." Sensors 24, no. 8 (2024): 2589. http://dx.doi.org/10.3390/s24082589.
Full textAbstract:
Teaching gesture recognition is a technique used to recognize the hand movements of teachers in classroom teaching scenarios. This technology is widely used in education, including for classroom teaching evaluation, enhancing online teaching, and assisting special education. However, current research on gesture recognition in teaching mainly focuses on detecting the static gestures of individual students and analyzing their classroom behavior. To analyze the teacher’s gestures and mitigate the difficulty of single-target dynamic gesture recognition in multi-person teaching scenarios, this paper proposes skeleton-based teaching gesture recognition (ST-TGR), which learns through spatio-temporal representation. This method mainly uses the human pose estimation technique RTMPose to extract the coordinates of the keypoints of the teacher’s skeleton and then inputs the recognized sequence of the teacher’s skeleton into the MoGRU action recognition network for classifying gesture actions. The MoGRU action recognition module mainly learns the spatio-temporal representation of target actions by stacking a multi-scale bidirectional gated recurrent unit (BiGRU) and using improved attention mechanism modules. To validate the generalization of the action recognition network model, we conducted comparative experiments on datasets including NTU RGB+D 60, UT-Kinect Action3D, SBU Kinect Interaction, and Florence 3D. The results indicate that, compared with most existing baseline models, the model proposed in this article exhibits better performance in recognition accuracy and speed.
8
Ahmed, Shahzad, and Sung Ho Cho. "Hand Gesture Recognition Using an IR-UWB Radar with an Inception Module-Based Classifier." Sensors 20, no. 2 (2020): 564. http://dx.doi.org/10.3390/s20020564.
Full textAbstract:
The emerging integration of technology in daily lives has increased the need for more convenient methods for human–computer interaction (HCI). Given that the existing HCI approaches exhibit various limitations, hand gesture recognition-based HCI may serve as a more natural mode of man–machine interaction in many situations. Inspired by an inception module-based deep-learning network (GoogLeNet), this paper presents a novel hand gesture recognition technique for impulse-radio ultra-wideband (IR-UWB) radars which demonstrates a higher gesture recognition accuracy. First, methodology to demonstrate radar signals as three-dimensional image patterns is presented and then, the inception module-based variant of GoogLeNet is used to analyze the pattern within the images for the recognition of different hand gestures. The proposed framework is exploited for eight different hand gestures with a promising classification accuracy of 95%. To verify the robustness of the proposed algorithm, multiple human subjects were involved in data acquisition.
9
Wang, Zhiyuan, Chongyuan Bi, Songhui You, and Junjie Yao. "Hidden Markov Model-Based Video Recognition for Sports." Advances in Mathematical Physics 2021 (December 20, 2021): 1–12. http://dx.doi.org/10.1155/2021/5183088.
Full textAbstract:
In this paper, we conduct an in-depth study and analysis of sports video recognition by improved hidden Markov model. The feature module is a complex gesture recognition module based on hidden Markov model gesture features, which applies the hidden Markov model features to gesture recognition and performs the recognition of complex gestures made by combining simple gestures based on simple gesture recognition. The combination of the two modules forms the overall technology of this paper, which can be applied to many scenarios, including some special scenarios with high-security levels that require real-time feedback and some public indoor scenarios, which can achieve different prevention and services for different age groups. With the increase of the depth of the feature extraction network, the experimental effect is enhanced; however, the two-dimensional convolutional neural network loses temporal information when extracting features, so the three-dimensional convolutional network is used in this paper to extract features from the video in time and space. Multiple binary classifications of the extracted features are performed to achieve the goal of multilabel classification. A multistream residual neural network is used to extract features from video data of three modalities, and the extracted feature vectors are fed into the attention mechanism network, then, the more critical information for video recognition is selected from a large amount of spatiotemporal information, further learning the temporal dependencies existing between consecutive video frames, and finally fusing the multistream network outputs to obtain the final prediction category. By training and optimizing the model in an end-to-end manner, recognition accuracies of 92.7% and 64.4% are achieved on the dataset, respectively.
10
Jing, Dong-Xing, Kui Huang, Shi-Jian Liu, Zheng Zou, and Chih-Yu Hsu. "Dynamic Hypergraph Convolutional Networks for Hand Motion Gesture Sequence Recognition." Technologies 13, no. 6 (2025): 257. https://doi.org/10.3390/technologies13060257.
Full textAbstract:
This paper introduces a novel approach to hand motion gesture recognition by integrating the Fourier transform with hypergraph convolutional networks (HGCNs). Traditional recognition methods often struggle to capture the complex spatiotemporal dynamics of hand gestures. HGCNs, which are capable of modeling intricate relationships among joints, are enhanced by Fourier transform to analyze gesture features in the frequency domain. A hypergraph is constructed to represent the interdependencies among hand joints, allowing for dynamic adjustments based on joint movements. Hypergraph convolution is applied to update node features, while the Fourier transform facilitates frequency-domain analysis. The T-Module, a multiscale temporal convolution module, aggregates features from multiple frames to capture gesture dynamics across different time scales. Experiments on the dynamic hypergraph (DHG14/28) and shape retrieval contest (SHREC’17) datasets demonstrate the effectiveness of the proposed method, achieving accuracies of 96.4% and 97.6%, respectively, and outperforming traditional gesture recognition algorithms. Ablation studies further validate the contributions of each component in enhancing recognition performance.
11
Baltazar, André. "ZatLab Gesture Recognition Framework." International Journal of Creative Interfaces and Computer Graphics 7, no. 2 (2016): 11–24. http://dx.doi.org/10.4018/ijcicg.2016070102.
Full textAbstract:
The main problem this work addresses is the real-time recognition of gestures, particularly in the complex domain of artistic performance. By recognizing the performer gestures, one is able to map them to diverse controls, from lightning control to the creation of visuals, sound control or even music creation, thus allowing performers real-time manipulation of creative events. The work presented here takes this challenge, using a multidisciplinary approach to the problem, based in some of the known principles of how humans recognize gesture, together with the computer science methods to successfully complete the task. This paper is a consequence of previous publications and presents in detail the Gesture Recognition Module of the ZatLab Framework and results obtained by its Machine Learning (ML) algorithms. One will provide a brief review the previous works done in the area, followed by the description of the framework design and the results of the recognition algorithms.
12
Wang, Huogen. "Two Stage Continuous Gesture Recognition Based on Deep Learning." Electronics 10, no. 5 (2021): 534. http://dx.doi.org/10.3390/electronics10050534.
Full textAbstract:
The paper proposes an effective continuous gesture recognition method, which includes two modules: segmentation and recognition. In the segmentation module, the video frames are divided into gesture frames and transitional frames by using the information of hand motion and appearance, and continuous gesture sequences are segmented into isolated sequences. In the recognition module, our method exploits the spatiotemporal information embedded in RGB and depth sequences. For the RGB modality, our method adopts Convolutional Long Short-Term Memory Networks to learn long-term spatiotemporal features from short-term spatiotemporal features obtained from a 3D convolutional neural network. For the depth modality, our method converts a sequence into Dynamic Images and Motion Dynamic Images through weighted rank pooling and feed them into Convolutional Neural Networks, respectively. Our method has been evaluated on both ChaLearn LAP Large-scale Continuous Gesture Dataset and Montalbano Gesture Dataset and achieved state-of-the-art performance.
13
Grabarczyk, Jakub, and Agnieszka Lazarowska. "Microcontroller Unit-Based Gesture Recognition System." Machines 13, no. 2 (2025): 90. https://doi.org/10.3390/machines13020090.
Full textAbstract:
This article describes the design, construction, and programming of a microcontroller-based system, which uses hand gestures with machine learning algorithms to control an unmanned aerial vehicle (UAV). A neural network is used as a model, and an IMU sensor detects the gestures. The developed gesture recognition system, besides the IMU sensor, is composed of a Raspberry Pi Pico and radio communication module. The benefits and drawbacks of deploying machine learning models on microcontrollers, as opposed to units superior in terms of clocking are also discussed.
14
ELBAHRI, Mohamed, Nasreddine TALEB, Sid Ahmed El Mehdi ARDJOUN, and Chakib Mustapha Anouar ZOUAOUI. "FEW-SHOT LEARNING WITH PRE-TRAINED LAYERS INTEGRATION APPLIED TO HAND GESTURE RECOGNITION FOR DISABLED PEOPLE." Applied Computer Science 20, no. 2 (2024): 1–23. http://dx.doi.org/10.35784/acs-2024-13.
Full textAbstract:
Employing vision-based hand gesture recognition for the interaction and communication of disabled individuals is highly beneficial. The hands and gestures of this category of people have a distinctive aspect, requiring the adaptation of a deep learning vision-based system with a dedicated dataset for each individual. To achieve this objective, the paper presents a novel approach for training gesture classification using few-shot samples. More specifically, the gesture classifiers are fine-tuned segments of a pre-trained deep network. The global framework consists of two modules. The first one is a base feature learner and a hand detector trained with normal people hand’s images; this module results in a hand detector ad hoc model. The second module is a learner sub-classifier; it is the leverage of the convolution layers of the hand detector feature extractor. It builds a shallow CNN trained with few-shot samples for gesture classification. The proposed approach enables the reuse of segments of a pre-trained feature extractor to build a new sub-classification model. The results obtained by varying the size of the training dataset have demonstrated the efficiency of our method compared to the ones of the literature.
15
Ahmed, Kadem Hamed AlSaedi, and H. Hassin AlAsadi Abbas. "A new hand gestures recognition system." Indonesian Journal of Electrical Engineering and Computer Science (IJEECS) 18, no. 1 (2020): 49–55. https://doi.org/10.11591/ijeecs.v18.i1.pp49-55.
Full textAbstract:
Talking about gestures makes us return to the historical beginning of human communication, because, in fact, there is no language completely free of gestures. People cannot communicate without gestures. Any action or movement without gestures is free of real feelings and cannot express the thoughts. The purpose from any hand gestures recognition system is to recognizes the hand gesture and used it to transfer a certain meaning or for computer control or and device. Our paper introduced a low cost system to recognize the hand gesture in real-time. Generally, the system divided into five steps, one to image acquisition, second to pre-processing the image, third for detection and segmentation of hand region, four to features extraction and five to count the numbers of fingers and gestures recognition. The system has coded by Python language, PyAutoGUI library, OS Module of Python and the OpenCV library.
16
Tong, Lina, Yunbo Li, Yixia Liang, and Chen Wang. "CAM-MR-MS based gesture recognition method using sEMG." Intelligence & Robotics 5, no. 2 (2025): 292–312. https://doi.org/10.20517/ir.2025.15.
Full textAbstract:
With the continuous concern for the disabled and the elderly, intelligent prosthetics and service robots have been widely applied. This paper provides a method for gesture recognition using forearm surface electromyography (sEMG), including an adaptive channel selection method to simplify the sEMG measurement. Based on the forearm muscle groups corresponding to different movements, surface skin areas are divided, and the Myo bracelet is used to collect sEMG signals from these areas. A method combined with channel attention module, multi-channel relationship feature extraction module and multi-scale skip connection module is built to adaptively select the signals from certain skin areas and recognize the seven gestures during experiment. The comparative experimental results indicate that this method can adaptively extract the optimal channel combination and show effective recognition results. It improved the practicability for the sEMG-based gesture recognition.
17
Hamed AlSaedi, Ahmed Kadem, and Abbas H. Hassin AlAsadi. "A new hand gestures recognition system." Indonesian Journal of Electrical Engineering and Computer Science 18, no. 1 (2020): 49. http://dx.doi.org/10.11591/ijeecs.v18.i1.pp49-55.
Full textAbstract:
<p>Talking about gestures makes us return to the historical beginning of human communication, because, in fact, there is no language completely free of gestures. People cannot communicate without gestures. Any action or movement without gestures is free of real feelings and cannot express the thoughts. The purpose from any hand gestures recognition system is to recognizes the hand gesture and used it to transfer a certain meaning or for computer control or and device. Our paper introduced a low cost system to recognize the hand gesture in real-time. Generally, the system divided into five steps, one to image acquisition, second to pre-processing the image, third for detection and segmentation of hand region, four to features extraction and five to count the numbers of fingers and gestures recognition. The system has coded by Python language, PyAutoGUI library, OS Module of Python and the OpenCV library.</p>
18
RAUTARAY, SIDDHARTH SWARUP, and ANUPAM AGRAWAL. "HAND GESTURE RECOGNITION TOWARDS VOCABULARY AND APPLICATION INDEPENDENCY." International Journal of Image and Graphics 13, no. 02 (2013): 1340001. http://dx.doi.org/10.1142/s0219467813400019.
Full textAbstract:
Traditional human–computer interaction devices such as the keyboard and mouse become ineffective for an effective interaction with the virtual environment applications because the 3D applications need a new interaction device. An efficient human interaction with the modern virtual environments requires more natural devices. Among them the "Hand Gesture" human–computer interaction modality has recently become of major interest. The main objective of gesture recognition research is to build a system which can recognize human gestures and utilize them to control an application. One of the drawbacks of present gesture recognition systems is being application-dependent which makes it difficult to transfer one gesture control interface into multiple applications. This paper focuses on designing a hand gesture recognition system which is vocabulary independent as well as adaptable to multiple applications. This makes the proposed system vocabulary independent and application independent. The designed system is comprised of the different processing steps like detection, segmentation, tracking, recognition, etc. Vocabulary independence has been incorporated in the proposed system with the help of a robust gesture mapping module that allows the user for cognitive mapping of different gestures to the same command and vice versa. For performance analysis of the proposed system accuracy, recognition rate and command response time have been compared. These parameters have been considered because they analyze the vital impact on the performance of the proposed vocabulary and application-independent hand gesture recognition system.
19
Li, Fenfang, Chujie Weng, and Yongguang Liang. "SE-WiGR: A WiFi Gesture Recognition Approach Incorporating the Squeeze–Excitation Mechanism and VGG16." Applied Sciences 15, no. 11 (2025): 6346. https://doi.org/10.3390/app15116346.
Full textAbstract:
With advancements in IoT and smart home tech, WiFi-driven gesture recognition is attracting more focus due to its non-contact nature and user-friendly design. However, WiFi signals are affected by multipath effects, attenuation, and interference, resulting in complex and variable signal patterns that pose challenges for accurately modeling gesture characteristics. This study proposes SE-WiGR, an innovative WiFi gesture recognition method to address these challenges. First, channel state information (CSI) related to gesture actions is collected using commercial WiFi devices. Next, the data is preprocessed, and Doppler-shift image data is extracted as input for the network model. Finally, the method integrates the squeeze-and-excitation (SE) mechanism with the VGG16 network to classify gestures. The method achieves a recognition accuracy of 94.12% across multiple scenarios, outperforming the standalone VGG16 network by 4.13%. This improvement confirms that the SE module effectively enhances gesture feature extraction while suppressing background noise.
20
Kenshimov, Chingis, Talgat Sundetov, Murat Kunelbayev, Zhazira Amirgaliyeva, Didar Yedilkhan, and Omirlan Auelbekov. "Development of a Verbal Robot Hand Gesture Recognition System." WSEAS TRANSACTIONS ON SYSTEMS AND CONTROL 16 (November 11, 2021): 573–83. http://dx.doi.org/10.37394/23203.2021.16.53.
Full textAbstract:
This article analyzes the most famous sign languages, the correlation of sign languages, and also considers the development of a verbal robot hand gesture recognition system in relation to the Kazakh language. The proposed system contains a touch sensor, in which the contact of the electrical property of the user's skin is measured, which provides more accurate information for simulating and indicating the gestures of the robot hand. Within the framework of the system, the speed and accuracy of recognition of each gesture of the verbal robot are calculated. The average recognition accuracy was over 98%. The detection time was 3ms on a 1.9 GHz Jetson Nano processor, which is enough to create a robot showing natural language gestures. A complete fingerprint of the Kazakh sign language for a verbal robot is also proposed. To improve the quality of gesture recognition, a machine learning method was used. The operability of the developed technique for recognizing gestures by a verbal robot was tested, and on the basis of computational experiments, the effectiveness of algorithms and software for responding to a verbal robot to a voice command was evaluated based on automatic recognition of a multilingual human voice. Thus, we can assume that the authors have proposed an intelligent verbal complex implemented in Python with the CMUSphinx communication module and the PyOpenGL graphical command execution simulator. Robot manipulation module based on 3D modeling from ABB.
21
Bhagwat, Prasad, Prashant Raut, Shubham Darade, Soham Gangurde, and Neha R. Hiray. "Hand Gesture Enabled Operation for Computer Using Deep Learning." INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT 08, no. 11 (2024): 1–7. http://dx.doi.org/10.55041/ijsrem38487.
Full textAbstract:
This project aims to develop a system that detects hand gestures and prints corresponding instructions for computer operators. The system utilizes Deep learning algorithms CNN and mediapipe to recognize hand gestures, and then prints instructions for the operator to perform specific computer operations.Operations like Notepad Open,Shutdown,Volume Up,Volume Down etc.The system will consist of a webcam, a Deep learning model, and a printer. The webcam will capture images of hand gestures, which will be processed using the Deep learning model to identify the specific gesture. The system will then print the corresponding instruction on the printer. The system will be designed to be user-friendly and adaptable to different environments and hardware configurations.The system aims to improve efficiency, reduce errors, and enhance accessibility for computer operators. The proposed system consists of a hand gesture detection module, instruction printing module, and computer operation module. The system is trained using a dataset of hand gestures and corresponding computer operations. Experimental results show that the system achieves high accuracy in gesture recognition and instruction printing. The system has potential applications in various fields, including gaming, education, and healthcare. Key Words: Hand gesture detection, Deep learning, instruction printing, computer operation, accessibility, Mediapipe, CNN.
22
Chen, Renxiang, and Xia Tian. "Gesture Detection and Recognition Based on Object Detection in Complex Background." Applied Sciences 13, no. 7 (2023): 4480. http://dx.doi.org/10.3390/app13074480.
Full textAbstract:
In practical human–computer interaction, a hand gesture recognition method based on improved YOLOv5 is proposed to address the problem of low recognition accuracy and slow speed with complex backgrounds. By replacing the CSP1_x module in the YOLOv5 backbone network with an efficient layer aggregation network, a richer combination of gradient paths can be obtained to improve the network’s learning and expressive capabilities and enhance recognition speed. The CBAM attention mechanism is introduced to filtering gesture features in channel and spatial dimensions, reducing various types of interference in complex background gesture images and enhancing the network’s robustness against complex backgrounds. Experimental verification was conducted on two complex background gesture datasets, EgoHands and TinyHGR, with recognition accuracies of mAP0.5:0.95 at 75.6% and 66.8%, respectively, and a recognition speed of 64 FPS for 640 × 640 input images. The results show that the proposed method can recognize gestures quickly and accurately with complex backgrounds, and has higher recognition accuracy and stronger robustness compared to YOLOv5l, YOLOv7, and other comparative algorithms.
23
Jiang, Hairong, Juan P. Wachs, and Bradley S. Duerstock. "Integrated vision-based system for efficient, semi-automated control of a robotic manipulator." International Journal of Intelligent Computing and Cybernetics 7, no. 3 (2014): 253–66. http://dx.doi.org/10.1108/ijicc-09-2013-0042.
Full textAbstract:
Purpose – The purpose of this paper is to develop an integrated, computer vision-based system to operate a commercial wheelchair-mounted robotic manipulator (WMRM). In addition, a gesture recognition interface system was developed specially for individuals with upper-level spinal cord injuries including object tracking and face recognition to function as an efficient, hands-free WMRM controller. Design/methodology/approach – Two Kinect® cameras were used synergistically to perform a variety of simple object retrieval tasks. One camera was used to interpret the hand gestures and locate the operator's face for object positioning, and then send those as commands to control the WMRM. The other sensor was used to automatically recognize different daily living objects selected by the subjects. An object recognition module employing the Speeded Up Robust Features algorithm was implemented and recognition results were sent as a commands for “coarse positioning” of the robotic arm near the selected object. Automatic face detection was provided as a shortcut enabling the positing of the objects close by the subject's face. Findings – The gesture recognition interface incorporated hand detection, tracking and recognition algorithms, and yielded a recognition accuracy of 97.5 percent for an eight-gesture lexicon. Tasks’ completion time were conducted to compare manual (gestures only) and semi-manual (gestures, automatic face detection, and object recognition) WMRM control modes. The use of automatic face and object detection significantly reduced the completion times for retrieving a variety of daily living objects. Originality/value – Integration of three computer vision modules were used to construct an effective and hand-free interface for individuals with upper-limb mobility impairments to control a WMRM.
24
Padmaja, M. "Wave Your Way: Navigation Through Hand Gestures." INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT 09, no. 03 (2025): 1–9. https://doi.org/10.55041/ijsrem42033.
Full textAbstract:
Several technologies are continuously developing in today’s technological world where human-computer interaction is very important. In human-computer interactions, hand gesture recognition is essential. We can control our system by showing our hands in front of a webcam, and hand gesture recognition can be useful for all kinds of people. A specific interactive module like a virtual mouse that makes use of Object Tracking and Gestures will help us interact and serve as an alternative way to the traditional touchscreen and physical mouse. The system allows people to control a computer cursor using hand movements recorded by a camera by utilizing computer vision techniques and machine learning algorithms. Three key components of the suggested system are hand detection, gesture recognition, and cursor control. Hand detection is the process of locating and tracking the user's hand within the camera's field of view. This process may employ methods such as deep learning-based object detection, backdrop subtraction, and skin colour segmentation. In this, we can also capture the gestures clearly even when the camera quality is poor, and it can provide better navigation without any clumsiness. KEYWORDS: Hand Gesture Recognition, Human-Computer Interaction, Computer Vision, video capturing, Python Libraries, Accessibility, Real-Time Systems, speech Recognition.
25
K., Selvakumar, Palanisamy R., Noyal Doss M.Arun, et al. "Gesture recognition vehicle using PIC microcontroller." Indonesian Journal of Electrical Engineering and Computer Science (IJEECS) 19, no. 1 (2020): 66–76. https://doi.org/10.11591/ijeecs.v19.i1.pp66-76.
Full textAbstract:
In this paper we introduce a hand controlled robotic vehicle. Hand control robotic vehicle consists of a transmitter module and a receiver module. The transmitter will be placed on a hand glove and the receiver will be placed on the motor drive along with PIC microcontroller and motor driver IC. The RF transmitter sends commands to the IC which then forwards the commands to RF receiver. The RF receiver then sends the commands to PIC microcontroller on the vehicle which processes the commands so that the vehicle moves in the specified desired direction. It is having proposed utility in field ofconstruction, hazardous waste disposal and field survey near borders etc. This project is developed as a travel buddy and industrial uses. Having future scope of advanced robotics that are designed and can be easily controlled using hand gesture only.
26
Gupta, Mr Deven, Ms Anamika Zagade, Ms Mansi Gawand, Ms Saloni Mhatre, and Prof J. W. Bakal. "Gesture Controlled Virtual Artboard." International Journal for Research in Applied Science and Engineering Technology 12, no. 2 (2024): 1557–65. http://dx.doi.org/10.22214/ijraset.2024.58680.
Full textAbstract:
Abstract: Hand Gesture Recognition has brought a brand new era to Artificial Intelligence. Gesture recognition is technology that interprets hand movements as commands. Gesture Controlled Virtual Artboard Project is an innovative and dynamic digital artboard. Use of this artboard can break various education barrier to provide students with fun and creative way of learning and hence It revolutionize the way of traditional teaching. Physically challenged or aged people find it difficult to identify and press the exact key on keyboard. Existing system allows user for free hand drawing with 3 different colors only. Overcoming all this difficulty stands as the ultimate goal for our proposed system. This system works on motion sensing technology which make use of OpenCV module & Mediapipe library to explicate results based on gestures of hand. Real time data is collected through webcam and this python application uses Mediapipe library to track hand gestures. Hence, this system allows user to navigate the fingers in mid air and as per the finger gestures they will draw different shapes or free hand drawing in various colors and also can erase.This Project also enables users to conduct PowerPoint presentations using gestures.
27
Selvakumar, K., Palanisamy R, M. Arun Noyal Doss, et al. "Gesture recognition vehicle using PIC microcontroller." Indonesian Journal of Electrical Engineering and Computer Science 19, no. 1 (2020): 66. http://dx.doi.org/10.11591/ijeecs.v19.i1.pp66-75.
Full textAbstract:
<p>In this paper we introduce a hand controlled robotic vehicle. Hand control robotic vehicle consists of a transmitter module and a receiver module. The transmitter will be placed on a hand glove and the receiver will be placed on the motor drive along with PIC microcontroller and motor driver IC. The RF transmitter sends commands to the IC which then forwards the commands to RF receiver. The RF receiver then sends the commands to PIC microcontroller on the vehicle which processes the commands so that the vehicle moves in the specified direction. It is having proposed utility in field ofconstruction, hazardous waste disposal and field survey near borders etc. This project is developed as a travel buddy and industrial uses. Having future scope of advanced robotics that are designed and can be easily controlled using hand gesture only.</p>
28
Chen, Si, Dandan Cheng, and Quan Zhou. "Design and Application of Interactive Algorithm for Advertising Media Screen Based on Smart Sensor." Scientific Programming 2022 (March 15, 2022): 1–10. http://dx.doi.org/10.1155/2022/4467739.
Full textAbstract:
Touch is one of the most important human senses. With the popularization of touch-screen mobile phones, tablet computers, and other devices, touch-screen interactive technology has become a norm in people’s daily lives, and advertisements that were once dominated by vision and hearing have added an interactive experience in the dimension of touch. Traditional advertising media screens can only complete simple information dissemination functions and cannot interact with users in a two-way manner. They can only receive information one-way and passively and lack interactivity. Touch-screen interactive advertising forms a good interaction with the target audience, thereby disseminating advertising information to achieve the purpose of promotion or brand image building. This paper designs a set of advertising media screen interaction systems based on smart sensors, including a gesture interaction module, a remote interaction module, and a touch interaction module. The gesture interaction module can recognize 5 static gestures and send gesture commands to control the advertising media screen. The remote interaction module can remotely control the advertising media screen, and the touch interaction module can control the advertising media screen through the touch screen. According to the functional requirements, the overall design of software and hardware is given, and the technical background of each module of the software is introduced. Next, the depth image-based gesture recognition method is studied. The number of fingers and the center distance feature are fused as feature vectors, and the weighted template matching method is used to classify and recognize gestures. Finally, the design and implementation of the interactive system are introduced.
29
Tu, Min. "Gesture Detection and Recognition Based on Pyramid Frequency Feature Fusion Module and Multiscale Attention in Human-Computer Interaction." Mathematical Problems in Engineering 2021 (May 7, 2021): 1–10. http://dx.doi.org/10.1155/2021/6043152.
Full textAbstract:
Aiming at the problem of the absence of detail texture and other high-frequency features in the feature extraction process of the deep network employing the upsampling operation, the accuracy of gesture recognition is seriously affected in complex scenes. This study integrates object detection and gesture recognition into one model and proposes a gesture detection and recognition based on the pyramid frequency feature fusion module and multiscale attention in human-computer interaction. Pyramid fusion module is used to perform efficient feature fusion and is proposed to obtain feature layers with rich details and semantic information, which is helpful to improve the efficiency and accuracy of gesture recognition. In addition, the multiscale attention module is further adopted to adaptively mine important and effective feature information from both temporal and spatial channels and embedded into the detection layer. Finally, our proposed network realizes the enhancement of the effective information and the suppression of the invalid information of the detection layer. Experimental results show that our proposed model makes full use of the high-low frequency feature fusion module without replacing the basic backbone network, which can greatly reduce the computational overhead while improving the detection accuracy.
30
Sravani, Jiripurapu, Soma Yagna Priya, Gowra Pavan Kumar, Chereddy Mohith Sankar, and KRMC Sekhar. "Hand Gesture Detection using Deep Learning with YOLOv5." International Journal of Multidisciplinary Research and Growth Evaluation. 6, no. 2 (2025): 742–50. https://doi.org/10.54660/.ijmrge.2025.6.2.742-750.
Full textAbstract:
Hand gesture recognition has become a significant technological advancement in assistive communication, offering a reliable means of interaction for individuals with hearing and speech impairments. This research introduces an intelligent gesture detection system powered by YOLOv5, a leading object detection model, to enable accurate and real-time recognition of Indian Sign Language (ISL) gestures. The system effectively handles diverse environmental conditions and user-specific variations using an extensive and well-annotated dataset. The methodology encompasses essential stages such as image preprocessing, data augmentation, and feature extraction to optimize model performance. Furthermore, a user-friendly web interface allows users to upload images for gesture detection, with corresponding text and audio outputs generated using a text-to-speech module. Designed for seamless scalability, the system can accommodate additional gestures and languages, making it a versatile solution for educational institutions, healthcare facilities, and public service sectors. By fostering greater inclusivity and accessibility, this approach represents a step forward in empowering the hearing-impaired community through innovative deep-learning applications.
31
Lu, Guangda, Wenhao Sun, Zhuanping Qin, and Tinghang Guo. "Real-Time Dynamic Gesture Recognition Algorithm Based on Adaptive Information Fusion and Multi-Scale Optimization Transformer." Journal of Advanced Computational Intelligence and Intelligent Informatics 27, no. 6 (2023): 1096–107. http://dx.doi.org/10.20965/jaciii.2023.p1096.
Full textAbstract:
Gesture recognition is a popular technology in the field of computer vision and an important technical mean of achieving human-computer interaction. To address problems such as the limited long-range feature extraction capability of existing dynamic gesture recognition networks based on convolutional operators, we propose a dynamic gesture recognition algorithm based on spatial pyramid pooling Transformer and optical flow information fusion. We take advantage of Transformer’s large receptive field to reduce model computation while improving the model’s ability to extract features at different scales by embedding spatial pyramid pooling. We use the optical flow algorithm with the global motion aggregation module to obtain an optical flow map of hand motion, and to extract the key frames based on the similarity minimization principle. We also design an adaptive feature fusion method to fuse the spatial and temporal features of the dual channels. Finally, we demonstrate the effectiveness of model components on model recognition enhancement through ablation experiments. We conduct training and validation on the SCUT-DHGA dynamic gesture dataset and on a dataset we collected, and we perform real-time dynamic gesture recognition tests using the trained model. The results show that our algorithm achieves high accuracy even while keeping the parameters balanced. It also achieves fast and accurate recognition of dynamic gestures in real-time tests.
32
Alange, Rutwik. "Hand Gesture Controller." International Journal for Research in Applied Science and Engineering Technology 12, no. 3 (2024): 2540–44. http://dx.doi.org/10.22214/ijraset.2024.59395.
Full textAbstract:
Abstract: The rapid evolution of human-computer interaction has spurred significant progress in gesture recognition technologies, placing a specific emphasis on diverse applications. This paper highlights key advancements in machine learning algorithms tailored for gesture recognition, including deep learning approaches that have notably improved the accuracy and robustness of hand tracking systems. Furthermore, the integration of hand gesture control into wearable devices and its implications for everyday technology usage are thoroughly examined. This paper relies on the formidable capabilities of Deep Learning, specifically Convolutional Neural Networks and Recurrent Neural Networks, to decipher and translate hand gestures into actionable directives. The implementation of the Hand Gesture Controller necessitates the integration of a camera or sensor module capable of capturing intricate hand movements. Real-time image processing and feature extraction are essential components, facilitating the provision of input data to the Deep Learning model. As technology progresses, this interface emerges as a versatile tool capable of enhancing productivity and inclusivity across various domains. The paper concludes with a discussion on future directions in hand gesture controller development, exploring anticipated technological advancements, novel use cases, and the potential for increased accessibility. In summary, the "Hand Gesture Controller using Deep Learning" project signifies a substantive stride forward in human-computer interaction, introducing an innovative interface that promises a future where devices seamlessly respond to innate gestures, thereby rendering technology more accessible and user-focused.
33
Chmurski, Mateusz, Gianfranco Mauro, Avik Santra, Mariusz Zubert, and Gökberk Dagasan. "Highly-Optimized Radar-Based Gesture Recognition System with Depthwise Expansion Module." Sensors 21, no. 21 (2021): 7298. http://dx.doi.org/10.3390/s21217298.
Full textAbstract:
The increasing integration of technology in our daily lives demands the development of more convenient human–computer interaction (HCI) methods. Most of the current hand-based HCI strategies exhibit various limitations, e.g., sensibility to variable lighting conditions and limitations on the operating environment. Further, the deployment of such systems is often not performed in resource-constrained contexts. Inspired by the MobileNetV1 deep learning network, this paper presents a novel hand gesture recognition system based on frequency-modulated continuous wave (FMCW) radar, exhibiting a higher recognition accuracy in comparison to the state-of-the-art systems. First of all, the paper introduces a method to simplify radar preprocessing while preserving the main information of the performed gestures. Then, a deep neural classifier with the novel Depthwise Expansion Module based on the depthwise separable convolutions is presented. The introduced classifier is optimized and deployed on the Coral Edge TPU board. The system defines and adopts eight different hand gestures performed by five users, offering a classification accuracy of 98.13% while operating in a low-power and resource-constrained environment.
34
Mrs A Praveena, Putala Balanarayana, M Ajay, M S HariharaSudhan, and P Johndev. "Voice Controlled EV System and Locking System with Gesture Based Authentication." International Journal of Scientific Research in Science and Technology 12, no. 1 (2025): 588–603. https://doi.org/10.32628/ijsrst25121193.
Full textAbstract:
The "Voice Controlled EV System and Locking System with Gesture-Based Authentication" introduces an advanced vehicle security and accessibility solution by integrating voice recognition using Mel-Frequency Cepstral Coefficients (MFCC) and gesture-based authentication via computer vision. This dual-layered system eliminates the need for traditional keys, enhancing both security and convenience by requiring specific voice commands and hand gestures for vehicle access and control. The primary objective is to develop a robust framework that ensures only registered users can unlock and operate the vehicle through authenticated voice and gesture inputs. By merging these technologies, the project creates an intuitive interface while improving system security and reliability. This aligns with the growing demand for smarter, connected vehicles, making it a timely innovation in the automotive sector. The methodology combines expertise in electrical engineering, computer science, signal processing, and motor control systems. The voice recognition module leverages MFCC for accurate speech feature extraction, while the gesture authentication system uses image processing and machine learning for reliable hand movement detection. These subsystems are integrated with a microcontroller-driven motor system that executes validated commands. The project follows a systematic development process, including standalone module creation, integration, and real-world testing to ensure efficiency and reliability. By replacing conventional access mechanisms with voice and gesture controls, this innovation enhances user experience, addresses modern security challenges, and sets a new benchmark for intuitive vehicle interaction.
35
Xu, Jun, Xiong Zhang, and Meng Zhou. "A High-Security and Smart Interaction System Based on Hand Gesture Recognition for Internet of Things." Security and Communication Networks 2018 (June 6, 2018): 1–11. http://dx.doi.org/10.1155/2018/4879496.
Full textAbstract:
In this work, we propose a vision-based hand gesture recognition system to provide a high-security and smart node in the application layer of Internet of Things. The system can be installed in any terminal device with a monocular camera and interact with users by recognizing pointing gestures in the captured images. The interaction information is determined by a straight line from the user’s eye to the tip of the index finger, which achieves real-time and authentic data communication. The system mainly contains two modules. The first module is an edge repair-based hand subpart segmentation algorithm which combines pictorial structures and edge information to extract hand regions from complex backgrounds. Second, the position which the user focuses on is located by an adaptive method of pointing gesture estimation, which adjusts the offsets between the target position and the calculated position due to lack of depth information.
36
Manoharan, Dr P. S., Dr B. Ashok Kumar, N. S. Indhumathi, L. Jeyaseelan, and A. Joseph Rajkumar. "CONTROL SYSTEM BASED ON HAND GESTURE RECOGNITION USING MEMS ACCELEROMETERS." International Journal of Engineering Applied Sciences and Technology 8, no. 6 (2023): 100–103. http://dx.doi.org/10.33564/ijeast.2023.v08i06.013.
Full textAbstract:
This paper aims to propose a methodology to assist individuals with physical disabilities by utilizing MEMS technology. This technology can be used to help non-verbal people communicate without words or assist those with partial paralysis to control their surroundings. The gesture identification is achieved using MEMS accelerometers. When a gesture is made, the accelerometers register the movement, which is then interpreted by a microcontroller. The microcontroller recognizes the gesture by comparing it to pre-programmed signals and assigns it a unique code. Additionally, there is a control module that includes an RF receiver. The RF receiver can detect the unique code from the user module. The ATMEGA microcontroller used in the control module to trigger specific functions based on the user's preferences and requirements. The proposed model is implemented in real time under various testing conditions.
37
Prof., C. D. Sawarkar Vivek Vaidya Vansh Sharma Samir Sheikh Aniket Neware Prathmesh Chaudhari. "AI Based Real Time Hand Gesture Recognition System." International Journal of Advanced Innovative Technology in Engineering 9, no. 3 (2024): 320–23. https://doi.org/10.5281/zenodo.12747525.
Full textAbstract:
This research presents a comprehensive approach for real-time hand gesture recognition using a synergistic combination of TensorFlow, OpenCV, and Media Pipe. Hand gesture recognition holds immense potential for natural and intuitive human-computer interaction in various applications, such as augmented reality, virtual reality, and human computer interfaces. The proposed system leverages the strengths of TensorFlow for deep learning-based model development, OpenCV for computer vision tasks, and Media Pipe for efficient hand landmark detection. The workflow begins with hand detection using OpenCV, followed by the extraction of hand landmarks through Media Pipe's hand tracking module. These landmarks serve as crucial input features for a custom trained TensorFlow model, designed to recognize a diverse set of hand gestures. The model is trained on a well- curated dataset, ensuring robust performance across different hand shapes, sizes, and orientations.
38
Potbhare, Tejas. "Hand Gesture Based PPT Controller." International Journal for Research in Applied Science and Engineering Technology 13, no. 4 (2025): 1040–44. https://doi.org/10.22214/ijraset.2025.68436.
Full textAbstract:
Presentations are essential in both academic and professional environments, yet conventional slide navigation methods often limit the presenter’s movement and interaction. Gesture recognition offers a more intuitive, hands-free alternative, removing the dependence on physical remotes or keyboards. This project presents a gesture-controlled system for managing PowerPoint slides and adjusting computer volume, leveraging embedded systems and wireless communication. The setup includes an APDS-9960 gesture sensor for hand gesture detection, an Arduino Uno for processing inputs, and an HC-05 Bluetooth module for transmitting commands to a computer. A custom-built desktop application developed with ASP.NET interprets these gestures in real time, allowing users to perform actions such as slide transitions, volume adjustments, and presentation pauses. The system is designed with accessibility in mind, aiming to support users with disabilities and improve the overall efficiency of presentations. By enabling touchless interaction, this solution enhances user experience and engagement. Future developments may include AI-based gesture recognition, support for multiple commands, and potential integration with AR/VR platforms to broaden its application in human-computer interaction.
39
Nagadeepa.Ch, Dr.N.Balaji, and Dr.V.Padmaja. "ANALYSIS OF INERTIAL SENSOR DATA USING TRAJECTORY RECOGNITION ALGORITHM." International Journal on Cybernetics & Informatics (IJCI) 5, no. 4 (2017): 101–7. https://doi.org/10.5121/ijci.2016.5412.
Full textAbstract:
This paper describes a digital pen based on IMU sensor for gesture and handwritten digit gesture trajectory recognition applications. This project allows human and Pc interaction. Handwriting Recognition is mainly used for applications in the field of security and authentication. By using embedded pen the user can make hand gesture or write a digit and also an alphabetical character. The embedded pen contains an inertial sensor, microcontroller and a module having Zigbee wireless transmitter for creating handwriting and trajectories using gestures. The propound trajectory recognition algorithm constitute the sensing signal attainment, pre-processing techniques, feature origination, feature extraction, classification technique. The user hand motion is measured using the sensor and the sensing information is wirelessly imparted to PC for recognition. In this process initially excerpt the time domain and frequency domain features from pre-processed signal, later it performs linear discriminant analysis in order to represent features with reduced dimension. The dimensionally reduced features are processed with two classifiers – State Vector Machine (SVM) and k-Nearest Neighbour (kNN). Through this algorithm with SVM classifier provides recognition rate is 98.5% and with kNN classifier recognition rate is 95.5% .
40
Liu, Xiaoguang, Mingjin Zhang, Jiawei Wang, et al. "Gesture recognition of continuous wavelet transform and deep convolution attention network." Mathematical Biosciences and Engineering 20, no. 6 (2023): 11139–54. http://dx.doi.org/10.3934/mbe.2023493.
Full textAbstract:
<abstract> <p>To solve the problem of missing data features using a deep convolutional neural network (DCNN), this paper proposes an improved gesture recognition method. The method first extracts the time-frequency spectrogram of surface electromyography (sEMG) using the continuous wavelet transform. Then, the Spatial Attention Module (SAM) is introduced to construct the DCNN-SAM model. The residual module is embedded to improve the feature representation of relevant regions, and reduces the problem of missing features. Finally, experiments with 10 different gestures are done for verification. The results validate that the recognition accuracy of the improved method is 96.1%. Compared with the DCNN, the accuracy is improved by about 6 percentage points.</p> </abstract>
41
Pan, Jingzhi. "Enhancing Gesture-Based Interactions for Individuals with Disabilities through Dual-Attention Wireless Sensing Networks." Highlights in Science, Engineering and Technology 119 (December 11, 2024): 199–206. https://doi.org/10.54097/vsjbt578.
Full textAbstract:
This paper investigates the application of wireless sensing technology to develop a more natural and intuitive gesture-based interaction framework for individuals with disabilities. The study begins by identifying the primary challenges faced by this demographic when interacting with conventional devices. It then introduces a novel approach for WiFi-enabled gesture recognition and interpretation using dual attention networks. This methodology comprises two essential components: Channel State Information (CSI) preprocessing and a gesture recognition module. The paper details the implementation of these modules and elucidates their roles in enhancing gesture detection accuracy. Furthermore, the discussion extends to the future prospects of wireless sensing technologies, envisioning their integration into smart home systems, public services, and enhanced social interactions. The research underscores the transformative potential of integrating gesture-based interactions with wireless sensing to significantly elevate the quality of life for people with disabilities, suggesting a paradigm shift in how assistive technologies are developed and utilized.
42
Sara, Meghana. "Gesture Controlled Robot for the Disabled using Arduino." International Journal for Research in Applied Science and Engineering Technology 12, no. 4 (2024): 5446–49. http://dx.doi.org/10.22214/ijraset.2024.61159.
Full textAbstract:
Abstract: The gesture-controlled robot project presents a novel approach to human-robot interaction, utilizing hand gestures as an intuitive and natural means of control. This project aims to develop a system where users can seamlessly communicate with a robot using simple hand movements. The project consists of two main components: a handheld device for gesture input and a robot equipped with sensors and actuators for movement. Through wireless communication, the handheld device transmits signals encoding the user's hand gestures to the robot's receiver module, which decodes the signals, processes them, and translates them into commands for controlling the robot's movements. Advantages of this system include intuitive control, wireless operation, real-time interaction, enhanced safety, adaptability, educational value, and versatility. However, challenges such as gesture recognition complexity and limited gesture vocabulary must be addressed to ensure the system's effectiveness. The project has numerous applications, including entertainment, assistive technology, industrial automation, education, home automation, IoT, security, and interactive exhibits. By leveraging gesture-controlled robotics, this project contributes to advancing human-robot interaction technology and fostering innovation in various domains.
43
Rupali, Shinganjude, Majrikar Dipak, Bawiskar Atharva, Bobhate Sayali, Chaurawar Jatin, and Ninawe Gaurav. "Virtual Mouse Using AI and Computer Vision." Virtual Mouse Using AI and Computer Vision 8, no. 11 (2023): 4. https://doi.org/10.5281/zenodo.10158688.
Full textAbstract:
A paper proposing a virtual mouse system controlled by hand gestures that use AI algorithms to recognize and translate them into mouse movements. In the human-computer interaction module, hand gestures played a significant role. The system is designed to provide an alternative interface for people who face difficulty using a traditional mouse or keyboard. This virtual mouse system uses a camera to capture images of the user's hand, which are processed by an AI algorithm to recognize the gestures made by the user. Once the gesture is recognized, it is translated to a corresponding mouse movement, which is then executed on the virtual screen. This model provides potential applications like enabling the hand-free operation of devices in hazardous environments and providing an alternative interface for hardware mouse. This hand gesture-controlled virtual mouse system offers a promising approach to enhance user experience and improve accessibility through human-computer interaction.Keywords:- Virtual Mouse, Hand Gesture Recognition, Computer Vision, Media-Pipe.
44
Gao, Zhongjian, Chien-Cheng Lee, Lianhui Zheng, Ruige Zhang, and Xiaofu Xu. "A Multitask Sign Language Recognition System Using Commodity Wi-Fi." Mobile Information Systems 2023 (February 10, 2023): 1–11. http://dx.doi.org/10.1155/2023/7959916.
Full textAbstract:
Wi-Fi sensing for gesture recognition systems is a fascinating and challenging research topic. We propose a multitask sign language recognition framework called Wi-SignFi, which accounts for gestures in the real world associated with various objects, actions, or scenes. The proposed framework comprises a convolutional neural network (CNN) and K-nearest neighbor (KNN) module. It is evaluated on the public SignFi dataset and achieves 98.91%, 86.67%, and 99.99% average gesture recognition accuracies on 276/150 activities, five users, and two environments, respectively. The experimental results show that the proposed gesture recognition method outperforms previous methods. Instead of converting the channel state information (CSI) data of multiple antennas into three-dimensional matrices (i.e., color images) as in the existing literature, we found that the CSI data can be converted into matrices (i.e., grayscale images) by concatenating different channels, allowing the Wi-SignFi model to balance between speed and accuracy. This finding facilitates deploying Wi-SignFi on Nvidia’s Jetson Nano edge embedded devices. We expect this work to promote the integration of Wi-Fi sensing and the Internet of Things (IoT) and improve the quality of life of the deaf community.
45
Li, Nan, Xiao Han, Xingguo Song, Xu Fang, Mengming Wu, and Qiulin Yu. "Lightweight RT-DETR with Attentional Up-Downsampling Pyramid Network." Applied Sciences 15, no. 6 (2025): 3309. https://doi.org/10.3390/app15063309.
Full textAbstract:
As healthcare costs rise due to aging populations and chronic illnesses, optimized care solutions are urgently needed. Gesture recognition and fall detection are critical for intelligent companion robots in healthcare. However, current deep learning models struggle with accuracy and real-time performance in complex backgrounds due to high computational demands. To address this, we propose an improved RT-DETR R18 model tailored for companion robots. This lightweight, efficient design integrates YOLOv9’s ADown module, the RepNCSPELAN4 module, and custom attention-based AdaptiveGateUpsample and AdaptiveGateDownsample modules for enhanced multi-scale feature fusion, reducing weight and complexity while optimizing real-time detection. Experiments show our model achieves a 51.7% reduction in parameters, a 46.7% decrease in GFLOPS, and higher FPS compared to RT-DETR R18, with mAP@0.5, mAP@0.5-0.95, precision, and recall improving to 99.4%, 86.4%, 99.6%, and 99.4%, respectively. Testing in complex indoor environments confirms its high accuracy for gesture recognition and fall detection, reducing manual workload and offering a novel solution for human behavior recognition in intelligent companionship.
46
Zhou, Benjia, Yunan Li, and Jun Wan. "Regional Attention with Architecture-Rebuilt 3D Network for RGB-D Gesture Recognition." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 4 (2021): 3563–71. http://dx.doi.org/10.1609/aaai.v35i4.16471.
Full textAbstract:
Human gesture recognition has drawn much attention in the area of computer vision. However, the performance of gesture recognition is always influenced by some gesture-irrelevant factors like the background and the clothes of performers. Therefore, focusing on the regions of hand/arm is important to the gesture recognition. Meanwhile, a more adaptive architecture-searched network structure can also perform better than the block-fixed ones like ResNet since it increases the diversity of features in different stages of the network better. In this paper, we propose a regional attention with architecture-rebuilt 3D network (RAAR3DNet) for gesture recognition. We replace the fixed Inception modules with the automatically rebuilt structure through the network via Neural Architecture Search (NAS), owing to the different shape and representation ability of features in the early, middle, and late stage of the network. It enables the network to capture different levels of feature representations at different layers more adaptively. Meanwhile, we also design a stackable regional attention module called Dynamic-Static Attention (DSA), which derives a Gaussian guidance heatmap and dynamic motion map to highlight the hand/arm regions and the motion information in the spatial and temporal domains, respectively. Extensive experiments on two recent large-scale RGB-D gesture datasets validate the effectiveness of the proposed method and show it outperforms state-of-the-art methods. The codes of our method are available at: https://github.com/zhoubenjia/RAAR3DNet.
47
Lei, Jingpeng. "Enhancing gesture recognition with multiscale feature extraction and spatial attention." PLOS One 20, no. 6 (2025): e0324050. https://doi.org/10.1371/journal.pone.0324050.
Full textAbstract:
Gesture recognition technology is a pivotal element in human-computer interaction, enabling users to communicate with machines in a natural and intuitive manner. This paper introduces a novel approach to gesture recognition that enhances accuracy and robustness by integrating multiscale feature extraction and spatial attention mechanisms. Specifically, we have developed a multiscale feature extraction module inspired by the Inception architecture, which captures comprehensive features across various scales, providing a more holistic feature representation. Additionally, We incorporate a spatial attention mechanism that focuses on image regions most relevant to the current gesture, thereby improving the discriminative power of the features. Extensive experiments conducted on multiple benchmark datasets demonstrate that our method significantly outperforms existing gesture recognition techniques in terms of accuracy.
48
Peng, Xiangdong, Xiao Zhou, Huaqiang Zhu, Zejun Ke, and Congcheng Pan. "MSFF-Net: Multi-Stream Feature Fusion Network for surface electromyography gesture recognition." PLOS ONE 17, no. 11 (2022): e0276436. http://dx.doi.org/10.1371/journal.pone.0276436.
Full textAbstract:
In the field of surface electromyography (sEMG) gesture recognition, how to improve recognition accuracy has been a research hotspot. The rapid development of deep learning provides a new solution to this problem. At present, the main applications of deep learning for sEMG gesture feature extraction are based on convolutional neural network (CNN) structures to capture spatial morphological information of the multichannel sEMG or based on long short-term memory network (LSTM) to extract time-dependent information of the single-channel sEMG. However, there are few methods to comprehensively consider the distribution area of the sEMG signal acquisition electrode sensor and the arrangement of the sEMG signal morphological features and electrode spatial features. In this paper, a novel multi-stream feature fusion network (MSFF-Net) model is proposed for sEMG gesture recognition. The model adopts a divide-and-conquer strategy to learn the relationship between different muscle regions and specific gestures. Firstly, a multi-stream convolutional neural network (Multi-stream CNN) and a convolutional block attention module integrated with a resblock (ResCBAM) are used to extract multi-dimensional spatial features from signal morphology, electrode space, and feature map space. Then the learned multi-view depth features are fused by a view aggregation network consisting of an early fusion network and a late fusion network. The results of all subjects and gesture movement validation experiments in the sEMG signal acquired from 12 sensors provided by NinaPro’s DB2 and DB4 sub-databases show that the proposed model in this paper has better performance in terms of gesture recognition accuracy compared with the existing models.
49
Jiang, Yujian, Lin Song, Junming Zhang, Yang Song, and Ming Yan. "Multi-Category Gesture Recognition Modeling Based on sEMG and IMU Signals." Sensors 22, no. 15 (2022): 5855. http://dx.doi.org/10.3390/s22155855.
Full textAbstract:
Gesture recognition based on wearable devices is one of the vital components of human–computer interaction systems. Compared with skeleton-based recognition in computer vision, gesture recognition using wearable sensors has attracted wide attention for its robustness and convenience. Recently, many studies have proposed deep learning methods based on surface electromyography (sEMG) signals for gesture classification; however, most of the existing datasets are built for surface EMG signals, and there is a lack of datasets for multi-category gestures. Due to model limitations and inadequate classification data, the recognition accuracy of these methods cannot satisfy multi-gesture interaction scenarios. In this paper, a multi-category dataset containing 20 gestures is recorded with the help of a wearable device that can acquire surface electromyographic and inertial (IMU) signals. Various two-stream deep learning models are established and improved further. The basic convolutional neural network (CNN), recurrent neural network (RNN), and Transformer models are experimented on with our dataset as the classifier. The CNN and the RNN models’ test accuracy is over 95%; however, the Transformer model has a lower test accuracy of 71.68%. After further improvements, the CNN model is introduced into the residual network and augmented to the CNN-Res model, achieving 98.24% accuracy; moreover, it has the shortest training and testing time. Then, after combining the RNN model and the CNN-Res model, the long short term memory (LSTM)-Res model and gate recurrent unit (GRU)-Res model achieve the highest classification accuracy of 99.67% and 99.49%, respectively. Finally, the fusion of the Transformer model and the CNN model enables the Transformer-CNN model to be constructed. Such improvement dramatically boosts the performance of the Transformer module, increasing the recognition accuracy from 71.86% to 98.96%.
50
Wang, Wenjie, Mengling He, Xiaohua Wang, Jianwei Ma, and Huajian Song. "Medical Gesture Recognition Method Based on Improved Lightweight Network." Applied Sciences 12, no. 13 (2022): 6414. http://dx.doi.org/10.3390/app12136414.
Full textAbstract:
Surgery is a compelling application field for collaborative control robots. This paper proposes a gesture recognition method applied to a medical assistant robot delivering instruments to collaborate with surgeons to complete surgeries. The key to assisting the surgeon in passing instruments in the operating room is the ability to recognize the surgeon’s hand gestures accurately and quickly. Existing gesture recognition techniques suffer from poor recognition accuracy and low rate. To address the existing shortcomings, we propose an improved lightweight convolutional neural network called E-MobileNetv2. The ECA module is added to the original MobileNetv2 network model to obtain more useful features by computing the information interactions between the current channel and the adjacent channels and between the current channel and the distant channels in the feature map. We add R6-SELU activation function to enhance the network’s ability to extract features. By adjusting the shrinkable hyper-parameters, the number of parameters of the network is reduced to improve the recognition speed. The improved network model achieves excellent performance on both the self-built dataset Gesture_II and the public dataset Jester. The recognition accuracy of the improved model is 96.82%, which is 3.17 % higher than that of the original model, achieving an increase in accuracy and recognition speed.