Log in

Relevant bibliographies by topics / Attention LSTM / Journal articles

To see the other types of publications on this topic, follow the link: Attention LSTM.

Journal articles on the topic 'Attention LSTM'

Author: Grafiati

Published: 8 July 2021

Last updated: 17 February 2022

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Attention LSTM.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

1

Li, Youru, Zhenfeng Zhu, Deqiang Kong, Hua Han, and Yao Zhao. "EA-LSTM: Evolutionary attention-based LSTM for time series prediction." Knowledge-Based Systems 181 (October 2019): 104785. http://dx.doi.org/10.1016/j.knosys.2019.05.028.

Full text

APA, Harvard, Vancouver, ISO, and other styles

2

Huang, Zhongzhan, Senwei Liang, Mingfu Liang, and Haizhao Yang. "DIANet: Dense-and-Implicit Attention Network." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 4206–14. http://dx.doi.org/10.1609/aaai.v34i04.5842.

Full text

Abstract:

Attention networks have successfully boosted the performance in various vision problems. Previous works lay emphasis on designing a new attention module and individually plug them into the networks. Our paper proposes a novel-and-simple framework that shares an attention module throughout different network layers to encourage the integration of layer-wise information and this parameter-sharing module is referred to as Dense-and-Implicit-Attention (DIA) unit. Many choices of modules can be used in the DIA unit. Since Long Short Term Memory (LSTM) has a capacity of capturing long-distance dependency, we focus on the case when the DIA unit is the modified LSTM (called DIA-LSTM). Experiments on benchmark datasets show that the DIA-LSTM unit is capable of emphasizing layer-wise feature interrelation and leads to significant improvement of image classification accuracy. We further empirically show that the DIA-LSTM has a strong regularization ability on stabilizing the training of deep networks by the experiments with the removal of skip connections (He et al. 2016a) or Batch Normalization (Ioffe and Szegedy 2015) in the whole residual network.

APA, Harvard, Vancouver, ISO, and other styles

3

Wang, Hao, Xiaofang Zhang, Bin Liang, Qian Zhou, and Baowen Xu. "Gated Hierarchical LSTMs for Target-Based Sentiment Analysis." International Journal of Software Engineering and Knowledge Engineering 28, no. 11n12 (November 2018): 1719–37. http://dx.doi.org/10.1142/s0218194018400259.

Full text

Abstract:

In the field of target-based sentiment analysis, the deep neural model combining attention mechanism is a remarkable success. In current research, it is commonly seen that attention mechanism is combined with Long Short-Term Memory (LSTM) networks. However, such neural network-based architectures generally rely on complex computation and only focus on single target. In this paper, we propose a gated hierarchical LSTM (GH-LSTMs) model which combines regional LSTM and sentence-level LSTM via a gated operation for the task of target-based sentiment analysis. This approach can distinguish different polarities of sentiment of different targets in the same sentence through a regional LSTM. Furthermore, it is able to concentrate on the long-distance dependency of target in the whole sentence via a sentence-level LSTM. The final results of our experiments on multi-domain datasets of two languages from SemEval 2016 indicate that our approach yields better performance than Support Vector Machine (SVM) and several typical neural network models. A case study of some typical examples also makes a supplement to this conclusion.

APA, Harvard, Vancouver, ISO, and other styles

4

Feng, Kaicheng, and Xiaobing Liu. "Adaptive Attention with Consumer Sentinel for Movie Box Office Prediction." Complexity 2020 (December 7, 2020): 1–9. http://dx.doi.org/10.1155/2020/6689304.

Full text

Abstract:

To improve the movie box office prediction accuracy, this paper proposes an adaptive attention with consumer sentinel (LSTM-AACS) for movie box office prediction. First, the influencing factors of the movie box office are analyzed. Tackling the problem of ignoring consumer groups in existing prediction models, we add consumer features and then quantitatively analyze and normalize the box office influence factors. Second, we establish an LSTM (Long Short-Term Memory) box office prediction model and inject the attention mechanism to construct an adaptive attention with consumer sentinel for movie box office prediction. Finally, 10,398 pieces of movie box office dataset are used in the Kaggle competition to compare the prediction results with the LSTM-AACS model, LSTM-Attention model, and LSTM model. The results show that the relative error of LSTM-AACS prediction is 6.58%, which is lower than other models used in the experiment.

APA, Harvard, Vancouver, ISO, and other styles

5

Jo, Youngki, and Hyunsoo Lee. "Electricity Demand Forecasting Framework using Modified Attention-based LSTM." Journal of Korean Institute of Intelligent Systems 30, no. 3 (June 30, 2020): 242–50. http://dx.doi.org/10.5391/jkiis.2020.30.3.242.

Full text

APA, Harvard, Vancouver, ISO, and other styles

6

Gallardo-Antolín, Ascensión, and Juan M. Montero. "Detecting Deception from Gaze and Speech Using a Multimodal Attention LSTM-Based Framework." Applied Sciences 11, no. 14 (July 11, 2021): 6393. http://dx.doi.org/10.3390/app11146393.

Full text

Abstract:

The automatic detection of deceptive behaviors has recently attracted the attention of the research community due to the variety of areas where it can play a crucial role, such as security or criminology. This work is focused on the development of an automatic deception detection system based on gaze and speech features. The first contribution of our research on this topic is the use of attention Long Short-Term Memory (LSTM) networks for single-modal systems with frame-level features as input. In the second contribution, we propose a multimodal system that combines the gaze and speech modalities into the LSTM architecture using two different combination strategies: Late Fusion and Attention-Pooling Fusion. The proposed models are evaluated over the Bag-of-Lies dataset, a multimodal database recorded in real conditions. On the one hand, results show that attentional LSTM networks are able to adequately model the gaze and speech feature sequences, outperforming a reference Support Vector Machine (SVM)-based system with compact features. On the other hand, both combination strategies produce better results than the single-modal systems and the multimodal reference system, suggesting that gaze and speech modalities carry complementary information for the task of deception detection that can be effectively exploited by using LSTMs.

APA, Harvard, Vancouver, ISO, and other styles

7

Zhang, Xuan, Xun Liang, Aakas Zhiyuli, Shusen Zhang, Rui Xu, and Bo Wu. "AT-LSTM: An Attention-based LSTM Model for Financial Time Series Prediction." IOP Conference Series: Materials Science and Engineering 569 (August 9, 2019): 052037. http://dx.doi.org/10.1088/1757-899x/569/5/052037.

Full text

APA, Harvard, Vancouver, ISO, and other styles

8

Yin, Helin, Dong Jin, Yeong Hyeon Gu, Chang Jin Park, Sang Keun Han, and Seong Joon Yoo. "STL-ATTLSTM: Vegetable Price Forecasting Using STL and Attention Mechanism-Based LSTM." Agriculture 10, no. 12 (December 8, 2020): 612. http://dx.doi.org/10.3390/agriculture10120612.

Full text

Abstract:

It is difficult to forecast vegetable prices because they are affected by numerous factors, such as weather and crop production, and the time-series data have strong non-linear and non-stationary characteristics. To address these issues, we propose the STL-ATTLSTM (STL-Attention-based LSTM) model, which integrates the seasonal trend decomposition using the Loess (STL) preprocessing method and attention mechanism based on long short-term memory (LSTM). The proposed STL-ATTLSTM forecasts monthly vegetable prices using various types of information, such as vegetable prices, weather information of the main production areas, and market trading volumes. The STL method decomposes time-series vegetable price data into trend, seasonality, and remainder components. It uses the remainder component by removing the trend and seasonality components. In the model training process, attention weights are assigned to all input variables; thus, the model’s prediction performance is improved by focusing on the variables that affect the prediction results. The proposed STL-ATTLSTM was applied to five crops, namely cabbage, radish, onion, hot pepper, and garlic, and its performance was compared to three benchmark models (i.e., LSTM, attention LSTM, and STL-LSTM). The performance results show that the LSTM model combined with the STL method (STL-LSTM) achieved a 12% higher prediction accuracy than the attention LSTM model that did not use the STL method and solved the prediction lag arising from high seasonality. The attention LSTM model improved the prediction accuracy by approximately 4% to 5% compared to the LSTM model. The STL-ATTLSTM model achieved the best performance, with an average root mean square error (RMSE) of 380, and an average mean absolute percentage error (MAPE) of 7%.

APA, Harvard, Vancouver, ISO, and other styles

9

Yang, Zhan, Chengliang Li, Zhongying Zhao, and Chao Li. "Sentiment classification based on dependency-relationship embedding and attention mechanism." Journal of Intelligent & Fuzzy Systems 41, no. 1 (August 11, 2021): 867–77. http://dx.doi.org/10.3233/jifs-202747.

Full text

Abstract:

Aspect-based sentiment classification, a fine-grained sentiment analysis task, aims to predict the sentiment polarity for a specified aspect. However, the existing aspect-based sentiment classification approaches cannot fully model the dependency-relationship between words and are easily disturbed by irrelevant aspects. To address this problem, we propose a novel approach named Dependency-Relationship Embedding and Attention Mechanism-based LSTM. DA-LSTM first merges the word hidden vector output by LSTM with the dependency-relationship embedding to form a combined vector. This vector is then fed into the attention mechanism together with the aspect information which can avoid interference to calculate the final word representation for sentiment classification. Our extensive experiments on benchmark data sets clearly show the effectiveness of DA-LSTM.

APA, Harvard, Vancouver, ISO, and other styles

10

Kim, Hong-In, and Rae-Hong Park. "Residual LSTM Attention Network for Object Tracking." IEEE Signal Processing Letters 25, no. 7 (July 2018): 1029–33. http://dx.doi.org/10.1109/lsp.2018.2835768.

Full text

APA, Harvard, Vancouver, ISO, and other styles

11

Xie, Yue, Ruiyu Liang, Zhenlin Liang, Chengwei Huang, Cairong Zou, and Bjorn Schuller. "Speech Emotion Classification Using Attention-Based LSTM." IEEE/ACM Transactions on Audio, Speech, and Language Processing 27, no. 11 (November 2019): 1675–85. http://dx.doi.org/10.1109/taslp.2019.2925934.

Full text

APA, Harvard, Vancouver, ISO, and other styles

12

Liu, Zhongyu, Tian Chen, Enjie Ding, Yafeng Liu, and Wanli Yu. "Attention-Based Convolutional LSTM for Describing Video." IEEE Access 8 (2020): 133713–24. http://dx.doi.org/10.1109/access.2020.3010872.

Full text

APA, Harvard, Vancouver, ISO, and other styles

13

Bin, Yi, Yang Yang, Fumin Shen, Ning Xie, Heng Tao Shen, and Xuelong Li. "Describing Video With Attention-Based Bidirectional LSTM." IEEE Transactions on Cybernetics 49, no. 7 (July 2019): 2631–41. http://dx.doi.org/10.1109/tcyb.2018.2831447.

Full text

APA, Harvard, Vancouver, ISO, and other styles

14

Li, Xiangpeng, Zhilong Zhou, Lijiang Chen, and Lianli Gao. "Residual attention-based LSTM for video captioning." World Wide Web 22, no. 2 (February 26, 2018): 621–36. http://dx.doi.org/10.1007/s11280-018-0531-z.

Full text

APA, Harvard, Vancouver, ISO, and other styles

15

Talafha, Bashar, Analle Abuammar, and Mahmoud Al-Ayyoub. "Atar: Attention-based LSTM for Arabizi transliteration." International Journal of Electrical and Computer Engineering (IJECE) 11, no. 3 (June 1, 2021): 2327. http://dx.doi.org/10.11591/ijece.v11i3.pp2327-2334.

Full text

Abstract:

A non-standard romanization of Arabic script, known as Arbizi, is widely used in Arabic online and SMS/chat communities. However, since state-of-the-art tools and applications for Arabic NLP expects Arabic to be written in Arabic script, handling contents written in Arabizi requires a special attention either by building customized tools or by transliterating them into Arabic script. The latter approach is the more common one and this work presents two significant contributions in this direction. The first one is to collect and publicly release the first large-scale “Arabizi to Arabic script” parallel corpus focusing on the Jordanian dialect and consisting of more than 25 k pairs carefully created and inspected by native speakers to ensure highest quality. Second, we present Atar, an attention-based encoder-decoder model for Arabizi transliteration. Training and testing this model on our dataset yields impressive accuracy (79%) and BLEU score (88.49).

APA, Harvard, Vancouver, ISO, and other styles

16

王, 彬. "Attention-Bi-LSTM Based Analysis of Weibo Comments." Computer Science and Application 10, no. 12 (2020): 2380–87. http://dx.doi.org/10.12677/csa.2020.1012252.

Full text

APA, Harvard, Vancouver, ISO, and other styles

17

Fang, Kuncheng, Lian Zhou, Cheng Jin, Yuejie Zhang, Kangnian Weng, Tao Zhang, and Weiguo Fan. "Fully Convolutional Video Captioning with Coarse-to-Fine and Inherited Attention." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 8271–78. http://dx.doi.org/10.1609/aaai.v33i01.33018271.

Full text

Abstract:

Automatically generating natural language description for video is an extremely complicated and challenging task. To tackle the obstacles of traditional LSTM-based model for video captioning, we propose a novel architecture to generate the optimal descriptions for videos, which focuses on constructing a new network structure that can generate sentences superior to the basic model with LSTM, and establishing special attention mechanisms that can provide more useful visual information for caption generation. This scheme discards the traditional LSTM, and exploits the fully convolutional network with coarse-to-fine and inherited attention designed according to the characteristics of fully convolutional structure. Our model cannot only outperform the basic LSTM-based model, but also achieve the comparable performance with those of state-of-the-art methods

APA, Harvard, Vancouver, ISO, and other styles

18

Li, Songzhou, Gang Xie, Jinchang Ren, Lei Guo, Yunyun Yang, and Xinying Xu. "Urban PM2.5 Concentration Prediction via Attention-Based CNN–LSTM." Applied Sciences 10, no. 6 (March 12, 2020): 1953. http://dx.doi.org/10.3390/app10061953.

Full text

Abstract:

Urban particulate matter forecasting is regarded as an essential issue for early warning and control management of air pollution, especially fine particulate matter (PM2.5). However, existing methods for PM2.5 concentration prediction neglect the effects of featured states at different times in the past on future PM2.5 concentration, and most fail to effectively simulate the temporal and spatial dependencies of PM2.5 concentration at the same time. With this consideration, we propose a deep learning-based method, AC-LSTM, which comprises a one-dimensional convolutional neural network (CNN), long short-term memory (LSTM) network, and attention-based network, for urban PM2.5 concentration prediction. Instead of only using air pollutant concentrations, we also add meteorological data and the PM2.5 concentrations of adjacent air quality monitoring stations as the input to our AC-LSTM. Hence, the spatiotemporal correlation and interdependence of multivariate air quality-related time-series data are learned by the CNN–LSTM network in AC-LSTM. The attention mechanism is applied to capture the importance degrees of the effects of featured states at different times in the past on future PM2.5 concentration. The attention-based layer can automatically weigh the past feature states to improve prediction accuracy. In addition, we predict the PM2.5 concentrations over the next 24 h by using air quality data in Taiyuan city, China, and compare it with six baseline methods. To compare the overall performance of each method, the mean absolute error (MAE), root-mean-square error (RMSE), and coefficient of determination (R2) are applied to the experiments in this paper. The experimental results indicate that our method is capable of dealing with PM2.5 concentration prediction with the highest performance.

APA, Harvard, Vancouver, ISO, and other styles

19

Lee, Yong-Hyeok, Dong-Won Jang, Jae-Bin Kim, Rae-Hong Park, and Hyung-Min Park. "Audio–Visual Speech Recognition Based on Dual Cross-Modality Attentions with the Transformer Model." Applied Sciences 10, no. 20 (October 17, 2020): 7263. http://dx.doi.org/10.3390/app10207263.

Full text

Abstract:

Since attention mechanism was introduced in neural machine translation, attention has been combined with the long short-term memory (LSTM) or replaced the LSTM in a transformer model to overcome the sequence-to-sequence (seq2seq) problems with the LSTM. In contrast to the neural machine translation, audio–visual speech recognition (AVSR) may provide improved performance by learning the correlation between audio and visual modalities. As a result that the audio has richer information than the video related to lips, AVSR is hard to train attentions with balanced modalities. In order to increase the role of visual modality to a level of audio modality by fully exploiting input information in learning attentions, we propose a dual cross-modality (DCM) attention scheme that utilizes both an audio context vector using video query and a video context vector using audio query. Furthermore, we introduce a connectionist-temporal-classification (CTC) loss in combination with our attention-based model to force monotonic alignments required in AVSR. Recognition experiments on LRS2-BBC and LRS3-TED datasets showed that the proposed model with the DCM attention scheme and the hybrid CTC/attention architecture achieved at least a relative improvement of 7.3% on average in the word error rate (WER) compared to competing methods based on the transformer model.

APA, Harvard, Vancouver, ISO, and other styles

20

XIE, Yue, Ruiyu LIANG, Zhenlin LIANG, and Li ZHAO. "Attention-Based Dense LSTM for Speech Emotion Recognition." IEICE Transactions on Information and Systems E102.D, no. 7 (July 1, 2019): 1426–29. http://dx.doi.org/10.1587/transinf.2019edl8019.

Full text

APA, Harvard, Vancouver, ISO, and other styles

21

NIZAMIDIN, Tashpolat, Li ZHAO, Ruiyu LIANG, Yue XIE, and Askar HAMDULLA. "Siamese Attention-Based LSTM for Speech Emotion Recognition." IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E103.A, no. 7 (July 1, 2020): 937–41. http://dx.doi.org/10.1587/transfun.2019eal2156.

Full text

APA, Harvard, Vancouver, ISO, and other styles

22

Shen, Yatian, Yan Li, Jun Sun, Wenke Ding, Xianjin Shi, Lei Zhang, Xiajiong Shen, and Jing He. "Hashtag Recommendation Using LSTM Networks with Self-Attention." Computers, Materials & Continua 61, no. 3 (2019): 1261–69. http://dx.doi.org/10.32604/cmc.2019.06104.

Full text

APA, Harvard, Vancouver, ISO, and other styles

23

Deng, Dong, Liping Jing, Jian Yu, and Shaolong Sun. "Sparse Self-Attention LSTM for Sentiment Lexicon Construction." IEEE/ACM Transactions on Audio, Speech, and Language Processing 27, no. 11 (November 2019): 1777–90. http://dx.doi.org/10.1109/taslp.2019.2933326.

Full text

APA, Harvard, Vancouver, ISO, and other styles

24

Wang, Ye, Xinxiang Zhang, Mi Lu, Han Wang, and Yoonsuck Choe. "Attention augmentation with multi-residual in bidirectional LSTM." Neurocomputing 385 (April 2020): 340–47. http://dx.doi.org/10.1016/j.neucom.2019.10.068.

Full text

APA, Harvard, Vancouver, ISO, and other styles

25

Yang, Liang, Haifeng Hu, Songlong Xing, and Xinlong Lu. "Constrained LSTM and Residual Attention for Image Captioning." ACM Transactions on Multimedia Computing, Communications, and Applications 16, no. 3 (September 4, 2020): 1–18. http://dx.doi.org/10.1145/3386725.

Full text

APA, Harvard, Vancouver, ISO, and other styles

26

Zhang, Tao, Xiao-Qing Zheng, and Ming-Xin Liu. "Multiscale attention-based LSTM for ship motion prediction." Ocean Engineering 230 (June 2021): 109066. http://dx.doi.org/10.1016/j.oceaneng.2021.109066.

Full text

APA, Harvard, Vancouver, ISO, and other styles

27

Yan, Le, Changwei Chen, Tingting Hang, and Youchuan Hu. "A stream prediction model based on attention-LSTM." Earth Science Informatics 14, no. 2 (February 16, 2021): 723–33. http://dx.doi.org/10.1007/s12145-021-00571-z.

Full text

APA, Harvard, Vancouver, ISO, and other styles

28

张, 玉铭. "Action Recognition Based on Attention and Bi-LSTM." Computer Science and Application 11, no. 06 (2021): 1607–16. http://dx.doi.org/10.12677/csa.2021.116166.

Full text

APA, Harvard, Vancouver, ISO, and other styles

29

Munir, Hafiz Shahbaz, Shengbing Ren, Mubashar Mustafa, Chaudry Naeem Siddique, and Shazib Qayyum. "Attention based GRU-LSTM for software defect prediction." PLOS ONE 16, no. 3 (March 4, 2021): e0247444. http://dx.doi.org/10.1371/journal.pone.0247444.

Full text

Abstract:

Software defect prediction (SDP) can be used to produce reliable, high-quality software. The current SDP is practiced on program granular components (such as file level, class level, or function level), which cannot accurately predict failures. To solve this problem, we propose a new framework called DP-AGL, which uses attention-based GRU-LSTM for statement-level defect prediction. By using clang to build an abstract syntax tree (AST), we define a set of 32 statement-level metrics. We label each statement, then make a three-dimensional vector and apply it as an automatic learning model, and then use a gated recurrent unit (GRU) with a long short-term memory (LSTM). In addition, the Attention mechanism is used to generate important features and improve accuracy. To verify our experiments, we selected 119,989 C/C++ programs in Code4Bench. The benchmark tests cover various programs and variant sets written by thousands of programmers. As an evaluation standard, compared with the state evaluation method, the recall, precision, accuracy and F1 measurement of our well-trained DP-AGL under normal conditions have increased by 1%, 4%, 5%, and 2% respectively.

APA, Harvard, Vancouver, ISO, and other styles

30

Mingkang Zhu, 朱铭康, and 卢先领 Xianling Lu. "Human Action Recognition Algorithm Based on Bi-LSTM-Attention Model." Laser & Optoelectronics Progress 56, no. 15 (2019): 151503. http://dx.doi.org/10.3788/lop56.151503.

Full text

APA, Harvard, Vancouver, ISO, and other styles

31

Kim, Mintae, Yeongtaek Oh, and Wooju Kim. "Sentence Similarity Prediction based on Siamese CNN-Bidirectional LSTM with Self-attention." Journal of KIISE 46, no. 3 (March 31, 2019): 241–45. http://dx.doi.org/10.5626/jok.2019.46.3.241.

Full text

APA, Harvard, Vancouver, ISO, and other styles

32

Cheng, Lin, Yuliang Shi, Kun Zhang, Xinjun Wang, and Zhiyong Chen. "GGATB-LSTM: Grouping and Global Attention-based Time-aware Bidirectional LSTM Medical Treatment Behavior Prediction." ACM Transactions on Knowledge Discovery from Data 15, no. 3 (May 2021): 1–16. http://dx.doi.org/10.1145/3441454.

Full text

Abstract:

In China, with the continuous development of national health insurance policies, more and more people have joined the health insurance. How to accurately predict patients future medical treatment behavior becomes a hotspot issue. The biggest challenge in this issue is how to improve the prediction performance by modeling health insurance data with high-dimensional time characteristics. At present, most of the research is to solve this issue by using Recurrent Neural Networks (RNNs) to construct an overall prediction model for the medical visit sequences. However, RNNs can not effectively solve the long-term dependence, and RNNs ignores the importance of time interval of the medical visit sequence. Additionally, the global model may lose some important content to different groups. In order to solve these problems, we propose a Grouping and Global Attention based Time-aware Bidirectional Long Short-Term Memory (GGATB-LSTM) model to achieve medical treatment behavior prediction. The model first constructs a heterogeneous information network based on health insurance data, and uses a tensor CANDECOMP/PARAFAC decomposition method to achieve similarity grouping. In terms of group prediction, a global attention and time factor are introduced to extend the bidirectional LSTM. Finally, the proposed model is evaluated by using real dataset, and conclude that GGATB-LSTM is better than other methods.

APA, Harvard, Vancouver, ISO, and other styles

33

Jang, Beakcheol, Myeonghwi Kim, Gaspard Harerimana, Sang-ug Kang, and Jong Wook Kim. "Bi-LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention Mechanism." Applied Sciences 10, no. 17 (August 24, 2020): 5841. http://dx.doi.org/10.3390/app10175841.

Full text

Abstract:

There is a need to extract meaningful information from big data, classify it into different categories, and predict end-user behavior or emotions. Large amounts of data are generated from various sources such as social media and websites. Text classification is a representative research topic in the field of natural-language processing that categorizes unstructured text data into meaningful categorical classes. The long short-term memory (LSTM) model and the convolutional neural network for sentence classification produce accurate results and have been recently used in various natural-language processing (NLP) tasks. Convolutional neural network (CNN) models use convolutional layers and maximum pooling or max-overtime pooling layers to extract higher-level features, while LSTM models can capture long-term dependencies between word sequences hence are better used for text classification. However, even with the hybrid approach that leverages the powers of these two deep-learning models, the number of features to remember for classification remains huge, hence hindering the training process. In this study, we propose an attention-based Bi-LSTM+CNN hybrid model that capitalize on the advantages of LSTM and CNN with an additional attention mechanism. We trained the model using the Internet Movie Database (IMDB) movie review data to evaluate the performance of the proposed model, and the test results showed that the proposed hybrid attention Bi-LSTM+CNN model produces more accurate classification results, as well as higher recall and F1 scores, than individual multi-layer perceptron (MLP), CNN or LSTM models as well as the hybrid models.

APA, Harvard, Vancouver, ISO, and other styles

34

Zhu, Xinxin, Lixiang Li, Jing Liu, Ziyi Li, Haipeng Peng, and Xinxin Niu. "Image captioning with triple-attention and stack parallel LSTM." Neurocomputing 319 (November 2018): 55–65. http://dx.doi.org/10.1016/j.neucom.2018.08.069.

Full text

APA, Harvard, Vancouver, ISO, and other styles

35

Wang, Bo, and Binwen Fan. "Attention-based Hierarchical LSTM Model for Document Sentiment Classification." IOP Conference Series: Materials Science and Engineering 435 (November 5, 2018): 012051. http://dx.doi.org/10.1088/1757-899x/435/1/012051.

Full text

APA, Harvard, Vancouver, ISO, and other styles

36

Fu, Xianghua, Jingying Yang, Jianqiang Li, Min Fang, and Huihui Wang. "Lexicon-Enhanced LSTM With Attention for General Sentiment Analysis." IEEE Access 6 (2018): 71884–91. http://dx.doi.org/10.1109/access.2018.2878425.

Full text

APA, Harvard, Vancouver, ISO, and other styles

37

Liang, Ruiyu, Fanliu Kong, Yue Xie, Guichen Tang, and Jiaming Cheng. "Real-Time Speech Enhancement Algorithm Based on Attention LSTM." IEEE Access 8 (2020): 48464–76. http://dx.doi.org/10.1109/access.2020.2979554.

Full text

APA, Harvard, Vancouver, ISO, and other styles

38

Ding, Yukai, Yuelong Zhu, Jun Feng, Pengcheng Zhang, and Zirun Cheng. "Interpretable spatio-temporal attention LSTM model for flood forecasting." Neurocomputing 403 (August 2020): 348–59. http://dx.doi.org/10.1016/j.neucom.2020.04.110.

Full text

APA, Harvard, Vancouver, ISO, and other styles

39

Kumar, Avinash, Vishnu Teja Narapareddy, Veerubhotla Aditya Srikanth, Aruna Malapati, and Lalita Bhanu Murthy Neti. "Sarcasm Detection Using Multi-Head Attention Based Bidirectional LSTM." IEEE Access 8 (2020): 6388–97. http://dx.doi.org/10.1109/access.2019.2963630.

Full text

APA, Harvard, Vancouver, ISO, and other styles

40

Lin, Zhifeng, Lianglun Cheng, and Guoheng Huang. "Electricity consumption prediction based on LSTM with attention mechanism." IEEJ Transactions on Electrical and Electronic Engineering 15, no. 4 (January 6, 2020): 556–62. http://dx.doi.org/10.1002/tee.23088.

Full text

APA, Harvard, Vancouver, ISO, and other styles

41

Zhu, Guangming, Liang Zhang, Lu Yang, Lin Mei, Syed Afaq Ali Shah, Mohammed Bennamoun, and Peiyi Shen. "Redundancy and Attention in Convolutional LSTM for Gesture Recognition." IEEE Transactions on Neural Networks and Learning Systems 31, no. 4 (April 2020): 1323–35. http://dx.doi.org/10.1109/tnnls.2019.2919764.

Full text

APA, Harvard, Vancouver, ISO, and other styles

42

Gao, Lianli, Zhao Guo, Hanwang Zhang, Xing Xu, and Heng Tao Shen. "Video Captioning With Attention-Based LSTM and Semantic Consistency." IEEE Transactions on Multimedia 19, no. 9 (September 2017): 2045–55. http://dx.doi.org/10.1109/tmm.2017.2729019.

Full text

APA, Harvard, Vancouver, ISO, and other styles

43

Jing, Ran. "A Self-attention Based LSTM Network for Text Classification." Journal of Physics: Conference Series 1207 (April 2019): 012008. http://dx.doi.org/10.1088/1742-6596/1207/1/012008.

Full text

APA, Harvard, Vancouver, ISO, and other styles

44

Zhong, Rui, Rui Wang, Yang Zou, Zhiqiang Hong, and Min Hu. "Graph Attention Networks Adjusted Bi-LSTM for Video Summarization." IEEE Signal Processing Letters 28 (2021): 663–67. http://dx.doi.org/10.1109/lsp.2021.3066349.

Full text

APA, Harvard, Vancouver, ISO, and other styles

45

Ullah, Mohib, Muhammad Mudassar Yamin, Ahmed Mohammed, Sultan Daud Khan, Habib Ullah, and Faouzi Alaya Cheikh. "ATTENTION-BASED LSTM NETWORK FOR ACTION RECOGNITION IN SPORTS." Electronic Imaging 2021, no. 6 (January 18, 2021): 302–1. http://dx.doi.org/10.2352/issn.2470-1173.2021.6.iriacv-302.

Full text

Abstract:

Understanding human action from the visual data is an important computer vision application for video surveillance, sports player performance analysis, and many IoT applications. The traditional approaches for action recognition used hand-crafted visual and temporal features for classifying specific actions. In this paper, we followed the standard deep learning framework for action recognition but introduced channel and spatial attention module sequentially in the network. In a nutshell, our network consists of four main components. First, the input frames are given to a pre-trained CNN for extracting the visual features and the visual features are passed through the attention module. The transformed features maps are given to the bi-directional LSTM network that exploits the temporal dependency among the frames for the underlying action in the scene. The output of bi-direction LSTM is given to a fully connected layer with a softmax classifier that assigns the probabilities to the actions of the subject in the scene. In addition to cross-entropy loss, the marginal loss function is used that penalizes the network for the inter action classes and complimenting the network for the intra action variations. The network is trained and validated on a tennis dataset and in total six tennis players' actions are focused. The network is evaluated on standard performance metrics (precision, recall) promising results are achieved.

APA, Harvard, Vancouver, ISO, and other styles

46

Yu, Yeonguk, and Yoon-Joong Kim. "Attention-LSTM-Attention Model for Speech Emotion Recognition and Analysis of IEMOCAP Database." Electronics 9, no. 5 (April 26, 2020): 713. http://dx.doi.org/10.3390/electronics9050713.

Full text

Abstract:

We propose a speech-emotion recognition (SER) model with an “attention-long Long Short-Term Memory (LSTM)-attention” component to combine IS09, a commonly used feature for SER, and mel spectrogram, and we analyze the reliability problem of the interactive emotional dyadic motion capture (IEMOCAP) database. The attention mechanism of the model focuses on emotion-related elements of the IS09 and mel spectrogram feature and the emotion-related duration from the time of the feature. Thus, the model extracts emotion information from a given speech signal. The proposed model for the baseline study achieved a weighted accuracy (WA) of 68% for the improvised dataset of IEMOCAP. However, the WA of the proposed model of the main study and modified models could not achieve more than 68% in the improvised dataset. This is because of the reliability limit of the IEMOCAP dataset. A more reliable dataset is required for a more accurate evaluation of the model’s performance. Therefore, in this study, we reconstructed a more reliable dataset based on the labeling results provided by IEMOCAP. The experimental results of the model for the more reliable dataset confirmed a WA of 73%.

APA, Harvard, Vancouver, ISO, and other styles

47

Li, Jiakang, Xiongwei Zhang, Meng Sun, Xia Zou, and Changyan Zheng. "Attention-Based LSTM Algorithm for Audio Replay Detection in Noisy Environments." Applied Sciences 9, no. 8 (April 13, 2019): 1539. http://dx.doi.org/10.3390/app9081539.

Full text

Abstract:

Even though audio replay detection has improved in recent years, its performance is known to severely deteriorate with the existence of strong background noises. Given the fact that different frames of an utterance have different impacts on the performance of spoofing detection, this paper introduces attention-based long short-term memory (LSTM) to extract representative frames for spoofing detection in noisy environments. With this attention mechanism, the specific and representative frame-level features will be automatically selected by adjusting their weights in the framework of attention-based LSTM. The experiments, conducted using the ASVspoof 2017 dataset version 2.0, show that the equal error rate (EER) of the proposed approach was about 13% lower than the constant Q cepstral coefficients-Gaussian mixture model (CQCC-GMM) baseline in noisy environments with four different signal-to-noise ratios (SNR). Meanwhile, the proposed algorithm also improved the performance of traditional LSTM on audio replay detection systems in noisy environments. Experiments using bagging with different frame lengths were also conducted to further improve the proposed approach.

APA, Harvard, Vancouver, ISO, and other styles

48

Cheng, Yepeng, Zuren Liu, and Yasuhiko Morimoto. "Attention-Based SeriesNet: An Attention-Based Hybrid Neural Network Model for Conditional Time Series Forecasting." Information 11, no. 6 (June 5, 2020): 305. http://dx.doi.org/10.3390/info11060305.

Full text

Abstract:

Traditional time series forecasting techniques can not extract good enough sequence data features, and their accuracies are limited. The deep learning structure SeriesNet is an advanced method, which adopts hybrid neural networks, including dilated causal convolutional neural network (DC-CNN) and Long-short term memory recurrent neural network (LSTM-RNN), to learn multi-range and multi-level features from multi-conditional time series with higher accuracy. However, they didn’t consider the attention mechanisms to learn temporal features. Besides, the conditioning method for CNN and RNN is not specific, and the number of parameters in each layer is tremendous. This paper proposes the conditioning method for two types of neural networks, and respectively uses the gated recurrent unit network (GRU) and the dilated depthwise separable temporal convolutional networks (DDSTCNs) instead of LSTM and DC-CNN for reducing the parameters. Furthermore, this paper presents the lightweight RNN-based hidden state attention module (HSAM) combined with the proposed CNN-based convolutional block attention module (CBAM) for time series forecasting. Experimental results show our model is superior to other models from the viewpoint of forecasting accuracy and computation efficiency.

APA, Harvard, Vancouver, ISO, and other styles

49

Wu, Yirui, Yukai Ding, Yuelong Zhu, Jun Feng, and Sifeng Wang. "Complexity to Forecast Flood: Problem Definition and Spatiotemporal Attention LSTM Solution." Complexity 2020 (March 26, 2020): 1–13. http://dx.doi.org/10.1155/2020/7670382.

Full text

Abstract:

With significant development of sensors and Internet of things, researchers nowadays can easily know what happens in physical space by acquiring time-varying values of various factors. Essentially, growing data category and size greatly contribute to solve problems happened in physical space. In this paper, we aim to solve a complex problem that affects both cities and villages, i.e., flood. To reduce impacts induced by floods, hydrological factors acquired from physical space and data-driven models in cyber space have been adopted to accurately forecast floods. Considering the significance of modeling attention capability among hydrology factors, we believe extraction of discriminative hydrology factors not only reflect natural rules in physical space, but also optimally model iterations of factors to forecast run-off values in cyber space. Therefore, we propose a novel data-driven model named as STA-LSTM by integrating Long Short-Term Memory (LSTM) structure and spatiotemporal attention module, which is capable of forecasting floods for small- and medium-sized rivers. The proposed spatiotemporal attention module firstly explores spatial relationship between input hydrological factors from different locations and run-off outputs, which assigns time-varying weights to various factors. Afterwards, the proposed attention module allocates temporal-dependent weights to hidden output of each LSTM cell, which describes significance of state output for final forecasting results. Taking Lech and Changhua river basins as cases of physical space, several groups of comparative experiments show that STA-LSTM is capable to optimize complexity of mathematically modeling floods in cyber space.

APA, Harvard, Vancouver, ISO, and other styles

50

Zou, Xiangyu, Jinjin Zhao, Duan Zhao, Bin Sun, Yongxin He, and Stelios Fuentes. "Air Quality Prediction Based on a Spatiotemporal Attention Mechanism." Mobile Information Systems 2021 (February 19, 2021): 1–12. http://dx.doi.org/10.1155/2021/6630944.

Full text

Abstract:

With the rapid development of the Internet of Things and Big Data, smart cities have received increasing attention. Predicting air quality accurately and efficiently is an important part of building a smart city. However, air quality prediction is very challenging because it is affected by many complex factors, such as dynamic spatial correlation between air quality detection sensors, dynamic temporal correlation, and external factors (such as road networks and points of interest). Therefore, this paper proposes a long short-term memory (LSTM) air quality prediction model based on a spatiotemporal attention mechanism (STA-LSTM). The model uses an encoder-decoder structure to model spatiotemporal features. A spatial attention mechanism is introduced in the encoder to capture the relative influence of surrounding sites on the prediction area. A temporal attention mechanism is introduced in the decoder to capture the time dependence of air quality. In addition, for spatial data such as point of interest (POI) and road networks, this paper uses the LINE graph embedding method to obtain a low-dimensional vector representation of spatial data to obtain abundant spatial features. This paper evaluates STA-LSTM on the Beijing dataset, and the root mean square error (RMSE) and R-squared ( R 2 ) indicators are used to compare with six benchmarks. The experimental results show that the model proposed in this paper can achieve better performance than the performances of other benchmarks.

APA, Harvard, Vancouver, ISO, and other styles

We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!