To see the other types of publications on this topic, follow the link: Seq2seq model.
Journal articles on the topic 'Seq2seq model'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Seq2seq model.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Aydın, Özlem, and Hüsein Kantarcı. "Türkçe Anahtar Sözcük Çıkarımında LSTM ve BERT Tabanlı Modellerin Karşılaştırılması." Bilgisayar Bilimleri ve Mühendisliği Dergisi 17, no. 1 (2024): 9–18. http://dx.doi.org/10.54525/bbmd.1454220.
Full textAbstract:
Günümüzde internet ortamında metne dayalı veri çok hızlı bir şekilde artış göstermektedir ve bu büyük veri içinden istenilen bilgiyi barındıran doğru içeriklere ulaşabilmek önemli bir ihtiyaçtır. İçeriklere ait anahtar sözcüklerin bilinmesi bu ihtiyacı karşılamada olumlu bir etki sağlayabilmektedir. Bu çalışmada, doğal dil işleme ve derin öğrenme modelleri ile Türkçe metinleri temsil eden anahtar sözcüklerin belirlenmesi amaçlanmıştır. Veri kümesi olarak Türkçe Etiketli Metin Derlemi ve Metin Özetleme-Anahtar Kelime Çıkarma Veri Kümesi birlikte kullanılmıştır. Derin öğrenme modeli olarak çalışmada iki farklı model ortaya konmuştur. İlk olarak Uzun Ömürlü Kısa Dönem Belleği ( LSTM) katmanlı bir Diziden Diziye (Seq2Seq) model tasarlanmıştır. Diğer model ise BERT (Transformatörler ile İki Yönlü Kodlayıcı Temsilleri) ile oluşturulmuş Seq2Seq bir modeldir. LSTM katmanlı Seq2seq modelin başarı değerlendirmesinde ROUGE-1 ölçütünde 0,38 F-1 değerine ulaşılmıştır. BERT tabanlı Seq2Seq modelde ROUGE-1 ölçütünde 0,399 F-1 değeri elde edilmiştir. Sonuç olarak dönüştürücü mimarisini temel alan BERT tabanlı Seq2Seq modelin, LSTM tabanlı Seq2seq modele görece daha başarılı olduğu gözlemlenmiştir.
2
Sak, Semih, and Mustafa Alper Akkaş. "6G'de Nesnelerin İnterneti Teknolojisinin Medikal Alandaki Gelişmeleri." Bilgisayar Bilimleri ve Mühendisliği Dergisi 17, no. 1 (2024): 1–8. http://dx.doi.org/10.54525/bbmd.1454186.
Full textAbstract:
Günümüzde internet ortamında metne dayalı veri çok hızlı bir şekilde artış göstermektedir ve bu büyük veri içinden istenilen bilgiyi barındıran doğru içeriklere ulaşabilmek önemli bir ihtiyaçtır. İçeriklere ait anahtar sözcüklerin bilinmesi bu ihtiyacı karşılamada olumlu bir etki sağlayabilmektedir. Bu çalışmada, doğal dil işleme ve derin öğrenme modelleri ile Türkçe metinleri temsil eden anahtar sözcüklerin belirlenmesi amaçlanmıştır. Veri kümesi olarak Türkçe Etiketli Metin Derlemi ve Metin Özetleme-Anahtar Kelime Çıkarma Veri Kümesi birlikte kullanılmıştır. Derin öğrenme modeli olarak çalışmada iki farklı model ortaya konmuştur. İlk olarak Uzun Ömürlü Kısa Dönem Belleği ( LSTM) katmanlı bir Diziden Diziye (Seq2Seq) model tasarlanmıştır. Diğer model ise BERT (Transformatörler ile İki Yönlü Kodlayıcı Temsilleri) ile oluşturulmuş Seq2Seq bir modeldir. LSTM katmanlı Seq2seq modelin başarı değerlendirmesinde ROUGE-1 ölçütünde 0,38 F-1 değerine ulaşılmıştır. BERT tabanlı Seq2Seq modelde ROUGE-1 ölçütünde 0,399 F-1 değeri elde edilmiştir. Sonuç olarak dönüştürücü mimarisini temel alan BERT tabanlı Seq2Seq modelin, LSTM tabanlı Seq2seq modele görece daha başarılı olduğu gözlemlenmiştir.
3
Palasundram, Kulothunkan, Nurfadhlina Mohd Sharef, Nurul Amelina Nasharuddin, Khairul Azhar Kasmiran, and Azreen Azman. "Sequence to Sequence Model Performance for Education Chatbot." International Journal of Emerging Technologies in Learning (iJET) 14, no. 24 (2019): 56. http://dx.doi.org/10.3991/ijet.v14i24.12187.
Full textAbstract:
Chatbot for education has great potential to complement human educators and education administrators. For example, it can be around the clock tutor to answer and clarify any questions from students who may have missed class. A chatbot can be implemented either by ruled based or artificial intel-ligence based. However, unlike the ruled-based chatbots, artificial intelli-gence based chatbots can learn and become smarter overtime and is more scalable and has become the popular choice for chatbot researchers recently. Recurrent Neural Network based Sequence-to-sequence (Seq2Seq) model is one of the most commonly researched model to implement artificial intelli-gence chatbot and has shown great progress since its introduction in 2014. However, it is still in infancy and has not been applied widely in educational chatbot development. Introduced originally for neural machine translation, the Seq2Seq model has been adapted for conversation modelling including question-answering chatbots. However, in-depth research and analysis of op-timal settings of the various components of Seq2Seq model for natural an-swer generation problem is very limited. Additionally, there has been no ex-periments and analysis conducted to understand how Seq2Seq model handles variations is questions posed to it to generate correct answers. Our experi-ments add to the empirical evaluations on Seq2Seq literature and provides insights to these questions. Additionally, we provide insights on how a cu-rated dataset can be developed and questions designed to train and test the performance of a Seq2Seq based question-answer model.
4
Palasundram, Kulothunkan, Nurfadhlina Mohd Sharef, Khairul Azhar Kasmiran, and Azreen Azman. "SEQ2SEQ++: A Multitasking-Based Seq2seq Model to Generate Meaningful and Relevant Answers." IEEE Access 9 (2021): 164949–75. http://dx.doi.org/10.1109/access.2021.3133495.
Full text
5
Jin, Weihua, Shijie Zhang, Bo Sun, Pengli Jin, and Zhidong Li. "An Analytical Investigation of Anomaly Detection Methods Based on Sequence to Sequence Model in Satellite Power Subsystem." Sensors 22, no. 5 (2022): 1819. http://dx.doi.org/10.3390/s22051819.
Full textAbstract:
The satellite power subsystem is responsible for all power supply in a satellite, and is an important component of it. The system’s performance has a direct impact on the operations of other systems as well as the satellite’s lifespan. Sequence to sequence (seq2seq) learning has recently advanced, gaining even more power in evaluating complicated and large-scale data. The potential of the seq2seq model in detecting anomalies in the satellite power subsystem is investigated in this work. A seq2seq-based scheme is given, with a thorough comparison of different neural-network cell types and levels of data smoothness. Three specific approaches were created to evaluate the seq2seq model performance, taking into account the unsupervised learning mechanism. The findings reveal that a CNN-based seq2seq with attention model under suitable data-smoothing conditions has a better ability to detect anomalies in the satellite power subsystem.
6
Zhou, Lijian, Lijun Wang, Zhiang Zhao, Yuwei Liu, and Xiwu Liu. "A Seq2Seq Model Improved by Transcendental Learning and Imaged Sequence Samples for Porosity Prediction." Mathematics 11, no. 1 (2022): 39. http://dx.doi.org/10.3390/math11010039.
Full textAbstract:
Since the accurate prediction of porosity is one of the critical factors for estimating oil and gas reservoirs, a novel porosity prediction method based on Imaged Sequence Samples (ISS) and a Sequence to Sequence (Seq2Seq) model fused by Transcendental Learning (TL) is proposed using well-logging data. Firstly, to investigate the correlation between logging features and porosity, the original logging features are normalized and selected by computing their correlation with porosity to obtain the point samples. Secondly, to better represent the depositional relations with depths, an ISS set is established by slidingly grouping sample points across depth, and the selected logging features are in a row. Therefore, spatial relations among the features are established along the vertical and horizontal directions. Thirdly, since the Seq2Seq model can better extract the spatio-temporal information of the input data than the Bidirectional Gate Recurrent Unit (BGRU), the Seq2Seq model is introduced for the first time to address the logging data and predict porosity. The experimental results show that it can achieve superior prediction results than state-of-the-art. However, the cumulative bias is likely to appear when using the Seq2Seq model. Motivated by teacher forcing, the idea of TL is proposed to be incorporated into the decoding process of Seq2Seq, named the TL-Seq2Seq model. The self-well and inter-well experimental results show that the proposed approach can significantly improve the accuracy of porosity prediction.
7
S., Keerthana, and Venkatesan R. "Abstractive Text Summarization using Seq2seq Model." International Journal of Computer Applications 176, no. 33 (2020): 24–26. http://dx.doi.org/10.5120/ijca2020920401.
Full text
8
Byambadorj, Zolzaya, Ryota Nishimura, Altangerel Ayush, and Norihide Kitaoka. "Normalization of Transliterated Mongolian Words Using Seq2Seq Model with Limited Data." ACM Transactions on Asian and Low-Resource Language Information Processing 20, no. 6 (2021): 1–19. http://dx.doi.org/10.1145/3464361.
Full textAbstract:
The huge increase in social media use in recent years has resulted in new forms of social interaction, changing our daily lives. Due to increasing contact between people from different cultures as a result of globalization, there has also been an increase in the use of the Latin alphabet, and as a result a large amount of transliterated text is being used on social media. In this study, we propose a variety of character level sequence-to-sequence (seq2seq) models for normalizing noisy, transliterated text written in Latin script into Mongolian Cyrillic script, for scenarios in which there is a limited amount of training data available. We applied performance enhancement methods, which included various beam search strategies, N-gram-based context adoption, edit distance-based correction and dictionary-based checking, in novel ways to two basic seq2seq models. We experimentally evaluated these two basic models as well as fourteen enhanced seq2seq models, and compared their noisy text normalization performance with that of a transliteration model and a conventional statistical machine translation (SMT) model. The proposed seq2seq models improved the robustness of the basic seq2seq models for normalizing out-of-vocabulary (OOV) words, and most of our models achieved higher normalization performance than the conventional method. When using test data during our text normalization experiment, our proposed method which included checking each hypothesis during the inference period achieved the lowest word error rate (WER = 13.41%), which was 4.51% fewer errors than when using the conventional SMT method.
9
Gong, Gangjun, Xiaonan An, Nawaraj Kumar Mahato, Shuyan Sun, Si Chen, and Yafeng Wen. "Research on Short-Term Load Prediction Based on Seq2seq Model." Energies 12, no. 16 (2019): 3199. http://dx.doi.org/10.3390/en12163199.
Full textAbstract:
Electricity load prediction is the primary basis on which power-related departments to make logical and effective generation plans and scientific scheduling plans for the most effective power utilization. The perpetual evolution of deep learning has recommended advanced and innovative concepts for short-term load prediction. Taking into consideration the time and nonlinear characteristics of power system load data and further considering the impact of historical and future information on the current state, this paper proposes a Seq2seq short-term load prediction model based on a long short-term memory network (LSTM). Firstly, the periodic fluctuation characteristics of users’ load data are analyzed, establishing a correlation of the load data so as to determine the model’s order in the time series. Secondly, the specifications of the Seq2seq model are given preference and a coalescence of the Residual mechanism (Residual) and the two Attention mechanisms (Attention) is developed. Then, comparing the predictive performance of the model under different types of Attention mechanism, this paper finally adopts the Seq2seq short-term load prediction model of Residual LSTM and the Bahdanau Attention mechanism. Eventually, the prediction model obtains better results when merging the actual power system load data of a certain place. In order to validate the developed model, the Seq2seq was compared with recurrent neural network (RNN), LSTM, and gated recurrent unit (GRU) algorithms. Last but not least, the performance indices were calculated. when training and testing the model with power system load data, it was noted that the root mean square error (RMSE) of Seq2seq was decreased by 6.61%, 16.95%, and 7.80% compared with RNN, LSTM, and GRU, respectively. In addition, a supplementary case study was carried out using data for a small power system considering different weather conditions and user behaviors in order to confirm the applicability and stability of the proposed model. The Seq2seq model for short-term load prediction can be reported to demonstrate superiority in all areas, exhibiting better prediction and stable performance.
10
Geng, Xiaoran, Yue Ma, Wennian Cai, et al. "Evaluation of models for multi-step forecasting of hand, foot and mouth disease using multi-input multi-output: A case study of Chengdu, China." PLOS Neglected Tropical Diseases 17, no. 9 (2023): e0011587. http://dx.doi.org/10.1371/journal.pntd.0011587.
Full textAbstract:
Background Hand, foot and mouth disease (HFMD) is a public health concern that threatens the health of children. Accurately forecasting of HFMD cases multiple days ahead and early detection of peaks in the number of cases followed by timely response are essential for HFMD prevention and control. However, many studies mainly predict future one-day incidence, which reduces the flexibility of prevention and control. Methods We collected the daily number of HFMD cases among children aged 0–14 years in Chengdu from 2011 to 2017, as well as meteorological and air pollutant data for the same period. The LSTM, Seq2Seq, Seq2Seq-Luong and Seq2Seq-Shih models were used to perform multi-step prediction of HFMD through multi-input multi-output. We evaluated the models in terms of overall prediction performance, the time delay and intensity of detection peaks. Results From 2011 to 2017, HFMD in Chengdu showed seasonal trends that were consistent with temperature, air pressure, rainfall, relative humidity, and PM10. The Seq2Seq-Shih model achieved the best performance, with RMSE, sMAPE and PCC values of 13.943~22.192, 17.880~27.937, and 0.887~0.705 for the 2-day to 15-day predictions, respectively. Meanwhile, the Seq2Seq-Shih model is able to detect peaks in the next 15 days with a smaller time delay. Conclusions The deep learning Seq2Seq-Shih model achieves the best performance in overall and peak prediction, and is applicable to HFMD multi-step prediction based on environmental factors.
11
Orduna-Cabrera, Fernando, Alejandro Rios-Ochoa, Federico Frank, et al. "Short-Term Forecasting Arabica Coffee Cherry Yields by Seq2Seq over LSTM for Smallholder Farmers." Sustainability 17, no. 9 (2025): 3888. https://doi.org/10.3390/su17093888.
Full textAbstract:
Coffee production is a vital source of income for smallholder farmers in Mexico’s Chiapas, Oaxaca, Puebla, and Veracruz regions. However, climate change, fluctuating yields, and the lack of decision-support tools pose challenges to the implementation of sustainable agricultural practices. The SABERES project aims to address these challenges through a Seq2Seq-LSTM model for predicting coffee yields in the short term, using datasets from Mexican national institutions, including the Agricultural Census (SIAP) and environmental data from the National Water Commission (CONAGUA). The model has demonstrated high accuracy in replicating historical yields for Chiapas and can forecast yields for the next two years. As a first step, we assessed coffee yield prediction for Bali, Indonesia, by comparing the LSTM, ARIMA, and Seq2Seq-LSTM models using historical data. The results show that the Seq2Seq-LSTM model provided the most accurate predictions, outperforming LSTM and ARIMA. Optimal performance was achieved using the maximum data sequence. Building on these findings, we aimed to apply the best configuration to forecast coffee yields in Chiapas, Mexico. The Seq2Seq-LSTM model achieved an average difference of only 0.000247, indicating near-perfect accuracy. It, therefore, demonstrated high accuracy in replicating historical yields for Chiapas, providing confidence for the next two years’ predictions. These results highlight the potential of Seq2Seq-LSTM to improve yield forecasts, support decision making, and enhance resilience in coffee production under climate change.
12
Zhang, Yong, and Weidong Xiao. "Keyphrase Generation Based on Deep Seq2seq Model." IEEE Access 6 (2018): 46047–57. http://dx.doi.org/10.1109/access.2018.2865589.
Full text
13
Nishtar, Zuhaib, and Jamil Afzal. "Seq2Seq-Based-Day-Ahead Scheduling for SCUC in Islanded Power Systems with Limited Intermittent Generation." Journal of Engineering, Science and Technological Trends 1, no. 1 (2024): 43–50. http://dx.doi.org/10.48112/jestt.v1i1.683.
Full textAbstract:
Due to their dependence on intermittent renewable energy sources, island power systems, which are generally located in remote places or on islands, offer particular issues for day-ahead scheduling. Using the capabilities of neural networks, we offer a Seq2Seq-based technique for day-ahead scheduling, which increases the precision and flexibility of unit commitment choices. The attention mechanisms in the Seq2Seq model are trained with historical data that includes projections for intermittent generation, demand, and unit commitment choices. The model is tested for its capacity to incorporate dynamic temporal relationships and deal with regenerative uncertainty. Seq2Seq models, a kind of deep learning approach, have shown impressive performance in several applications requiring sequence prediction. Uncertainty in renewable energy production, energy demand forecasts, and security limitations are all addressed in this work as Seq2Seq algorithms are applied to microgrid SCUC. In comparison to conventional scheduling approaches, the results show potential gains in prediction accuracy and operational efficiency. This study demonstrates how Seq2Seq models may be used to improve the longevity and dependability of isolated electrical grids and the way for the development of more effective, sustainable, and resilient energy infrastructure by contributing to the advancement of the area of microgrid optimization.
14
Chen, Xingguo, Yang Li, Xiaoyan Xu, and Min Shao. "A Novel Interpretable Deep Learning Model for Ozone Prediction." Applied Sciences 13, no. 21 (2023): 11799. http://dx.doi.org/10.3390/app132111799.
Full textAbstract:
Due to the limited understanding of the physical and chemical processes involved in ozone formation, as well as the large uncertainties surrounding its precursors, commonly used methods often result in biased predictions. Deep learning, as a powerful tool for fitting data, offers an alternative approach. However, most deep learning-based ozone-prediction models only take into account temporality and have limited capacity. Existing spatiotemporal deep learning models generally suffer from model complexity and inadequate spatiality learning. Thus, we propose a novel spatiotemporal model, namely the Spatiotemporal Attentive Gated Recurrent Unit (STAGRU). STAGRU uses a double attention mechanism, which includes temporal and spatial attention layers. It takes historical sequences from a target monitoring station and its neighboring stations as input to capture temporal and spatial information, respectively. This approach enables the achievement of more accurate results. The novel model was evaluated by comparing it to ozone observations in five major cities, Nanjing, Chengdu, Beijing, Guangzhou and Wuhan. All of these cities experience severe ozone pollution. The comparison involved Seq2Seq models, Seq2Seq+Attention models and our models. The experimental results show that our algorithm performs 14% better than Seq2Seq models and 4% better than Seq2Seq+Attention models. We also discuss the interpretability of our method, which reveals that temporality involves short-term dependency and long-term periodicity, while spatiality is mainly reflected in the transportation of ozone with the wind. This study emphasizes the significant impact of transportation on the implementation of ozone-pollution-control measures by the Chinese government.
15
Jeon, Wang-Su, and Sang-Yong Rhee. "Tool Wear Monitoring System Using Seq2Seq." Machines 12, no. 3 (2024): 169. http://dx.doi.org/10.3390/machines12030169.
Full textAbstract:
The advancement of smart factories has brought about small quantity batch production. In multi-variety production, both materials and processing methods change constantly, resulting in irregular changes in the progression of tool wear, which is often affected by processing methods. This leads to changes in the timing of tool replacement, and failure to correctly determine this timing may result in substantial damage and financial loss. In this study, we sought to address the issue of incorrect timing for tool replacement by using a Seq2Seq model to predict tool wear. We also trained LSTM and GRU models to compare performance by using R2, mean absolute error (MAE), and mean squared error (MSE). The Seq2Seq model outperformed LSTM and GRU with an R2 of approximately 0.03~0.037 in step drill data, 0.540.57 in top metal data, and 0.16~0.45 in low metal data. Confirming that Seq2Seq exhibited the best performance, we established a real-time monitoring system to verify the prediction results obtained using the Seq2Seq model. It is anticipated that this monitoring system will help prevent accidents in advance.
16
You, Lan, Siyu Xiao, Qingxi Peng, et al. "ST-Seq2Seq: A Spatio-Temporal Feature-Optimized Seq2Seq Model for Short-Term Vessel Trajectory Prediction." IEEE Access 8 (2020): 218565–74. http://dx.doi.org/10.1109/access.2020.3041762.
Full text
17
Guo, Yinuo, Tao Ge, and Furu Wei. "Fact-Aware Sentence Split and Rephrase with Permutation Invariant Training." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (2020): 7855–62. http://dx.doi.org/10.1609/aaai.v34i05.6291.
Full textAbstract:
Sentence Split and Rephrase aims to break down a complex sentence into several simple sentences with its meaning preserved. Previous studies tend to address the issue by seq2seq learning from parallel sentence pairs, which takes a complex sentence as input and sequentially generates a series of simple sentences. However, the conventional seq2seq learning has two limitations for this task: (1) it does not take into account the facts stated in the long sentence; As a result, the generated simple sentences may miss or inaccurately state the facts in the original sentence. (2) The order variance of the simple sentences to be generated may confuse the seq2seq model during training because the simple sentences derived from the long source sentence could be in any order.To overcome the challenges, we first propose the Fact-aware Sentence Encoding, which enables the model to learn facts from the long sentence and thus improves the precision of sentence split; then we introduce Permutation Invariant Training to alleviate the effects of order variance in seq2seq learning for this task. Experiments on the WebSplit-v1.0 benchmark dataset show that our approaches can largely improve the performance over the previous seq2seq learning approaches. Moreover, an extrinsic evaluation on oie-benchmark verifies the effectiveness of our approaches by an observation that splitting long sentences with our state-of-the-art model as preprocessing is helpful for improving OpenIE performance.
18
Li, Bo, Dingyao Yu, Wei Ye, Jinglei Zhang, and Shikun Zhang. "Sequence Generation with Label Augmentation for Relation Extraction." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 11 (2023): 13043–50. http://dx.doi.org/10.1609/aaai.v37i11.26532.
Full textAbstract:
Sequence generation demonstrates promising performance in recent information extraction efforts, by incorporating large-scale pre-trained Seq2Seq models. This paper investigates the merits of employing sequence generation in relation extraction, finding that with relation names or synonyms as generation targets, their textual semantics and the correlation (in terms of word sequence pattern) among them affect model performance. We then propose Relation Extraction with Label Augmentation (RELA), a Seq2Seq model with automatic label augmentation for RE. By saying label augmentation, we mean prod semantically synonyms for each relation name as the generation target. Besides, we present an in-depth analysis of the Seq2Seq model's behavior when dealing with RE. Experimental results show that RELA achieves competitive results compared with previous methods on four RE datasets.
19
Jin, Guozhe, and Zhezhou Yu. "A Hierarchical Sequence-to-Sequence Model for Korean POS Tagging." ACM Transactions on Asian and Low-Resource Language Information Processing 20, no. 2 (2021): 1–13. http://dx.doi.org/10.1145/3421762.
Full textAbstract:
Part-of-speech (POS) tagging is a fundamental task in natural language processing. Korean POS tagging consists of two subtasks: morphological analysis and POS tagging. In recent years, scholars have tended to use the seq2seq model to solve this problem. The full context of a sentence is considered in these seq2seq-based Korean POS tagging methods. However, Korean morphological analysis relies more on local contextual information, and in many cases, there exists one-to-one matching between morpheme surface form and base form. To make better use of these characteristics, we propose a hierarchical seq2seq model. In our model, the low-level Bi-LSTM encodes the syllable sequence, whereas the high-level Bi-LSTM models the context information of the whole sentence, and the decoder generates the morpheme base form syllables as well as the POS tags. To improve the accuracy of the morpheme base form recovery, we introduced the convolution layer and the attention mechanism to our model. The experimental results on the Sejong corpus show that our model outperforms strong baseline systems in both morpheme-level F1-score and eojeol-level accuracy, achieving state-of-the-art performance.
20
Zhang, Yong, Dan Li, Yuheng Wang, Yang Fang, and Weidong Xiao. "Abstract Text Summarization with a Convolutional Seq2seq Model." Applied Sciences 9, no. 8 (2019): 1665. http://dx.doi.org/10.3390/app9081665.
Full textAbstract:
Abstract text summarization aims to offer a highly condensed and valuable information that expresses the main ideas of the text. Most previous researches focus on extractive models. In this work, we put forward a new generative model based on convolutional seq2seq architecture. A hierarchical CNN framework is much more efficient than the conventional RNN seq2seq models. We also equip our model with a copying mechanism to deal with the rare or unseen words. Additionally, we incorporate a hierarchical attention mechanism to model the keywords and key sentences simultaneously. Finally we verify our model on two real-life datasets, GigaWord and DUC corpus. The experiment results verify the effectiveness of our model as it outperforms state-of-the-art alternatives consistently and statistical significantly.
21
Singh, Aditya, Tessy Mariam Thomas, Nitin Tandon, and Jinlong (Torres) Li. "1003 Dissecting Speech Planning and Articulation Circuits Using Seq2Seq Models." Neurosurgery 71, Supplement_1 (2025): 129. https://doi.org/10.1227/neu.0000000000003360_1003.
Full textAbstract:
INTRODUCTION: Understanding the cortical mechanisms underlying speech articulation is crucial for developing efficient, tractable speech brain-computer interface (BCI) devices. This study investigates pre-articulatory and articulatory kinematics during single-word production, utilizing sequence-to-sequence (Seq2Seq) models to elucidate the spatiotemporal signatures of articulatory trajectories. METHODS: Intracranial recordings were obtained from patients implanted with depth electrodes in the subcentral and pre-central gyri during single-word speech production. A Seq2Seq model was employed to reconstruct phonemic sequences from neural activity during and before articulation. For higher, clinically translatable results using solely the neural data, we can employ Bayes’ rule and solve for the probability of phoneme sequences given neural data by tying together a phoneme probability model and a reverse Seq2Seq model that predicts neural activity given a phoneme sequence. RESULTS: Seq2Seq models leveraging phonemic transition probabilities model achieved a 21.63% accuracy in word classification for 33 words using pre-articulatory data, compared to 6.5% with a linear classifier. Distinct pre-articulatory and articulatory channels were identified in precentral and subcentral gyri, and temporal dynamics showed separable peaks around 250 milliseconds (ms) prior to articulation and 100ms post-articulation for these cortical regions. We then employed Bayesian decoding methods to improve pre-articulatory word classification decoding performances for words outside the training set. CONCLUSIONS: These findings advance our understanding of speech planning and production, demonstrating that recurrent Seq2Seq models capture intricate neural dynamics and reveal distinct roles for different cortical areas. This architecture provides interpretable knowledge crucial for neurosurgeons in functional surgery and speech mapping. Moreover, it creates an infrastructure for clinically translatable, high-performance brain-computer interfaces with generalizable decoders, enhancing the robustness and applicability of neural decoding models for improved patient outcomes in speech restoration therapies.
22
Zhang, Gang. "A study of Grammar Analysis in English Teaching With Deep Learning Algorithm." International Journal of Emerging Technologies in Learning (iJET) 15, no. 18 (2020): 20. http://dx.doi.org/10.3991/ijet.v15i18.15425.
Full textAbstract:
In English teaching, grammar is a very important part. Based on the seq2seq model, a grammar analysis method combining the attention mechanism, word embedding and CNN seq2seq was designed using the deep learning algorithm, then the algorithm training was completed on NUCLE, and it was tested on CoNIL-2014. The experimental results showed that of seq2seq+attention improved 33.43% compared to the basic seq2seq; in the comparison between the method proposed in this study and CAMB, the P value of the former was 59.33% larger than that of CAMB, the R value was 8.9% larger, and the value of was 42.91% larger. Finally, in the analysis of the actual students' grammar homework, the proposed method also showed a good performance. The experimental results show that the method designed in this study is effective in grammar analysis and can be applied and popularized in actual English teaching.
23
Chung, Euisok, and Jeon Gue Park. "Sentence-Chain Based Seq2seq Model for Corpus Expansion." ETRI Journal 39, no. 4 (2017): 455–66. http://dx.doi.org/10.4218/etrij.17.0116.0074.
Full text
24
Sun, Ke, Tieyun Qian, Xu Chen, and Ming Zhong. "Context-aware seq2seq translation model for sequential recommendation." Information Sciences 581 (December 2021): 60–72. http://dx.doi.org/10.1016/j.ins.2021.09.001.
Full text
25
Sun, Haotai, Xiaodong Zhu, Yuanning Liu, and Wentao Liu. "WiFi Based Fingerprinting Positioning Based on Seq2seq Model." Sensors 20, no. 13 (2020): 3767. http://dx.doi.org/10.3390/s20133767.
Full textAbstract:
Indoor positioning technologies are of great use in GPS-denied areas. They can be partitioned into two types of systems—infrastructure-free based and infrastructure-dependent based. WiFi based indoor positioning system is somewhere between the infrastructure-free and infrastructure-dependent systems. The reason is that in WiFi based systems, Access Points (APs) as pre-installed infrastructures are necessary. However, the APs do not need to be specially installed, because WiFi APs are already widely deployed in many indoor areas, for example, offices, malls and airports. This feature makes WiFi based indoor positioning suitable for many practical applications. In this paper, a seq2seq model based, deep learning method is proposed for WiFi based fingerprinting. The model can learn from different length of training sequences, and thus can exploit the context information for positioning. The context information denotes the information contained in the sequence, which can help finding the correspondences between RSS fingerprints and the coordinate positions. A simple example piece of context information is human walking routine (such as no sharp turns). The proposed method shows an improvement with an open source dataset, when compared against deep learning based counterpart methods.
26
Li, Shuyu, and Yunsick Sung. "Transformer-Based Seq2Seq Model for Chord Progression Generation." Mathematics 11, no. 5 (2023): 1111. http://dx.doi.org/10.3390/math11051111.
Full textAbstract:
Machine learning is widely used in various practical applications with deep learning models demonstrating advantages in handling huge data. Treating music as a special language and using deep learning models to accomplish melody recognition, music generation, and music analysis has proven feasible. In certain music-related deep learning research, recurrent neural networks have been replaced with transformers. This has achieved significant results. In traditional approaches with recurrent neural networks, input sequences are limited in length. This paper proposes a method to generate chord progressions for melodies using a transformer-based sequence-to-sequence model, which is divided into a pre-trained encoder and decoder. A pre-trained encoder extracts contextual information from melodies, whereas a decoder uses this information to produce chords asynchronously and finally outputs chord progressions. The proposed method addresses length limitation issues while considering the harmony between chord progressions and melodies. Chord progressions can be generated for melodies in practical music composition applications. Evaluation experiments are conducted using the proposed method and three baseline models. The baseline models included the bidirectional long short-term memory (BLSTM), bidirectional encoder representation from transformers (BERT), and generative pre-trained transformer (GPT2). The proposed method outperformed the baseline models in Hits@k (k = 1) by 25.89, 1.54, and 2.13 %, respectively.
27
Colombo, Pierre, Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni, and Chloe Clavel. "Guiding Attention in Sequence-to-Sequence Models for Dialogue Act Prediction." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (2020): 7594–601. http://dx.doi.org/10.1609/aaai.v34i05.6259.
Full textAbstract:
The task of predicting dialog acts (DA) based on conversational dialog is a key component in the development of conversational agents. Accurately predicting DAs requires a precise modeling of both the conversation and the global tag dependencies. We leverage seq2seq approaches widely adopted in Neural Machine Translation (NMT) to improve the modelling of tag sequentiality. Seq2seq models are known to learn complex global dependencies while currently proposed approaches using linear conditional random fields (CRF) only model local tag dependencies. In this work, we introduce a seq2seq model tailored for DA classification using: a hierarchical encoder, a novel guided attention mechanism and beam search applied to both training and inference. Compared to the state of the art our model does not require handcrafted features and is trained end-to-end. Furthermore, the proposed approach achieves an unmatched accuracy score of 85% on SwDA, and state-of-the-art accuracy score of 91.6% on MRDA.
28
Yang, Qun, and Dejian Shen. "Learning Damage Representations with Sequence-to-Sequence Models." Sensors 22, no. 2 (2022): 452. http://dx.doi.org/10.3390/s22020452.
Full textAbstract:
Natural hazards have caused damages to structures and economic losses worldwide. Post-hazard responses require accurate and fast damage detection and assessment. In many studies, the development of data-driven damage detection within the research community of structural health monitoring has emerged due to the advances in deep learning models. Most data-driven models for damage detection focus on classifying different damage states and hence damage states cannot be effectively quantified. To address such a deficiency in data-driven damage detection, we propose a sequence-to-sequence (Seq2Seq) model to quantify a probability of damage. The model was trained to learn damage representations with only undamaged signals and then quantify the probability of damage by feeding damaged signals into models. We tested the validity of our proposed Seq2Seq model with a signal dataset which was collected from a two-story timber building subjected to shake table tests. Our results show that our Seq2Seq model has a strong capability of distinguishing damage representations and quantifying the probability of damage in terms of highlighting the regions of interest.
29
Loh, Zheung Yik, Wan Mohd Nasir Wan Kadir, and Noraini Ibrahim. "A Comparative Evaluation of Transformers in Seq2Seq Code Mutation: Non-Pre-trained Vs. Pre-trained Variants." Journal of Advanced Research Design 123, no. 1 (2024): 45–65. https://doi.org/10.37934/ard.123.1.4565.
Full textAbstract:
Mutation testing (MT) is a gold standard way to assess the efficacy of software test suites. However, the accuracy of mutation score is affected by the presence of trivial mutants which can be “killed” by even the simplest and most basic test suites. Since the existence of trivial mutants is due to the fixed set of mutation operators that constraints the complexity of code mutations, state-of-the-art recurrent neural network (RNN) model is used for sequence-to-sequence (seq2seq) code mutation without relying on mutation operators. However, the quality of the produced mutants is affected by the limitation of RNN in interpreting the relationships between far-apart tokens of the code to be mutated. Transformers that do not have this limitation, have superseded RNN in seq2seq machine translation domains such as natural language processing (NLP). However, to the best of our knowledge, there is still no research that investigates the performance of transformers in seq2seq code mutation. This paper presents a comparison study that involves different variants of the non-pre-trained transformers, the transformers pre-trained with source code, the transformers pre-trained with natural language, and the state-of-the-art RNN model in seq2seq code mutation. The results show that transformers pre-trained with source code, especially CodeT5, demonstrated the best performance, achieving an average character n-gram F-score (CHRF) of 82.89 and superior code mutation complexity. Since the performance of transformers in seq2seq code mutation has not been previously investigated, the primary contribution of this paper is the best performing transformer for seq2seq code mutation. It establishes the foundation for the future research that proposes an integrated solution which addresses both the high-cost problem and the inaccurate mutation score problem of MT simultaneously, unlike existing solutions which only tackle one of the MT problems and give rise to other MT problems.
30
Wang, Guoju, Rongjie Zhu, Xiang Gong, et al. "A New Hybrid Deep Sequence Model for Decomposing, Interpreting, and Predicting Sulfur Dioxide Decline in Coastal Cities of Northern China." Sustainability 17, no. 6 (2025): 2546. https://doi.org/10.3390/su17062546.
Full textAbstract:
The recent success of emission reduction policies in China has significantly lowered sulfur dioxide (SO2) levels. However, accurately forecasting these concentrations remains challenging due to their inherent non-stationary tendency. This study introduces an innovative hybrid deep learning model, RF-VMD-Seq2Seq, combining the Random Forest (RF) algorithm, Variational Mode Decomposition (VMD), and the Sequence-to-Sequence (Seq2Seq) framework to improve SO2 concentration forecasting in five coastal cities of northern China. Our results show that the predicted SO2 concentrations closely align with observed values, effectively capturing fluctuations, outliers, and extreme events—such as sharp declines the Novel Coronavirus Pneumonia (COVID-19) pandemic in 2020—along with the upper 5% of SO2 levels. The model achieved high coefficients of determination (>0.91) and Pearson’s correlation (>0.96), with low prediction errors (RMSE < 1.35 μg/m3, MAE < 0.94 μg/m3, MAPE < 15%). The low-frequency band decomposing from VMD showed a notable long-term decrease in SO2 concentrations from 2013 to 2020, with a sharp decline since 2018 during heating seasons, probably due to the ‘Coal-to-Natural Gas’ policy in northern China. The input sequence length of seven steps was recommended for the prediction model, based on high-frequency periodicities extracted through VMD, which significantly improved our model performance. This highlights the critical role of weekly-cycle variations in SO2 levels, driven by anthropogenic activities, in enhancing the accuracy of one-day-ahead SO2 predictions across northern China’s coastal regions. The results of the RF model further reveal that CO and NO2, sharing common anthropogenic sources with SO2, contribute over 50% to predicting SO2 concentrations, while meteorological factors—relative humidity (RH) and air temperature—contribute less than 20%. Additionally, the integration of VMD outperformed both the standard Seq2Seq and Ensemble Empirical Mode Decomposition (EEMD)-enhanced Seq2Seq models, showcasing the advantages of VMD in predicting SO2 decline. This research highlights the potential of the RF-VMD-Seq2Seq model for non-stationary SO2 prediction and its relevance for environmental protection and public health management.
31
Kim, Eun-Ah. "A Study on the Implementation of Educational Chatbot Using Seq2Seq Model." Korea Industrial Technology Convergence Society 28, no. 4 (2023): 135–43. http://dx.doi.org/10.29279/jitr.2023.28.4.135.
Full textAbstract:
With the development of mobile devices and the activation of messenger use, it is necessary to use chatbot as a communication tool with students. If individuals can ask various questions and receive customized answers owing to differences in levels among students, it will help improve academic achievement. For students to naturally communicate with chatbot, they must be able to provide familiar and accurate answers similar to those of Georgia Tech's Jill Watson. In this study, a chatbot capable of natural language processing and improving the Seq2Seq algorithm of the long short-term memory structure was implemented. The implemented chatbot learned from the open daily life conversation data, and consequently, it exhibited high accuracy and low loss rate within a short time, and the answer was excellent.
32
Jia, Xingbin, Xiang Gong, Xiaohuan Liu, et al. "Deep Sequence Learning for Prediction of Daily NO2 Concentration in Coastal Cities of Northern China." Atmosphere 14, no. 3 (2023): 467. http://dx.doi.org/10.3390/atmos14030467.
Full textAbstract:
Nitrogen dioxide (NO2) is an important precursor of atmospheric aerosol. Forecasting urban NO2 concentration is vital for effective control of air pollution. This paper proposes a hybrid deep learning model for predicting daily average NO2 concentrations on the next day, based on atmospheric pollutants, meteorological data, and historical data during 2014 to 2020 in five coastal cities of Shandong peninsula, northern China. A random Forest (RF) algorithm was used to select input variables to reduce data dimensionality trained by the sequence to sequence (Seq2Seq) the model and describe how the Seq2Seq model understands each predictor variable. The hybrid model combining an RF with Seq2Seq network (RF-S2S) was evaluated and achieved a Pearson’s correlation coefficient of 0.93, a Nash–Sutcliffe coefficient (NS) of 0.79, a Root Mean Square Error (RMSE) of 5.85 µg/m3, a Mean Absolute Error (MAE) of 4.50 µg/m3, and a Mean Absolute Percentage Error (MAPE) of 20.86%. Feature selection by an RF model improves the performance of the Seq2Seq model, reducing errors by 19.7% (RMSE), 20.3% (MAE), and 29.3% (MAPE), respectively. Carbon monoxide (CO) and PM10 are two common, important features influencing the prediction of NO2 concentrations in coastal areas of northern China. The results of RF-S2S models can capture general trends and disruptions more accurately than can long-short term memory (LSTM) models with and without feature selection. The decreasing tendency of NO2 from 2014 to 2020 illustrated by the empirical mode decomposition (EMD) method is one important obstacle to improving the RF-S2S prediction accuracy. An EMD-based RF-S2S model could help to perform the short-term forecast of NO2 concentrations efficiently.
33
Han, Xiaoming, Zhentao Dai, Mifeng Ren, Jing Cui, and Yunfeng Shi. "One-Time Prediction of Battery Capacity Fade Curve under Multiple Fast Charging Strategies." Batteries 10, no. 3 (2024): 74. http://dx.doi.org/10.3390/batteries10030074.
Full textAbstract:
Using different fast charging strategies for lithium-ion batteries can affect the degradation rate of the batteries. In this case, predicting the capacity fade curve can facilitate the application of new batteries. Considering the impact of fast charging strategies on battery aging, a battery capacity degradation trajectory prediction method based on the TM-Seq2Seq (Trend Matching—Sequence-to-Sequence) model is proposed. This method uses data from the first 100 cycles to predict the future capacity fade curve and EOL (end of life) in one-time. First, features are extracted from the discharge voltage-capacity curve. Secondly, a sequence-to-sequence model based on CNN, SE-net, and GRU is designed. Finally, a trend matching loss function is designed based on the common characteristics of capacity fade curves to constrain the encoding features of the sequence-to-sequence model, facilitating the learning of the underlying relationship between inputs and outputs. TM-Seq2Seq model is verified on a public dataset with 132 battery cells and multiple fast charging strategies. The experimental results indicate that, compared to other popular models, the TM-Seq2Seq model has lower prediction errors.
34
Dai, Zhihui, Ming Zhang, Nadia Nedjah, Dong Xu, and Feng Ye. "A Hydrological Data Prediction Model Based on LSTM with Attention Mechanism." Water 15, no. 4 (2023): 670. http://dx.doi.org/10.3390/w15040670.
Full textAbstract:
With the rapid development of IoT, big data and artificial intelligence, the research and application of data-driven hydrological models are increasing. However, when conducting time series analysis, many prediction models are often directly based on the following assumptions: hydrologic time series are normal, homogeneous, smooth and non-trending, which are not always all true. To address the related issues, a solution for short-term hydrological forecasting is proposed. Firstly, a feature test is conducted to verify whether the hydrological time series are normal, homogeneous, smooth and non-trending; secondly, a sequence-to-sequence (seq2seq)-based short-term water level prediction model (LSTM-seq2seq) is proposed to improve the accuracy of hydrological prediction. The model uses a long short-term memory neural network (LSTM) as an encoding layer to encode the historical flow sequence into a context vector, and another LSTM as a decoding layer to decode the context vector in order to predict the target runoff, by superimposing on the attention mechanism, aiming at improving the prediction accuracy. Using the experimental data regarding the water level of the Chu River, the model is compared to other models based on the analysis of normality, smoothness, homogeneity and trending of different water level data. The results show that the prediction accuracy of the proposed model is greater than that of the data set without these characteristics for the data set with normality, smoothness, homogeneity and trend. Flow data at Runcheng, Wuzhi, Baima Temple, Longmen Town, Dongwan, Lu’s and Tongguan are used as input data sets to train and evaluate the model. Metrics RMSE and NSE are used to evaluate the prediction accuracy and convergence speed of the model. The results show that the prediction accuracy of LSTM-seq2seq and LSTM-BP models is higher than other models. Furthermore, the convergence process of the LSTM-seq2seq model is the fastest among the compared models.
35
Bae, Yong Seok, Sungwon Lee, and Janghyuk Moon. "Developing an Innovative Seq2Seq Model to Predict the Remaining Useful Life of Low-Charged Battery Performance Using High-Speed Degradation Data." Batteries 10, no. 11 (2024): 389. http://dx.doi.org/10.3390/batteries10110389.
Full textAbstract:
This study introduces a novel Sequence-to-Sequence (Seq2Seq) deep learning model for predicting lithium-ion batteries’ remaining useful life. We address the challenge of extrapolating battery performance from high-rate to low-rate charging conditions, a significant limitation in previous studies. Experiments were also conducted on commercial cells using charge rates from 1C to 3C. Comparative analysis of fully connected neural networks, convolutional neural networks, and long short-term memory networks revealed their limitations in extrapolating to untrained conditions. Our Seq2Seq model overcomes these limitations, predicting charging profiles and discharge capacity for untrained, low-rate conditions using only high-rate charging data. The Seq2Seq model demonstrated superior performance with low error and high curve-fitting accuracy for 1C and 1.2C untrained data. Unlike traditional models, it predicts complete charging profiles (voltage, current, temperature) for subsequent cycles, offering a comprehensive view of battery degradation. This method significantly reduces battery life testing time while maintaining high prediction accuracy. The findings have important implications for lithium-ion battery development, potentially accelerating advancements in electric vehicle technology and energy storage.
36
Du, Muyuan, Zhimeng Zhang, and Chunning Ji. "Prediction for Coastal Wind Speed Based on Improved Variational Mode Decomposition and Recurrent Neural Network." Energies 18, no. 3 (2025): 542. https://doi.org/10.3390/en18030542.
Full textAbstract:
Accurate and comprehensive wind speed forecasting is crucial for improving efficiency in offshore wind power operation systems in coastal regions. However, raw wind speed data often suffer from noise and missing values, which can undermine the prediction performance. This study proposes a systematic framework, termed VMD-RUN-Seq2Seq-Attention, for noise reduction, outlier detection, and wind speed prediction by integrating Variational Mode Decomposition (VMD), the Runge–Kutta optimization algorithm (RUN), and a Sequence-to-Sequence model with an Attention mechanism (Seq2Seq-Attention). Using wind speed data from the Shidao, Xiaomaidao, and Lianyungang stations as case studies, a fitness function based on the Pearson correlation coefficient was developed to optimize the VMD mode count and penalty factor. A comparative analysis of different Intrinsic Mode Function (IMF) selection ratios revealed that selecting a 50% IMF ratio effectively retains the intrinsic information of the raw data while minimizing noise. For outlier detection, statistical methods were employed, followed by a comparative evaluation of three models—LSTM, LSTM-KAN, and Seq2Seq-Attention—for multi-step wind speed forecasting over horizons ranging from 1 to 12 h. The results consistently showed that the Seq2Seq-Attention model achieved superior predictive accuracy across all forecast horizons, with the correlation coefficient of its prediction results greater than 0.9 in all cases. The proposed VMD-RUN-Seq2Seq-Attention framework outperformed other methods in the denoising, data cleansing, and reconstruction of the original wind speed dataset, with a maximum improvement of 21% in accuracy, producing highly accurate and reliable results. This approach offers a robust methodology for improving data quality and enhancing wind speed forecasting accuracy in coastal environments.
37
Sulisetyo Puji Widodo and Adila Alfa Krisnadhi. "Enhancing Table-to-Text Generation with Numerical Reasoning Using Graph2Seq Models." International Journal of Innovation in Enterprise System 8, no. 2 (2024): 11–21. https://doi.org/10.25124/ijies.v8i02.236.
Full textAbstract:
Interpreting data in tables into narratives is necessary because tables cannot explain their own data.Additionally, there is a need to produce more analytic narratives from the results of numericalreasoning on data from tables. The sequence-to-sequence (Seq2Seq) encoder-decoder structure is themost widely used in table-to-text generation (T2XG). However, Seq2Seq requires the linearization oftables, which can omit structural information and create hallucination problems. Alternatively, thegraph-to-sequence (Graph2Seq) encoder-decoder structure utilizes a graph encoder to better captureimportant data information. Several studies have shown that Graph2Seq outperforms Seq2Seq. Thus,this study applies Graph2Seq to T2XG, leveraging the structured nature of tables that can berepresented by graphs. This research initiates the use of Graph2Seq in T2XG with GCN-RNN andGraphSage-RNN, aiming to improve narrative generation from tables through enhanced numericalreasoning. Based on the automatic evaluation of the application of Graph2Seq on the T2XG task, ithas the same performance as the baseline model. Meanwhile, in human evaluation, Graphsage-RNNis better able to reduce the possibility of hallucinations in text compared to the baseline model andGCN-RNN.
38
Yu, Tianxiang, Yang Xin, Hongliang Zhu, Qifeng Tang, and Yuling Chen. "Network Penetration Intrusion Prediction Based on Attention Seq2seq Model." Security and Communication Networks 2022 (May 4, 2022): 1–19. http://dx.doi.org/10.1155/2022/6012232.
Full textAbstract:
Intrusion detection is a critical component of network security. However, intrusion detection cannot play a very good role in the face of APT and 0 day. It needs to combine intrusion prevention, deception defense, and other technologies to ensure network security. Intrusion prediction is an important part of intrusion prevention and deception defense. Only by predicting the next possible attack can we prevent the corresponding intrusion or cheat adversary more efficiently. However, the current research on intrusion prediction has not received much attention. Most of the existing intrusion prediction research focuses on the prediction of security situation, specific security events, system calls, etc., having limitation in applicability and sequence dependency. In order to supplement this part of research, this paper reports the prediction of network penetration intrusion sequence for the first time. By introducing the ATT&CK framework, this paper builds a dictionary for the penetration intrusion types and builds three different seq2seq models. The experiment runs on the public and generated sequence data based on real APT events and adversary groups resulting that the model can predict future penetration intrusion sequence with an accuracy of up to 0.90.
39
Wang, Boran, and Yingxiang Sun. "SSM-Seq2Seq: A Novel Speaking Style Neural Conversation Model." Journal of Physics: Conference Series 1576 (June 2020): 012001. http://dx.doi.org/10.1088/1742-6596/1576/1/012001.
Full text
40
Meenu, Gupta1 Prince Kumar2. "Robust Neural Language Translation Model Formulation using Seq2seq approach." Fusion: Practice and Applications (FPA) 5, no. 2 (2021): 61–67. https://doi.org/10.5281/zenodo.5270391.
Full textAbstract:
<em>Fusion: Practice and Applications (FPA) Vol. 5, No.2, PP. 61-67, 2021</em>
41
Zhang, Hanqin. "Application of LSTM-Based Seq2Seq Models in Natural Language to SQL Conversion in Financial Domain." Science, Technology and Social Development Proceedings Series 2 (November 10, 2024): 110–16. http://dx.doi.org/10.70088/n3mbj650.
Full textAbstract:
As a crucial branch of artificial intelligence, Natural Language Processing (NLP) enables computers to understand, interpret, and generate human language, significantly enhancing the efficiency of information retrieval and search. Given the growing demand for data processing in the financial sector, this paper proposes and implements a Seq2Seq model based on the LSTM algorithm to convert natural language queries into SQL statements (NL2SQL) for application in finance. The model demonstrates stable and significant performance improvements over 10 training epochs, with accuracy increasing from 0.75 to 0.9877 and the loss value decreasing from 1.5 to 0.4978. These results validate the accuracy and effectiveness of the proposed LSTM-based Seq2Seq model in handling NLP tasks within the financial domain.
42
Xie, Yuhong, Yuzuru Ueda, and Masakazu Sugiyama. "A Two-Stage Short-Term Load Forecasting Method Using Long Short-Term Memory and Multilayer Perceptron." Energies 14, no. 18 (2021): 5873. http://dx.doi.org/10.3390/en14185873.
Full textAbstract:
Load forecasting is an essential task in the operation management of a power system. Electric power companies utilize short-term load forecasting (STLF) technology to make reasonable power generation plans. A forecasting model with low prediction errors helps reduce operating costs and risks for the operators. In recent years, machine learning has become one of the most popular technologies for load forecasting. In this paper, a two-stage STLF model based on long short-term memory (LSTM) and multilayer perceptron (MLP), which improves the forecasting accuracy over the entire time horizon, is proposed. In the first stage, a sequence-to-sequence (seq2seq) architecture, which can handle a multi-sequence of input to extract more features of historical data than that of single sequence, is used to make multistep predictions. In the second stage, the MLP is used for residual modification by perceiving other information that the LSTM cannot. To construct the model, we collected the electrical load, calendar, and meteorological records of Kanto region in Japan for four years. Unlike other LSTM-based hybrid architectures, the proposed model uses two independent neural networks instead of making the neural network deeper by concatenating a series of LSTM cells and convolutional neural networks (CNNs). Therefore, the proposed model is easy to be trained and more interpretable. The seq2seq module performs well in the first few hours of the predictions. The MLP inherits the advantage of the seq2seq module and improves the results by feeding artificially selected features both from historical data and information of the target day. Compared to the LSTM-AM model and single MLP model, the mean absolute percentage error (MAPE) of the proposed model decreases from 2.82% and 2.65% to 2%, respectively. The results demonstrate that the MLP helps improve the prediction accuracy of seq2seq module and the proposed model achieves better performance than other popular models. In addition, this paper also reveals the reason why the MLP achieves the improvement.
43
Wu, Wenxiong, Pengfei Chen, Linying Chen, and Junmin Mou. "Ship Trajectory Prediction: An Integrated Approach Using ConvLSTM-Based Sequence-to-Sequence Model." Journal of Marine Science and Engineering 11, no. 8 (2023): 1484. http://dx.doi.org/10.3390/jmse11081484.
Full textAbstract:
Maritime transportation is one of the major contributors to the development of the global economy. To ensure its safety and reduce the occurrence of a maritime accident, intelligent maritime monitoring and ship behavior identification have been drawing much attention from industry and academia, among which, the accurate prediction of ship trajectory is one of the key questions. This paper proposed a trajectory prediction model integrating the Convolutional LSTM (ConvLSTM) and Sequence to Sequence (Seq2Seq) models to facilitate simultaneous extraction of temporal and spatial features of ship trajectories, thereby enhancing the accuracy of prediction. Firstly, the trajectories are preprocessed using kinematic-based anomaly removal and Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) to improve the data quality for the training process of trajectory prediction. Secondly, the ConvLSTM-based Seq2seq model is designed to extract temporal and spatial features of the ship trajectory and improve the performance of long-time prediction. Finally, by using real AIS data, the proposed model is compared with the Seq2Seq and Bidirectional LSTM based on attention mechanism (Bi-Attention-LSTM) models to verify its effectiveness. The experimental results demonstrate that the proposed model achieves excellent performance in predicting turning trajectories, good predictive accuracy on straight line motions, and greater improvement in prediction accuracy compared to the other two benchmark models. Overall, the proposed model represents a promising contribution to improving ship trajectory prediction accuracy and may enhance the safety and quality of ship navigation in complex and volatile marine environments.
44
BUDAEV, E. S. "DEVELOPMENT OF A WEB APPLICATION FOR INTELLIGENT ANALYSIS OF CUSTOMER REVIEWS USING A MODIFIED SEQ2SEQ MODEL WITH AN ATTENTION MECHANISM." Computational nanotechnology 11, no. 1 (2024): 151–61. http://dx.doi.org/10.33693/2313-223x-2024-11-1-151-161.
Full textAbstract:
Machine learning, and neural networks in particular, are having a huge impact on business and marketing by providing convenient tools for analytics and customer feedback. The article proposes an intelligent analysis of customer feedback based on the use of a modified seq2seq deep learning model. Since the basic seq2seq model has a significant disadvantage - the inability to concentrate on the main parts of the input sequence, the results of machine learning may give an inadequate assessment of customer feedback. This disadvantage is eliminated by means of a model proposed in the work called the “attention mechanism”. The model formed the basis for the development of a web application that solves the problem of flexible interaction with customers by parsing new reviews, analyzing them and generating a response to a review using a neural network.
45
Nayan Banik, Chayti Saha, Chayti Saha, Ikbal Ahmed, Ikbal Ahmed, and Kulsum Akter Shapna. "Bangla text generation system by incorporating attention in sequence-to-sequence model." World Journal of Advanced Research and Reviews 14, no. 1 (2022): 080–94. http://dx.doi.org/10.30574/wjarr.2022.14.1.0292.
Full textAbstract:
In this AI-driven digital era, the pervasive nature of digital data is possible due to the widespread and cheap access to the Internet. Internet is continuously flourishing with data in many forms. Among them, textual data are a great source of information where people share their expressions in written format. Social media, blogs, online newspapers, government documents are some notable textual data sources. Information extraction from this enormous amount of data by manual inspection is time-consuming, cumbersome, and sometimes impossible. Natural Language Processing (NLP) is the computational domain for addressing those limitations by solving human language-related problems. Text summarization, Named entity recognition, Question answering are some of them where a common task for a machine is to generate coherent text. In such scenarios, the input is a sequence of text, and the output is also a sequence, but they differ in length. Sequence-to-Sequence (Seq2Seq) is an algorithmic approach to address that scenario by utilizing layers of recurrent units. However, the simple Seq2Seq model fails to capture the long-term relationship on the input sequence. Research shows that the attention mechanism guides the model to concentrate on specific inputs. Existing literature shows a lack of quality research on this text generation problem in the Bangla language, whereas many languages show excellent results. This work aims to develop such a system by incorporating attention to the Seq2Seq model and justifying its applicability by comparing it with baseline models. The model perplexity shows that the system can generate human-level readable text using a preprocessed dataset.
46
Nayan, Banik, Saha Chayti, Ahmed Ikbal, and Akter Shapna Kulsum. "Bangla text generation system by incorporating attention in sequence-to-sequence model." World Journal of Advanced Research and Reviews 14, no. 1 (2022): 080–94. https://doi.org/10.5281/zenodo.7009006.
Full textAbstract:
In this AI-driven digital era, the pervasive nature of digital data is possible due to the widespread and cheap access to the Internet. Internet is continuously flourishing with data in many forms. Among them, textual data are a great source of information where people share their expressions in written format. Social media, blogs, online newspapers, government documents are some notable textual data sources. Information extraction from this enormous amount of data by manual inspection is time-consuming, cumbersome, and sometimes impossible. Natural Language Processing (NLP) is the computational domain for addressing those limitations by solving human language-related problems. Text summarization, Named entity recognition, Question answering are some of them where a common task for a machine is to generate coherent text. In such scenarios, the input is a sequence of text, and the output is also a sequence, but they differ in length. Sequence-to-Sequence (Seq2Seq) is an algorithmic approach to address that scenario by utilizing layers of recurrent units. However, the simple Seq2Seq model fails to capture the long-term relationship on the input sequence. Research shows that the attention mechanism guides the model to concentrate on specific inputs. Existing literature shows a lack of quality research on this text generation problem in the Bangla language, whereas many languages show excellent results. This work aims to develop such a system by incorporating attention to the Seq2Seq model and justifying its applicability by comparing it with baseline models. The model perplexity shows that the system can generate human-level readable text using a preprocessed dataset.
47
Oh, Jiun, and Yong-Suk Choi. "Reusing Monolingual Pre-Trained Models by Cross-Connecting Seq2seq Models for Machine Translation." Applied Sciences 11, no. 18 (2021): 8737. http://dx.doi.org/10.3390/app11188737.
Full textAbstract:
This work uses sequence-to-sequence (seq2seq) models pre-trained on monolingual corpora for machine translation. We pre-train two seq2seq models with monolingual corpora for the source and target languages, then combine the encoder of the source language model and the decoder of the target language model, i.e., the cross-connection. We add an intermediate layer between the pre-trained encoder and the decoder to help the mapping of each other since the modules are pre-trained completely independently. These monolingual pre-trained models can work as a multilingual pre-trained model because one model can be cross-connected with another model pre-trained on any other language, while their capacity is not affected by the number of languages. We will demonstrate that our method improves the translation performance significantly over the random baseline. Moreover, we will analyze the appropriate choice of the intermediate layer, the importance of each part of a pre-trained model, and the performance change along with the size of the bitext.
48
Masood, Zaki, Rahma Gantassi, Ardiansyah, and Yonghoon Choi. "A Multi-Step Time-Series Clustering-Based Seq2Seq LSTM Learning for a Single Household Electricity Load Forecasting." Energies 15, no. 7 (2022): 2623. http://dx.doi.org/10.3390/en15072623.
Full textAbstract:
The deep learning (DL) approaches in smart grid (SG) describes the possibility of shifting the energy industry into a modern era of reliable and sustainable energy networks. This paper proposes a time-series clustering framework with multi-step time-series sequence to sequence (Seq2Seq) long short-term memory (LSTM) load forecasting strategy for households. Specifically, we investigate a clustering-based Seq2Seq LSTM electricity load forecasting model to undertake an energy load forecasting problem, where information input to the model contains individual appliances and aggregate energy as historical data of households. The original dataset is preprocessed, and forwarded to a multi-step time-series learning model which reduces the training time and guarantees convergence for energy forecasting. Furthermore, simulation results show the accuracy performance of the proposed model by validation and testing cluster data, which shows a promising potential of the proposed predictive model.
49
Han, Yerim, and Woohyun Kim. "Development and Validation of Building Control Algorithm Energy Management." Buildings 11, no. 3 (2021): 131. http://dx.doi.org/10.3390/buildings11030131.
Full textAbstract:
In this paper, a building control algorithm is proposed to reduce the electricity consumption of a building with a variable refrigerant flow (VRF) system. The algorithm uses sequence-to-sequence long short-term memory (seq2seq LSTM) to set target electricity consumption, and uses a VRF air conditioner system to reduce electricity consumption. After setting target electricity consumption, the algorithm is applied as a method of updating target electricity consumption. In addition, we propose two methods to increase the performance of the seq2seq LSTM model. First, among the feature selection methods, random forest is used to select, among the numerous features of the data, only those features that are most relevant to the predicted value. Second, we use Bayesian optimization, which selects the optimal hyperparameter that shows the best model performance. In order to control the air conditioners, the priority of air conditioners is designated, the method of prioritization being the analytical hierarchy process (AHP). In this study, comparison of the performance of seq2seq LSTM model with and without Bayesian optimization proved that the use of Bayesian optimization achieved good performance. Simulation and demonstration experiments using the algorithm were also conducted, and showed that building electricity consumption decreased in a similar manner to the reduction rate by means of the algorithm.
50
Wang, Lei, Jun Hu, Rundong Jiang, and Zengping Chen. "A Deep Long-Term Joint Temporal–Spectral Network for Spectrum Prediction." Sensors 24, no. 5 (2024): 1498. http://dx.doi.org/10.3390/s24051498.
Full textAbstract:
Spectrum prediction is a promising technique to release spectrum resources and plays an essential role in cognitive radio networks and spectrum situation generating. Traditional algorithms normally focus on one-dimensional or predict spectrum values in a slot-by-slot manner and thus cannot fully perceive the spectrum states in complex environments and lack timeliness. In this paper, a deep learning-based prediction method with a simple structure is developed for temporal–spectral and multi-slot spectrum prediction simultaneously. Specifically, we first analyze and construct spectrum data suitable for the model to simultaneously achieve long-term and multi-dimensional spectrum prediction. Then, a hierarchical spectrum prediction system is developed that takes advantage of the advanced Bi-ConvLSTM and the seq2seq framework. The Bi-ConvLSTM captures time–frequency characteristics of spectrum data, and the seq2seq framework is used for long-term spectrum prediction. Furthermore, the attention mechanism is used to address the limitations of the seq2seq framework that compresses all inputs into fixed-length vectors, resulting in information loss. Finally, the experimental results have shown that the proposed model has a significant advantage over the benchmark schemes. Especially, the proposed spectrum prediction model achieves 6.15%, 0.7749, 1.0978, and 0.9628 in MAPE, MAE, RMSE, and R2, respectively, which is better than all the baseline deep learning models.