Добірка наукової літератури з теми "Natural language processing biomedical nlp deep learning transfer learning"
Оформте джерело за APA, MLA, Chicago, Harvard та іншими стилями
Ознайомтеся зі списками актуальних статей, книг, дисертацій, тез та інших наукових джерел на тему "Natural language processing biomedical nlp deep learning transfer learning".
Біля кожної праці в переліку літератури доступна кнопка «Додати до бібліографії». Скористайтеся нею – і ми автоматично оформимо бібліографічне посилання на обрану працю в потрібному вам стилі цитування: APA, MLA, «Гарвард», «Чикаго», «Ванкувер» тощо.
Також ви можете завантажити повний текст наукової публікації у форматі «.pdf» та прочитати онлайн анотацію до роботи, якщо відповідні параметри наявні в метаданих.
Статті в журналах з теми "Natural language processing biomedical nlp deep learning transfer learning":
1
Vaghasia, Rishil. "An Improvised Approach of Deep Learning Neural Networks in NLP Applications." International Journal for Research in Applied Science and Engineering Technology 11, no. 1 (January 31, 2023): 1599–603. http://dx.doi.org/10.22214/ijraset.2023.48884.
Анотація:
Abstract: In recent years, natural language processing (NLP) has drawn a lot of interest for its ability to computationally represent and analyze human language. Its uses have expanded to include machine translation, email spam detection, information extraction, summarization, medical diagnosis, and question answering, among other areas. The purpose of this research is to investigate how deep learning and neural networks are used to analyze the syntax of natural language. This research first investigates a feed-forward neural network-based classifier for a transfer-based dependent syntax analyzer. This study presents a long-term memory neural network-based dependent syntactic analysis paradigm. This model, which will serve as a feature extractor, is based on the feed-forward neural network model mentioned before. After the feature extractor is learned, we train a recursive neural network classifier that is optimized by sentences using a long short-term memory neural network as a classifier of the transfer action and the characteristics retrieved by the syntactic analyzer as its input. Syntactic analysis replaces the method of modeling independent analysis with one that models the analysis of the entire sentence as a whole. The experimental findings demonstrate that the model has improved its performance more than the benchmark techniques.
2
Garrido-Muñoz , Ismael, Arturo Montejo-Ráez , Fernando Martínez-Santiago , and L. Alfonso Ureña-López . "A Survey on Bias in Deep NLP." Applied Sciences 11, no. 7 (April 2, 2021): 3184. http://dx.doi.org/10.3390/app11073184.
Анотація:
Deep neural networks are hegemonic approaches to many machine learning areas, including natural language processing (NLP). Thanks to the availability of large corpora collections and the capability of deep architectures to shape internal language mechanisms in self-supervised learning processes (also known as “pre-training”), versatile and performing models are released continuously for every new network design. These networks, somehow, learn a probability distribution of words and relations across the training collection used, inheriting the potential flaws, inconsistencies and biases contained in such a collection. As pre-trained models have been found to be very useful approaches to transfer learning, dealing with bias has become a relevant issue in this new scenario. We introduce bias in a formal way and explore how it has been treated in several networks, in terms of detection and correction. In addition, available resources are identified and a strategy to deal with bias in deep NLP is proposed.
3
Guarasci, Raffaele, Giuseppe De Pietro, and Massimo Esposito. "Quantum Natural Language Processing: Challenges and Opportunities." Applied Sciences 12, no. 11 (June 2, 2022): 5651. http://dx.doi.org/10.3390/app12115651.
Анотація:
The meeting between Natural Language Processing (NLP) and Quantum Computing has been very successful in recent years, leading to the development of several approaches of the so-called Quantum Natural Language Processing (QNLP). This is a hybrid field in which the potential of quantum mechanics is exploited and applied to critical aspects of language processing, involving different NLP tasks. Approaches developed so far span from those that demonstrate the quantum advantage only at the theoretical level to the ones implementing algorithms on quantum hardware. This paper aims to list the approaches developed so far, categorizing them by type, i.e., theoretical work and those implemented on classical or quantum hardware; by task, i.e., general purpose such as syntax-semantic representation or specific NLP tasks, like sentiment analysis or question answering; and by the resource used in the evaluation phase, i.e., whether a benchmark dataset or a custom one has been used. The advantages offered by QNLP are discussed, both in terms of performance and methodology, and some considerations about the possible usage QNLP approaches in the place of state-of-the-art deep learning-based ones are given.
4
Ok, Changwon, Geonseok Lee, and Kichun Lee. "Informative Language Encoding by Variational Autoencoders Using Transformer." Applied Sciences 12, no. 16 (August 9, 2022): 7968. http://dx.doi.org/10.3390/app12167968.
Анотація:
In natural language processing (NLP), Transformer is widely used and has reached the state-of-the-art level in numerous NLP tasks such as language modeling, summarization, and classification. Moreover, a variational autoencoder (VAE) is an efficient generative model in representation learning, combining deep learning with statistical inference in encoded representations. However, the use of VAE in natural language processing often brings forth practical difficulties such as a posterior collapse, also known as Kullback–Leibler (KL) vanishing. To mitigate this problem, while taking advantage of the parallelization of language data processing, we propose a new language representation model as the integration of two seemingly different deep learning models, which is a Transformer model solely coupled with a variational autoencoder. We compare the proposed model with previous works, such as a VAE connected with a recurrent neural network (RNN). Our experiments with four real-life datasets show that implementation with KL annealing mitigates posterior collapses. The results also show that the proposed Transformer model outperforms RNN-based models in reconstruction and representation learning, and that the encoded representations of the proposed model are more informative than other tested models.
5
Gupta, Manish, and Puneet Agrawal. "Compression of Deep Learning Models for Text: A Survey." ACM Transactions on Knowledge Discovery from Data 16, no. 4 (August 31, 2022): 1–55. http://dx.doi.org/10.1145/3487045.
Анотація:
In recent years, the fields of natural language processing (NLP) and information retrieval (IR) have made tremendous progress thanks to deep learning models like Recurrent Neural Networks (RNNs), Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTMs) networks, and Transformer [ 121 ] based models like Bidirectional Encoder Representations from Transformers (BERT) [ 24 ], Generative Pre-training Transformer (GPT-2) [ 95 ], Multi-task Deep Neural Network (MT-DNN) [ 74 ], Extra-Long Network (XLNet) [ 135 ], Text-to-text transfer transformer (T5) [ 96 ], T-NLG [ 99 ], and GShard [ 64 ]. But these models are humongous in size. On the other hand, real-world applications demand small model size, low response times, and low computational power wattage. In this survey, we discuss six different types of methods (Pruning, Quantization, Knowledge Distillation (KD), Parameter Sharing, Tensor Decomposition, and Sub-quadratic Transformer-based methods) for compression of such models to enable their deployment in real industry NLP projects. Given the critical need of building applications with efficient and small models, and the large amount of recently published work in this area, we believe that this survey organizes the plethora of work done by the “deep learning for NLP” community in the past few years and presents it as a coherent story.
6
Schomacker, Thorben, and Marina Tropmann-Frick. "Language Representation Models: An Overview." Entropy 23, no. 11 (October 28, 2021): 1422. http://dx.doi.org/10.3390/e23111422.
Анотація:
In the last few decades, text mining has been used to extract knowledge from free texts. Applying neural networks and deep learning to natural language processing (NLP) tasks has led to many accomplishments for real-world language problems over the years. The developments of the last five years have resulted in techniques that have allowed for the practical application of transfer learning in NLP. The advances in the field have been substantial, and the milestone of outperforming human baseline performance based on the general language understanding evaluation has been achieved. This paper implements a targeted literature review to outline, describe, explain, and put into context the crucial techniques that helped achieve this milestone. The research presented here is a targeted review of neural language models that present vital steps towards a general language representation model.
7
Laparra, Egoitz, Aurelie Mascio, Sumithra Velupillai, and Timothy Miller. "A Review of Recent Work in Transfer Learning and Domain Adaptation for Natural Language Processing of Electronic Health Records." Yearbook of Medical Informatics 30, no. 01 (August 2021): 239–44. http://dx.doi.org/10.1055/s-0041-1726522.
Анотація:
Summary Objectives: We survey recent work in biomedical NLP on building more adaptable or generalizable models, with a focus on work dealing with electronic health record (EHR) texts, to better understand recent trends in this area and identify opportunities for future research. Methods: We searched PubMed, the Institute of Electrical and Electronics Engineers (IEEE), the Association for Computational Linguistics (ACL) anthology, the Association for the Advancement of Artificial Intelligence (AAAI) proceedings, and Google Scholar for the years 2018-2020. We reviewed abstracts to identify the most relevant and impactful work, and manually extracted data points from each of these papers to characterize the types of methods and tasks that were studied, in which clinical domains, and current state-of-the-art results. Results: The ubiquity of pre-trained transformers in clinical NLP research has contributed to an increase in domain adaptation and generalization-focused work that uses these models as the key component. Most recently, work has started to train biomedical transformers and to extend the fine-tuning process with additional domain adaptation techniques. We also highlight recent research in cross-lingual adaptation, as a special case of adaptation. Conclusions: While pre-trained transformer models have led to some large performance improvements, general domain pre-training does not always transfer adequately to the clinical domain due to its highly specialized language. There is also much work to be done in showing that the gains obtained by pre-trained transformers are beneficial in real world use cases. The amount of work in domain adaptation and transfer learning is limited by dataset availability and creating datasets for new domains is challenging. The growing body of research in languages other than English is encouraging, and more collaboration between researchers across the language divide would likely accelerate progress in non-English clinical NLP.
8
Sarhan, Injy, and Marco Spruit. "Can We Survive without Labelled Data in NLP? Transfer Learning for Open Information Extraction." Applied Sciences 10, no. 17 (August 20, 2020): 5758. http://dx.doi.org/10.3390/app10175758.
Анотація:
Various tasks in natural language processing (NLP) suffer from lack of labelled training data, which deep neural networks are hungry for. In this paper, we relied upon features learned to generate relation triples from the open information extraction (OIE) task. First, we studied how transferable these features are from one OIE domain to another, such as from a news domain to a bio-medical domain. Second, we analyzed their transferability to a semantically related NLP task, namely, relation extraction (RE). We thereby contribute to answering the question: can OIE help us achieve adequate NLP performance without labelled data? Our results showed comparable performance when using inductive transfer learning in both experiments by relying on a very small amount of the target data, wherein promising results were achieved. When transferring to the OIE bio-medical domain, we achieved an F-measure of 78.0%, only 1% lower when compared to traditional learning. Additionally, transferring to RE using an inductive approach scored an F-measure of 67.2%, which was 3.8% lower than training and testing on the same task. Hereby, our analysis shows that OIE can act as a reliable source task.
9
Peña-Torres, Jefferson A., Raúl E. Gutiérrez, Víctor A. Bucheli, and Fabio A. González. "How to Adapt Deep Learning Models to a New Domain: The Case of Biomedical Relation Extraction." TecnoLógicas 22 (December 5, 2019): 49–62. http://dx.doi.org/10.22430/22565337.1483.
Анотація:
In this article, we study the relation extraction problem from Natural Language Processing (NLP) implementing a domain adaptation setting without external resources. We trained a Deep Learning (DL) model for Relation Extraction (RE), which extracts semantic relations in the biomedical domain. However, can the model be applied to different domains? The model should be adaptable to automatically extract relationships across different domains using the DL network. Completely training DL models in a short time is impractical because the models should quickly adapt to different datasets in several domains without delay. Therefore, adaptation is crucial for intelligent systems, where changing factors and unanticipated perturbations are common. In this study, we present a detailed analysis of the problem, as well as preliminary experimentation, results, and their evaluation.
10
Son, Suhyune, Seonjeong Hwang, Sohyeun Bae, Soo Jun Park, and Jang-Hwan Choi. "A Sequential and Intensive Weighted Language Modeling Scheme for Multi-Task Learning-Based Natural Language Understanding." Applied Sciences 11, no. 7 (March 31, 2021): 3095. http://dx.doi.org/10.3390/app11073095.
Анотація:
Multi-task learning (MTL) approaches are actively used for various natural language processing (NLP) tasks. The Multi-Task Deep Neural Network (MT-DNN) has contributed significantly to improving the performance of natural language understanding (NLU) tasks. However, one drawback is that confusion about the language representation of various tasks arises during the training of the MT-DNN model. Inspired by the internal-transfer weighting of MTL in medical imaging, we introduce a Sequential and Intensive Weighted Language Modeling (SIWLM) scheme. The SIWLM consists of two stages: (1) Sequential weighted learning (SWL), which trains a model to learn entire tasks sequentially and concentrically, and (2) Intensive weighted learning (IWL), which enables the model to focus on the central task. We apply this scheme to the MT-DNN model and call this model the MTDNN-SIWLM. Our model achieves higher performance than the existing reference algorithms on six out of the eight GLUE benchmark tasks. Moreover, our model outperforms MT-DNN by 0.77 on average on the overall task. Finally, we conducted a thorough empirical investigation to determine the optimal weight for each GLUE task.
Дисертації з теми "Natural language processing biomedical nlp deep learning transfer learning":
1
Ramponi, Alan. "Knowledge Extraction from Biomedical Literature with Symbolic and Deep Transfer Learning Methods." Doctoral thesis, Università degli studi di Trento, 2021. http://hdl.handle.net/11572/310787.
Анотація:
The available body of biomedical literature is increasing at a high pace, exceeding the ability of researchers to promptly leverage this knowledge-rich amount of information. Although the outstanding progress in natural language processing (NLP) we observed in the past few years, current technological advances in the field mainly concern newswire and web texts, and do not directly translate in good performance on highly specialized domains such as biomedicine due to linguistic variations along surface, syntax and semantic levels. Given the advances in NLP and the challenges the biomedical domain exhibits, and the explosive growth of biomedical knowledge being currently published, in this thesis we contribute to the biomedical NLP field by providing efficient means for extracting semantic relational information from biomedical literature texts. To this end, we made the following contributions towards the real-world adoption of knowledge extraction methods to support biomedicine: (i) we propose a symbolic high-precision biomedical relation extraction approach to reduce the time-consuming manual curation efforts of extracted relational evidence (Chapter 3), (ii) we conduct a thorough cross-domain study to quantify the drop in performance of deep learning methods for biomedical edge detection shedding lights on the importance of linguistic varieties in biomedicine (Chapter 4), and (iii) we propose a fast and accurate end-to-end solution for biomedical event extraction, leveraging sequential transfer learning and multi-task learning, making it a viable approach for real-world large-scale scenarios (Chapter 5). We then outline the conclusions by highlighting challenges and providing future research directions in the field.
2
Ramponi, Alan. "Knowledge Extraction from Biomedical Literature with Symbolic and Deep Transfer Learning Methods." Doctoral thesis, Università degli studi di Trento, 2021. http://hdl.handle.net/11572/310787.
Анотація:
The available body of biomedical literature is increasing at a high pace, exceeding the ability of researchers to promptly leverage this knowledge-rich amount of information. Although the outstanding progress in natural language processing (NLP) we observed in the past few years, current technological advances in the field mainly concern newswire and web texts, and do not directly translate in good performance on highly specialized domains such as biomedicine due to linguistic variations along surface, syntax and semantic levels. Given the advances in NLP and the challenges the biomedical domain exhibits, and the explosive growth of biomedical knowledge being currently published, in this thesis we contribute to the biomedical NLP field by providing efficient means for extracting semantic relational information from biomedical literature texts. To this end, we made the following contributions towards the real-world adoption of knowledge extraction methods to support biomedicine: (i) we propose a symbolic high-precision biomedical relation extraction approach to reduce the time-consuming manual curation efforts of extracted relational evidence (Chapter 3), (ii) we conduct a thorough cross-domain study to quantify the drop in performance of deep learning methods for biomedical edge detection shedding lights on the importance of linguistic varieties in biomedicine (Chapter 4), and (iii) we propose a fast and accurate end-to-end solution for biomedical event extraction, leveraging sequential transfer learning and multi-task learning, making it a viable approach for real-world large-scale scenarios (Chapter 5). We then outline the conclusions by highlighting challenges and providing future research directions in the field.
3
Cozzi, Riccardo. "Using semantic entities to improve the distillation of transformers." Master's thesis, Alma Mater Studiorum - Università di Bologna, 2022.
Анотація:
In the last decade, the size of deep neural architectures implied in Natural Language Processing (NLP) has increased exponentially, reaching in some cases with hundreds of billions of parameters. Although, training and deploying these huge architectures is an extremely resource-demanding process and the costs are often not affordable in real-world applications. For these reasons, lots of research and industrial efforts are investigating solutions to reduce the size of these models but at the same time maintain high performance.
This work was about studying and experimenting Knowledge Distillation techniques with the goal of training smaller and cheaper models while attempting to produce a good approximation of large pre-trained ones. The conducted experiments consist of a first reproduction of a recent promising work of DistilBERT while trying to further reduce the resources implied in the process. In fact, we discovered it is possible to achieve approximately the same score of the state-of-the-art but involving only a small fraction of data and training resources.
The second proposed experiment consists of an attempt of performing the same distillation task with an architecture based on LUKE, a powerful entity-aware transformer that has recently shown how injecting semantic entities can positively influence the training of these models. Unfortunately, this second experiment, as we will see, did not give us the result we hoped, meaning that the task needs additional research effort.
4
Sagen, Markus. "Large-Context Question Answering with Cross-Lingual Transfer." Thesis, Uppsala universitet, Institutionen för informationsteknologi, 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-440704.
Анотація:
Models based around the transformer architecture have become one of the most prominent for solving a multitude of natural language processing (NLP)tasks since its introduction in 2017. However, much research related to the transformer model has focused primarily on achieving high performance and many problems remain unsolved. Two of the most prominent currently are the lack of high performing non-English pre-trained models, and the limited number of words most trained models can incorporate for their context. Solving these problems would make NLP models more suitable for real-world applications, improving information retrieval, reading comprehension, and more. All previous research has focused on incorporating long-context for English language models. This thesis investigates the cross-lingual transferability between languages when only training for long-context in English. Training long-context models in English only could make long-context in low-resource languages, such as Swedish, more accessible since it is hard to find such data in most languages and costly to train for each language. This could become an efficient method for creating long-context models in other languages without the need for such data in all languages or pre-training from scratch. We extend the models’ context using the training scheme of the Longformer architecture and fine-tune on a question-answering task in several languages. Our evaluation could not satisfactorily confirm nor deny if transferring long-term context is possible for low-resource languages. We believe that using datasets that require long-context reasoning, such as a multilingual TriviaQAdataset, could demonstrate our hypothesis’s validity.
Частини книг з теми "Natural language processing biomedical nlp deep learning transfer learning":
1
Schölly, Reto, Suhail Yazijy, and Philipp Kellmeyer. "MedSentinel – A Smart Sentinel for Biomedical Online Search Demonstrated by a COVID-19 Search." In MEDINFO 2021: One World, One Health – Global Partnership for Digital Innovation. IOS Press, 2022. http://dx.doi.org/10.3233/shti220078.
Анотація:
We present a work-in-progress software project which aims to assist cross-database medical research and knowledge acquisition from heterogeneous sources. Using a Natural Language Processing (NLP) model based on deep learning algorithms, topical similarities are detected, going beyond measures of connectivity via citation or database suggestion algorithms. A network is generated based on the NLP-similarities between them, and then presented within an explorable 3D environment. Our software will then generate a list of publications and datasets which pertain to a certain topic of interest, based on their level of similarity in terms of knowledge representation.
Тези доповідей конференцій з теми "Natural language processing biomedical nlp deep learning transfer learning":
1
Gupta, Aman, and Yadul Raghav. "Deep Learning Roles based Approach to Link Prediction in Networks." In 9th International Conference on Natural Language Processing (NLP 2020). AIRCC Publishing Corporation, 2020. http://dx.doi.org/10.5121/csit.2020.101416.
Анотація:
The problem of predicting links has gained much attention in recent years due to its vast application in various domains such as sociology, network analysis, information science, etc. Many methods have been proposed for link prediction such as RA, AA, CCLP, etc. These methods required hand-crafted structural features to calculate the similarity scores between a pair of nodes in a network. Some methods use local structural information while others use global information of a graph. These methods do not tell which properties are better than others. With an in-depth analysis of these methods, we understand that one way to overcome this problem is to consider network structure and node attribute information to capture the discriminative features for link prediction tasks. We proposed a deep learning Autoencoder based Link Prediction (ALP) architecture for the latent representation of a graph, unified with non-negative matrix factorization to automatically determine the underlying roles in a network, after that assigning a mixed-membership of these roles to each node in the network. The idea is to transfer these roles as a feature vector for the link prediction task in the network. Further, cosine similarity is applied after getting the required features to compute the pairwise similarity score between the nodes. We present the performance of the algorithm on the real-world datasets, where it gives the competitive result compared to other algorithms.
2
Kang, Huay Wen, Kah Kien Chye, Zi Yuan Ong, and Chi Wee Tan. "The Science of Emotion: Malaysian Airlines Sentiment Analysis using BERT Approach." In International Conference on Digital Transformation and Applications (ICDXA 2021). Tunku Abdul Rahman University College, 2021. http://dx.doi.org/10.56453/icdxa.2021.1013.
Анотація:
Sentiment analysis has grown to be one of the most active research areas in Natural Language Processing (NLP). Sentiment analysis, also known as opinion mining, uses a series of methods, techniques and tools to study people’s opinions, views and sentiment towards a wide range of topics such as products, services, events and issues. In the airline industry, millions of people today use social networking sites such Twitter, Skytrax, TripAdvisor to express their emotions, opinions, and share information about the aircraft service. It is a hidden gem to the airline company to gain valuable insight from this data and have the broadest possible view into what people are saying about the airline’ brand online. Hence, this paper explores six different sentiment analysis models: Random Forest, Multinomial Naive Bayes, Linear Support Vector Classifier, Ensemble Method, Bidirectional Long Term Short Memory (Bi-LSTM) and BERT model, in order to determine and develop the best model to be used. The best model was then used to determine the social status, company reputation, and brand image of Malaysian airline companies. In conclusion, the BERT model was found to perform the best out of the six models tested, scoring an accuracy of 86%. Keywords: Supervised Learning, Ensemble Learning, Deep Learning, Transfer Learning, Airline Sentiment