Log in

Relevant bibliographies by topics / Contextualized Language Models / Journal articles

To see the other types of publications on this topic, follow the link: Contextualized Language Models.

Journal articles on the topic 'Contextualized Language Models'

Author: Grafiati

Published: 25 June 2021

Last updated: 29 July 2025

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Contextualized Language Models.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

1

El Adlouni, Yassine, Noureddine En Nahnahi, Said Ouatik El Alaoui, Mohammed Meknassi, Horacio Rodríguez, and Nabil Alami. "Arabic Biomedical Community Question Answering Based on Contextualized Embeddings." International Journal of Intelligent Information Technologies 17, no. 3 (2021): 13–29. http://dx.doi.org/10.4018/ijiit.2021070102.

Full text

Abstract:

Community question answering has become increasingly important as they are practical for seeking and sharing information. Applying deep learning models often leads to good performance, but it requires an extensive amount of annotated data, a problem exacerbated for languages suffering a scarcity of resources. Contextualized language representation models have gained success due to promising results obtained on a wide array of downstream natural language processing tasks such as text classification, textual entailment, and paraphrase identification. This paper presents a novel approach by fine-

APA, Harvard, Vancouver, ISO, and other styles

2

Zhou, Xuhui, Yue Zhang, Leyang Cui, and Dandan Huang. "Evaluating Commonsense in Pre-Trained Language Models." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (2020): 9733–40. http://dx.doi.org/10.1609/aaai.v34i05.6523.

Full text

Abstract:

Contextualized representations trained over large raw text data have given remarkable improvements for NLP tasks including question answering and reading comprehension. There have been works showing that syntactic, semantic and word sense knowledge are contained in such representations, which explains why they benefit such tasks. However, relatively little work has been done investigating commonsense knowledge contained in contextualized representations, which is crucial for human question answering and reading comprehension. We study the commonsense ability of GPT, BERT, XLNet, and RoBERTa by

APA, Harvard, Vancouver, ISO, and other styles

3

Myagmar, Batsergelen, Jie Li, and Shigetomo Kimura. "Cross-Domain Sentiment Classification With Bidirectional Contextualized Transformer Language Models." IEEE Access 7 (2019): 163219–30. http://dx.doi.org/10.1109/access.2019.2952360.

Full text

APA, Harvard, Vancouver, ISO, and other styles

4

Li, Yichen, Yintong Huo, Renyi Zhong, et al. "Go Static: Contextualized Logging Statement Generation." Proceedings of the ACM on Software Engineering 1, FSE (2024): 609–30. http://dx.doi.org/10.1145/3643754.

Full text

Abstract:

Logging practices have been extensively investigated to assist developers in writing appropriate logging statements for documenting software behaviors. Although numerous automatic logging approaches have been proposed, their performance remains unsatisfactory due to the constraint of the single-method input, without informative programming context outside the method. Specifically, we identify three inherent limitations with single-method context: limited static scope of logging statements, inconsistent logging styles, and missing type information of logging variables. To tackle these limitatio

APA, Harvard, Vancouver, ISO, and other styles

5

Yan, Huijiong, Tao Qian, Liang Xie, and Shanguang Chen. "Unsupervised cross-lingual model transfer for named entity recognition with contextualized word representations." PLOS ONE 16, no. 9 (2021): e0257230. http://dx.doi.org/10.1371/journal.pone.0257230.

Full text

Abstract:

Named entity recognition (NER) is one fundamental task in the natural language processing (NLP) community. Supervised neural network models based on contextualized word representations can achieve highly-competitive performance, which requires a large-scale manually-annotated corpus for training. While for the resource-scarce languages, the construction of such as corpus is always expensive and time-consuming. Thus, unsupervised cross-lingual transfer is one good solution to address the problem. In this work, we investigate the unsupervised cross-lingual NER with model transfer based on contex

APA, Harvard, Vancouver, ISO, and other styles

6

Xu, Yifei, Jingqiao Zhang, Ru He, et al. "SAS: Self-Augmentation Strategy for Language Model Pre-training." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 10 (2022): 11586–94. http://dx.doi.org/10.1609/aaai.v36i10.21412.

Full text

Abstract:

The core of self-supervised learning for pre-training language models includes pre-training task design as well as appropriate data augmentation. Most data augmentations in language model pre-training are context-independent. A seminal contextualized augmentation was recently proposed in ELECTRA and achieved state-of-the-art performance by introducing an auxiliary generation network (generator) to produce contextualized data augmentation for the training of a main discrimination network (discriminator). This design, however, introduces extra computation cost of the generator and a need to adju

APA, Harvard, Vancouver, ISO, and other styles

7

Cong, Yan. "AI Language Models: An Opportunity to Enhance Language Learning." Informatics 11, no. 3 (2024): 49. http://dx.doi.org/10.3390/informatics11030049.

Full text

Abstract:

AI language models are increasingly transforming language research in various ways. How can language educators and researchers respond to the challenge posed by these AI models? Specifically, how can we embrace this technology to inform and enhance second language learning and teaching? In order to quantitatively characterize and index second language writing, the current work proposes the use of similarities derived from contextualized meaning representations in AI language models. The computational analysis in this work is hypothesis-driven. The current work predicts how similarities should

APA, Harvard, Vancouver, ISO, and other styles

8

Zhang, Shuiliang, Hai Zhao, Junru Zhou, Xi Zhou, and Xiang Zhou. "Semantics-Aware Inferential Network for Natural Language Understanding." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 16 (2021): 14437–45. http://dx.doi.org/10.1609/aaai.v35i16.17697.

Full text

Abstract:

For natural language understanding tasks, either machine reading comprehension or natural language inference, both semantics-aware and inference are favorable features of the concerned modeling for better understanding performance. Thus we propose a Semantics-Aware Inferential Network (SAIN) to meet such a motivation. Taking explicit contextualized semantics as a complementary input, the inferential module of SAIN enables a series of reasoning steps over semantic clues through an attention mechanism. By stringing these steps, the inferential network effectively learns to perform iterative reas

APA, Harvard, Vancouver, ISO, and other styles

9

Schumacher, Elliot, and Mark Dredze. "Learning unsupervised contextual representations for medical synonym discovery." JAMIA Open 2, no. 4 (2019): 538–46. http://dx.doi.org/10.1093/jamiaopen/ooz057.

Full text

Abstract:

Abstract Objectives An important component of processing medical texts is the identification of synonymous words or phrases. Synonyms can inform learned representations of patients or improve linking mentioned concepts to medical ontologies. However, medical synonyms can be lexically similar (“dilated RA” and “dilated RV”) or dissimilar (“cerebrovascular accident” and “stroke”); contextual information can determine if 2 strings are synonymous. Medical professionals utilize extensive variation of medical terminology, often not evidenced in structured medical resources. Therefore, the ability to

APA, Harvard, Vancouver, ISO, and other styles

10

Zhang, Yuhan, Wenqi Chen, Ruihan Zhang, and Xiajie Zhang. "Representing affect information in word embeddings." Experiments in Linguistic Meaning 2 (January 27, 2023): 310. http://dx.doi.org/10.3765/elm.2.5391.

Full text

Abstract:

A growing body of research in natural language processing (NLP) and natural language understanding (NLU) is investigating human-like knowledge learned or encoded in the word embeddings from large language models. This is a step towards understanding what knowledge language models capture that resembles human understanding of language and communication. Here, we investigated whether and how the affect meaning of a word (i.e., valence, arousal, dominance) is encoded in word embeddings pre-trained in large neural networks. We used the human-labeled dataset (Mohammad 2018) as the ground truth and

APA, Harvard, Vancouver, ISO, and other styles

11

Schick, Timo, and Hinrich Schütze. "Rare Words: A Major Problem for Contextualized Embeddings and How to Fix it by Attentive Mimicking." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (2020): 8766–74. http://dx.doi.org/10.1609/aaai.v34i05.6403.

Full text

Abstract:

Pretraining deep neural network architectures with a language modeling objective has brought large improvements for many natural language processing tasks. Exemplified by BERT, a recently proposed such architecture, we demonstrate that despite being trained on huge amounts of data, deep language models still struggle to understand rare words. To fix this problem, we adapt Attentive Mimicking, a method that was designed to explicitly learn embeddings for rare words, to deep language models. In order to make this possible, we introduce one-token approximation, a procedure that enables us to use

APA, Harvard, Vancouver, ISO, and other styles

12

Gatto, Joseph, Madhusudan Basak, and Sarah Masud Preum. "Scope of Pre-trained Language Models for Detecting Conflicting Health Information." Proceedings of the International AAAI Conference on Web and Social Media 17 (June 2, 2023): 221–32. http://dx.doi.org/10.1609/icwsm.v17i1.22140.

Full text

Abstract:

An increasing number of people now rely on online platforms to meet their health information needs. Thus identifying inconsistent or conflicting textual health information has become a safety-critical task. Health advice data poses a unique challenge where information that is accurate in the context of one diagnosis can be conflicting in the context of another. For example, people suffering from diabetes and hypertension often receive conflicting health advice on diet. This motivates the need for technologies which can provide contextualized, user-specific health advice. A crucial step towards

APA, Harvard, Vancouver, ISO, and other styles

13

Dev, Sunipa, Tao Li, Jeff M. Phillips, and Vivek Srikumar. "On Measuring and Mitigating Biased Inferences of Word Embeddings." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (2020): 7659–66. http://dx.doi.org/10.1609/aaai.v34i05.6267.

Full text

Abstract:

Word embeddings carry stereotypical connotations from the text they are trained on, which can lead to invalid inferences in downstream models that rely on them. We use this observation to design a mechanism for measuring stereotypes using the task of natural language inference. We demonstrate a reduction in invalid inferences via bias mitigation strategies on static word embeddings (GloVe). Further, we show that for gender bias, these techniques extend to contextualized embeddings when applied selectively only to the static components of contextualized embeddings (ELMo, BERT).

APA, Harvard, Vancouver, ISO, and other styles

14

Alshattnawi, Sawsan, Amani Shatnawi, Anas M. R. AlSobeh, and Aws A. Magableh. "Beyond Word-Based Model Embeddings: Contextualized Representations for Enhanced Social Media Spam Detection." Applied Sciences 14, no. 6 (2024): 2254. http://dx.doi.org/10.3390/app14062254.

Full text

Abstract:

As social media platforms continue their exponential growth, so do the threats targeting their security. Detecting disguised spam messages poses an immense challenge owing to the constant evolution of tactics. This research investigates advanced artificial intelligence techniques to significantly enhance multiplatform spam classification on Twitter and YouTube. The deep neural networks we use are state-of-the-art. They are recurrent neural network architectures with long- and short-term memory cells that are powered by both static and contextualized word embeddings. Extensive comparative exper

APA, Harvard, Vancouver, ISO, and other styles

15

Lin, Guanjun, Heming Jia, and Di Wu. "Distilled and Contextualized Neural Models Benchmarked for Vulnerable Function Detection." Mathematics 10, no. 23 (2022): 4482. http://dx.doi.org/10.3390/math10234482.

Full text

Abstract:

Detecting vulnerabilities in programs is an important yet challenging problem in cybersecurity. The recent advancement in techniques of natural language understanding enables the data-driven research on automated code analysis to embrace Pre-trained Contextualized Models (PCMs). These models are pre-trained on the large corpus and can be fine-tuned for various downstream tasks, but their feasibility and effectiveness for software vulnerability detection have not been systematically studied. In this paper, we explore six prevalent PCMs and compare them with three mainstream Non-Contextualized M

APA, Harvard, Vancouver, ISO, and other styles

16

Ciucă, Ioana, and Yuan-Sen Ting. "Galactic ChitChat: Using Large Language Models to Converse with Astronomy Literature." Research Notes of the AAS 7, no. 9 (2023): 193. http://dx.doi.org/10.3847/2515-5172/acf85f.

Full text

Abstract:

Abstract We demonstrate the potential of the state-of-the-art OpenAI GPT-4 large language model to engage in meaningful interactions with Astronomy papers using in-context prompting. To optimize for efficiency, we employ a distillation technique that effectively reduces the size of the original input paper by 50%, while maintaining the paragraph structure and overall semantic integrity. We then explore the model’s responses using a multi-document context (ten distilled documents). Our findings indicate that GPT-4 excels in the multi-document domain, providing detailed answers contextualized wi

APA, Harvard, Vancouver, ISO, and other styles

17

Chen, Zeming, and Qiyue Gao. "Probing Linguistic Information for Logical Inference in Pre-trained Language Models." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 10 (2022): 10509–17. http://dx.doi.org/10.1609/aaai.v36i10.21294.

Full text

Abstract:

Progress in pre-trained language models has led to a surge of impressive results on downstream tasks for natural language understanding. Recent work on probing pre-trained language models uncovered a wide range of linguistic properties encoded in their contextualized representations. However, it is unclear whether they encode semantic knowledge that is crucial to symbolic inference methods. We propose a methodology for probing knowledge for inference that logical systems require but often lack in pre-trained language model representations. Our probing datasets cover a list of key types of know

APA, Harvard, Vancouver, ISO, and other styles

18

Strokach, Alexey, Tian Yu Lu, and Philip M. Kim. "ELASPIC2 (EL2): Combining Contextualized Language Models and Graph Neural Networks to Predict Effects of Mutations." Journal of Molecular Biology 433, no. 11 (2021): 166810. http://dx.doi.org/10.1016/j.jmb.2021.166810.

Full text

APA, Harvard, Vancouver, ISO, and other styles

19

Theuner, Katharina, Tomas Mikael Elmgren, Axel Götling, Marvin Carl May, and Haluk Akay. "Weaving Knowledge Graphs and Large Language Models (LLMs): Leveraging Semantics for Contextualized Design Knowledge Retrieval." Procedia CIRP 134 (2025): 1125–30. https://doi.org/10.1016/j.procir.2025.03.073.

Full text

APA, Harvard, Vancouver, ISO, and other styles

20

Flemotomos, Nikolaos, Victor R. Martinez, Zhuohao Chen, Torrey A. Creed, David C. Atkins, and Shrikanth Narayanan. "Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations." PLOS ONE 16, no. 10 (2021): e0258639. http://dx.doi.org/10.1371/journal.pone.0258639.

Full text

Abstract:

During a psychotherapy session, the counselor typically adopts techniques which are codified along specific dimensions (e.g., ‘displays warmth and confidence’, or ‘attempts to set up collaboration’) to facilitate the evaluation of the session. Those constructs, traditionally scored by trained human raters, reflect the complex nature of psychotherapy and highly depend on the context of the interaction. Recent advances in deep contextualized language models offer an avenue for accurate in-domain linguistic representations which can lead to robust recognition and scoring of such psychotherapy-rel

APA, Harvard, Vancouver, ISO, and other styles

21

Murthy, G. S. N., K. Anshu, T. Srilakshmi, CH Sumanth, and M. Naga Mounika. "AI POWERED TRANSLATOR: TRANSFORMING NATURAL LANGUAGE TO DATABASE QUERIES." International Journal of Engineering Applied Sciences and Technology 09, no. 11 (2025): 39–43. https://doi.org/10.33564/ijeast.2025.v09i11.006.

Full text

Abstract:

In the modern digital landscape, interacting with databases often requires structured query languages SQL or NoSQL syntax, which can be a barrier for non-technical users. This project introduces an intelligent AI-powered system that seamlessly converts natural language questions into executable database queries, bridging the gap between human communication and database management. Leveraging advanced natural language processing (NLP) models, large language models (LLMs), the system dynamically interprets user queries and translates them into optimized SQL or NoSQL commands. Our approach involv

APA, Harvard, Vancouver, ISO, and other styles

22

Tang, Xiaobin, Nuo Lei, Manru Dong, and Dan Ma. "Stock Price Prediction Based on Natural Language Processing1." Complexity 2022 (May 6, 2022): 1–15. http://dx.doi.org/10.1155/2022/9031900.

Full text

Abstract:

The keywords used in traditional stock price prediction are mainly based on literature and experience. This study designs a new text mining method for keywords augmentation based on natural language processing models including Bidirectional Encoder Representation from Transformers (BERT) and Neural Contextualized Representation for Chinese Language Understanding (NEZHA) natural language processing models. The BERT vectorization and the NEZHA keyword discrimination models extend the seed keywords from two dimensions of similarity and importance, respectively, thus constructing the keyword thesa

APA, Harvard, Vancouver, ISO, and other styles

23

Davagdorj, Khishigsuren, Ling Wang, Meijing Li, Van-Huy Pham, Keun Ho Ryu, and Nippon Theera-Umpon. "Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering." International Journal of Environmental Research and Public Health 19, no. 10 (2022): 5893. http://dx.doi.org/10.3390/ijerph19105893.

Full text

Abstract:

The increasing expansion of biomedical documents has increased the number of natural language textual resources related to the current applications. Meanwhile, there has been a great interest in extracting useful information from meaningful coherent groupings of textual content documents in the last decade. However, it is challenging to discover informative representations and define relevant articles from the rapidly growing biomedical literature due to the unsupervised nature of document clustering. Moreover, empirical investigations demonstrated that traditional text clustering methods prod

APA, Harvard, Vancouver, ISO, and other styles

24

Huang, Jiayang, Yue Huang, David Yip, and Varvara Guljajeva. "Ephemera: Language as a Virus - AI-driven Interactive and Immersive Art Installation." Proceedings of the ACM on Computer Graphics and Interactive Techniques 7, no. 4 (2024): 1–8. http://dx.doi.org/10.1145/3664219.

Full text

Abstract:

In this paper, we introduce the speech-based interactive and immersive installation, Ephemera, as an artistic response to the linguistic taboos encountered in daily communication, prompting audience reflection and thoughts. Within this project, we symbolize the dissemination chain of language through a computational ecosystem. Utilizing the surreal 'virus' as an embodiment of banned words, we employ generative models for visual representation, leverage large language models for communicative agents, and use machine learning for behavioral engines, ultimately simulating a digitally autonomous m

APA, Harvard, Vancouver, ISO, and other styles

25

Bian, Yifan, Dennis Küster, Hui Liu, and Eva G. Krumhuber. "Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models." Sensors 24, no. 1 (2023): 126. http://dx.doi.org/10.3390/s24010126.

Full text

Abstract:

This paper provides a comprehensive overview of affective computing systems for facial expression recognition (FER) research in naturalistic contexts. The first section presents an updated account of user-friendly FER toolboxes incorporating state-of-the-art deep learning models and elaborates on their neural architectures, datasets, and performances across domains. These sophisticated FER toolboxes can robustly address a variety of challenges encountered in the wild such as variations in illumination and head pose, which may otherwise impact recognition accuracy. The second section of this pa

APA, Harvard, Vancouver, ISO, and other styles

26

Soler, Aina Garí, Matthieu Labeau, and Chloé Clavel. "The Impact of Word Splitting on the Semantic Content of Contextualized Word Representations." Transactions of the Association for Computational Linguistics 12 (2024): 299–320. http://dx.doi.org/10.1162/tacl_a_00647.

Full text

Abstract:

Abstract When deriving contextualized word representations from language models, a decision needs to be made on how to obtain one for out-of-vocabulary (OOV) words that are segmented into subwords. What is the best way to represent these words with a single vector, and are these representations of worse quality than those of in-vocabulary words? We carry out an intrinsic evaluation of embeddings from different models on semantic similarity tasks involving OOV words. Our analysis reveals, among other interesting findings, that the quality of representations of words that are split is often, but

APA, Harvard, Vancouver, ISO, and other styles

27

Araujo, Vladimir, Marie-Francine Moens, and Alvaro Soto. "Learning Sentence-Level Representations with Predictive Coding." Machine Learning and Knowledge Extraction 5, no. 1 (2023): 59–77. http://dx.doi.org/10.3390/make5010005.

Full text

Abstract:

Learning sentence representations is an essential and challenging topic in the deep learning and natural language processing communities. Recent methods pre-train big models on a massive text corpus, focusing mainly on learning the representation of contextualized words. As a result, these models cannot generate informative sentence embeddings since they do not explicitly exploit the structure and discourse relationships existing in contiguous sentences. Drawing inspiration from human language processing, this work explores how to improve sentence-level representations of pre-trained models by

APA, Harvard, Vancouver, ISO, and other styles

28

Al-Ghamdi, Sharefah, Hend Al-Khalifa, and Abdulmalik Al-Salman. "Fine-Tuning BERT-Based Pre-Trained Models for Arabic Dependency Parsing." Applied Sciences 13, no. 7 (2023): 4225. http://dx.doi.org/10.3390/app13074225.

Full text

Abstract:

With the advent of pre-trained language models, many natural language processing tasks in various languages have achieved great success. Although some research has been conducted on fine-tuning BERT-based models for syntactic parsing, and several Arabic pre-trained models have been developed, no attention has been paid to Arabic dependency parsing. In this study, we attempt to fill this gap and compare nine Arabic models, fine-tuning strategies, and encoding methods for dependency parsing. We evaluated three treebanks to highlight the best options and methods for fine-tuning Arabic BERT-based

APA, Harvard, Vancouver, ISO, and other styles

29

Qarah, Faisal, and Tawfeeq Alsanoosy. "A Comprehensive Analysis of Various Tokenizers for Arabic Large Language Models." Applied Sciences 14, no. 13 (2024): 5696. http://dx.doi.org/10.3390/app14135696.

Full text

Abstract:

Pretrained language models have achieved great success in various natural language understanding (NLU) tasks due to their capacity to capture deep contextualized information in text using pretraining on large-scale corpora. Tokenization plays a significant role in the process of lexical analysis. Tokens become the input for other natural language processing (NLP) tasks, like semantic parsing and language modeling. However, there is a lack of research on the evaluation of the impact of tokenization on the Arabic language model. Therefore, this study aims to address this gap in the literature by

APA, Harvard, Vancouver, ISO, and other styles

30

Sabbeh, Sahar F., and Heba A. Fasihuddin. "A Comparative Analysis of Word Embedding and Deep Learning for Arabic Sentiment Classification." Electronics 12, no. 6 (2023): 1425. http://dx.doi.org/10.3390/electronics12061425.

Full text

Abstract:

Sentiment analysis on social media platforms (i.e., Twitter or Facebook) has become an important tool to learn about users’ opinions and preferences. However, the accuracy of sentiment analysis is disrupted by the challenges of natural language processing (NLP). Recently, deep learning models have proved superior performance over statistical- and lexical-based approaches in NLP-related tasks. Word embedding is an important layer of deep learning models to generate input features. Many word embedding models have been presented for text representation of both classic and context-based word embed

APA, Harvard, Vancouver, ISO, and other styles

31

Garí Soler, Aina, and Marianna Apidianaki. "Let’s Play Mono-Poly: BERT Can Reveal Words’ Polysemy Level and Partitionability into Senses." Transactions of the Association for Computational Linguistics 9 (2021): 825–44. http://dx.doi.org/10.1162/tacl_a_00400.

Full text

Abstract:

Pre-trained language models (LMs) encode rich information about linguistic structure but their knowledge about lexical polysemy remains unclear. We propose a novel experimental setup for analyzing this knowledge in LMs specifically trained for different languages (English, French, Spanish, and Greek) and in multilingual BERT. We perform our analysis on datasets carefully designed to reflect different sense distributions, and control for parameters that are highly correlated with polysemy such as frequency and grammatical category. We demonstrate that BERT-derived representations reflect words’

APA, Harvard, Vancouver, ISO, and other styles

32

Zhurko, Dmytro, and Iryna Bilous. "Using word embedding models in natural language processing." Technical sciences and technologies, no. 1 (39) (May 22, 2025): 151–60. https://doi.org/10.25140/2411-5363-2025-1(39)-151-160.

Full text

Abstract:

The paper presents findings of the scientific and methodological investigation of word embedding models for application in natural language processing (NLP). The research is timely due to rapid advancements in computational power, enabling large-scale text analysis. The study is focused on adapting existing word embedding models — Word2Vec, GloVe, FastText, ELMo and BERT — to the under-represented Ukrainian language. While effective for English texts, these models are difficult to use for Ukrainian due to linguistic specifics and lack of quality resources. The objective is to compare these mod

APA, Harvard, Vancouver, ISO, and other styles

33

Zhang, Zhuosheng, Hai Zhao, Masao Utiyama, and Eiichiro Sumita. "Language Model Pre-training on True Negatives." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 11 (2023): 14002–10. http://dx.doi.org/10.1609/aaai.v37i11.26639.

Full text

Abstract:

Discriminative pre-trained language models (PrLMs) learn to predict original texts from intentionally corrupted ones. Taking the former text as positive and the latter as negative samples, the PrLM can be trained effectively for contextualized representation. However, the training of such a type of PrLMs highly relies on the quality of the automatically constructed samples. Existing PrLMs simply treat all corrupted texts as equal negative without any examination, which actually lets the resulting model inevitably suffer from the false negative issue where training is carried out on pseudo-nega

APA, Harvard, Vancouver, ISO, and other styles

34

Saha, Koustuv, Ted Grover, Stephen M. Mattingly, et al. "Person-Centered Predictions of Psychological Constructs with Social Media Contextualized by Multimodal Sensing." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, no. 1 (2021): 1–32. http://dx.doi.org/10.1145/3448117.

Full text

Abstract:

Personalized predictions have shown promises in various disciplines but they are fundamentally constrained in their ability to generalize across individuals. These models are often trained on limited datasets which do not represent the fluidity of human functioning. In contrast, generalized models capture normative behaviors between individuals but lack precision in predicting individual outcomes. This paper aims to balance the tradeoff between one-for-each and one-for-all models by clustering individuals on mutable behaviors and conducting cluster-specific predictions of psychological constru

APA, Harvard, Vancouver, ISO, and other styles

35

Pezzelle, Sandro, Ece Takmaz, and Raquel Fernández. "Word Representation Learning in Multimodal Pre-Trained Transformers: An Intrinsic Evaluation." Transactions of the Association for Computational Linguistics 9 (2021): 1563–79. http://dx.doi.org/10.1162/tacl_a_00443.

Full text

Abstract:

Abstract This study carries out a systematic intrinsic evaluation of the semantic representations learned by state-of-the-art pre-trained multimodal Transformers. These representations are claimed to be task-agnostic and shown to help on many downstream language-and-vision tasks. However, the extent to which they align with human semantic intuitions remains unclear. We experiment with various models and obtain static word representations from the contextualized ones they learn. We then evaluate them against the semantic judgments provided by human speakers. In line with previous evidence, we o

APA, Harvard, Vancouver, ISO, and other styles

36

Lin, Sheng-Chieh, Minghan Li, and Jimmy Lin. "Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval." Transactions of the Association for Computational Linguistics 11 (2023): 436–52. http://dx.doi.org/10.1162/tacl_a_00556.

Full text

Abstract:

Abstract Pre-trained language models have been successful in many knowledge-intensive NLP tasks. However, recent work has shown that models such as BERT are not “structurally ready” to aggregate textual information into a [CLS] vector for dense passage retrieval (DPR). This “lack of readiness” results from the gap between language model pre-training and DPR fine-tuning. Previous solutions call for computationally expensive techniques such as hard negative mining, cross-encoder distillation, and further pre-training to learn a robust DPR model. In this work, we instead propose to fully exploit

APA, Harvard, Vancouver, ISO, and other styles

37

Ye, Yilin, Qian Zhu, Shishi Xiao, Kang Zhang, and Wei Zeng. "The Contemporary Art of Image Search: Iterative User Intent Expansion via Vision-Language Model." Proceedings of the ACM on Human-Computer Interaction 8, CSCW1 (2024): 1–31. http://dx.doi.org/10.1145/3641019.

Full text

Abstract:

Image search is an essential and user-friendly method to explore vast galleries of digital images. However, existing image search methods heavily rely on proximity measurements like tag matching or image similarity, requiring precise user inputs for satisfactory results. To meet the growing demand for a contemporary image search engine that enables accurate comprehension of users' search intentions, we introduce an innovative user intent expansion framework. Our framework leverages visual-language models to parse and compose multi-modal user inputs to provide more accurate and satisfying resul

APA, Harvard, Vancouver, ISO, and other styles

38

Zhang, Zhuosheng, Yuwei Wu, Hai Zhao, et al. "Semantics-Aware BERT for Language Understanding." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (2020): 9628–35. http://dx.doi.org/10.1609/aaai.v34i05.6510.

Full text

Abstract:

The latest work on language representations carefully integrates contextualized features into language model training, which enables a series of success especially in various machine reading comprehension and natural language inference tasks. However, the existing language representation models including ELMo, GPT and BERT only exploit plain context-sensitive features such as character or word embeddings. They rarely consider incorporating structured semantic information which can provide rich semantics for language representation. To promote natural language understanding, we propose to incor

APA, Harvard, Vancouver, ISO, and other styles

39

Mercer, Sarah, and Peter D. MacIntyre. "Introducing positive psychology to SLA." Studies in Second Language Learning and Teaching 4, no. 2 (2014): 153–72. http://dx.doi.org/10.14746/ssllt.2014.4.2.2.

Full text

Abstract:

Positive psychology is a rapidly expanding subfield in psychology that has important implications for the field of second language acquisition (SLA). This paper introduces positive psychology to the study of language by describing its key tenets. The potential contributions of positive psychology are contextualized with reference to prior work, including the humanistic movement in language teaching, models of motivation, the concept of an affective filter, studies of the good language learner, and the concepts related to the self. There are reasons for both encouragement and caution as studies

APA, Harvard, Vancouver, ISO, and other styles

40

Dragomir, Isabela-Anda, and Brânduşa-Oana Niculescu. "Packing and Unpacking Grammar – Towards a Communicative Approach to Teaching Language Structures." Scientific Bulletin 26, no. 2 (2021): 121–28. http://dx.doi.org/10.2478/bsaft-2021-0014.

Full text

Abstract:

Abstract Creating classroom activities which really get students to communicate in a natural and meaningful way remains a timely challenge when it comes to teaching a foreign language. This article sets out to revisit the traditional trifecta based on the three Ps (presentation, practice, production) and reincorporate it in a set of specific elements that ensure a successful outcome of language learning activities. With particular focus on grammar, the paper aims to deconstruct the steps and procedures behind five important aspects to be taken into consideration when planning and conducting ac

APA, Harvard, Vancouver, ISO, and other styles

41

Zeng, Ziheng, and Suma Bhat. "Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions." Transactions of the Association for Computational Linguistics 10 (2022): 1120–37. http://dx.doi.org/10.1162/tacl_a_00510.

Full text

Abstract:

Abstract Idiomatic expressions (IEs), characterized by their non-compositionality, are an important part of natural language. They have been a classical challenge to NLP, including pre-trained language models that drive today’s state-of-the-art. Prior work has identified deficiencies in their contextualized representation stemming from the underlying compositional paradigm of representation. In this work, we take a first-principles approach to build idiomaticity into BART using an adapter as a lightweight non-compositional language expert trained on idiomatic sentences. The improved capability

APA, Harvard, Vancouver, ISO, and other styles

42

Berlec, Tomaž, Marko Corn, Sergej Varljen, and Primož Podržaj. "Exploring Decentralized Warehouse Management Using Large Language Models: A Proof of Concept." Applied Sciences 15, no. 10 (2025): 5734. https://doi.org/10.3390/app15105734.

Full text

Abstract:

The Fourth Industrial Revolution has introduced “shared manufacturing” as a key concept that leverages digitalization, IoT, blockchain, and robotics to redefine the production and delivery of manufacturing services. This paper presents a novel approach to decentralized warehouse management integrating Large Language Models (LLMs) into the decision-making processes of autonomous agents, which serves as a proof of concept for shared manufacturing. A multi-layered system architecture consisting of physical, digital shadow, organizational, and protocol layers was developed to enable seamless inter

APA, Harvard, Vancouver, ISO, and other styles

43

Syed, Muzamil Hussain, and Sun-Tae Chung. "MenuNER: Domain-Adapted BERT Based NER Approach for a Domain with Limited Dataset and Its Application to Food Menu Domain." Applied Sciences 11, no. 13 (2021): 6007. http://dx.doi.org/10.3390/app11136007.

Full text

Abstract:

Entity-based information extraction is one of the main applications of Natural Language Processing (NLP). Recently, deep transfer-learning utilizing contextualized word embedding from pre-trained language models has shown remarkable results for many NLP tasks, including Named-entity recognition (NER). BERT (Bidirectional Encoder Representations from Transformers) is gaining prominent attention among various contextualized word embedding models as a state-of-the-art pre-trained language model. It is quite expensive to train a BERT model from scratch for a new application domain since it needs a

APA, Harvard, Vancouver, ISO, and other styles

44

Şahin, Gözde Gül. "To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP." Computational Linguistics 48, no. 1 (2022): 5–42. http://dx.doi.org/10.1162/coli_a_00425.

Full text

Abstract:

Abstract Data-hungry deep neural networks have established themselves as the de facto standard for many NLP tasks, including the traditional sequence tagging ones. Despite their state-of-the-art performance on high-resource languages, they still fall behind their statistical counterparts in low-resource scenarios. One methodology to counterattack this problem is text augmentation, that is, generating new synthetic training data points from existing data. Although NLP has recently witnessed several new textual augmentation techniques, the field still lacks a systematic performance analysis on a

APA, Harvard, Vancouver, ISO, and other styles

45

Yoo, Yongseok. "Automated Think-Aloud Protocol for Identifying Students with Reading Comprehension Impairment Using Sentence Embedding." Applied Sciences 14, no. 2 (2024): 858. http://dx.doi.org/10.3390/app14020858.

Full text

Abstract:

The think-aloud protocol is a valuable tool for investigating readers’ cognitive processes during reading. However, its reliance on experienced human evaluators poses challenges in terms of efficiency and scalability. To address this limitation, this study proposes a novel application of natural language processing to automate the think-aloud protocol. Specifically, we use a sentence embedding technique to encode the stimulus text and corresponding readers’ responses into high-dimensional vectors, and the similarity between these embeddings serves as a feature. The properties of the feature ar

APA, Harvard, Vancouver, ISO, and other styles

46

Karaoğlan, Kürşat Mustafa. "Novel approaches for fake news detection based on attention-based deep multiple-instance learning using contextualized neural language models." Neurocomputing 602 (October 2024): 128263. http://dx.doi.org/10.1016/j.neucom.2024.128263.

Full text

APA, Harvard, Vancouver, ISO, and other styles

47

MacAvaney, Sean, Sergey Feldman, Nazli Goharian, Doug Downey, and Arman Cohan. "ABNIRML: Analyzing the Behavior of Neural IR Models." Transactions of the Association for Computational Linguistics 10 (2022): 224–39. http://dx.doi.org/10.1162/tacl_a_00457.

Full text

Abstract:

Abstract Pretrained contextualized language models such as BERT and T5 have established a new state-of-the-art for ad-hoc search. However, it is not yet well understood why these methods are so effective, what makes some variants more effective than others, and what pitfalls they may have. We present a new comprehensive framework for Analyzing the Behavior of Neural IR ModeLs (ABNIRML), which includes new types of diagnostic probes that allow us to test several characteristics—such as writing styles, factuality, sensitivity to paraphrasing and word order—that are not addressed by previous tech

APA, Harvard, Vancouver, ISO, and other styles

48

Liu, Jiaqing, Chong Deng, Qinglin Zhang, et al. "Recording for Eyes, Not Echoing to Ears: Contextualized Spoken-to-Written Conversion of ASR Transcripts." Proceedings of the AAAI Conference on Artificial Intelligence 39, no. 23 (2025): 24623–31. https://doi.org/10.1609/aaai.v39i23.34642.

Full text

Abstract:

Automatic Speech Recognition (ASR) transcripts exhibit recognition errors and various spoken language phenomena such as disfluencies, ungrammatical sentences, and incomplete sentences, hence suffering from poor readability. To improve readability, we propose a Contextualized Spoken-to-Written conversion (CoS2W) task to address ASR and grammar errors and also transfer the informal text into the formal style with content preserved, utilizing contexts and auxiliary information. This task naturally matches the in-context learning capabilities of Large Language Models (LLMs). To facilitate comprehe

APA, Harvard, Vancouver, ISO, and other styles

49

Jouffroy, Jordan, Sarah F. Feldman, Ivan Lerner, Bastien Rance, Anita Burgun, and Antoine Neuraz. "Hybrid Deep Learning for Medication-Related Information Extraction From Clinical Texts in French: MedExt Algorithm Development Study." JMIR Medical Informatics 9, no. 3 (2021): e17934. http://dx.doi.org/10.2196/17934.

Full text

Abstract:

Background Information related to patient medication is crucial for health care; however, up to 80% of the information resides solely in unstructured text. Manual extraction is difficult and time-consuming, and there is not a lot of research on natural language processing extracting medical information from unstructured text from French corpora. Objective We aimed to develop a system to extract medication-related information from clinical text written in French. Methods We developed a hybrid system combining an expert rule–based system, contextual word embedding (embedding for language model)

APA, Harvard, Vancouver, ISO, and other styles

50

Gamallo, Pablo. "Compositional Distributional Semantics with Syntactic Dependencies and Selectional Preferences." Applied Sciences 11, no. 12 (2021): 5743. http://dx.doi.org/10.3390/app11125743.

Full text

Abstract:

This article describes a compositional model based on syntactic dependencies which has been designed to build contextualized word vectors, by following linguistic principles related to the concept of selectional preferences. The compositional strategy proposed in the current work has been evaluated on a syntactically controlled and multilingual dataset, and compared with Transformer BERT-like models, such as Sentence BERT, the state-of-the-art in sentence similarity. For this purpose, we created two new test datasets for Portuguese and Spanish on the basis of that defined for the English langu

APA, Harvard, Vancouver, ISO, and other styles

We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!