Segui questo link per vedere altri tipi di pubblicazioni sul tema: State Token Mechanism.
Articoli di riviste sul tema "State Token Mechanism"
Cita una fonte nei formati APA, MLA, Chicago, Harvard e in molti altri stili
Vedi i top-50 articoli di riviste per l'attività di ricerca sul tema "State Token Mechanism".
Accanto a ogni fonte nell'elenco di riferimenti c'è un pulsante "Aggiungi alla bibliografia". Premilo e genereremo automaticamente la citazione bibliografica dell'opera scelta nello stile citazionale di cui hai bisogno: APA, MLA, Harvard, Chicago, Vancouver ecc.
Puoi anche scaricare il testo completo della pubblicazione scientifica nel formato .pdf e leggere online l'abstract (il sommario) dell'opera se è presente nei metadati.
Vedi gli articoli di riviste di molte aree scientifiche e compila una bibliografia corretta.
1
Hu, Anwen, Zhicheng Dou, Jian-Yun Nie e Ji-Rong Wen. "Leveraging Multi-Token Entities in Document-Level Named Entity Recognition". Proceedings of the AAAI Conference on Artificial Intelligence 34, n. 05 (3 aprile 2020): 7961–68. http://dx.doi.org/10.1609/aaai.v34i05.6304.
Testo completoAbstract (sommario):
Most state-of-the-art named entity recognition systems are designed to process each sentence within a document independently. These systems are easy to confuse entity types when the context information in a sentence is not sufficient enough. To utilize the context information within the whole document, most document-level work let neural networks on their own to learn the relation across sentences, which is not intuitive enough for us humans. In this paper, we divide entities to multi-token entities that contain multiple tokens and single-token entities that are composed of a single token. We propose that the context information of multi-token entities should be more reliable in document-level NER for news articles. We design a fusion attention mechanism which not only learns the semantic relevance between occurrences of the same token, but also focuses more on occurrences belonging to multi-tokens entities. To identify multi-token entities, we design an auxiliary task namely ‘Multi-token Entity Classification’ and perform this task simultaneously with document-level NER. This auxiliary task is simplified from NER and doesn't require extra annotation. Experimental results on the CoNLL-2003 dataset and OntoNotesnbm dataset show that our model outperforms state-of-the-art sentence-level and document-level NER methods.
2
Yang, Yixiao, Xiang Chen e Jiaguang Sun. "Improve Language Modeling for Code Completion Through Learning General Token Repetition of Source Code with Optimized Memory". International Journal of Software Engineering and Knowledge Engineering 29, n. 11n12 (novembre 2019): 1801–18. http://dx.doi.org/10.1142/s0218194019400229.
Testo completoAbstract (sommario):
In last few years, applying language model to source code is the state-of-the-art method for solving the problem of code completion. However, compared with natural language, code has more obvious repetition characteristics. For example, a variable can be used many times in the following code. Variables in source code have a high chance to be repetitive. Cloned code and templates, also have the property of token repetition. Capturing the token repetition of source code is important. In different projects, variables or types are usually named differently. This means that a model trained in a finite data set will encounter a lot of unseen variables or types in another data set. How to model the semantics of the unseen data and how to predict the unseen data based on the patterns of token repetition are two challenges in code completion. Hence, in this paper, token repetition is modelled as a graph, we propose a novel REP model which is based on deep neural graph network to learn the code toke repetition. The REP model is to identify the edge connections of a graph to recognize the token repetition. For predicting the token repetition of token [Formula: see text], the information of all the previous tokens needs to be considered. We use memory neural network (MNN) to model the semantics of each distinct token to make the framework of REP model more targeted. The experiments indicate that the REP model performs better than LSTM model. Compared with Attention-Pointer network, we also discover that the attention mechanism does not work in all situations. The proposed REP model could achieve similar or slightly better prediction accuracy compared to Attention-Pointer network and consume less training time. We also find other attention mechanism which could further improve the prediction accuracy.
3
Kim, Jinsu, Eunsun Choi, Byung-Gyu Kim e Namje Park. "Proposal of a Token-Based Node Selection Mechanism for Node Distribution of Mobility IoT Blockchain Nodes". Sensors 23, n. 19 (5 ottobre 2023): 8259. http://dx.doi.org/10.3390/s23198259.
Testo completoAbstract (sommario):
Various elements, such as evolutions in IoT services resulting from sensoring by vehicle parts and advances in small communication technology devices, have significantly impacted the mass spread of mobility services that are provided to users in need of limited resources. In particular, business models are progressing away from one-off costs towards longer-term costs, as represented by shared services utilizing kick-boards or bicycles and subscription services for vehicle software. Advances in shared mobility services, as described, are calling for solutions that can enhance the reliability of data aggregated by users leveraging mobility services in the next-generation mobility areas. However, the mining process to renew status ensures continued network communication, and block creation demands high performance in the public block chain. Therefore, easing the mining process for state updates in public blockchains is a way to alleviate the high-performance process requirements of public blockchains. The proposed mechanism assigns token-based block creation authority instead of the mining method, which provides block creation authority to nodes that provide many resources. Blocks are created only by a group of participants with tokens, and after creation, tokens are updated and delivered to new nodes to form a new token group. Additionally, tokens are updated in each block after their initial creation, making it difficult to disguise the tokens and preventing resource-centered centralization.
4
Bai, He, Peng Shi, Jimmy Lin, Yuqing Xie, Luchen Tan, Kun Xiong, Wen Gao e Ming Li. "Segatron: Segment-Aware Transformer for Language Modeling and Understanding". Proceedings of the AAAI Conference on Artificial Intelligence 35, n. 14 (18 maggio 2021): 12526–34. http://dx.doi.org/10.1609/aaai.v35i14.17485.
Testo completoAbstract (sommario):
Transformers are powerful for sequence modeling. Nearly all state-of-the-art language models and pre-trained language models are based on the Transformer architecture. However, it distinguishes sequential tokens only with the token position index. We hypothesize that better contextual representations can be generated from the Transformer with richer positional information. To verify this, we propose a segment-aware Transformer (Segatron), by replacing the original token position encoding with a combined position encoding of paragraph, sentence, and token. We first introduce the segment-aware mechanism to Transformer-XL, which is a popular Transformer-based language model with memory extension and relative position encoding. We find that our method can further improve the Transformer-XL base model and large model, achieving 17.1 perplexity on the WikiText-103 dataset. We further investigate the pre-training masked language modeling task with Segatron. Experimental results show that BERT pre-trained with Segatron (SegaBERT) can outperform BERT with vanilla Transformer on various NLP tasks, and outperforms RoBERTa on zero-shot sentence representation learning. Our code is available on GitHub.
5
Sitnik, A. A. "NFT as an Object of Legal Regulation". Actual Problems of Russian Law 17, n. 12 (19 novembre 2022): 84–93. http://dx.doi.org/10.17803/1994-1471.2022.145.12.084-093.
Testo completoAbstract (sommario):
The paper is devoted to the study of the legal nature of a non-fungible token — NFT. The paper discusses the concept and types of tokens. The author defines a token as a unit of accounting in a distributed ledger that digitally represents financial instruments or other assets that expresses the economic value of the objects being represented and allows the rights associated with them to be exercised. According to a common point of view, NFT serves as a means of digital expression of a particular object, it has characteristics (signs) inherent exclusively to it, by virtue of which it cannot be exchanged for another token, and the cost of one NFT is not conditioned by the cost of other tokens. The author notes that the listed features are not inherent in NFT in all cases. In addition, using the example of NFT, the author draws attention to the problem of artificial limitations of the mechanism of legal regulation of fundamentally new digital objects. It is determined that, with regard to NFT, today in the Russian Federation, both the legislator and the financial market regulator maintain the status quo: the state intervenes in public relations that develop during the turnover of non-fungible tokens only if transactions involving them violate the law. Meanwhile, it can be expected that eventually the problems of the issue and circulation of NFT in the financial market will receive their regulatory and legal resolution.
6
Huang, Lingbo, Yushi Chen e Xin He. "Spectral-Spatial Mamba for Hyperspectral Image Classification". Remote Sensing 16, n. 13 (3 luglio 2024): 2449. http://dx.doi.org/10.3390/rs16132449.
Testo completoAbstract (sommario):
Recently, transformer has gradually attracted interest for its excellence in modeling the long-range dependencies of spatial-spectral features in HSI. However, transformer has the problem of the quadratic computational complexity due to the self-attention mechanism, which is heavier than other models and thus has limited adoption in HSI processing. Fortunately, the recently emerging state space model-based Mamba shows great computational efficiency while achieving the modeling power of transformers. Therefore, in this paper, we first proposed spectral-spatial Mamba (SS-Mamba) for HSI classification. Specifically, SS-Mamba mainly includes a spectral-spatial token generation module and several stacked spectral-spatial Mamba blocks. Firstly, the token generation module converts any given HSI cube to spatial and spectral tokens as sequences. And then these tokens are sent to stacked spectral-spatial mamba blocks (SS-MB). Each SS-MB includes two basic mamba blocks and a spectral-spatial feature enhancement module. The spatial and spectral tokens are processed separately by the two basic mamba blocks, correspondingly. Moreover, the feature enhancement module modulates spatial and spectral tokens using HSI sample’s center region information. Therefore, the spectral and spatial tokens cooperate with each other and achieve information fusion within each block. The experimental results conducted on widely used HSI datasets reveal that the proposed SS-Mamba requires less processing time compared with transformer. The Mamba-based method thus opens a new window for HSI classification.
7
Liu, Huey-Ing, e Wei-Lin Chen. "X-Transformer: A Machine Translation Model Enhanced by the Self-Attention Mechanism". Applied Sciences 12, n. 9 (29 aprile 2022): 4502. http://dx.doi.org/10.3390/app12094502.
Testo completoAbstract (sommario):
Machine translation has received significant attention in the field of natural language processing not only because of its challenges but also due to the translation needs that arise in the daily life of modern people. In this study, we design a new machine translation model named X-Transformer, which refines the original Transformer model regarding three aspects. First, the model parameter of the encoder is compressed. Second, the encoder structure is modified by adopting two layers of the self-attention mechanism consecutively and reducing the point-wise feed forward layer to help the model understand the semantic structure of sentences precisely. Third, we streamline the decoder model size, while maintaining the accuracy. Through experiments, we demonstrate that having a large number of decoder layers not only affects the performance of the translation model but also increases the inference time. The X-Transformer reaches the state-of-the-art result of 46.63 and 55.63 points in the BiLingual Evaluation Understudy (BLEU) metric of the World Machine Translation (WMT), from 2014, using the English–German and English–French translation corpora, thus outperforming the Transformer model with 19 and 18 BLEU points, respectively. The X-Transformer significantly reduces the training time to only 1/3 times that of the Transformer. In addition, the heat maps of the X-Transformer reach token-level precision (i.e., token-to-token attention), while the Transformer model remains at the sentence level (i.e., token-to-sentence attention).
8
Guo, Chaopeng, Pengyi Zhang, Bangyao Lin e Jie Song. "A Dual Incentive Value-Based Paradigm for Improving the Business Market Profitability in Blockchain Token Economy". Mathematics 10, n. 3 (29 gennaio 2022): 439. http://dx.doi.org/10.3390/math10030439.
Testo completoAbstract (sommario):
Blockchain solves the problem of mutual trust and consensus in the business market of the token economy. In the existing paradigm of blockchain token economy, there are disadvantages of lacking the incentive mechanism, business applications and virtual token value. These shortcomings reduce consumers’ willingness to consume and the profits of the merchants. In this paper, we propose a novel “Dual incentive value-based” paradigm to improve the business market profitability in blockchain token economy. To evaluate our proposed paradigm, we propose a business study case for improving merchants’ environment state. In this case, we set up two economic models and make simulations to validate the profitability. The result shows that merchants with the novel paradigm have 32% more profit compared with those without the paradigm and at most 10% more profitable than those in existing paradigms.
9
Khoo, Ling Min Serena, Hai Leong Chieu, Zhong Qian e Jing Jiang. "Interpretable Rumor Detection in Microblogs by Attending to User Interactions". Proceedings of the AAAI Conference on Artificial Intelligence 34, n. 05 (3 aprile 2020): 8783–90. http://dx.doi.org/10.1609/aaai.v34i05.6405.
Testo completoAbstract (sommario):
We address rumor detection by learning to differentiate between the community's response to real and fake claims in microblogs. Existing state-of-the-art models are based on tree models that model conversational trees. However, in social media, a user posting a reply might be replying to the entire thread rather than to a specific user. We propose a post-level attention model (PLAN) to model long distance interactions between tweets with the multi-head attention mechanism in a transformer network. We investigated variants of this model: (1) a structure aware self-attention model (StA-PLAN) that incorporates tree structure information in the transformer network, and (2) a hierarchical token and post-level attention model (StA-HiTPLAN) that learns a sentence representation with token-level self-attention. To the best of our knowledge, we are the first to evaluate our models on two rumor detection data sets: the PHEME data set as well as the Twitter15 and Twitter16 data sets. We show that our best models outperform current state-of-the-art models for both data sets. Moreover, the attention mechanism allows us to explain rumor detection predictions at both token-level and post-level.
10
Keerthana, R. L., Awadhesh Kumar Singh, Poonam Saini e Diksha Malhotra. "Explaining Sarcasm of Tweets using Attention Mechanism". Scalable Computing: Practice and Experience 24, n. 4 (17 novembre 2023): 787–96. http://dx.doi.org/10.12694/scpe.v24i4.2166.
Testo completoAbstract (sommario):
Emotion identification from text can help boost the effectiveness of sentiment analysis models. Sarcasm is one of the more difficult emotions to detect, particularly in textual data. Even though several models for detecting sarcasm have been presented, their performance falls way short of that of other emotion detection models. As a result, few strategies have been introduced in the paper that helped to enhance the performance of sarcasm detection models. To compare performance, the model was tested using the TweetEval benchmark dataset. On the TweetEval benchmark, the technique proposed in this paper has established a new state-of-the-art. Besides the low performance, interpretability of existing sarcasm detection models are lacking compared to other emotion detection models like hate speech and anger. Therefore, an attention-based interpretability technique has been proposed in this paper that interprets the token importance for a certain decision of sarcasm detection model. The results of the interpretability technique aid in our comprehension of the contextual embeddings of the input tokens that the model has paid the greatest attention to while making a particular decision which outperforms existing transformer-based interpretability techniques, particularly in terms of visualisations.
11
Subramanian, Hemang. "Security tokens: architecture, smart contract applications and illustrations using SAFE". Managerial Finance 46, n. 6 (13 agosto 2019): 735–48. http://dx.doi.org/10.1108/mf-09-2018-0467.
Testo completoAbstract (sommario):
Purpose Blockchain technologies have pervaded modern crowdfunding and capital sourcing through a variety of financial instruments implemented as smart contracts. Smart contracts provide a unique mechanism not only to create a unique one-of-a-type financial instrument, but also to enable unique innovations atop existing financial instruments due to underlying efficiencies. The smartness comes from the flexibility that programs provide which can create extremely unique financial instruments that are often complex to implement, yet easy to create, maintain through versioning, trade and destroy. The purpose of this paper is to describe the security token architecture as an application of smart contracts. Further, the author illustrates the implementation and design of a commonly used financial instrument known as Simple Agreement for Future Equity (SAFE) using the security token architecture proposed and smart contract functionality. The author then models the transaction using relational algebra, and, models the utility maximization. The author shows how on account of reduced information asymmetry between the investors and SAFE users (i.e. startups) utility is positive when smart contract-based security tokens are deployed for each state in the SAFE contract. Design/methodology/approach Using an existing well-adopted instrument called a SAFE contract, the author illustrates the architecture of a smart contract-based security token system. The author illustrates how different components of a SAFE contract can be implemented as a smart contract and discusses the advantages and disadvantages of applying blockchain-based smart contracts to design SAFE instruments. The author deploys two methods: a state space diagram to explain state transitions and a utility model to explain the utilities. Findings The key findings of this research study are the design of a security token architecture, which can be used to convert any the physical or contract-based financial instrument to a smart contract that runs on the blockchain. However, there are limitations to the implementation of the same which can be overcome. The model illustrates the positive utilities derived for all economic actors, i.e. the contractors, the utility providers, etc., in the market. Originality/value This paper is an original paper. For the very first time, the author explored the architecture of a security token system. Using a well-known financial instrument, namely the SAFE, the author describes various components, e.g. the four contracts that form SAFE and then model the utilities for the system.
12
Et. al., M. Pushpalatha,. "Deep Learning Strategy to Recognize Kannada Named Entities". Turkish Journal of Computer and Mathematics Education (TURCOMAT) 12, n. 10 (28 aprile 2021): 5731–37. http://dx.doi.org/10.17762/turcomat.v12i10.5387.
Testo completoAbstract (sommario):
Entity representatives are useful in understanding the natural language tasks including the semantics of the Kannada sentences into various entities. In this paper, we have come up with new pertained tag based representative learning of words and entities based on the bidirectional parsing. The proposed research works on segmenting the sentences of Kannada words into various taken, where every token makes various contributions in understanding the semantics of Kannada Sentences which treats words and entities in a given text as independent tokens, and outputs tagged entities based on representative learning mechanism. The research also has focused its attention towards achieving the results of good classification accuracy while recognizing the entities are through the tagging mechanism that is an extension of the general self-tagging mechanism of the Supervised Machine Learning Technique, and considers the types of tokens (words or entities) when computing attention scores. The erected research work has given its significant contribution in terms of good results over a standard benchmark datasets. In particular, it obtains state-of-the-art results on five well-known datasets: Open Entity (entity typing), TACRED (relation classification), CoNLL-2003 (named entity recognition), ReCoRD (cloze-style question answering), and SQuAD 1.1 (extractive question answering) as well as Kannada Named Entity Recognition of Central Institute of Indian Languages.
13
Fasiku, Ayodeji Ireti, Muhammad Nadzir Bin Marsono, Paulson Eberechukwu Numan, Asrani Lit e Shahrizal Rusli. "Wireless Network On-Chips History-Based Traffic Prediction for Token Flow Control and Allocation". ELEKTRIKA- Journal of Electrical Engineering 18, n. 3 (19 dicembre 2019): 21–26. http://dx.doi.org/10.11113/elektrika.v18n3.162.
Testo completoAbstract (sommario):
Wireless network-on-chip (WiNoC) uses a wireless backbone on top of the traditional wired-based NoC which demonstrated high scalability. WiNoC introduces long-range single-hop link connecting distanced core and high bandwidth radio frequency interconnects that reduces multi-hop communication in conventional wired-based NoC. However, to ensure full benefits of WiNoC technology, there is a need for fair and efficient Medium Access Control (MAC) mechanism to enhance communication in the wireless Network-on-Chip. To adapt to the varying traffic demands from the applications running on a multicore environment, MAC mechanisms should dynamically adjust the transmission slots of the wireless interfaces (WIs), to ensure efficient utilization of the wireless medium in a WiNoC. This work presents a prediction model that improves MAC mechanism to predict the traffic demand of the WIs and respond accordingly by adjusting transmission slots of the WIs. This research aims to reduce token waiting time and inefficient decision making for radio hub-to-hub communication and congestion-aware routing in WiNoC to enhance end to end latency. Through system level simulation, we will show that the dynamic MAC using an History-based prediction mechanism can significantly improve the performance of a WiNoC in terms of latency and network throughput compared to the state-of-the-art dynamic MAC mechanisms.
14
Lu, Hui, Albert Ali Salah e Ronald Poppe. "TCNet: Continuous Sign Language Recognition from Trajectories and Correlated Regions". Proceedings of the AAAI Conference on Artificial Intelligence 38, n. 4 (24 marzo 2024): 3891–99. http://dx.doi.org/10.1609/aaai.v38i4.28181.
Testo completoAbstract (sommario):
A key challenge in continuous sign language recognition (CSLR) is to efficiently capture long-range spatial interactions over time from the video input. To address this challenge, we propose TCNet, a hybrid network that effectively models spatio-temporal information from Trajectories and Correlated regions. TCNet's trajectory module transforms frames into aligned trajectories composed of continuous visual tokens. This facilitates extracting region trajectory patterns. In addition, for a query token, self-attention is learned along the trajectory. As such, our network can also focus on fine-grained spatio-temporal patterns, such as finger movement, of a region in motion. TCNet's correlation module utilizes a novel dynamic attention mechanism that filters out irrelevant frame regions. Additionally, it assigns dynamic key-value tokens from correlated regions to each query. Both innovations significantly reduce the computation cost and memory. We perform experiments on four large-scale datasets: PHOENIX14, PHOENIX14-T, CSL, and CSL-Daily. Our results demonstrate that TCNet consistently achieves state-of-the-art performance. For example, we improve over the previous state-of-the-art by 1.5\% and 1.0\% word error rate on PHOENIX14 and PHOENIX14-T, respectively. Code is available at https://github.com/hotfinda/TCNet
15
Zeng, Daojian, Haoran Zhang e Qianying Liu. "CopyMTL: Copy Mechanism for Joint Extraction of Entities and Relations with Multi-Task Learning". Proceedings of the AAAI Conference on Artificial Intelligence 34, n. 05 (3 aprile 2020): 9507–14. http://dx.doi.org/10.1609/aaai.v34i05.6495.
Testo completoAbstract (sommario):
Joint extraction of entities and relations has received significant attention due to its potential of providing higher performance for both tasks. Among existing methods, CopyRE is effective and novel, which uses a sequence-to-sequence framework and copy mechanism to directly generate the relation triplets. However, it suffers from two fatal problems. The model is extremely weak at differing the head and tail entity, resulting in inaccurate entity extraction. It also cannot predict multi-token entities (e.g. Steven Jobs). To address these problems, we give a detailed analysis of the reasons behind the inaccurate entity extraction problem, and then propose a simple but extremely effective model structure to solve this problem. In addition, we propose a multi-task learning framework equipped with copy mechanism, called CopyMTL, to allow the model to predict multi-token entities. Experiments reveal the problems of CopyRE and show that our model achieves significant improvement over the current state-of-the-art method by 9% in NYT and 16% in WebNLG (F1 score). Our code is available at https://github.com/WindChimeRan/CopyMTL
16
He, Ju, Jie-Neng Chen, Shuai Liu, Adam Kortylewski, Cheng Yang, Yutong Bai e Changhu Wang. "TransFG: A Transformer Architecture for Fine-Grained Recognition". Proceedings of the AAAI Conference on Artificial Intelligence 36, n. 1 (28 giugno 2022): 852–60. http://dx.doi.org/10.1609/aaai.v36i1.19967.
Testo completoAbstract (sommario):
Fine-grained visual classification (FGVC) which aims at recognizing objects from subcategories is a very challenging task due to the inherently subtle inter-class differences. Most existing works mainly tackle this problem by reusing the backbone network to extract features of detected discriminative regions. However, this strategy inevitably complicates the pipeline and pushes the proposed regions to contain most parts of the objects thus fails to locate the really important parts. Recently, vision transformer (ViT) shows its strong performance in the traditional classification task. The self-attention mechanism of the transformer links every patch token to the classification token. In this work, we first evaluate the effectiveness of the ViT framework in the fine-grained recognition setting. Then motivated by the strength of the attention link can be intuitively considered as an indicator of the importance of tokens, we further propose a novel Part Selection Module that can be applied to most of the transformer architectures where we integrate all raw attention weights of the transformer into an attention map for guiding the network to effectively and accurately select discriminative image patches and compute their relations. A contrastive loss is applied to enlarge the distance between feature representations of confusing classes. We name the augmented transformer-based model TransFG and demonstrate the value of it by conducting experiments on five popular fine-grained benchmarks where we achieve state-of-the-art performance. Qualitative results are presented for better understanding of our model.
17
Tian, Jialin, Xing Xu, Fumin Shen, Yang Yang e Heng Tao Shen. "TVT: Three-Way Vision Transformer through Multi-Modal Hypersphere Learning for Zero-Shot Sketch-Based Image Retrieval". Proceedings of the AAAI Conference on Artificial Intelligence 36, n. 2 (28 giugno 2022): 2370–78. http://dx.doi.org/10.1609/aaai.v36i2.20136.
Testo completoAbstract (sommario):
In this paper, we study the zero-shot sketch-based image retrieval (ZS-SBIR) task, which retrieves natural images related to sketch queries from unseen categories. In the literature, convolutional neural networks (CNNs) have become the de-facto standard and they are either trained end-to-end or used to extract pre-trained features for images and sketches. However, CNNs are limited in modeling the global structural information of objects due to the intrinsic locality of convolution operations. To this end, we propose a Transformer-based approach called Three-Way Vision Transformer (TVT) to leverage the ability of Vision Transformer (ViT) to model global contexts due to the global self-attention mechanism. Going beyond simply applying ViT to this task, we propose a token-based strategy of adding fusion and distillation tokens and making them complementary to each other. Specifically, we integrate three ViTs, which are pre-trained on data of each modality, into a three-way pipeline through the processes of distillation and multi-modal hypersphere learning. The distillation process is proposed to supervise fusion ViT (ViT with an extra fusion token) with soft targets from modality-specific ViTs, which prevents fusion ViT from catastrophic forgetting. Furthermore, our method learns a multi-modal hypersphere by performing inter- and intra-modal alignment without loss of uniformity, which aims to bridge the modal gap between modalities of sketch and image and avoid the collapse in dimensions. Extensive experiments on three benchmark datasets, i.e., Sketchy, TU-Berlin, and QuickDraw, demonstrate the superiority of our TVT method over the state-of-the-art ZS-SBIR methods.
18
Bo, Zeng, Yabo Dong, Jie He e Lu Dongming. "An Energy-Efficient One-Shot Scheduling Algorithm for Wireless Sensor Networks". Journal of Sensors 2021 (22 novembre 2021): 1–15. http://dx.doi.org/10.1155/2021/9999403.
Testo completoAbstract (sommario):
In low-load wireless sensor networks, the power consumption of the node consists mainly of two parts: data transmission and node state switching. The lower node workload causes low energy consumption on data transmission, and the state switching energy of the node cannot be ignored. This paper proposes a one-shot time division multiple access (TMDA) scheduling with unlimited channels (SUC) on the assumption that the number of available channels is unlimited. SUC combines the receiver-based consecutive slot allocation with channel allocation, which minimises the number of node state switching and optimizes energy efficiency. Theoretical analysis demonstrates that the number of channels required by SUC does not exceed log 2 N + 1 , where N indicates the number of nodes. Seeing that the number of available wireless channels is limited in practice, the paper proposes the scheduling with limited channels (SLC) and uses a Lookahead Search mechanism to solve slot conflict. For the scalability of the algorithm, a distributed implementation based on the token change is proposed. The algorithm uses the depth-first-search (DFS) to pass the token to all nodes and terminates slot and channel assignment. The simulation results show our algorithm can reduce the energy consumption by minimizing the number of state switching and shorten the data aggregation time by reusing slots among nodes.
19
Ke, Qingtian, e Peng Zhang. "Hybrid-TransCD: A Hybrid Transformer Remote Sensing Image Change Detection Network via Token Aggregation". ISPRS International Journal of Geo-Information 11, n. 4 (17 aprile 2022): 263. http://dx.doi.org/10.3390/ijgi11040263.
Testo completoAbstract (sommario):
Existing optical remote sensing image change detection (CD) methods aim to learn an appropriate discriminate decision by analyzing the feature information of bitemporal images obtained at the same place. However, the complex scenes in high-resolution (HR) remote images cause unsatisfied results, especially for some irregular and occluded objects. Although recent self-attention-driven change detection models with CNN achieve promising effects, the computational and consumed parameters costs emerge as an impassable gap for HR images. In this paper, we utilize a transformer structure replacing self-attention to learn stronger feature representations per image. In addition, concurrent vision transformer models only consider tokenizing single-dimensional image tokens, thus failing to build multi-scale long-range interactions among features. Here, we propose a hybrid multi-scale transformer module for HR remote images change detection, which fully models representation attentions at hybrid scales of each image via a fine-grained self-attention mechanism. The key idea of the hybrid transformer structure is to establish heterogeneous semantic tokens containing multiple receptive fields, thus simultaneously preserving large object and fine-grained features. For building relationships between features without embedding with token sequences from the Siamese tokenizer, we also introduced a hybrid difference transformer decoder (HDTD) layer to further strengthen multi-scale global dependencies of high-level features. Compared to capturing single-stream tokens, our HDTD layer directly focuses representing differential features without increasing exponential computational cost. Finally, we propose a cascade feature decoder (CFD) for aggregating different-dimensional upsampling features by establishing difference skip-connections. To evaluate the effectiveness of the proposed method, experiments on two HR remote sensing CD datasets are conducted. Compared to state-of-the-art methods, our Hybrid-TransCD achieved superior performance on both datasets (i.e., LEVIR-CD, SYSU-CD) with improvements of 0.75% and 1.98%, respectively.
20
Fei, Hao, Donghong Ji, Bobo Li, Yijiang Liu, Yafeng Ren e Fei Li. "Rethinking Boundaries: End-To-End Recognition of Discontinuous Mentions with Pointer Networks". Proceedings of the AAAI Conference on Artificial Intelligence 35, n. 14 (18 maggio 2021): 12785–93. http://dx.doi.org/10.1609/aaai.v35i14.17513.
Testo completoAbstract (sommario):
A majority of research interests in irregular (e.g., nested or discontinuous) named entity recognition (NER) have been paid on nested entities, while discontinuous entities received limited attention. Existing work for discontinuous NER, however, either suffers from decoding ambiguity or predicting using token-level local features. In this work, we present an innovative model for discontinuous NER based on pointer networks, where the pointer simultaneously decides whether a token at each decoding frame constitutes an entity mention and where the next constituent token is. Our model has three major merits compared with previous work: (1) The pointer mechanism is memory-augmented, which enhances the mention boundary detection and interactions between the current decision and prior recognized mentions. (2) The encoder-decoder architecture can linearize the complexity of structure prediction, and thus reduce search costs. (3) The model makes every decision using global information, i.e., by consulting all the input, encoder and previous decoder output in a global view. Experimental results on the CADEC and ShARe13 datasets show that our model outperforms flat and hypergraph models as well as a state-of-the-art transition-based model for discontinuous NER. Further in-depth analysis demonstrates that our model performs well in recognizing various entities including flat, overlapping and discontinuous ones. More crucially, our model is effective on boundary detection, which is the kernel source to NER.
21
Zhou, Qianrui, Hua Xu, Hao Li, Hanlei Zhang, Xiaohan Zhang, Yifan Wang e Kai Gao. "Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition". Proceedings of the AAAI Conference on Artificial Intelligence 38, n. 15 (24 marzo 2024): 17114–22. http://dx.doi.org/10.1609/aaai.v38i15.29656.
Testo completoAbstract (sommario):
Multimodal intent recognition aims to leverage diverse modalities such as expressions, body movements and tone of speech to comprehend user's intent, constituting a critical task for understanding human language and behavior in real-world multimodal scenarios. Nevertheless, the majority of existing methods ignore potential correlations among different modalities and own limitations in effectively learning semantic features from nonverbal modalities. In this paper, we introduce a token-level contrastive learning method with modality-aware prompting (TCL-MAP) to address the above challenges. To establish an optimal multimodal semantic environment for text modality, we develop a modality-aware prompting module (MAP), which effectively aligns and fuses features from text, video and audio modalities with similarity-based modality alignment and cross-modality attention mechanism. Based on the modality-aware prompt and ground truth labels, the proposed token-level contrastive learning framework (TCL) constructs augmented samples and employs NT-Xent loss on the label token. Specifically, TCL capitalizes on the optimal textual semantic insights derived from intent labels to guide the learning processes of other modalities in return. Extensive experiments show that our method achieves remarkable improvements compared to state-of-the-art methods. Additionally, ablation analyses demonstrate the superiority of the modality-aware prompt over the handcrafted prompt, which holds substantial significance for multimodal prompt learning. The codes are released at https://github.com/thuiar/TCL-MAP.
22
Kidambi, Jayakrishna, Dipak Ghosal e Biswanath Mukherjee. "Dynamic token bucket (DTB): a fair bandwidth allocation algorithm for high‐speed networks". Journal of High Speed Networks 9, n. 2 (gennaio 2000): 67–87. https://doi.org/10.3233/hsn-2000-180.
Testo completoAbstract (sommario):
Fair allocation of available bandwidth to competing flows is a simple form of quality of service (QoS) that can be provided to customers in public networks. A number of packet‐scheduling and buffer‐management techniques have been proposed in the literature to achieve this goal efficiently. However, the complexity of the existing algorithms prevents a high‐speed implementation with the current state of router technology. We propose a computationally simpler mechanism based on token‐bucket policing to achieve almost equal bandwidth allocation for a set of competing flows. The proposed method adjusts the token‐bucket threshold dynamically and measures the instantaneous arrival rate of flows. It uses this information to decide whether or not to admit a packet arriving at the network edge. With minor modifications, our framework can be used in a variety of practical network environments ranging from the Internet to virtual private networks (VPNs) over Frame Relay. We present a detailed simulation study that evaluates the performance of our algorithm. The simulation results indicate that DTB is fair, efficient, and robust.
23
Chen, Zewei, Hang Lei, Maolin Yang, Yong Liao e Lei Qiao. "A Hierarchical Hybrid Locking Protocol for Parallel Real-Time Tasks". ACM Transactions on Embedded Computing Systems 20, n. 5s (31 ottobre 2021): 1–22. http://dx.doi.org/10.1145/3477017.
Testo completoAbstract (sommario):
Parallel tasks have been paid growing attention in recent years, and the scheduling with shared resources is of significant importance to real-time systems. As an efficient mechanism to provide mutual exclusion for parallel processing, spin-locks are ubiquitous in multi-processor real-time systems. However, the spin-locks suffer the scalability problem, and the intra-task parallelism further exacerbates the analytical pessimism. To overcome such deficiencies, we propose a Hierarchical Hybrid Locking Protocol (H2LP) under federated scheduling. The proposed H2LP integrates the classical Multiprocessor Stack Resource Policy (MSRP) and uses a token mechanism to reduce global contentions. We provide a complete analysis framework supporting both heavy and light tasks under federated scheduling and develop a blocking analysis with the state-of-the-art linear optimization technique. Empirical evaluations showed that the H2LP outperformed the other state-of-the-art locking protocols in at least configurations when considering exclusive clustering. Furthermore, our partitioned approach for light tasks can substantially improve schedulability by mitigating the over-provisioning problem.
24
Pakhomov, Valeriy Nikolaevich. "Blockchain as a technological means of ensuring copyright protection of the results of intellectual activity". Право и политика, n. 3 (marzo 2025): 90–99. https://doi.org/10.7256/2454-0706.2025.3.71379.
Testo completoAbstract (sommario):
The subject of the article is the legal forms of using blockchain as an independent technology that ensures the protection of originality and confirmation of authorship in relation to intellectual property objects. The article reveals the current areas of blockchain use, within which this technology allows solving traditional problems of information security and identification of copyright objects. As part of the improvement of the mechanism of private law regulation in this area, it is proposed to use blockchain to create public registers of copyright objects, which will contain information about the created work, the presence of legal disputes in relation to these works, as well as other information that will reflect the main characteristics of the work as an object enjoying copyright protection. This requires the development of legislative initiatives that form unified state standards for the placement of information about copyright objects in a distributed database. Based on the use of a systematic approach and formal legal analysis, the article discusses specific ways to introduce blockchain technology into the mechanism of copyright protection. Blockchain technology can be used to create public registers of copyright objects, which will contain information about the created work, the existence of legal disputes in relation to these works, as well as other information that will reflect the main characteristics of the work as an object enjoying copyright protection. The creation of such a registry is possible in both single and multiple versions. If there are several registers of copyright objects, it is advisable to create mechanisms that exclude the possibility of duplication of the same work. Blockchain technology allows to confirm the authenticity and uniqueness of a copy of a work, but does not provide the buyer of the token with automatic rights to use the work outside the framework established by the copyright holder. Thus, in order to strengthen trust and protect the rights of buyers of NFT tokens, comprehensive solutions aimed at verification and confirmation of authorship are needed. In order to ensure to users that the issue of the NFT token is actually carried out by the author, there are several solutions. One of the most reliable methods is to use the services of a notary, who can officially confirm the creation of a work of art or any other object presented in the form of an NFT by its author.
25
Jia, Xiaodong, e Gang Tan. "V-Star: Learning Visibly Pushdown Grammars from Program Inputs". Proceedings of the ACM on Programming Languages 8, PLDI (20 giugno 2024): 2003–26. http://dx.doi.org/10.1145/3656458.
Testo completoAbstract (sommario):
Accurate description of program inputs remains a critical challenge in the field of programming languages. Active learning, as a well-established field, achieves exact learning for regular languages. We offer an innovative grammar inference tool, V-Star, based on the active learning of visibly pushdown automata. V-Star deduces nesting structures of program input languages from sample inputs, employing a novel inference mechanism based on nested patterns. This mechanism identifies token boundaries and converts languages such as XML documents into VPLs. We then adapted Angluin's L-Star, an exact learning algorithm, for VPA learning, which improves the precision of our tool. Our evaluation demonstrates that V-Star effectively and efficiently learns a variety of practical grammars, including S-Expressions, JSON, and XML, and outperforms other state-of-the-art tools.
26
Xie, Tianming, Zhonghao Zhang, Jing Tian e Lihong Ma. "Focal DETR: Target-Aware Token Design for Transformer-Based Object Detection". Sensors 22, n. 22 (10 novembre 2022): 8686. http://dx.doi.org/10.3390/s22228686.
Testo completoAbstract (sommario):
In this paper, we propose a novel target-aware token design for transformer-based object detection. To tackle the target attribute diffusion challenge of transformer-based object detection, we propose two key components in the new target-aware token design mechanism. Firstly, we propose a target-aware sampling module, which forces the sampling patterns to converge inside the target region and obtain its representative encoded features. More specifically, a set of four sampling patterns are designed, including small and large patterns, which focus on the detailed and overall characteristics of a target, respectively, as well as the vertical and horizontal patterns, which handle the object’s directional structures. Secondly, we propose a target-aware key-value matrix. This is a unified, learnable, feature-embedding matrix which is directly weighted on the feature map to reduce the interference of non-target regions. With such a new design, we propose a new variant of the transformer-based object-detection model, called Focal DETR, which achieves superior performance over the state-of-the-art transformer-based object-detection models on the COCO object-detection benchmark dataset. Experimental results demonstrate that our Focal DETR achieves a 44.7 AP in the coco2017 test set, which is 2.7 AP and 0.9 AP higher than the DETR and deformable DETR using the same training strategy and the same feature-extraction network.
27
Tang, Chuanxin, Yucheng Zhao, Guangting Wang, Chong Luo, Wenxuan Xie e Wenjun Zeng. "Sparse MLP for Image Recognition: Is Self-Attention Really Necessary?" Proceedings of the AAAI Conference on Artificial Intelligence 36, n. 2 (28 giugno 2022): 2344–51. http://dx.doi.org/10.1609/aaai.v36i2.20133.
Testo completoAbstract (sommario):
Transformers have sprung up in the field of computer vision. In this work, we explore whether the core self-attention module in Transformer is the key to achieving excellent performance in image recognition. To this end, we build an attention-free network called sMLPNet based on the existing MLP-based vision models. Specifically, we replace the MLP module in the token-mixing step with a novel sparse MLP (sMLP) module. For 2D image tokens, sMLP applies 1D MLP along the axial directions and the parameters are shared among rows or columns. By sparse connection and weight sharing, sMLP module significantly reduces the number of model parameters and computational complexity, avoiding the common over-fitting problem that plagues the performance of MLP-like models. When only trained on the ImageNet-1K dataset, the proposed sMLPNet achieves 81.9% top-1 accuracy with only 24M parameters, which is much better than most CNNs and vision Transformers under the same model size constraint. When scaling up to 66M parameters, sMLPNet achieves 83.4% top-1 accuracy, which is on par with the state-of-the-art Swin Transformer. The success of sMLPNet suggests that the self-attention mechanism is not necessarily a silver bullet in computer vision. The code and models are publicly available at https://github.com/microsoft/SPACH.
28
MOHAMAD, ISTI ANJELINA, ERMAN I. RAHIM e ABDUL HAMID TOME. "REKONSTRUKSI PENGISIAN JABATAN KEMENTERIAN NEGARA DI INDONESIA MELALUI PERBANDINGAN DI NEGARA-NEGARA LAIN". GANEC SWARA 18, n. 2 (6 giugno 2024): 624. http://dx.doi.org/10.35327/gara.v18i2.839.
Testo completoAbstract (sommario):
The elected president appoints state ministers as a token of appreciation to the supporting coalition parties. The total percentage of post-reform for appointing state ministers from political party caolitions is 55,7%. In addition, the Indonesian Survey Institute (LSI) denotes that state ministers from professionals by 78,3% and from political parties or coalitions of political parties by 4.1%. specifically, this research was conducted using normative methods with a statue approach and a comparative approach. The research finding was obtained by comparing two parliamentary systems, namely the Netherlands and England, and two presidential systems, namely the United States and South Korea. The appointment of state ministers in Indonesia post reform until now has main characteristics apparent in the appointment proves, namely the interest in coalition parties that have supported the party of the current elected president. Therefore, the ideal mechanism for appointing state ministers is to reconstruct Act Number 39 of 2008 concerning State Ministers, establish a special selection, team for ministerial appointments, the selection team proceeds with a sequence of selection, selection based on quality and integrity, and the appointment of state minister candidate by the president.
29
Liang, Ke, Sifan Wu e Jiayi Gu. "MKA: A Scalable Medical Knowledge-Assisted Mechanism for Generative Models on Medical Conversation Tasks". Computational and Mathematical Methods in Medicine 2021 (23 dicembre 2021): 1–10. http://dx.doi.org/10.1155/2021/5294627.
Testo completoAbstract (sommario):
Using natural language processing (NLP) technologies to develop medical chatbots makes the diagnosis of the patient more convenient and efficient, which is a typical application in healthcare AI. Because of its importance, lots of researches have come out. Recently, the neural generative models have shown their impressive ability as the core of chatbot, while it cannot scale well when directly applied to medical conversation due to the lack of medical-specific knowledge. To address the limitation, a scalable medical knowledge-assisted mechanism (MKA) is proposed in this paper. The mechanism is aimed at assisting general neural generative models to achieve better performance on the medical conversation task. The medical-specific knowledge graph is designed within the mechanism, which contains 6 types of medical-related information, including department, drug, check, symptom, disease, and food. Besides, the specific token concatenation policy is defined to effectively inject medical information into the input data. Evaluation of our method is carried out on two typical medical datasets, MedDG and MedDialog-CN. The evaluation results demonstrate that models combined with our mechanism outperform original methods in multiple automatic evaluation metrics. Besides, MKA-BERT-GPT achieves state-of-the-art performance.
30
Zhu, Qi, Chenyu Gao, Peng Wang e Qi Wu. "Simple is not Easy: A Simple Strong Baseline for TextVQA and TextCaps". Proceedings of the AAAI Conference on Artificial Intelligence 35, n. 4 (18 maggio 2021): 3608–15. http://dx.doi.org/10.1609/aaai.v35i4.16476.
Testo completoAbstract (sommario):
Texts appearing in daily scenes that can be recognized by OCR (Optical Character Recognition) tools contain significant information, such as street name, product brand and prices. Two tasks -- text-based visual question answering and text-based image captioning, with a text extension from existing vision-language applications, are catching on rapidly. To address these problems, many sophisticated multi-modality encoding frameworks (such as heterogeneous graph structure) are being used. In this paper, we argue that a simple attention mechanism can do the same or even better job without any bells and whistles. Under this mechanism, we simply split OCR token features into separate visual- and linguistic-attention branches, and send them to a popular Transformer decoder to generate answers or captions. Surprisingly, we find this simple baseline model is rather strong -- it consistently outperforms state-of-the-art (SOTA) models on two popular benchmarks, TextVQA and all three tasks of ST-VQA, although these SOTA models use far more complex encoding mechanisms. Transferring it to text-based image captioning, we also surpass the TextCaps Challenge 2020 winner. We wish this work to set the new baseline for these two OCR text related applications and to inspire new thinking of multi-modality encoder design. Code is available at https://github.com/ZephyrZhuQi/ssbaseline
31
Khafidin, Ahmad, Tatyantoro Andrasto e Suryono Suryono. "Implementation flow control to improve quality of service on computer networks". Indonesian Journal of Electrical Engineering and Computer Science 16, n. 3 (1 dicembre 2019): 1474. http://dx.doi.org/10.11591/ijeecs.v16.i3.pp1474-1481.
Testo completoAbstract (sommario):
<p>Quality of Service (QoS) is the collective effect of service performances, which determine the degree of satisfaction of a user of the service. In addition, QoS defined as the ability of a network to provide good service. QoS aims to provide different quality of services for various needs in the IP network. QoS parameters that can be used to analyze the data communication services are jitter, packet loss, throughput, and delay. The quality of QoS parameters in the network is affected by congestion. Congestion occurs because there is an excessive queue in the network. Congestion can be prevented by implementing flow control on network. Flow control is a method to control the data packet flow in a network. By controlling of the data packet flow, it can improve of QoS. This study intends to find out value of QoS on the internet network at Faculty Engineering, State University of Semarang by measuring network performance using QoS parameters. Then, in this research will be implemented the token bucket method as a flow control mechanism at the network to improve the QoS. After research and data analysis, internet network at Faculty Engineering State University of Semarang has QoS value was 3,5 with 87,5 % of percentage and classified in satisfying of category. When measuring the network performance, there are decreases of performance at access point that having data rates 150 Mbps with many users connected. It has 9,0 ms of delay value, 0.046 ms of jitter, 16,6% of packet loss and, 1293407 bps of throughput. After token bucket was applied as flow control mechanism that be simulated on Graphical Network Simulator 3, the internet network has QoS values 3,75 with 93,75 % of percentage and classified as “satisfying” category. Furthermore, the percentage of the throughput value obtained on network by implementing flow control is 62%, while on the existing network is 41%.</p>
32
Sitnik, Aleksandr Aleksandrovich. "Juridical facts within the mechanism of legal regulation of public relations in the sphere of money circulation". Финансы и управление, n. 1 (gennaio 2020): 13–22. http://dx.doi.org/10.25136/2409-7802.2020.1.32063.
Testo completoAbstract (sommario):
The research explores public relations emerging in the process of money issuance, circulation of cash, credit and electronic currency, accounting and reporting on operations involving finances, currency regulation, organization of national payment system, as well as financial control. The financial control includes control of the solvency of token money; control of adherence to the order of cash transactions and cash register operations; control and oversight of adherence to requirements towards check out equipment; monitoring and supervision within the national payment system; currency control; control in the sphere of counteraction of money laundering, and various forms of financing of terrorism. The scientific novelty consists in the fact that based on the general theoretic positions on juridical facts, the author formulates a concept of juridical facts, which bring forth emergence, changes, and termination of financial legal relations in the sphere of money circulation. The work delineates financial operations from civil law dealings, and the conclusion is made on the possibility of examining financial reform as a relative event. The author highlights the juridical facts of the event and juridical facts of the state.
33
Mohapatra, Nilamadhaba, Namrata Sarraf e Swapna sarit Sahu. "Domain based Chunking". International Journal on Natural Language Computing 10, n. 04 (30 agosto 2021): 1–14. http://dx.doi.org/10.5121/ijnlc.2021.10401.
Testo completoAbstract (sommario):
Chunking means splitting the sentences into tokens and then grouping them in a meaningful way. When it comes to high-performance chunking systems, transformer models have proved to be the state of the art benchmarks. To perform chunking as a task it requires a large-scale high quality annotated corpus where each token is attached with a particular tag similar as that of Named Entity Recognition Tasks. Later these tags are used in conjunction with pointer frameworks to find the final chunk. To solve this for a specific domain problem, it becomes a highly costly affair in terms of time and resources to manually annotate and produce a large-high-quality training set. When the domain is specific and diverse, then cold starting becomes even more difficult because of the expected large number of manually annotated queries to cover all aspects. To overcome the problem, we applied a grammar-based text generation mechanism where instead of annotating a sentence we annotate using grammar templates. We defined various templates corresponding to different grammar rules. To create a sentence we used these templates along with the rules where symbol or terminal values were chosen from the domain data catalog. It helped us to create a large number of annotated queries. These annotated queries were used for training the machine learning model using an ensemble transformer-based deep neural network model [24.] We found that grammar-based annotation was useful to solve domain-based chunks in input query sentences without any manual annotation where it was found to achieve a classification F1 score of 96.97% in classifying the tokens for the out of template queries.
34
Buoy, Rina, Nguonly Taing, Sovisal Chenda e Sokchea Kor. "Khmer printed character recognition using attention-based Seq2Seq network". HO CHI MINH CITY OPEN UNIVERSITY JOURNAL OF SCIENCE - ENGINEERING AND TECHNOLOGY 12, n. 1 (20 aprile 2022): 3–16. http://dx.doi.org/10.46223/hcmcoujs.tech.en.12.1.2217.2022.
Testo completoAbstract (sommario):
This paper presents an end-to-end deep convolutional recurrent neural network solution for Khmer optical character recognition (OCR) task. The proposed solution uses a sequence-to-sequence (Seq2Seq) architecture with attention mechanism. The encoder extracts visual features from an input text-line image via layers of convolutional blocks and a layer of gated recurrent units (GRU). The features are encoded in a single context vector and a sequence of hidden states which are fed to the decoder for decoding one character at a time until a special end-of-sentence (EOS) token is reached. The attention mechanism allows the decoder network to adaptively select relevant parts of the input image while predicting a target character. The Seq2Seq Khmer OCR network is trained on a large collection of computer-generated text-line images for multiple common Khmer fonts. Complex data augmentation is applied on both train and validation dataset. The proposed model’s performance outperforms the state-of-art Tesseract OCR engine for Khmer language on the validation set of 6400 augmented images by achieving a character error rate (CER) of 0.7% vs 35.9%.
35
Ren, Pengjie, Zhumin Chen, Christof Monz, Jun Ma e Maarten De Rijke. "Thinking Globally, Acting Locally: Distantly Supervised Global-to-Local Knowledge Selection for Background Based Conversation". Proceedings of the AAAI Conference on Artificial Intelligence 34, n. 05 (3 aprile 2020): 8697–704. http://dx.doi.org/10.1609/aaai.v34i05.6395.
Testo completoAbstract (sommario):
Background Based Conversation (BBCs) have been introduced to help conversational systems avoid generating overly generic responses. In a BBC, the conversation is grounded in a knowledge source. A key challenge in BBCs is Knowledge Selection (KS): given a conversational context, try to find the appropriate background knowledge (a text fragment containing related facts or comments, etc.) based on which to generate the next response. Previous work addresses KS by employing attention and/or pointer mechanisms. These mechanisms use a local perspective, i.e., they select a token at a time based solely on the current decoding state. We argue for the adoption of a global perspective, i.e., pre-selecting some text fragments from the background knowledge that could help determine the topic of the next response. We enhance KS in BBCs by introducing a Global-to-Local Knowledge Selection (GLKS) mechanism. Given a conversational context and background knowledge, we first learn a topic transition vector to encode the most likely text fragments to be used in the next response, which is then used to guide the local KS at each decoding timestamp. In order to effectively learn the topic transition vector, we propose a distantly supervised learning schema. Experimental results show that the GLKS model significantly outperforms state-of-the-art methods in terms of both automatic and human evaluation. More importantly, GLKS achieves this without requiring any extra annotations, which demonstrates its high degree of scalability.
36
Luo, Ying, Fengshun Xiao e Hai Zhao. "Hierarchical Contextualized Representation for Named Entity Recognition". Proceedings of the AAAI Conference on Artificial Intelligence 34, n. 05 (3 aprile 2020): 8441–48. http://dx.doi.org/10.1609/aaai.v34i05.6363.
Testo completoAbstract (sommario):
Named entity recognition (NER) models are typically based on the architecture of Bi-directional LSTM (BiLSTM). The constraints of sequential nature and the modeling of single input prevent the full utilization of global information from larger scope, not only in the entire sentence, but also in the entire document (dataset). In this paper, we address these two deficiencies and propose a model augmented with hierarchical contextualized representation: sentence-level representation and document-level representation. In sentence-level, we take different contributions of words in a single sentence into consideration to enhance the sentence representation learned from an independent BiLSTM via label embedding attention mechanism. In document-level, the key-value memory network is adopted to record the document-aware information for each unique word which is sensitive to similarity of context information. Our two-level hierarchical contextualized representations are fused with each input token embedding and corresponding hidden state of BiLSTM, respectively. The experimental results on three benchmark NER datasets (CoNLL-2003 and Ontonotes 5.0 English datasets, CoNLL-2002 Spanish dataset) show that we establish new state-of-the-art results.
37
KADDOURI, Amar, e Nawel BOUAZIZ. "The Guarantee Fund FGAR As A Public Mechanism To Promote Entrepreneurship Beyond The Hydrocarbon Sector". Journal of Finance & Corporate Governance 1, n. 1 (30 giugno 2017): 16–30. http://dx.doi.org/10.54960/jfcg.v1i1.2.
Testo completoAbstract (sommario):
The promotion of non-hydrocarbon investments remains the main occupation of the Algerian government. The Algerian economy's dependence on hydrocarbons sector remains with 95% of export earnings, despite all the efforts and attempts token by the Algerian state. Over the last few decades, several measures have been adopted to encourage entrepreneurship and boost investment, by lessening the constraints that face the entrepreneur, in particular, the access of finance. To alleviate this problem, the Algerian government has created, in 20014, the Guarantee Fund for Loans to SMEs (FGAR). It provides guarantees on loans to borrowers by covering a share of the default risk. The aim of this paper is to evaluate the performance of this fund, as a guarantee provider, by analyzing its data and examining its effect on the level of the development of entrepreneurship in Algeria since its creation. The main result is that this guarantee fund has a positive tangible impact on the development of entrepreneurship. Though it has grown in terms of number, entrepreneurship still face various challenges and issues in both enterprise and environment levels that undermine their business conduct. Therefore, much effort is needed in the future to promote and boost the entrepreneurship in order to benefit from all the advantages it gives.
38
Wang, Yikai, Xin Yin, Xianggen Yin, Jian Qiao e Liming Tan. "A Petri Net-Based Power Supply Recovery Strategy for the Electric Power System of Floating Nuclear Power Plant". Applied Sciences 12, n. 18 (8 settembre 2022): 9026. http://dx.doi.org/10.3390/app12189026.
Testo completoAbstract (sommario):
Floating nuclear power plants contain sensitive loads of nuclear reactors. After equipment faults, fast and efficient power supply recovery should be realized. To realize the unified analysis of system topology and power flow distribution, a power supply recovery strategy based on Petri nets is proposed. Considering that systems of different voltage levels cannot be connected instantaneously, a two-stage power supply recovery mode is adopted. Emergency power supply is put in first, and then the whole network is reconstructed. In the network reconstruction process, load transfer is realized through switching the transformation to redistribute the load of each switchboard and adjust the power output of each power source. Corresponding to the Petri net model, the above process is similar to the dynamic transmission process of a token in each library by firing the transition. Therefore, the topological model of system is constructed based on the Petri net, and a power flow analysis is proposed through its dynamic updating mechanism. The objective function of the network reconstruction is established by integrating load recovery amount, switch operation cost and generator operation efficiency, and the optimal switching state combination scheme that satisfies the system constraints is obtained by the multi-population genetic algorithm (MPGA). Simulation results show that the proposed method can provide complete power supply recovery.
39
Chen, Lin, Zhijie Jia, Lechao Cheng, Yang Gao, Jie Lei, Yijun Bei e Zunlei Feng. "ViT-Calibrator: Decision Stream Calibration for Vision Transformer". Proceedings of the AAAI Conference on Artificial Intelligence 38, n. 2 (24 marzo 2024): 1147–55. http://dx.doi.org/10.1609/aaai.v38i2.27876.
Testo completoAbstract (sommario):
A surge of interest has emerged in utilizing Transformers in diverse vision tasks owing to its formidable performance. However, existing approaches primarily focus on optimizing internal model architecture designs that often entail significant trial and error with high burdens. In this work, we propose a new paradigm dubbed Decision Stream Calibration that boosts the performance of general Vision Transformers. To achieve this, we shed light on the information propagation mechanism in the learning procedure by exploring the correlation between different tokens and the relevance coefficient of multiple dimensions. Upon further analysis, it was discovered that 1) the final decision is associated with tokens of foreground targets, while token features of foreground target will be transmitted into the next layer as much as possible, and the useless token features of background area will be eliminated gradually in the forward propagation. 2) Each category is solely associated with specific sparse dimensions in the tokens. Based on the discoveries mentioned above, we designed a two-stage calibration scheme, namely ViT-Calibrator, including token propagation calibration stage and dimension propagation calibration stage. Extensive experiments on commonly used datasets show that the proposed approach can achieve promising results.
40
Muyambo, Edmore, e Stacey O. Baror. "Digital Forensic Readiness Model for Internet Voting". European Conference on Cyber Warfare and Security 22, n. 1 (19 giugno 2023): 657–67. http://dx.doi.org/10.34190/eccws.22.1.1186.
Testo completoAbstract (sommario):
Voting is an exercise of choosing a preferred candidate through a process called an election. In many countries, this exercise is a basic human right. In every election process, there are some pre-requisite processes and procedures which must be set up first. These are essential in the pre-vote-casting stage, during vote-casting and post-vote-casting stage. Electoral disagreements amongst stakeholders and parties of interest are usually experienced in each of the above-mentioned voting process stages. The main points of conflict in an election process are vote rigging and vote fraud. Failure to amicably mitigate these issues can result in a criticised/rejected election result. Therefore, this research aims to address the problem of vote rigging and vote fraud allegations in an election process. The resolution thereof is achieved through the introduction of an online based voting system which is supported by a digital forensic readiness mechanism. Online voting system gives citizens the flexibility to use internet-enabled devices such as cell phones and laptops to cast their votes in a safe, secrete and secure protocol. To address the problem of vote rigging and vote fraud, the online voting system is integrated with cyber security and vote protection mechanisms. The cyber security and vote protection mechanism is based on Blockchain algorithms. A Blockchain-based voting process is a peer-to-peer mechanism where a decentralised database is used to store data. Tokens move directly from one peer (voter) to another peer (candidate). The results are tallied by counting the number of tokens paid to each candidate. Each voter is allocated a Bitcoin token and each candidate is allocated a Bitcoin address. During vote casting, the voter transfers their Bitcoin token into the wallet of a registered candidate. At the end of the voting process, the total number of Bitcoin tokens transferred to each candidate is counted and tallied up. The wallet is loaded with only one Bitcoin token, hence there is no possibility of double voting. The model ensures vote security, anonymity, auditability, accountability, accuracy and uniqueness.
41
Yu, Junxiao, Zhengyuan Xu, Xu He, Jian Wang, Bin Liu, Rui Feng, Songsheng Zhu, Wei Wang e Jianqing Li. "DIA-TTS: Deep-Inherited Attention-Based Text-to-Speech Synthesizer". Entropy 25, n. 1 (26 dicembre 2022): 41. http://dx.doi.org/10.3390/e25010041.
Testo completoAbstract (sommario):
Text-to-speech (TTS) synthesizers have been widely used as a vital assistive tool in various fields. Traditional sequence-to-sequence (seq2seq) TTS such as Tacotron2 uses a single soft attention mechanism for encoder and decoder alignment tasks, which is the biggest shortcoming that incorrectly or repeatedly generates words when dealing with long sentences. It may also generate sentences with run-on and wrong breaks regardless of punctuation marks, which causes the synthesized waveform to lack emotion and sound unnatural. In this paper, we propose an end-to-end neural generative TTS model that is based on the deep-inherited attention (DIA) mechanism along with an adjustable local-sensitive factor (LSF). The inheritance mechanism allows multiple iterations of the DIA by sharing the same training parameter, which tightens the token–frame correlation, as well as fastens the alignment process. In addition, LSF is adopted to enhance the context connection by expanding the DIA concentration region. In addition, a multi-RNN block is used in the decoder for better acoustic feature extraction and generation. Hidden-state information driven from the multi-RNN layers is utilized for attention alignment. The collaborative work of the DIA and multi-RNN layers contributes to outperformance in the high-quality prediction of the phrase breaks of the synthesized speech. We used WaveGlow as a vocoder for real-time, human-like audio synthesis. Human subjective experiments show that the DIA-TTS achieved a mean opinion score (MOS) of 4.48 in terms of naturalness. Ablation studies further prove the superiority of the DIA mechanism for the enhancement of phrase breaks and attention robustness.
42
Ramtohul, Avinash, e K. M. S. Soyjaudah. "Information security governance for e-services in southern African developing countries e-Government projects". Journal of Science & Technology Policy Management 7, n. 1 (7 marzo 2016): 26–42. http://dx.doi.org/10.1108/jstpm-04-2014-0014.
Testo completoAbstract (sommario):
Purpose – Highly sensitive information pertaining to citizens and government transactions is processed in an electronic format, making information security a critical part of e-Government applications and architectures. Information security measures should ideally span from authentication to authorisation and from logical/physical access control to auditing of electronic transactions and log books. The lack of such measures compromises confidentiality, integrity and availability of information. Today, most e-Government projects in developing countries in Southern Africa Developing Community (SADC) face challenges in two main areas, namely, information security and application software integration. This paper aims to discuss and analyse the information security requirements for e-Government projects and proposes an information security governance model for service-based architectures (SBAs). Design/methodology/approach – The current state of information security in emerging economies in SADC countries was researched. The main problems identified were the lack of software integration and information security governance, policy and administration. The design consists of three basic layers: information security governance defined at the strategic level of the government; information security policy/management defined at the management/operational level; and information security measures, implemented at the technical level. This section also proposes a policy for implementing public key infrastructures to protect information, transactions and e-services. A Token-Ring-based mechanism for implementing Single-Sign-On has also been developed as part of this study. Findings – The main problems identified were the lack of software integration and information security governance, policy and administration. These challenges are causing e-government projects to stagnate. Practical implications – The proposed approach for implementing information security in e-Government systems will ensure a holistic approach to ensuring confidentiality, integrity and non-repudiation, allowing e-Government maturity to progress from “interaction” to “online transaction” stage in emerging economies. Originality/value – Research has not focused on developing a solution for emerging economies which are facing difficulties in integration software applications to deploy end-to-end e-services and to produce an underlying identity management architecture and information security governance to secure the e-services developed and deployed using an SBA. The work produced in this paper is specific to SBAs in e-government environments where legacy systems already exist. The work includes: information security governance defined at the strategic level of the government; information security policy/management defined at the management/operational level; and information security measures implemented at the technical level. This section also proposes a policy for implementing public key infrastructures to protect information, transactions and e-services. A Token-Ring-based mechanism for implementing Single-Sign-On has also been developed as part of this study.
43
Xu, Yifei, Jingqiao Zhang, Ru He, Liangzhu Ge, Chao Yang, Cheng Yang e Ying Nian Wu. "SAS: Self-Augmentation Strategy for Language Model Pre-training". Proceedings of the AAAI Conference on Artificial Intelligence 36, n. 10 (28 giugno 2022): 11586–94. http://dx.doi.org/10.1609/aaai.v36i10.21412.
Testo completoAbstract (sommario):
The core of self-supervised learning for pre-training language models includes pre-training task design as well as appropriate data augmentation. Most data augmentations in language model pre-training are context-independent. A seminal contextualized augmentation was recently proposed in ELECTRA and achieved state-of-the-art performance by introducing an auxiliary generation network (generator) to produce contextualized data augmentation for the training of a main discrimination network (discriminator). This design, however, introduces extra computation cost of the generator and a need to adjust the relative capability between the generator and the discriminator. In this paper, we propose a self-augmentation strategy (SAS) where a single network is utilized for both regular pre-training and contextualized data augmentation for the training in later epochs. Essentially, this strategy eliminates a separate generator and uses the single network to jointly conduct two pre-training tasks with MLM (Masked Language Modeling) and RTD (Replaced Token Detection) heads. It avoids the challenge to search for an appropriate size of the generator, which is critical to the performance as evidenced in ELECTRA and its subsequent variant models. In addition, SAS is a general strategy that can be seamlessly combined with many new techniques emerging recently or in the future, such as the disentangled attention mechanism from DeBERTa. Our experiments show that SAS is able to outperform ELECTRA and other state-of-the-art models in the GLUE tasks with similar or less computation cost.
44
Li, Jinghui, Feng Shao, Qiang Liu e Xiangchao Meng. "Global-Local Collaborative Learning Network for Optical Remote Sensing Image Change Detection". Remote Sensing 16, n. 13 (27 giugno 2024): 2341. http://dx.doi.org/10.3390/rs16132341.
Testo completoAbstract (sommario):
Due to the widespread applications of change detection technology in urban change analysis, environmental monitoring, agricultural surveillance, disaster detection, and other domains, the task of change detection has become one of the primary applications of Earth orbit satellite remote sensing data. However, the analysis of dual-temporal change detection (CD) remains a challenge in high-resolution optical remote sensing images due to the complexities in remote sensing images, such as intricate textures, seasonal variations in imaging time, climatic differences, and significant differences in the sizes of various objects. In this paper, we propose a novel U-shaped architecture for change detection. In the encoding stage, a multi-branch feature extraction module is employed by combining CNN and transformer networks to enhance the network’s perception capability for objects of varying sizes. Furthermore, a multi-branch aggregation module is utilized to aggregate features from different branches, providing the network with global attention while preserving detailed information. For dual-temporal features, we introduce a spatiotemporal discrepancy perception module to model the context of dual-temporal images. Particularly noteworthy is the construction of channel attention and token attention modules based on the transformer attention mechanism to facilitate information interaction between multi-level features, thereby enhancing the network’s contextual awareness. The effectiveness of the proposed network is validated on three public datasets, demonstrating its superior performance over other state-of-the-art methods through qualitative and quantitative experiments.
45
Skowroński, Rafał, e Jerzy Brzeziński. "UI dApps Meet Decentralized Operating Systems". Electronics 11, n. 19 (22 settembre 2022): 3004. http://dx.doi.org/10.3390/electronics11193004.
Testo completoAbstract (sommario):
The advent of Ethereum opened up a pandora box of decentralized possibilities. While allowing for the replicated, decentralized computation of Turing-complete instructions, platforms such as Ethereum do not offer the possibility of direct, interactive, real-time processing of users’ inputs that could later affect the decentralized state machine. They cannot directly observe, replicate and authenticate users’ actions performed in real-time while presenting the results of these. They lack mechanics that would incentivize full-nodes to provide low-latency-constrained services to users in-between epochs of a decentralized state machine, thus pushing dApps’ developers towards hybrid architectures—ones employing centralized servers while not even considering certain applications, due to the aforementioned limitations. In this research paper, we explore our results of an attempt to create a ‘decentralized operating system’ user experience a reality. We propose an architecture which solves the problems of the responsiveness and finalization of multiple actions performed by users in real-time—without the need for users to pre-authenticate but after having presented a single, unitary consent to commit—through the hereby proposed Deferred Authentication mechanism. To allow for this, we employ an in-house developed #GridScript programming language, used by our decentralized state machine, along with a computer-vision-enabled and AI-aided mobile app (available for both iOS and Android). We introduce the concept of Decentralized Processing Threads (DPTs) and see how these enable fascinating possibilities. In addition, we look into how Access-Control-Lists (ACLs)-enabled, incentivized storage, incentivized Sybil-proof communication, embedded firewall apparatus, integrated off-the-chain payments, and crypto-incentivized off-the-chain storage aid such a system and thus render it as feasible. We highlight various interesting troubles we have encountered, such as state recovery after disconnects of the UI and the replication of its state across both nodes maintaining the network and web browsers. We depict ‘off-the-chain’ mechanics, which we use to reward for real-time services provided to users by nodes maintaining the network. We tackle crypto-incentivized WebRTC swarms not needing centralized servers for signaling. We look into a user-friendly approach to Non-Fungible Tokens (NFTs). The test-bed is readily available with multiple functional UI dApps already in place. Indeed, the paper presents UI and UX design decisions we have undertaken based on conclusions from statistical research results on a group of 50,341 volunteers over 4 years, which we have used to formulate what we codenamed as the Venice UI/UX design paradigm. We extend upon the notion of Token Pools to allow for the Sybil-proof incentivization of multiple-peers from a single data structure stored on the decentralized state machine.
46
Zhang, Haiyang, Guanqun Zhang e Ricardo Ma. "Syntax-Informed Self-Attention Network for Span-Based Joint Entity and Relation Extraction". Applied Sciences 11, n. 4 (6 febbraio 2021): 1480. http://dx.doi.org/10.3390/app11041480.
Testo completoAbstract (sommario):
Current state-of-the-art joint entity and relation extraction framework is based on span-level entity classification and relation identification between pairs of entity mentions. However, while maintaining an efficient exhaustive search on spans, the importance of syntactic features is not taken into consideration. It will lead to a problem that the prediction of a relation between two entities is related based on corresponding entity types, but in fact they are not related in the sentence. In addition, although previous works have proven that extract local context is beneficial for the task, it still lacks in-depth learning of contextual features in local context. In this paper, we propose to incorporate syntax knowledge into multi-head self-attention by employing part of heads to focus on syntactic parents of each token from pruned dependency trees, and we use it to model the global context to fuse syntactic and semantic features. In addition, in order to get richer contextual features from the local context, we apply local focus mechanism on entity pairs and corresponding context. Based on applying the two strategies, we perform joint entity and relation extraction on span-level. Experimental results show that our model achieves significant improvements on both Conll04 and SciERC dataset compared to strong competitors.
47
Celik, Mete, Fehim Koylu e Dervis Karaboga. "CoABCMiner: An Algorithm for Cooperative Rule Classification System Based on Artificial Bee Colony". International Journal on Artificial Intelligence Tools 25, n. 01 (febbraio 2016): 1550028. http://dx.doi.org/10.1142/s0218213015500281.
Testo completoAbstract (sommario):
In data mining, classification rule learning extracts the knowledge in the representation of IF_THEN rule which is comprehensive and readable. It is a challenging problem due to the complexity of data sets. Various meta-heuristic machine learning algorithms are proposed for rule learning. Cooperative rule learning is the discovery process of all classification rules with a single run concurrently. In this paper, a novel cooperative rule learning algorithm, called CoABCMiner, based on Artificial Bee Colony is introduced. The proposed algorithm handles the training data set and discovers the classification model containing the rule list. Token competition, new updating strategy used in onlooker and employed phases, and new scout bee mechanism are proposed in CoABCMiner to achieve cooperative learning of different rules belonging to different classes. We compared the results of CoABCMiner with several state-of-the-art algorithms using 14 benchmark data sets. Non parametric statistical tests, such as Friedman test, post hoc test, and contrast estimation based on medians are performed. Nonparametric tests determine the similarity of control algorithm among other algorithms on multiple problems. Sensitivity analysis of CoABCMiner is conducted. It is concluded that CoABCMiner can be used to discover classification rules for the data sets used in experiments, efficiently.
48
Qin, Guanyi, Runze Hu, Yutao Liu, Xiawu Zheng, Haotian Liu, Xiu Li e Yan Zhang. "Data-Efficient Image Quality Assessment with Attention-Panel Decoder". Proceedings of the AAAI Conference on Artificial Intelligence 37, n. 2 (26 giugno 2023): 2091–100. http://dx.doi.org/10.1609/aaai.v37i2.25302.
Testo completoAbstract (sommario):
Blind Image Quality Assessment (BIQA) is a fundamental task in computer vision, which however remains unresolved due to the complex distortion conditions and diversified image contents. To confront this challenge, we in this paper propose a novel BIQA pipeline based on the Transformer architecture, which achieves an efficient quality-aware feature representation with much fewer data. More specifically, we consider the traditional fine-tuning in BIQA as an interpretation of the pre-trained model. In this way, we further introduce a Transformer decoder to refine the perceptual information of the CLS token from different perspectives. This enables our model to establish the quality-aware feature manifold efficiently while attaining a strong generalization capability. Meanwhile, inspired by the subjective evaluation behaviors of human, we introduce a novel attention panel mechanism, which improves the model performance and reduces the prediction uncertainty simultaneously. The proposed BIQA method maintains a light-weight design with only one layer of the decoder, yet extensive experiments on eight standard BIQA datasets (both synthetic and authentic) demonstrate its superior performance to the state-of-the-art BIQA methods, i.e., achieving the SRCC values of 0.875 (vs. 0.859 in LIVEC) and 0.980 (vs. 0.969 in LIVE). Checkpoints, logs and code will be available at https://github.com/narthchin/DEIQT.
49
Ishida, Takeshi. "Emergence of Turing Patterns in a Simple Cellular Automata-Like Model via Exchange of Integer Values between Adjacent Cells". Discrete Dynamics in Nature and Society 2020 (28 gennaio 2020): 1–12. http://dx.doi.org/10.1155/2020/2308074.
Testo completoAbstract (sommario):
The Turing pattern model is one of the theories used to describe organism formation patterns. Using this model, self-organized patterns emerge due to differences in the concentrations of activators and inhibitors. Here a cellular automata (CA)-like model was constructed wherein the Turing patterns emerged via the exchange of integer values between adjacent cells. In this simple hexagonal grid model, each cell state changed according to information exchanged from the six adjacent cells. The distinguishing characteristic of this model is that it presents a different pattern formation mechanism using only one kind of token, such as a chemical agent that ages via spatial diffusion. Using this CA-like model, various Turing-like patterns (spots or stripes) emerge when changing two of four parameters. This model has the ability to support Turing instability that propagates in the neighborhood space; global patterns are observed to spread from locally limited patterns. This model is not a substitute for a conventional Turing model but rather is a simplified Turing model. Using this model, it is possible to control the formation of multiple robots into such forms as circle groups or dividing a circle group into two groups, for example. In the field of information networks, the presented model could be applied to groups of Internet-of-Things devices to create macroscopic spatial structures to control data traffic.
50
Zhang, Zheng, Chunle Miao, Changan Liu, Qing Tian e Yongsheng Zhou. "HA-RoadFormer: Hybrid Attention Transformer with Multi-Branch for Large-Scale High-Resolution Dense Road Segmentation". Mathematics 10, n. 11 (2 giugno 2022): 1915. http://dx.doi.org/10.3390/math10111915.
Testo completoAbstract (sommario):
Road segmentation is one of the essential tasks in remote sensing. Large-scale high-resolution remote sensing images originally have larger pixel sizes than natural images, while the existing models based on Transformer have the high computational cost of square complexity, leading to more extended model training and inference time. Inspired by the long text Transformer model, this paper proposes a novel hybrid attention mechanism to improve the inference speed of the model. By calculating several diagonals and random blocks of the attention matrix, hybrid attention achieves linear time complexity in the token sequence. Using the superposition of adjacent and random attention, hybrid attention introduces the inductive bias similar to convolutional neural networks (CNNs) and retains the ability to acquire long-distance dependence. In addition, the dense road segmentation result of remote sensing image still has the problem of insufficient continuity. However, multiscale feature representation is an effective means in the network based on CNNs. Inspired by this, we propose a multi-scale patch embedding module, which divides images by patches with different scales to obtain coarse-to-fine feature representations. Experiments on the Massachusetts dataset show that the proposed HA-RoadFormer could effectively preserve the integrity of the road segmentation results, achieving a higher Intersection over Union (IoU) 67.36% of road segmentation compared to other state-of-the-art (SOTA) methods. At the same time, the inference speed has also been greatly improved compared with other Transformer based models.