To see the other types of publications on this topic, follow the link: Pytorch geometric.
Journal articles on the topic 'Pytorch geometric'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 23 journal articles for your research on the topic 'Pytorch geometric.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Sønstebø, Jakob L., Herman Brunborg, and Mikkel Elle Lepperød. "Spikeometric: Linear Non-Linear Cascade Spiking Neural Networks with Pytorch Geometric." Journal of Open Source Software 8, no. 89 (2023): 5451. http://dx.doi.org/10.21105/joss.05451.
Full text
2
Sineglazov, Victor, and Kyrylo Bylym. "Twitter Fake News Detection Using Graph Neural Networks." Electronics and Control Systems 4, no. 78 (2023): 26–33. http://dx.doi.org/10.18372/1990-5548.78.18259.
Full textAbstract:
This article is devoted to the intellectual processing of text information for the purpose of detecting rail news. To solve the given task, the use of deep graph neural networks is proposed. Fake news detection based on user preferences is augmented with deeper graph neural network topologies, including Hierarchical Graph Pooling with Structure Learning, to improve the graph convolution operation and capture richer contextual relationships in news graphs. The paper presents the possibilities of extending the framework of fake news detection based on user preferences using deep graph neural networks to improve fake news recognition. Evaluation on the FakeNewsNet dataset (a subset of Gossipcop) using the PyTorch Geometric and PyTorch Lightning frameworks demonstrates that the developed deep graph neural network model achieves 94% accuracy in fake news classification. The results show that deeper graph neural networks with integrated text and graph features offer promising options for reliable and accurate fake news detection, paving the way for improved information quality in social networks and beyond.
3
Betkier, Igor, Mateusz Oszczypała, Janusz Pobożniak, Sergiusz Sobieski, and Przemysław Betkier. "PocketFinderGNN: A manufacturing feature recognition software based on Graph Neural Networks (GNNs) using PyTorch Geometric and NetworkX." SoftwareX 23 (July 2023): 101466. http://dx.doi.org/10.1016/j.softx.2023.101466.
Full text
4
Villanueva-Domingo, Pablo, Francisco Villaescusa-Navarro, Daniel Anglés-Alcázar, et al. "Inferring Halo Masses with Graph Neural Networks." Astrophysical Journal 935, no. 1 (2022): 30. http://dx.doi.org/10.3847/1538-4357/ac7aa3.
Full textAbstract:
Abstract Understanding the halo–galaxy connection is fundamental in order to improve our knowledge on the nature and properties of dark matter. In this work, we build a model that infers the mass of a halo given the positions, velocities, stellar masses, and radii of the galaxies it hosts. In order to capture information from correlations among galaxy properties and their phase space, we use Graph Neural Networks (GNNs), which are designed to work with irregular and sparse data. We train our models on galaxies from more than 2000 state-of-the-art simulations from the Cosmology and Astrophysics with MachinE Learning Simulations project. Our model, which accounts for cosmological and astrophysical uncertainties, is able to constrain the masses of the halos with a ∼0.2 dex accuracy. Furthermore, a GNN trained on a suite of simulations is able to preserve part of its accuracy when tested on simulations run with a different code that utilizes a distinct subgrid physics model, showing the robustness of our method. The PyTorch Geometric implementation of the GNN is publicly available on GitHub (https://github.com/PabloVD/HaloGraphNet).
5
Tang, T., T. Chen, B. Zhu, and Y. Ye. "MU-NET: A MULTISCALE UNSUPERVISED NETWORK FOR REMOTE SENSING IMAGE REGISTRATION." International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLIII-B3-2022 (May 30, 2022): 537–44. http://dx.doi.org/10.5194/isprs-archives-xliii-b3-2022-537-2022.
Full textAbstract:
Abstract. Registration for multi-sensor or multi-modal image pairs with a large degree of distortions is a fundamental task for many remote sensing applications. To achieve accurate and low-cost remote sensing image registration, we propose a multiscale unsupervised network (MU-Net). Without costly ground truth labels, MU-Net directly learns the end-to-end mapping from the image pairs to their transformation parameters. MU-Net performs a coarse-to-fine registration pipeline by stacking several deep neural network models on multiple scales, which prevents the backpropagation being falling into a local extremum and resists significant image distortions. In addition, a novel loss function paradigm is designed based on structural similarity, which makes MU-Net suitable for various types of multi-modal images. MU-Net is compared with traditional feature-based and area-based methods, as well as supervised and other unsupervised learning methods on the Optical-Optical, Optical-Infrared, Optical-SAR and Optical-Map datasets. Experimental results show that MU-Net achieves more robust and accurate registration performance between these image pairs with geometric and radiometric distortions.We share the datasets and the code implemented by Pytorch at https://github.com/yeyuanxin110/MU-Net.
6
Yendrapati Ravindra Babu. "Enhancing Distributed Workflow Optimization with Graph Neural Networks and Deep Learning Techniques." Journal of Information Systems Engineering and Management 10, no. 3 (2025): 1255–70. https://doi.org/10.52783/jisem.v10i3.8862.
Full textAbstract:
This research presents a distributed processing framework using Graph Neural Networks (GNNs) for workflow scheduling and data routing in large-scale systems. Our adaptive GNN architecture dynamically models computing workflows, where nodes represent tasks and edges capture dependencies and communication patterns. Evaluated on Microsoft Azure Datacenter Traces (25 days, 11,000 machines) and Amazon AWS CloudWatch Metrics (10 days, 5,000 machines), our framework achieves 43% lower processing latency and 39% reduced memory footprint compared to DAG-based schedulers, maintaining 99.7% accuracy. The GNN-based topology optimization predicts optimal data routing paths with 91% accuracy, reducing storage overhead by 45% versus shortest-path algorithms (62% accuracy). Using PyTorch Geometric on a 180-node cluster, the system reduces network congestion by 35% and improves space utilization by 42% over baseline methods. Our multi-layer graph attention mechanism with dynamic edge weight updates accelerates workflow optimization by 46% while using 33% less memory. Under sudden workload variations, the framework sustains 92% performance stability and 98.5% data accuracy, surpassing traditional systems (68% stability). It achieves a time-space optimization ratio of 0.85 (vs. 0.62 in conventional systems), processing 1,100 tasks/hr with 95% resource efficiency. Additionally, it improves memory utilization by 41%, maintaining a ±0.3% accuracy deviation across workloads, setting new benchmarks in distributed processing.
7
Lin, Yi-Hsin, Yu-Hung Ting, Yi-Cyun Huang, Kai-Lun Cheng, and Wen-Ren Jong. "Integration of Deep Learning for Automatic Recognition of 2D Engineering Drawings." Machines 11, no. 8 (2023): 802. http://dx.doi.org/10.3390/machines11080802.
Full textAbstract:
In an environment where manufacturing precision requirements are increasing, complete project plans can consist of hundreds of engineering drawings. The presentation of these drawings often varies based on personal preferences, leading to inconsistencies in format and symbols. The lack of standardization in these aspects can result in inconsistent interpretations during subsequent analysis. Therefore, proper annotation of engineering drawings is crucial as it determines product quality, subsequent inspections, and processing costs. To reduce the time and cost associated with interpreting and analyzing drawings, as well as to minimize human errors in judgment, we developed an engineering drawing recognition system. This study employs geometric dimensioning and tolerancing (GD&T) in accordance with the ASME (American Society of Mechanical Engineers) Y14.5 2018 specification to describe the language of engineering drawings. Additionally, PyTorch, OpenCV, and You Only Look Once (YOLO) are utilized for training. Existing 2D engineering drawings serve as the training data, and image segmentation is performed to identify objects such as dimensions, tolerances, functional frames, and geometric symbols in the drawings using the network model. By reading the coordinates corresponding to each object, the correct values are displayed. Real-world cases are utilized to train the model with multiple engineering drawings containing mixed features, resulting in recognition capabilities surpassing those of single-feature identification. This approach improves the recognition accuracy of deep learning models and makes engineering drawing and image recognition more practical. The recognition results are directly stored in a database, reducing product verification time and preventing errors that may occur due to manual data entry, thereby avoiding subsequent quality control issues. The accuracy rates achieved are as follows: 85% accuracy in detecting views in 2D engineering drawings, 70% accuracy in detecting annotation groups and annotations, and 80% accuracy in text and symbol recognition.
8
Frank, Franz, and Fernando Bacao. "Advanced Genetic Programming vs. State-of-the-Art AutoML in Imbalanced Binary Classification." Emerging Science Journal 7, no. 4 (2023): 1349–63. http://dx.doi.org/10.28991/esj-2023-07-04-021.
Full textAbstract:
The objective of this article is to provide a comparative analysis of two novel genetic programming (GP) techniques, differentiable Cartesian genetic programming for artificial neural networks (DCGPANN) and geometric semantic genetic programming (GSGP), with state-of-the-art automated machine learning (AutoML) tools, namely Auto-Keras, Auto-PyTorch and Auto-Sklearn. While all these techniques are compared to several baseline algorithms upon their introduction, research still lacks direct comparisons between them, especially of the GP approaches with state-of-the-art AutoML. This study intends to fill this gap in order to analyze the true potential of GP for AutoML. The performances of the different tools are assessed by applying them to 20 benchmark datasets of the imbalanced binary classification field, thus an area that is a frequent and challenging problem. The tools are compared across the four categories average performance, maximum performance, standard deviation within performance, and generalization ability, whereby the metrics F1-score, G-mean, and AUC are used for evaluation. The analysis finds that the GP techniques, while unable to completely outperform state-of-the-art AutoML, are indeed already a very competitive alternative. Therefore, these advanced GP tools prove that they are able to provide a new and promising approach for practitioners developing machine learning (ML) models. Doi: 10.28991/ESJ-2023-07-04-021 Full Text: PDF
9
ЗАЦ, Олександр, Сергій ШМАТКОВ, Вікторія СТРІЛЕЦЬ та Григорій ЛІТВІНОВ. "МОДЕЛЬ МАРШРУТИЗАЦІЇ В КОМП’ЮТЕРНИХ МЕРЕЖАХ НА ОСНОВІ ГРАФОВИХ НЕЙРОННИХ МЕРЕЖ". Information Technology: Computer Science, Software Engineering and Cyber Security, № 1 (30 квітня 2025): 73–81. https://doi.org/10.32782/it/2025-1-11.
Full textAbstract:
У комп’ютерних мережах ефективна маршрутизація є ключовим елементом для забезпечення надійної та швидкої передачі даних. Останнім часом набуває популярності використання графових нейронних мереж для вирішення задач маршрутизації. Графові нейронні мережі дозволяють моделювати складні взаємозв’язки в мережах та адаптуватися до змінних умов, що робить їх перспективним інструментом для оптимізації процесів передачі даних. Мета роботи полягає у розробці ефективної моделі маршрутизації в комп’ютерних мережах для забезпечення зниження середньої затримки, покращення пропускної здатності та рівномірного розподілу навантаження між вузлами, з урахуванням динамічних умов мережі та складної топології. Методологія. У дослідженні проаналізовано застосування графових нейронних мереж для розв’язання задачі маршрутизації. Задача маршрутизації представлена як задача машинного навчання, а саме класифікації ребер графа мережі, які формують оптимальний маршрут. Для побудови моделі оптимізації використані сучасні інструменти і технології, такі як Mininet, PyTorch Geometric та Ryu SDN Controller. Наукова новизна. Запропонована модель маршрутизації в комп’ютерній мережі створена на основі згорткової графової нейронної мережі з двома згортковими шарами, що дозволяє врахувати складні топології мереж. Додавання шару регуляризації дозволило запобігти перенавчанню моделі. Висновки. Розроблено модель маршрутизації в комп’ютерних мережах з використанням графових нейронних мереж. Результати тестування та інтеграції моделі довели, що GNN здатні ефективно вирішувати складні задачі маршрутизації, враховуючи динамічні зміни в мережі та складну топологію. Модель може бути використана для моніторингу та управління трафіком у реальних комп’ютерних мережах, особливо за умов великих навантажень або потенційних атак.
10
Park, Yeonhong, Sunhong Min, and Jae W. Lee. "Ginex." Proceedings of the VLDB Endowment 15, no. 11 (2022): 2626–39. http://dx.doi.org/10.14778/3551793.3551819.
Full textAbstract:
Graph Neural Networks (GNNs) are receiving a spotlight as a powerful tool that can effectively serve various inference tasks on graph structured data. As the size of real-world graphs continues to scale, the GNN training system faces a scalability challenge. Distributed training is a popular approach to address this challenge by scaling out CPU nodes. However, not much attention has been paid to disk-based GNN training, which can scale up the single-node system in a more cost-effective manner by leveraging high-performance storage devices like NVMe SSDs. We observe that the data movement between the main memory and the disk is the primary bottleneck in the SSD-based training system, and that the conventional GNN training pipeline is sub-optimal without taking this overhead into account. Thus, we propose Ginex, the first SSD-based GNN training system that can process billion-scale graph datasets on a single machine. Inspired by the inspector-executor execution model in compiler optimization, Ginex restructures the GNN training pipeline by separating sample and gather stages. This separation enables Ginex to realize a provably optimal replacement algorithm, known as Belady's algorithm , for caching feature vectors in memory, which account for the dominant portion of I/O accesses. According to our evaluation with four billion-scale graph datasets and two GNN models, Ginex achieves 2.11X higher training throughput on average (2.67X at maximum) than the SSD-extended PyTorch Geometric.
11
Ma, Ning, Xiangrui Weng, Yunfeng Cao, and Linbin Wu. "Monocular-Vision-Based Precise Runway Detection Applied to State Estimation for Carrier-Based UAV Landing." Sensors 22, no. 21 (2022): 8385. http://dx.doi.org/10.3390/s22218385.
Full textAbstract:
Improving the level of autonomy during the landing phase helps promote the full-envelope autonomous flight capability of unmanned aerial vehicles (UAVs). Aiming at the identification of potential landing sites, an end-to-end state estimation method for the autonomous landing of carrier-based UAVs based on monocular vision is proposed in this paper, which allows them to discover landing sites in flight by using equipped optical sensors and avoid a crash or damage during normal and emergency landings. This scheme aims to solve two problems: the requirement of accuracy for runway detection and the requirement of precision for UAV state estimation. First, we design a robust runway detection framework on the basis of YOLOv5 (you only look once, ver. 5) with four modules: a data augmentation layer, a feature extraction layer, a feature aggregation layer and a target prediction layer. Then, the corner prediction method based on geometric features is introduced into the prediction model of the detection framework, which enables the landing field prediction to more precisely fit the runway appearance. In simulation experiments, we developed datasets applied to carrier-based UAV landing simulations based on monocular vision. In addition, our method was implemented with help of the PyTorch deep learning tool, which supports the dynamic and efficient construction of a detection network. Results showed that the proposed method achieved a higher precision and better performance on state estimation during carrier-based UAV landings.
12
Merkel, Nikolai, Pierre Toussing, Ruben Mayer, and Hans-Arno Jacobsen. "Can Graph Reordering Speed Up Graph Neural Network Training? An Experimental Study." Proceedings of the VLDB Endowment 18, no. 2 (2024): 293–307. https://doi.org/10.14778/3705829.3705846.
Full textAbstract:
Graph neural networks (GNNs) are a type of neural network capable of learning on graph-structured data. However, training GNNs on large-scale graphs is challenging due to iterative aggregations of high-dimensional features from neighboring vertices within sparse graph structures combined with neural network operations. The sparsity of graphs frequently results in suboptimal memory access patterns and longer training time. Graph reordering is an optimization strategy aiming to improve the graph data layout. It has shown to be effective to speed up graph analytics workloads, but its effect on the performance of GNN training has not been investigated yet. The generalization of reordering to GNN performance is nontrivial, as multiple aspects must be considered: GNN hyper-parameters such as the number of layers, the number of hidden dimensions, and the feature size used in the GNN model, neural network operations, large intermediate vertex states, and GPU acceleration. In our work, we close this gap by performing an empirical evaluation of 12 reordering strategies in two state-of-the-art GNN systems, PyTorch Geometric and Deep Graph Library. Our results show that graph reordering is effective in reducing training time for CPU- and GPU-based training, respectively. Further, we find that GNN hyper-parameters influence the effectiveness of reordering, that reordering metrics play an important role in selecting a reordering strategy, that lightweight reordering performs better for GPU-based than for CPU-based training, and that invested reordering time can in many cases be amortized.
13
Smorodin, Andrii V. "The use of control theory methods in neural networks’ training based on a handwritten text." Applied Aspects of Information Technology 4, no. 3 (2021): 243–49. http://dx.doi.org/10.15276/aait.03.2021.3.
Full textAbstract:
The paper shows the importance of reducing the neural networks’ training time at present stage and the role of new optimization methods in neural networks’ training. The paper researches a modification of stochastic gradient descent, which is based on the idea of gradient descent representation as a discrete dynamical system. The connection between the extreme points, to which the gradient descent iterations tend, and the stationary points of the corresponding discrete dynamicalsystem is a consequence of this representation. The further applied stabilizing scheme with predictive control, for which a theoretical apparatus was developed bymeans of geometric complex analysis together with solving optimization tasks in a set of polynomials with real coefficients, was able to train a multilevel perceptron for recognizing handwritten numbers many times faster. The new algorithm software implementation used the PyTorch library, created for researches in the field of neural networks. All experiments were run on NVidia graphical processing unit to check the processing unit’s resource consumption. The numerical experiments did not reveal any deviation in training time. There was a slight increase in the used video memory, which was expected as the new algorithm retains one additional copy of perceptron internal parameters. The importance of this result is associated with the growth in the useof deep neural network technology, which has grown three hundred thousand times from 2012 till 2018, and the associated resource consumption. This situation forces the industry to consider training optimization issues as well as their accuracy. Therefore, any training process acceleration that reduces the time or resources of the clusters is a desirableand important result, which was achieved in this article. The results obtained discover a new area of theoretical and practical research, since the stabilization usedis only one ofthe methods of stabilization and search for cycles in control theory. Such good practical results confirm the need to add the lagging control and the additional experiments with both predictive and lagging control elements
14
Ermin, Omeragić, and Đuranović Vuk. "[Re] G-Mixup: Graph Data Augmentation for Graph Classification." ReScience C 9, no. 2 (2023): #1. https://doi.org/10.5281/zenodo.8173650.
Full text
15
Nikitina, N. N., N. Bursa, A. Satici, and G. Uzer. "DATA DRIVEN AND CELL SPECIFIC DETERMINATION OF NUCLEI-ASSOCIATED ACTIN STRUCTURE." Eurasian Journal of Applied Biotechnology, no. 3S (September 12, 2024): 52. http://dx.doi.org/10.11134/btp.3s.2024.40.
Full textAbstract:
In the specialized field of cellular mechanosignaling, the intricate relationship between the cytoskeleton and nucleus is crucial for determining the behavior and fate of mesenchymal stem cells (MSCs). Existing analytical methods for filamentous actin fibers (F-actin) suffer from issues of accuracy and reproducibility. To address this, our research introduces a significant advancement in image analysis by utilizing two deep-learning segmentation Convolutional Neural Networks (CNNs) based on U-Net architecture. This algorithm precisely quantifies F-actin and offers reproducible data from 3D confocal microscopy images. Our CNNs automatically extract the 3D shape of both the nucleus and individual actin fibers, thereby enabling the geometric reconstruction of these critical cellular components. We applied this methodology to MSCs exposed to low-intensity vibration, and hypothesized that mechanical stimuli will induce perinuclear actin remodeling. Confocal Z-stack images of MSCs were captured using a 40x oil lens and a Zeiss LSM 900 microscope. Nucleus and F-actin structures were stained with Hoechst 33342 and Phalloidin, respectively. Nucleus-based images were employed to mask and isolate actin fibers within the nuclear vicinity. A specialized deep-learning segmentation model was engineered within the PyTorch framework. For model training, a selective 2% of nuclei and F-actin images were annotated. Specifically, three layers from each of six representative nuclei, among approximately 500 cross-sectional layers, were meticulously selected; a similar approach was employed for actin. The model underwent 200 epochs of training. Post-training, the deep-learning model was employed to process all collected cross-sectional images of F-actin and nuclei. A subsequent analytical step involved the scrutiny of intersecting segmented fibers between consecutive layers for geometrically accurate actin fiber reconstruction. We analyzed 102 control and 110 vibration-treated mesenchymal stem cells (MSCs) by utilizing our deep-learning-based reconstruction algorithm on confocal Z-stack images. Our data revealed an 8% decrease in nuclear height from 5.73 µm to 5.28 µm (p < 0.001) upon application of low-intensity vibration. In contrast, the two groups observed no significant difference in nuclear volume. Accompanying these changes at the nuclear level, the following changes in the F-actin cytoskeleton were observed upon LIV application. The total number of actin filaments per cell increased from 117 to 150. The average volume of individual actin fibers rose from 0.40 µm³ to 0.43 µm³. Notably, the volume of fibers at the apical nuclear surface increased from 0.35 µm³ to 0.46 µm³. While the average length of actin fibers in the apical nuclear surface remained largely unchanged due to the addition of smaller fibers in LIV groups, the LIV group had a larger number of fibers that were larger than 15µm.
16
Emmerson, Parker Yeshuason, and Rooter Lulu Pig. "Mathematical Formalization of Non-Commutative Tensor Fields and Neural Manifold Evolution." March 23, 2025. https://doi.org/10.5281/zenodo.15070142.
Full textAbstract:
This paper formalizes non-commutative tensor calculus applied to evolving neuro-semantic fields. We introduce a novel computational framework where quantum-inspired synaptic operators drive the evolution of fractal cognition patterns. Weextend classical commutator structures into neural space using geometric alge-bra, yielding new insights into phase-encoded neural structures. Implementationis computationally achieved through PyTorch, deriving effective field equations vianeuro-geometric programming. Applications encompass advanced artificial-intelligence cognition, quantum computing, and non-commutative gravity.
17
Bigi, Filippo, Guillaume Fraux, Nicholas J. Browning, and Michele Ceriotti. "Fast evaluation of spherical harmonics with sphericart." Journal of Chemical Physics 159, no. 6 (2023). http://dx.doi.org/10.1063/5.0156307.
Full textAbstract:
Spherical harmonics provide a smooth, orthogonal, and symmetry-adapted basis to expand functions on a sphere, and they are used routinely in physical and theoretical chemistry as well as in different fields of science and technology, from geology and atmospheric sciences to signal processing and computer graphics. More recently, they have become a key component of rotationally equivariant models in geometric machine learning, including applications to atomic-scale modeling of molecules and materials. We present an elegant and efficient algorithm for the evaluation of the real-valued spherical harmonics. Our construction features many of the desirable properties of existing schemes and allows us to compute Cartesian derivatives in a numerically stable and computationally efficient manner. To facilitate usage, we implement this algorithm in sphericart, a fast C++ library that also provides C bindings, a Python API, and a PyTorch implementation that includes a GPU kernel.
18
Baldassarre, Federico, David Menéndez Hurtado, Arne Elofsson, and Hossein Azizpour. "GraphQA: protein model quality assessment using graph convolutional networks." Bioinformatics, August 11, 2020. http://dx.doi.org/10.1093/bioinformatics/btaa714.
Full textAbstract:
Abstract Motivation Proteins are ubiquitous molecules whose function in biological processes is determined by their 3D structure. Experimental identification of a protein’s structure can be time-consuming, prohibitively expensive and not always possible. Alternatively, protein folding can be modeled using computational methods, which however are not guaranteed to always produce optimal results. GraphQA is a graph-based method to estimate the quality of protein models, that possesses favorable properties such as representation learning, explicit modeling of both sequential and 3D structure, geometric invariance and computational efficiency. Results GraphQA performs similarly to state-of-the-art methods despite using a relatively low number of input features. In addition, the graph network structure provides an improvement over the architecture used in ProQ4 operating on the same input features. Finally, the individual contributions of GraphQA components are carefully evaluated. Availability and implementation PyTorch implementation, datasets, experiments and link to an evaluation server are available through this GitHub repository: github.com/baldassarreFe/graphqa. Supplementary information Supplementary data are available at Bioinformatics online.
19
Tan, Yang, Bingxin Zhou, Lirong Zheng, Guisheng Fan, and Liang Hong. "Semantical and geometrical protein encoding toward enhanced bioactivity and thermostability." eLife 13 (May 2, 2025). https://doi.org/10.7554/elife.98033.4.
Full textAbstract:
Protein engineering is a pivotal aspect of synthetic biology, involving the modification of amino acids within existing protein sequences to achieve novel or enhanced functionalities and physical properties. Accurate prediction of protein variant effects requires a thorough understanding of protein sequence, structure, and function. Deep learning methods have demonstrated remarkable performance in guiding protein modification for improved functionality. However, existing approaches predominantly rely on protein sequences, which face challenges in efficiently encoding the geometric aspects of amino acids’ local environment and often fall short in capturing crucial details related to protein folding stability, internal molecular interactions, and bio-functions. Furthermore, there lacks a fundamental evaluation for developed methods in predicting protein thermostability, although it is a key physical property that is frequently investigated in practice. To address these challenges, this article introduces a novel pre-training framework that integrates sequential and geometric encoders for protein primary and tertiary structures. This framework guides mutation directions toward desired traits by simulating natural selection on wild-type proteins and evaluates variant effects based on their fitness to perform specific functions. We assess the proposed approach using three benchmarks comprising over 300 deep mutational scanning assays. The prediction results showcase exceptional performance across extensive experiments compared to other zero-shot learning methods, all while maintaining a minimal cost in terms of trainable parameters. This study not only proposes an effective framework for more accurate and comprehensive predictions to facilitate efficient protein engineering, but also enhances the in silico assessment system for future deep learning models to better align with empirical requirements. The PyTorch implementation is available at https://github.com/ai4protein/ProtSSN.
20
Tennyson, Mark, Pranav Singh, Sergey Dolgov, and Tobias Jawecki. "Optimal Poles for shift-and-invert Krylov Methods." May 27, 2025. https://doi.org/10.5281/zenodo.15526382.
Full textAbstract:
This paper presents an adaptive approach for optimising repeated poles in the shift-and-invert Krylov subspace method for approximating the matrix exponential of skew-symmetricand skew-Hermitian matrices, as well as mild perturbations of these. The use of the Rayleighquotient, an orthonormal projection of the exponent-matrix onto the shift-and-invert Krylovsubspace, in the exponential approximation is explored in this paper. The Rayleigh quotientapproach has an advantage over the typical shift-inverted approach in that properties such asnorm and energy remain conserved when the exponent-matrix is skew-Hermitian. The methodadaptively selects poles by minimising a defect-integral based error estimate, dynamically ad-justing the poles during the time-stepping procedure. The defect-integral based estimate tothe error is computed by integrating the small problem, and so is cheap to compute evenwhen a large number of quadrature points are required. We demonstrate the efficacy of thisapproach with numerical examples, including an ODE on skew-symmetric sparsely-connectedblock-diagonal graphs, as well as two PDE examples: the time-dependent Schrödinger equationwith Coulomb potential with softening term β = 1 and β = 0.1. In practice, we find that theapproach allows the use of cheaper surrogates when identifying the optimal poles significantlyreducing the overheads from the optimisation procedure. Results show that adaptive pole se-lection improves computational accuracy and efficiency, enabling larger time steps and feweriterations compared to standard polynomial Krylov methods. A PyTorch-based implementationis provided for facilitating efficient gradient-based optimisation of the poles.
21
McLaughlin, Connor, and Yi Lu. "Multi-class vulnerability prediction using value flow and graph neural networks." Neural Computing and Applications, May 20, 2024. http://dx.doi.org/10.1007/s00521-024-09819-3.
Full textAbstract:
AbstractIn recent years, machine learning models have been increasingly used to detect security vulnerabilities in software, due to their ability to achieve high performance and lower false positive rates compared to traditional program analysis tools. However, these models often lack the capability to provide a clear explanation for why a program has been flagged as vulnerable, leaving developers with little reasoning to work with. We present a new method which not only identifies the presence of vulnerabilities in a program, but also the specific type of error, considering the whole program rather than just individual functions. Our approach utilizes graph neural networks that employ inter-procedural value flow graphs, and instruction embedding from the LLVM Intermediate Representation, to predict a class. By mapping these classes to the Common Weakness Enumeration list, we provide a clear indication of the security issue found, saving developers valuable time which would otherwise be spent analyzing a binary vulnerable/non-vulnerable label. To evaluate our method’s effectiveness, we used two datasets: one containing memory-related errors (out of bound array accesses), and the other a range of vulnerabilities from the Juliet Test Suite, including buffer and integer overflows, format strings, and invalid frees. Our model, implemented using PyTorch and the Gated Graph Sequence Neural Network from Torch-Geometric, achieved a precision of 96.35 and 91.59% on the two datasets, respectively. Compared to common static analysis tools, our method produced roughly half the number of false positives, while identifying approximately three times the number of vulnerable samples. Compared to recent machine learning systems, we achieve similar performance while offering the added benefit of differentiating between classes. Overall, our approach represents a meaningful improvement in software vulnerability detection, providing developers with valuable insights to better secure their code.
22
El-Mhamdi, El-Mahdi, Rachid Guerraoui, Arsany Guirguis, Lê-Nguyên Hoang, and Sébastien Rouault. "Genuinely distributed Byzantine machine learning." Distributed Computing, May 26, 2022. http://dx.doi.org/10.1007/s00446-022-00427-9.
Full textAbstract:
AbstractMachine learning (ML) solutions are nowadays distributed, according to the so-called server/worker architecture. One server holds the model parameters while several workers train the model. Clearly, such architecture is prone to various types of component failures, which can be all encompassed within the spectrum of a Byzantine behavior. Several approaches have been proposed recently to tolerate Byzantine workers. Yet all require trusting a central parameter server. We initiate in this paper the study of the “general” Byzantine-resilient distributed machine learning problem where no individual component is trusted. In particular, we distribute the parameter server computation on several nodes. We show that this problem can be solved in an asynchronous system, despite the presence of $$\frac{1}{3}$$ 1 3 Byzantine parameter servers (i.e., $$n_{ps} > 3f_{ps}+1$$ n ps > 3 f ps + 1 ) and $$\frac{1}{3}$$ 1 3 Byzantine workers (i.e., $$n_w > 3f_w$$ n w > 3 f w ), which is asymptotically optimal. We present a new algorithm, ByzSGD, which solves the general Byzantine-resilient distributed machine learning problem by relying on three major schemes. The first, scatter/gather, is a communication scheme whose goal is to bound the maximum drift among models on correct servers. The second, distributed median contraction (DMC), leverages the geometric properties of the median in high dimensional spaces to bring parameters within the correct servers back close to each other, ensuring safe and lively learning. The third, Minimum-diameter averaging (MDA), is a statistically-robust gradient aggregation rule whose goal is to tolerate Byzantine workers. MDA requires a loose bound on the variance of non-Byzantine gradient estimates, compared to existing alternatives [e.g., Krum (Blanchard et al., in: Neural information processing systems, pp 118-128, 2017)]. Interestingly, ByzSGD ensures Byzantine resilience without adding communication rounds (on a normal path), compared to vanilla non-Byzantine alternatives. ByzSGD requires, however, a larger number of messages which, we show, can be reduced if we assume synchrony. We implemented ByzSGD on top of both TensorFlow and PyTorch, and we report on our evaluation results. In particular, we show that ByzSGD guarantees convergence with around 32% overhead compared to vanilla SGD. Furthermore, we show that ByzSGD’s throughput overhead is 24–176% in the synchronous case and 28–220% in the asynchronous case.
23
Smets, Bart M. N., Jim Portegies, Erik J. Bekkers, and Remco Duits. "PDE-Based Group Equivariant Convolutional Neural Networks." Journal of Mathematical Imaging and Vision, July 27, 2022. http://dx.doi.org/10.1007/s10851-022-01114-x.
Full textAbstract:
AbstractWe present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs). In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer’s trainable weights. Formulating our PDEs on homogeneous spaces allows these networks to be designed with built-in symmetries such as rotation in addition to the standard translation equivariance of CNNs. Having all the desired symmetries included in the design obviates the need to include them by means of costly techniques such as data augmentation. We will discuss our PDE-based G-CNNs (PDE-G-CNNs) in a general homogeneous space setting while also going into the specifics of our primary case of interest: roto-translation equivariance. We solve the PDE of interest by a combination of linear group convolutions and nonlinear morphological group convolutions with analytic kernel approximations that we underpin with formal theorems. Our kernel approximations allow for fast GPU-implementation of the PDE-solvers; we release our implementation with this article in the form of the LieTorch extension to PyTorch, available at https://gitlab.com/bsmetsjr/lietorch. Just like for linear convolution, a morphological convolution is specified by a kernel that we train in our PDE-G-CNNs. In PDE-G-CNNs, we do not use non-linearities such as max/min-pooling and ReLUs as they are already subsumed by morphological convolutions. We present a set of experiments to demonstrate the strength of the proposed PDE-G-CNNs in increasing the performance of deep learning-based imaging applications with far fewer parameters than traditional CNNs.