Search Results for author:
Found 96 papers, 50 papers with code
On Isotropy Calibration of Transformer Models
Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone.
TempCaps: A Capsule Network-based Embedding Model for Temporal Knowledge Graph Completion
In this paper, we tackle the temporal knowledge graph completion task by proposing TempCaps, which is a Capsule network-based embedding model for Temporal knowledge graph completion.
Seeing Through the Mask: Rethinking Adversarial Examples for CAPTCHAs
However, these methods are model-specific and thus can not aid CAPTCHAs in fooling all models.
GraphFSA: A Finite State Automaton Framework for Algorithmic Learning on Graphs
Therefore, we propose a novel framework: GraphFSA (Graph Finite State Automaton).
AEye: A Visualization Tool for Image Datasets
Image datasets serve as the foundation for machine learning models in computer vision, significantly influencing model capabilities, performance, and biases alongside architectural considerations.
Graph Dimension Attention Networks for Enterprise Credit Assessment
Enterprise credit assessment is critical for evaluating financial risk, and Graph Neural Networks (GNNs), with their advanced capability to model inter-entity relationships, are a natural tool to get a deeper understanding of these financial networks.
Cue Point Estimation using Object Detection
Cue points indicate possible temporal boundaries in a transition between two pieces of music in DJ mixing and constitute a crucial element in autonomous DJ systems as well as for live mixing.
Benchmarking GNNs Using Lightning Network Data
Nodes announce their channels to the network, forming a graph with channels as edges.
PUZZLES: A Benchmark for Neural Algorithmic Reasoning
Algorithmic reasoning is a fundamental cognitive ability that plays a pivotal role in problem-solving and decision-making processes.
Towards Learning Abductive Reasoning using VSA Distributed Representations
We introduce the Abductive Rule Learner with Context-awareness (ARLC), a model that solves abstract reasoning tasks based on Learn-VRF.
Next Level Message-Passing with Hierarchical Support Graphs
Message-Passing Neural Networks (MPNNs) are extensively employed in graph learning tasks but suffer from limitations such as the restricted scope of information exchange, by being confined to neighboring nodes during each round of message passing.
Ranked #1 on
Graph Regression
on Peptides-struct
SUBER: An RL Environment with Simulated Human Behavior for Recommender Systems
However, the successful implementation of RL in recommender systems is challenging because of several factors, including the limited availability of online data for training on-policy methods.
Assessing Adversarial Robustness of Large Language Models: An Empirical Study
Large Language Models (LLMs) have revolutionized natural language processing, but their robustness against adversarial attacks remains a critical concern.
CAESAR: Enhancing Federated RL in Heterogeneous MDPs through Convergence-Aware Sampling with Screening
CAESAR is an aggregation strategy used by the server that combines convergence-aware sampling with a screening mechanism.
Bridging Diversity and Uncertainty in Active learning with Self-Supervised Pre-Training
This study addresses the integration of diversity-based and uncertainty-based sampling strategies in active learning, particularly within the context of self-supervised pre-trained models.
SUPClust: Active Learning at the Boundaries
Active learning is a machine learning paradigm designed to optimize model performance in a setting where labeled data is expensive to acquire.
CoRe-GD: A Hierarchical Framework for Scalable Graph Visualization with GNNs
We introduce a scalable Graph Neural Network (GNN) based Graph Drawing framework with sub-quadratic runtime that can learn to optimize stress.
Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergence
We provide the first decentralized Byzantine fault-tolerant FRL method.
Efficient and Scalable Graph Generation through Iterative Local Expansion
However, most existing methods struggle with large graphs due to the complexity of representing the entire joint distribution across all node pairs and capturing both global and local graph structures simultaneously.
Exponentially Faster Language Modelling
Language models only really need to use an exponential fraction of their neurons for individual inferences.
SURF: A Generalization Benchmark for GNNs Predicting Fluid Dynamics
We propose SURF, a benchmark designed to test the $\textit{generalization}$ of learned graph-based fluid simulators.
Flood and Echo Net: Algorithmically Aligned GNNs that Generalize
We test the Flood and Echo Net on a variety of synthetic tasks and the SALSA-CLRS benchmark and find that the algorithmic alignment of the execution improves generalization to larger graph sizes.
What Determines the Price of NFTs?
In the evolving landscape of digital art, Non-Fungible Tokens (NFTs) have emerged as a groundbreaking platform, bridging the realms of art and technology.
SALSA-CLRS: A Sparse and Scalable Benchmark for Algorithmic Reasoning
We introduce an extension to the CLRS algorithmic learning benchmark, prioritizing scalability and the utilization of sparse representations.
Fast Feedforward Networks
We break the linear link between the layer size and its inference cost by introducing the fast feedforward (FFF) architecture, a log-time alternative to feedforward networks.
An Interpretable and Attention-based Method for Gaze Estimation Using Electroencephalography
Eye movements can reveal valuable insights into various aspects of human mental processes, physical well-being, and actions.
Graphtester: Exploring Theoretical Boundaries of GNNs on Graph Datasets
We use Graphtester to analyze over 40 different graph datasets, determining upper bounds on the performance of various GNNs based on the number of layers.
Siamese SIREN: Audio Compression with Implicit Neural Representations
Implicit Neural Representations (INRs) have emerged as a promising method for representing diverse data modalities, including 3D shapes, images, and audio.
Provably Powerful Graph Neural Networks for Directed Multigraphs
This paper analyses a set of simple adaptations that transform standard message-passing Graph Neural Networks (GNN) into provably powerful directed multigraph neural networks.
Examining the Emergence of Deductive Reasoning in Generative Language Models
We conduct a preliminary inquiry into the ability of generative transformer models to deductively reason from premises provided.
Cascaded Beam Search: Plug-and-Play Terminology-Forcing For Neural Machine Translation
Our plug-and-play approach performs on par with the winning submissions without using a domain-specific language model and with no additional training.
Discovering Graph Generation Algorithms
We provide a novel approach to construct generative models for graphs.
Abstract Visual Reasoning Enabled by Language
While artificial intelligence (AI) models have achieved human or even superhuman performance in many well-defined applications, they still struggle to show signs of broad and flexible intelligence.
DAVA: Disentangling Adversarial Variational Autoencoder
We compare DAVA to models with optimal hyperparameters.
Electrode Clustering and Bandpass Analysis of EEG Data for Gaze Estimation
In this study, we validate the findings of previously published papers, showing the feasibility of an Electroencephalography (EEG) based gaze estimation.
FedHQL: Federated Heterogeneous Q-Learning
Federated Reinforcement Learning (FedRL) encourages distributed agents to learn collectively from each other's experience to improve their performance without exchanging their raw trajectories.
DeFi Lending During The Merge
They feared spiking ETH borrowing rates would lead to mass liquidations which could undermine their viability.
Learning Graph Algorithms With Recurrent Graph Neural Networks
In order to scale, we focus on a recurrent architecture design that can learn simple graph problems end to end on smaller graphs and then extrapolate to larger instances.
Automating Rigid Origami Design
Rigid origami has shown potential in large diversity of practical applications.
Neural Combinatorial Logic Circuit Synthesis from Input-Output Examples
We propose a novel, fully explainable neural approach to synthesis of combinatorial logic circuits from input-output examples.
Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations
We further explore the applicability of our clustering approach by evaluating it on 14 datasets with more diverse topics, text lengths, and numbers of classes.
Diffusion Models for Graphs Benefit From Discrete State Spaces
Denoising diffusion probabilistic models and score-matching models have proven to be very powerful for generative tasks.
A Neural Model for Regular Grammar Induction
Grammatical inference is a classical problem in computational learning theory and a topic of wider influence in natural language processing.
Periodic Extrapolative Generalisation in Neural Networks
The learning of the simplest possible computational pattern -- periodicity -- is an open problem in the research of strong generalisation in neural networks.
FACT: Learning Governing Abstractions Behind Integer Sequences
Integer sequences are of central importance to the modeling of concepts admitting complete finitary descriptions.
Deterministic Graph-Walking Program Mining
These programs characterise linear long-distance relationships between the given two vertex sets in the context of the whole graph.
Exploring Price Accuracy on Uniswap V3 in Times of Distress
Financial markets have evolved over centuries, and exchanges have converged to rely on the order book mechanism for market making.
Agent-based Graph Neural Networks
AgentNet is inspired by sublinear algorithms, featuring a computational complexity that is independent of the graph size.
A Deep Learning Approach for the Segmentation of Electroencephalography Data in Eye Tracking Applications
Here, we introduce DETRtime, a novel framework for time-series segmentation that creates ocular event detectors that do not require additionally recorded eye-tracking modality and rely solely on EEG data.
The Economics of Automated Market Makers
This paper studies the question whether automated market maker protocols such as Uniswap can sustainably retain a portion of their trading fees for the protocol.
Graph Neural Networks with Precomputed Node Features
Most Graph Neural Networks (GNNs) cannot distinguish some graphs or indeed some pairs of nodes within a graph.
DT+GNN: A Fully Explainable Graph Neural Network using Decision Trees
We propose the fully explainable Decision Tree Graph Neural Network (DT+GNN) architecture.
Asynchronous Neural Networks for Learning in Graphs
This paper studies asynchronous message passing (AMP), a new paradigm for applying neural network based learning to graphs.
Risks and Returns of Uniswap V3 Liquidity Providers
However, Uniswap V3 requires far more decisions from liquidity providers than previous DEX designs.
SPECTRE: Spectral Conditioning Helps to Overcome the Expressivity Limits of One-shot Graph Generators
We approach the graph generation problem from a spectral perspective by first generating the dominant parts of the graph Laplacian spectrum and then building a graph matching these eigenvalues and eigenvectors.
A Theoretical Comparison of Graph Neural Network Extensions
We study and compare different Graph Neural Network extensions that increase the expressive power of GNNs beyond the Weisfeiler-Leman test.
The Price of Majority Support
We consider the ratio between this number and the number of matches of the overall best outcome which may not have majority support.
DropGNN: Random Dropouts Increase the Expressiveness of Graph Neural Networks
In DropGNNs, we execute multiple runs of a GNN on the input graph, with some of the nodes randomly and independently dropped in each of these runs.
Ranked #11 on
Graph Classification
on IMDb-B
EEGEyeNet: a Simultaneous Electroencephalography and Eye-tracking Dataset and Benchmark for Eye Movement Prediction
We present a new dataset and benchmark with the goal of advancing research in the intersection of brain activities and eye movements.
3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers
3D-RETR is capable of 3D reconstruction from a single view or multiple views.
On Isotropy Calibration of Transformers
Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone.
A Plug-and-Play Method for Controlled Text Generation
Large pre-trained language models have repeatedly shown their ability to produce fluent text.
BERT is Robust! A Case Against Synonym-Based Adversarial Examples in Text Classification
We combine a human evaluation of individual word substitutions and a probabilistic analysis to show that between 96% and 99% of the analyzed attacks do not preserve semantics, indicating that their success is mainly based on feeding poor data to the model.
Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based Attacks
In this paper, we present an approach to improve the robustness of BERT language models against word substitution-based adversarial attacks by leveraging adversarial perturbations for self-supervised contrastive learning.
Debt Swapping for Risk Mitigation in Financial Networks
We first show that there can be no positive swap for any pair of banks in a static financial system, or when a shock hits each bank in the network proportionally.
Behavior of Liquidity Providers in Decentralized Exchanges
In this paper, we aim to understand how liquidity providers react to market information and how they benefit from providing liquidity in DEXes.
Cyclic Arbitrage in Decentralized Exchanges
We find that traders have executed 292, 606 cyclic arbitrages over eleven months and exploited more than 138 million USD in revenue.
Towards BERT-based Automatic ICD Coding: Limitations and Opportunities
Unlike the general trend in language processing, no transformer model has been reported to reach high performance on this task.
Should Graph Neural Networks Use Features, Edges, Or Both?
We find that for graph classification, a GNN is not more than the sum of its parts.
Towards Robust Graph Contrastive Learning
We study the problem of adversarially robust self-supervised learning on graphs.
Of Non-Linearity and Commutativity in BERT
In this work we provide new insights into the transformer architecture, and in particular, its best-known variant, BERT.
KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation
We present Knowledge Enhanced Multimodal BART (KM-BART), which is a Transformer-based sequence-to-sequence model capable of reasoning about commonsense knowledge from multimodal inputs of images and texts.
Unsupervised Task Clustering for Multi-Task Reinforcement Learning
We show the generality of our approach by evaluating on simple discrete and continuous control tasks, as well as complex bipedal walker tasks and Atari games.
Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation
Given that state-of-the-art language models are too large to be trained from scratch in a manageable time, it is desirable to control these models without re-training them.
Contrastive Graph Neural Network Explanation
Graph Neural Networks achieve remarkable results on problems with structured data but come as black-box predictors.
Learning Lower Bounds for Graph Exploration With Reinforcement Learning
Given there are quadratically many possible edges in a graph and each subset of edges is a possible solution, this yields unfeasibly large search spaces even for few nodes.
A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples
Generating adversarial examples for natural language is hard, as natural language consists of discrete symbols, and examples are often of variable lengths.
Brain2Word: Decoding Brain Activity for Language Generation
In the case of language stimuli, recent studies have shown that it is possible to decode fMRI scans into an embedding of the word a subject is reading.
Medley2K: A Dataset of Medley Transitions
The automatic generation of medleys, i. e., musical pieces formed by different songs concatenated via smooth transitions, is not well studied in the current literature.
Normalized Attention Without Probability Cage
Attention architectures are widely used; they recently gained renewed popularity with Transformers yielding a streak of state of the art results.
Neural Status Registers
Standard Neural Networks can learn mathematical operations, but they do not extrapolate.
Telling BERT's full story: from Local Attention to Global Aggregation
This way, we propose a distinction between local patterns revealed by attention and global patterns that refer back to the input, and analyze BERT from both angles.
Divide and Scale: Formalization and Roadmap to Robust Sharding
Sharding distributed ledgers is a promising on-chain solution for scaling blockchains but lacks formal grounds, nurturing skepticism on whether such complex systems can scale blockchains securely.
Distributed, Parallel, and Cluster Computing
The Frechet Distance of training and test distribution predicts the generalization gap
Empirically across several data domains, we substantiate this viewpoint by showing that test performance correlates strongly with the distance in data distributions between training and test set.
ABC: Asynchronous Blockchain without Consensus
There is a preconception that a blockchain needs consensus.
Cryptography and Security
On Identifiability in Transformers
We show that, for sequences longer than the attention head dimension, attention weights are not identifiable.
Synthetic Epileptic Brain Activities Using Generative Adversarial Networks
In this work, we generate synthetic seizure-like brain electrical activities, i. e., EEG signals, that can be used to train seizure detection algorithms, alleviating the need for recorded data.
Attentive Multi-Task Deep Reinforcement Learning
Sharing knowledge between tasks is vital for efficient learning in a multi-task setting.
Learning Policies through Quantile Regression
Policy gradient based reinforcement learning algorithms coupled with neural networks have shown success in learning complex policies in the model free continuous action space control setting.
Using State Predictions for Value Regularization in Curiosity Driven Deep Reinforcement Learning
Learning in sparse reward settings remains a challenge in Reinforcement Learning, which is often addressed by using intrinsic rewards.
The Urban Last Mile Problem: Autonomous Drone Delivery to Your Balcony
The drop-off location could, e. g., be on a balcony or porch, and simply needs to be indicated by a visual marker on the wall or window.
Robotics Systems and Control
MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer
The interpolations smoothly change pitches, dynamics and instrumentation to create a harmonic bridge between two music pieces.
Symbolic Music Genre Transfer with CycleGAN
In this paper we apply such a model to symbolic music and show the feasibility of our approach for music genre transfer.
Natural Language Multitasking: Analyzing and Improving Syntactic Saliency of Hidden Representations
We train multi-task autoencoders on linguistic tasks and analyze the learned hidden sentence representations.
JamBot: Music Theory Aware Chord Based Generation of Polyphonic Music with LSTMs
First, a chord LSTM predicts a chord progression based on a chord embedding.
Teaching a Machine to Read Maps with Deep Reinforcement Learning
Localization and navigation is also an important problem in domains such as robotics, and has recently become a focus of the deep reinforcement learning community.
