Search Results for author:
Found 85 papers, 42 papers with code
Exponentially Faster Language Modelling
Language models only really need to use an exponential fraction of their neurons for individual inferences.
Symbolic Music Genre Transfer with CycleGAN
In this paper we apply such a model to symbolic music and show the feasibility of our approach for music genre transfer.
Fast Feedforward Networks
We break the linear link between the layer size and its inference cost by introducing the fast feedforward (FFF) architecture, a log-time alternative to feedforward networks.
The Urban Last Mile Problem: Autonomous Drone Delivery to Your Balcony
The drop-off location could, e. g., be on a balcony or porch, and simply needs to be indicated by a visual marker on the wall or window.
Robotics Systems and Control
A Plug-and-Play Method for Controlled Text Generation
Large pre-trained language models have repeatedly shown their ability to produce fluent text.
JamBot: Music Theory Aware Chord Based Generation of Polyphonic Music with LSTMs
First, a chord LSTM predicts a chord progression based on a chord embedding.
3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers
3D-RETR is capable of 3D reconstruction from a single view or multiple views.
EEGEyeNet: a Simultaneous Electroencephalography and Eye-tracking Dataset and Benchmark for Eye Movement Prediction
We present a new dataset and benchmark with the goal of advancing research in the intersection of brain activities and eye movements.
Teaching a Machine to Read Maps with Deep Reinforcement Learning
Localization and navigation is also an important problem in domains such as robotics, and has recently become a focus of the deep reinforcement learning community.
KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation
We present Knowledge Enhanced Multimodal BART (KM-BART), which is a Transformer-based sequence-to-sequence model capable of reasoning about commonsense knowledge from multimodal inputs of images and texts.
Diffusion Models for Graphs Benefit From Discrete State Spaces
Denoising diffusion probabilistic models and score-matching models have proven to be very powerful for generative tasks.
DropGNN: Random Dropouts Increase the Expressiveness of Graph Neural Networks
In DropGNNs, we execute multiple runs of a GNN on the input graph, with some of the nodes randomly and independently dropped in each of these runs.
Ranked #11 on
Graph Classification
on IMDb-B
SPECTRE: Spectral Conditioning Helps to Overcome the Expressivity Limits of One-shot Graph Generators
We approach the graph generation problem from a spectral perspective by first generating the dominant parts of the graph Laplacian spectrum and then building a graph matching these eigenvalues and eigenvectors.
Normalized Attention Without Probability Cage
Attention architectures are widely used; they recently gained renewed popularity with Transformers yielding a streak of state of the art results.
Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations
We further explore the applicability of our clustering approach by evaluating it on 14 datasets with more diverse topics, text lengths, and numbers of classes.
Contrastive Graph Neural Network Explanation
Graph Neural Networks achieve remarkable results on problems with structured data but come as black-box predictors.
A Deep Learning Approach for the Segmentation of Electroencephalography Data in Eye Tracking Applications
Here, we introduce DETRtime, a novel framework for time-series segmentation that creates ocular event detectors that do not require additionally recorded eye-tracking modality and rely solely on EEG data.
Attentive Multi-Task Deep Reinforcement Learning
Sharing knowledge between tasks is vital for efficient learning in a multi-task setting.
Synthetic Epileptic Brain Activities Using Generative Adversarial Networks
In this work, we generate synthetic seizure-like brain electrical activities, i. e., EEG signals, that can be used to train seizure detection algorithms, alleviating the need for recorded data.
Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation
Given that state-of-the-art language models are too large to be trained from scratch in a manageable time, it is desirable to control these models without re-training them.
SALSA-CLRS: A Sparse and Scalable Benchmark for Algorithmic Reasoning
We introduce an extension to the CLRS algorithmic learning benchmark, prioritizing scalability and the utilization of sparse representations.
Using State Predictions for Value Regularization in Curiosity Driven Deep Reinforcement Learning
Learning in sparse reward settings remains a challenge in Reinforcement Learning, which is often addressed by using intrinsic rewards.
Agent-based Graph Neural Networks
AgentNet is inspired by sublinear algorithms, featuring a computational complexity that is independent of the graph size.
Automating Rigid Origami Design
Rigid origami has shown potential in large diversity of practical applications.
Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergence
We provide the first decentralized Byzantine fault-tolerant FRL method.
ABC: Asynchronous Blockchain without Consensus
There is a preconception that a blockchain needs consensus.
Cryptography and Security
Brain2Word: Decoding Brain Activity for Language Generation
In the case of language stimuli, recent studies have shown that it is possible to decode fMRI scans into an embedding of the word a subject is reading.
Of Non-Linearity and Commutativity in BERT
In this work we provide new insights into the transformer architecture, and in particular, its best-known variant, BERT.
Periodic Extrapolative Generalisation in Neural Networks
The learning of the simplest possible computational pattern -- periodicity -- is an open problem in the research of strong generalisation in neural networks.
A Neural Model for Regular Grammar Induction
Grammatical inference is a classical problem in computational learning theory and a topic of wider influence in natural language processing.
Neural Combinatorial Logic Circuit Synthesis from Input-Output Examples
We propose a novel, fully explainable neural approach to synthesis of combinatorial logic circuits from input-output examples.
Graphtester: Exploring Theoretical Boundaries of GNNs on Graph Datasets
We use Graphtester to analyze over 40 different graph datasets, determining upper bounds on the performance of various GNNs based on the number of layers.
SURF: A Generalization Benchmark for GNNs Predicting Fluid Dynamics
We propose SURF, a benchmark designed to test the $\textit{generalization}$ of learned graph-based fluid simulators.
Efficient and Scalable Graph Generation through Iterative Local Expansion
However, most existing methods struggle with large graphs due to the complexity of representing the entire joint distribution across all node pairs and capturing both global and local graph structures simultaneously.
Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based Attacks
In this paper, we present an approach to improve the robustness of BERT language models against word substitution-based adversarial attacks by leveraging adversarial perturbations for self-supervised contrastive learning.
DAVA: Disentangling Adversarial Variational Autoencoder
We compare DAVA to models with optimal hyperparameters.
Unsupervised Task Clustering for Multi-Task Reinforcement Learning
We show the generality of our approach by evaluating on simple discrete and continuous control tasks, as well as complex bipedal walker tasks and Atari games.
TempCaps: A Capsule Network-based Embedding Model for Temporal Knowledge Graph Completion
In this paper, we tackle the temporal knowledge graph completion task by proposing TempCaps, which is a Capsule network-based embedding model for Temporal knowledge graph completion.
Learning Graph Algorithms With Recurrent Graph Neural Networks
In order to scale, we focus on a recurrent architecture design that can learn simple graph problems end to end on smaller graphs and then extrapolate to larger instances.
CoRe-GD: A Hierarchical Framework for Scalable Graph Visualization with GNNs
We introduce a scalable Graph Neural Network (GNN) based Graph Drawing framework with sub-quadratic runtime that can learn to optimize stress.
CAESAR: Enhancing Federated RL in Heterogeneous MDPs through Convergence-Aware Sampling with Screening
CAESAR is an aggregation strategy used by the server that combines convergence-aware sampling with a screening mechanism.
Natural Language Multitasking: Analyzing and Improving Syntactic Saliency of Hidden Representations
We train multi-task autoencoders on linguistic tasks and analyze the learned hidden sentence representations.
MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer
The interpolations smoothly change pitches, dynamics and instrumentation to create a harmonic bridge between two music pieces.
Learning Policies through Quantile Regression
Policy gradient based reinforcement learning algorithms coupled with neural networks have shown success in learning complex policies in the model free continuous action space control setting.
On Identifiability in Transformers
We show that, for sequences longer than the attention head dimension, attention weights are not identifiable.
Telling BERT's full story: from Local Attention to Global Aggregation
This way, we propose a distinction between local patterns revealed by attention and global patterns that refer back to the input, and analyze BERT from both angles.
Neural Status Registers
Standard Neural Networks can learn mathematical operations, but they do not extrapolate.
Medley2K: A Dataset of Medley Transitions
The automatic generation of medleys, i. e., musical pieces formed by different songs concatenated via smooth transitions, is not well studied in the current literature.
A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples
Generating adversarial examples for natural language is hard, as natural language consists of discrete symbols, and examples are often of variable lengths.
Divide and Scale: Formalization and Roadmap to Robust Sharding
Sharding distributed ledgers is a promising on-chain solution for scaling blockchains but lacks formal grounds, nurturing skepticism on whether such complex systems can scale blockchains securely.
Distributed, Parallel, and Cluster Computing
Towards Robust Graph Contrastive Learning
We study the problem of adversarially robust self-supervised learning on graphs.
Should Graph Neural Networks Use Features, Edges, Or Both?
We find that for graph classification, a GNN is not more than the sum of its parts.
Towards BERT-based Automatic ICD Coding: Limitations and Opportunities
Unlike the general trend in language processing, no transformer model has been reported to reach high performance on this task.
Cyclic Arbitrage in Decentralized Exchanges
We find that traders have executed 292, 606 cyclic arbitrages over eleven months and exploited more than 138 million USD in revenue.
Behavior of Liquidity Providers in Decentralized Exchanges
In this paper, we aim to understand how liquidity providers react to market information and how they benefit from providing liquidity in DEXes.
Debt Swapping for Risk Mitigation in Financial Networks
We first show that there can be no positive swap for any pair of banks in a static financial system, or when a shock hits each bank in the network proportionally.
BERT is Robust! A Case Against Synonym-Based Adversarial Examples in Text Classification
We combine a human evaluation of individual word substitutions and a probabilistic analysis to show that between 96% and 99% of the analyzed attacks do not preserve semantics, indicating that their success is mainly based on feeding poor data to the model.
On Isotropy Calibration of Transformers
Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone.
The Frechet Distance of training and test distribution predicts the generalization gap
Empirically across several data domains, we substantiate this viewpoint by showing that test performance correlates strongly with the distance in data distributions between training and test set.
Learning Lower Bounds for Graph Exploration With Reinforcement Learning
Given there are quadratically many possible edges in a graph and each subset of edges is a possible solution, this yields unfeasibly large search spaces even for few nodes.
The Price of Majority Support
We consider the ratio between this number and the number of matches of the overall best outcome which may not have majority support.
A Theoretical Comparison of Graph Neural Network Extensions
We study and compare different Graph Neural Network extensions that increase the expressive power of GNNs beyond the Weisfeiler-Leman test.
On Isotropy Calibration of Transformer Models
Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone.
Risks and Returns of Uniswap V3 Liquidity Providers
However, Uniswap V3 requires far more decisions from liquidity providers than previous DEX designs.
Asynchronous Neural Networks for Learning in Graphs
This paper studies asynchronous message passing (AMP), a new paradigm for applying neural network based learning to graphs.
DT+GNN: A Fully Explainable Graph Neural Network using Decision Trees
We propose the fully explainable Decision Tree Graph Neural Network (DT+GNN) architecture.
Graph Neural Networks with Precomputed Node Features
Most Graph Neural Networks (GNNs) cannot distinguish some graphs or indeed some pairs of nodes within a graph.
The Economics of Automated Market Makers
This paper studies the question whether automated market maker protocols such as Uniswap can sustainably retain a portion of their trading fees for the protocol.
Deterministic Graph-Walking Program Mining
These programs characterise linear long-distance relationships between the given two vertex sets in the context of the whole graph.
Exploring Price Accuracy on Uniswap V3 in Times of Distress
Financial markets have evolved over centuries, and exchanges have converged to rely on the order book mechanism for market making.
FACT: Learning Governing Abstractions Behind Integer Sequences
Integer sequences are of central importance to the modeling of concepts admitting complete finitary descriptions.
FedHQL: Federated Heterogeneous Q-Learning
Federated Reinforcement Learning (FedRL) encourages distributed agents to learn collectively from each other's experience to improve their performance without exchanging their raw trajectories.
Electrode Clustering and Bandpass Analysis of EEG Data for Gaze Estimation
In this study, we validate the findings of previously published papers, showing the feasibility of an Electroencephalography (EEG) based gaze estimation.
Abstract Visual Reasoning Enabled by Language
While artificial intelligence (AI) models have achieved human or even superhuman performance in many well-defined applications, they still struggle to show signs of broad and flexible intelligence.
DeFi Lending During The Merge
They feared spiking ETH borrowing rates would lead to mass liquidations which could undermine their viability.
Discovering Graph Generation Algorithms
We provide a novel approach to construct generative models for graphs.
Cascaded Beam Search: Plug-and-Play Terminology-Forcing For Neural Machine Translation
Our plug-and-play approach performs on par with the winning submissions without using a domain-specific language model and with no additional training.
Examining the Emergence of Deductive Reasoning in Generative Language Models
We conduct a preliminary inquiry into the ability of generative transformer models to deductively reason from premises provided.
Provably Powerful Graph Neural Networks for Directed Multigraphs
This paper analyses a set of simple adaptations that transform standard message-passing Graph Neural Networks (GNN) into provably powerful directed multigraph neural networks.
Siamese SIREN: Audio Compression with Implicit Neural Representations
Implicit Neural Representations (INRs) have emerged as a promising method for representing diverse data modalities, including 3D shapes, images, and audio.
An Interpretable and Attention-based Method for Gaze Estimation Using Electroencephalography
Eye movements can reveal valuable insights into various aspects of human mental processes, physical well-being, and actions.
What Determines the Price of NFTs?
In the evolving landscape of digital art, Non-Fungible Tokens (NFTs) have emerged as a groundbreaking platform, bridging the realms of art and technology.
Flood and Echo: Algorithmic Alignment of GNNs with Distributed Computing
However, this raises two core questions i) How can we enable nodes to gather the required information in a given graph ($\textit{information exchange}$), even if is far away and ii) How can we design an execution framework which enables this information exchange for extrapolation to larger graph sizes ($\textit{algorithmic alignment for extrapolation}$).
SUPClust: Active Learning at the Boundaries
Active learning is a machine learning paradigm designed to optimize model performance in a setting where labeled data is expensive to acquire.
Bridging Diversity and Uncertainty in Active learning with Self-Supervised Pre-Training
This study addresses the integration of diversity-based and uncertainty-based sampling strategies in active learning, particularly within the context of self-supervised pre-trained models.
