Search Results for author: Roger Wattenhofer

Found 68 papers, 32 papers with code

TempCaps: A Capsule Network-based Embedding Model for Temporal Knowledge Graph Completion

1 code implementation spnlp (ACL) 2022 Guirong Fu, Zhao Meng, Zhen Han, Zifeng Ding, Yunpu Ma, Matthias Schubert, Volker Tresp, Roger Wattenhofer

In this paper, we tackle the temporal knowledge graph completion task by proposing TempCaps, which is a Capsule network-based embedding model for Temporal knowledge graph completion.

Entity Embeddings Temporal Knowledge Graph Completion

On Isotropy Calibration of Transformer Models

no code implementations insights (ACL) 2022 Yue Ding, Karolis Martinkus, Damian Pascual, Simon Clematide, Roger Wattenhofer

Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone.

Cascaded Beam Search: Plug-and-Play Terminology-Forcing For Neural Machine Translation

no code implementations23 May 2023 Frédéric Odermatt, Béni Egressy, Roger Wattenhofer

Our plug-and-play approach performs on par with the winning submissions without using a domain-specific language model and with no additional training.

Language Modelling Machine Translation +1

Discovering Graph Generation Algorithms

no code implementations25 Apr 2023 Mihai Babiac, Karolis Martinkus, Roger Wattenhofer

We provide a novel approach to construct generative models for graphs.

Graph Generation

Visual Abstraction and Reasoning through Language

no code implementations7 Mar 2023 Giacomo Camposampiero, Loic Houmard, Benjamin Estermann, Joël Mathys, Roger Wattenhofer

While Artificial Intelligence (AI) models have achieved human or even superhuman performance in narrowly defined applications, they still struggle to show signs of broader and more flexible intelligence.

Electrode Clustering and Bandpass Analysis of EEG Data for Gaze Estimation

no code implementations19 Feb 2023 Ard Kastrati, Martyna Beata Plomecka, Joël Küchler, Nicolas Langer, Roger Wattenhofer

In this study, we validate the findings of previously published papers, showing the feasibility of an Electroencephalography (EEG) based gaze estimation.

Electroencephalogram (EEG) Gaze Estimation

FedHQL: Federated Heterogeneous Q-Learning

no code implementations26 Jan 2023 Flint Xiaofeng Fan, Yining Ma, Zhongxiang Dai, Cheston Tan, Bryan Kian Hsiang Low, Roger Wattenhofer

Federated Reinforcement Learning (FedRL) encourages distributed agents to learn collectively from each other's experience to improve their performance without exchanging their raw trajectories.

Q-Learning reinforcement-learning +1

DeFi Lending During The Merge

no code implementations23 Jan 2023 Lioba Heimbach, Eric Schertenleib, Roger Wattenhofer

In the lead-up to The Merge: Ethereum blockchain's transition from proof-of-work (PoW) to proof-of-stake (PoS), a fraction of the Ethereum ecosystem announced plans of continuing with a PoW-chain.

POS

Learning Graph Algorithms With Recurrent Graph Neural Networks

1 code implementation9 Dec 2022 Florian Grötschla, Joël Mathys, Roger Wattenhofer

In order to scale, we focus on a recurrent architecture design that can learn simple graph problems end to end on smaller graphs and then extrapolate to larger instances.

Automating Rigid Origami Design

1 code implementation20 Nov 2022 Jeremia Geiger, Karolis Martinkus, Oliver Richter, Roger Wattenhofer

Rigid origami has shown potential in large diversity of practical applications.

Neural Combinatorial Logic Circuit Synthesis from Input-Output Examples

1 code implementation29 Oct 2022 Peter Belcak, Roger Wattenhofer

We propose a novel, fully explainable neural approach to synthesis of combinatorial logic circuits from input-output examples.

Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations

1 code implementation29 Oct 2022 Yu Fei, Ping Nie, Zhao Meng, Roger Wattenhofer, Mrinmaya Sachan

We further explore the applicability of our clustering approach by evaluating it on 14 datasets with more diverse topics, text lengths, and numbers of classes.

Sentence Embeddings Sentiment Analysis +5

Diffusion Models for Graphs Benefit From Discrete State Spaces

1 code implementation4 Oct 2022 Kilian Konstantin Haefeli, Karolis Martinkus, Nathanaël Perraudin, Roger Wattenhofer

Denoising diffusion probabilistic models and score matching models have proven to be very powerful for generative tasks.

Denoising

A Neural Model for Regular Grammar Induction

1 code implementation23 Sep 2022 Peter Belcák, David Hofer, Roger Wattenhofer

Grammatical inference is a classical problem in computational learning theory and a topic of wider influence in natural language processing.

Learning Theory

Periodic Extrapolative Generalisation in Neural Networks

1 code implementation21 Sep 2022 Peter Belcák, Roger Wattenhofer

The learning of the simplest possible computational pattern -- periodicity -- is an open problem in the research of strong generalisation in neural networks.

Benchmarking

FACT: Learning Governing Abstractions Behind Integer Sequences

no code implementations20 Sep 2022 Peter Belcák, Ard Kastrati, Flavio Schenker, Roger Wattenhofer

Integer sequences are of central importance to the modeling of concepts admitting complete finitary descriptions.

Benchmarking

Deterministic Graph-Walking Program Mining

no code implementations22 Aug 2022 Peter Belcak, Roger Wattenhofer

These programs characterise linear long-distance relationships between the given two vertex sets in the context of the whole graph.

Exploring Price Accuracy on Uniswap V3 in Times of Distress

no code implementations20 Aug 2022 Lioba Heimbach, Eric Schertenleib, Roger Wattenhofer

Financial markets have evolved over centuries, and exchanges have converged to rely on the order book mechanism for market making.

Agent-based Graph Neural Networks

1 code implementation22 Jun 2022 Karolis Martinkus, Pál András Papp, Benedikt Schesch, Roger Wattenhofer

AgentNet is inspired by sublinear algorithms, featuring a computational complexity that is independent of the graph size.

Graph Classification

A Deep Learning Approach for the Segmentation of Electroencephalography Data in Eye Tracking Applications

1 code implementation17 Jun 2022 Lukas Wolf, Ard Kastrati, Martyna Beata Płomecka, Jie-Ming Li, Dustin Klebe, Alexander Veicht, Roger Wattenhofer, Nicolas Langer

Here, we introduce DETRtime, a novel framework for time-series segmentation that creates ocular event detectors that do not require additionally recorded eye-tracking modality and rely solely on EEG data.

Electroencephalogram (EEG) Event Detection +1

The Economics of Automated Market Makers

no code implementations9 Jun 2022 Robin Fritsch, Samuel Käser, Roger Wattenhofer

This paper studies the question whether automated market maker protocols such as Uniswap can sustainably retain a portion of their trading fees for the protocol.

Graph Neural Networks with Precomputed Node Features

no code implementations1 Jun 2022 Beni Egressy, Roger Wattenhofer

Most Graph Neural Networks (GNNs) cannot distinguish some graphs or indeed some pairs of nodes within a graph.

Asynchronous Neural Networks for Learning in Graphs

no code implementations24 May 2022 Lukas Faber, Roger Wattenhofer

This paper studies asynchronous message passing (AMP), a new paradigm for applying neural network based learning to graphs.

Distributed Computing Graph Classification

Risks and Returns of Uniswap V3 Liquidity Providers

no code implementations18 May 2022 Lioba Heimbach, Eric Schertenleib, Roger Wattenhofer

However, Uniswap V3 requires far more decisions from liquidity providers than previous DEX designs.

Management

SPECTRE: Spectral Conditioning Helps to Overcome the Expressivity Limits of One-shot Graph Generators

1 code implementation4 Apr 2022 Karolis Martinkus, Andreas Loukas, Nathanaël Perraudin, Roger Wattenhofer

We approach the graph generation problem from a spectral perspective by first generating the dominant parts of the graph Laplacian spectrum and then building a graph matching these eigenvalues and eigenvectors.

Graph Generation Graph Matching

A Theoretical Comparison of Graph Neural Network Extensions

no code implementations30 Jan 2022 Pál András Papp, Roger Wattenhofer

We study and compare different Graph Neural Network extensions that increase the expressive power of GNNs beyond the Weisfeiler-Leman test.

The Price of Majority Support

no code implementations28 Jan 2022 Robin Fritsch, Roger Wattenhofer

We consider the ratio between this number and the number of matches of the overall best outcome which may not have majority support.

DropGNN: Random Dropouts Increase the Expressiveness of Graph Neural Networks

1 code implementation NeurIPS 2021 Pál András Papp, Karolis Martinkus, Lukas Faber, Roger Wattenhofer

In DropGNNs, we execute multiple runs of a GNN on the input graph, with some of the nodes randomly and independently dropped in each of these runs.

Graph Classification Graph Regression

On Isotropy Calibration of Transformers

no code implementations27 Sep 2021 Yue Ding, Karolis Martinkus, Damian Pascual, Simon Clematide, Roger Wattenhofer

Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone.

BERT is Robust! A Case Against Synonym-Based Adversarial Examples in Text Classification

no code implementations15 Sep 2021 Jens Hauser, Zhao Meng, Damián Pascual, Roger Wattenhofer

We combine a human evaluation of individual word substitutions and a probabilistic analysis to show that between 96% and 99% of the analyzed attacks do not preserve semantics, indicating that their success is mainly based on feeding poor data to the model.

Data Augmentation text-classification +1

Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based Attacks

1 code implementation Findings (NAACL) 2022 Zhao Meng, Yihan Dong, Mrinmaya Sachan, Roger Wattenhofer

In this paper, we present an approach to improve the robustness of BERT language models against word substitution-based adversarial attacks by leveraging adversarial perturbations for self-supervised contrastive learning.

Adversarial Attack Contrastive Learning +1

Debt Swapping for Risk Mitigation in Financial Networks

no code implementations1 Jun 2021 Pál András Papp, Roger Wattenhofer

We first show that there can be no positive swap for any pair of banks in a static financial system, or when a shock hits each bank in the network proportionally.

Behavior of Liquidity Providers in Decentralized Exchanges

no code implementations28 May 2021 Lioba Heimbach, Ye Wang, Roger Wattenhofer

In this paper, we aim to understand how liquidity providers react to market information and how they benefit from providing liquidity in DEXes.

Cyclic Arbitrage in Decentralized Exchanges

no code implementations21 Apr 2021 Ye Wang, Yan Chen, Haotian Wu, Liyi Zhou, Shuiguang Deng, Roger Wattenhofer

We find that traders have executed 292, 606 cyclic arbitrages over eleven months and exploited more than 138 million USD in revenue.

Towards BERT-based Automatic ICD Coding: Limitations and Opportunities

no code implementations NAACL (BioNLP) 2021 Damian Pascual, Sandro Luck, Roger Wattenhofer

Unlike the general trend in language processing, no transformer model has been reported to reach high performance on this task.

Of Non-Linearity and Commutativity in BERT

1 code implementation12 Jan 2021 Sumu Zhao, Damian Pascual, Gino Brunner, Roger Wattenhofer

In this work we provide new insights into the transformer architecture, and in particular, its best-known variant, BERT.

Inductive Bias

KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation

1 code implementation ACL 2021 Yiran Xing, Zai Shi, Zhao Meng, Gerhard Lakemeyer, Yunpu Ma, Roger Wattenhofer

We present Knowledge Enhanced Multimodal BART (KM-BART), which is a Transformer-based sequence-to-sequence model capable of reasoning about commonsense knowledge from multimodal inputs of images and texts.

Knowledge Graphs Language Modelling

Unsupervised Task Clustering for Multi-Task Reinforcement Learning

1 code implementation1 Jan 2021 Johannes Ackermann, Oliver Paul Richter, Roger Wattenhofer

We show the generality of our approach by evaluating on simple discrete and continuous control tasks, as well as complex bipedal walker tasks and Atari games.

Atari Games Continuous Control +4

Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation

1 code implementation31 Dec 2020 Damian Pascual, Beni Egressy, Florian Bolli, Roger Wattenhofer

Given that state-of-the-art language models are too large to be trained from scratch in a manageable time, it is desirable to control these models without re-training them.

Language Modelling Machine Translation +2

Contrastive Graph Neural Network Explanation

1 code implementation26 Oct 2020 Lukas Faber, Amin K. Moghaddam, Roger Wattenhofer

Graph Neural Networks achieve remarkable results on problems with structured data but come as black-box predictors.

Learning Lower Bounds for Graph Exploration With Reinforcement Learning

no code implementations NeurIPS Workshop LMCA 2020 Jorel Elmiger, Lukas Faber, Pankaj Khanchandani, Oliver Paul Richter, Roger Wattenhofer

Given there are quadratically many possible edges in a graph and each subset of edges is a possible solution, this yields unfeasibly large search spaces even for few nodes.

reinforcement-learning Reinforcement Learning (RL)

A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples

no code implementations COLING 2020 Zhao Meng, Roger Wattenhofer

Generating adversarial examples for natural language is hard, as natural language consists of discrete symbols, and examples are often of variable lengths.

Brain2Word: Decoding Brain Activity for Language Generation

1 code implementation10 Sep 2020 Nicolas Affolter, Beni Egressy, Damian Pascual, Roger Wattenhofer

In the case of language stimuli, recent studies have shown that it is possible to decode fMRI scans into an embedding of the word a subject is reading.

Brain Decoding Text Generation +1

Medley2K: A Dataset of Medley Transitions

no code implementations25 Aug 2020 Lukas Faber, Sandro Luck, Damian Pascual, Andreas Roth, Gino Brunner, Roger Wattenhofer

The automatic generation of medleys, i. e., musical pieces formed by different songs concatenated via smooth transitions, is not well studied in the current literature.

Normalized Attention Without Probability Cage

2 code implementations19 May 2020 Oliver Richter, Roger Wattenhofer

Attention architectures are widely used; they recently gained renewed popularity with Transformers yielding a streak of state of the art results.

Neural Status Registers

no code implementations15 Apr 2020 Lukas Faber, Roger Wattenhofer

Standard Neural Networks can learn mathematical operations, but they do not extrapolate.

Telling BERT's full story: from Local Attention to Global Aggregation

no code implementations EACL 2021 Damian Pascual, Gino Brunner, Roger Wattenhofer

This way, we propose a distinction between local patterns revealed by attention and global patterns that refer back to the input, and analyze BERT from both angles.

Divide and Scale: Formalization and Roadmap to Robust Sharding

no code implementations23 Oct 2019 Georgia Avarikioti, Eleftherios Kokoris-Kogias, Roger Wattenhofer

Sharding distributed ledgers is a promising on-chain solution for scaling blockchains but lacks formal grounds, nurturing skepticism on whether such complex systems can scale blockchains securely.

Distributed, Parallel, and Cluster Computing

The Frechet Distance of training and test distribution predicts the generalization gap

no code implementations25 Sep 2019 Julian Zilly, Hannes Zilly, Oliver Richter, Roger Wattenhofer, Andrea Censi, Emilio Frazzoli

Empirically across several data domains, we substantiate this viewpoint by showing that test performance correlates strongly with the distance in data distributions between training and test set.

Learning Theory Transfer Learning

ABC: Asynchronous Blockchain without Consensus

1 code implementation24 Sep 2019 Jakub Sliwinski, Roger Wattenhofer

There is a preconception that a blockchain needs consensus.

Cryptography and Security

On Identifiability in Transformers

no code implementations ICLR 2020 Gino Brunner, Yang Liu, Damián Pascual, Oliver Richter, Massimiliano Ciaramita, Roger Wattenhofer

We show that, for sequences longer than the attention head dimension, attention weights are not identifiable.

Synthetic Epileptic Brain Activities Using Generative Adversarial Networks

1 code implementation22 Jul 2019 Damian Pascual, Amir Aminifar, David Atienza, Philippe Ryvlin, Roger Wattenhofer

In this work, we generate synthetic seizure-like brain electrical activities, i. e., EEG signals, that can be used to train seizure detection algorithms, alleviating the need for recorded data.

Electroencephalogram (EEG) Seizure Detection

Learning Policies through Quantile Regression

no code implementations27 Jun 2019 Oliver Richter, Roger Wattenhofer

Policy gradient based reinforcement learning algorithms coupled with neural networks have shown success in learning complex policies in the model free continuous action space control setting.

regression

Using State Predictions for Value Regularization in Curiosity Driven Deep Reinforcement Learning

1 code implementation30 Sep 2018 Gino Brunner, Manuel Fritsche, Oliver Richter, Roger Wattenhofer

Learning in sparse reward settings remains a challenge in Reinforcement Learning, which is often addressed by using intrinsic rewards.

reinforcement-learning Reinforcement Learning (RL)

The Urban Last Mile Problem: Autonomous Drone Delivery to Your Balcony

1 code implementation21 Sep 2018 Gino Brunner, Bence Szebedy, Simon Tanner, Roger Wattenhofer

The drop-off location could, e. g., be on a balcony or porch, and simply needs to be indicated by a visual marker on the wall or window.

Robotics Systems and Control

Symbolic Music Genre Transfer with CycleGAN

5 code implementations20 Sep 2018 Gino Brunner, Yuyi Wang, Roger Wattenhofer, Sumu Zhao

In this paper we apply such a model to symbolic music and show the feasibility of our approach for music genre transfer.

Music Genre Transfer Style Transfer

MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer

no code implementations20 Sep 2018 Gino Brunner, Andres Konrad, Yuyi Wang, Roger Wattenhofer

The interpolations smoothly change pitches, dynamics and instrumentation to create a harmonic bridge between two music pieces.

Style Transfer

Natural Language Multitasking: Analyzing and Improving Syntactic Saliency of Hidden Representations

no code implementations18 Jan 2018 Gino Brunner, Yuyi Wang, Roger Wattenhofer, Michael Weigelt

We train multi-task autoencoders on linguistic tasks and analyze the learned hidden sentence representations.

Translation

Teaching a Machine to Read Maps with Deep Reinforcement Learning

1 code implementation20 Nov 2017 Gino Brunner, Oliver Richter, Yuyi Wang, Roger Wattenhofer

Localization and navigation is also an important problem in domains such as robotics, and has recently become a focus of the deep reinforcement learning community.

Navigate reinforcement-learning +1

Cannot find the paper you are looking for? You can Submit a new open access paper.