Search Results for author:
Found 81 papers, 43 papers with code
CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
With this dataset, we train a return-conditioned multi-agent behaviour model that allows for fine-grained manipulation of agent behaviours by modifying the desired returns for the various reward components.
LitLLM: A Toolkit for Scientific Literature Review
Conducting literature reviews for scientific papers is essential for understanding research, its limitations, and building on existing work.
Capture the Flag: Uncovering Data Insights with Large Language Models
The extraction of a small number of relevant insights from vast amounts of data is a crucial component of data-driven decision-making.
StarVector: Generating Scalable Vector Graphics Code from Images
These visual tokens are pre-pended to the SVG token embeddings, and the sequence is modeled by the StarCoder model using next-token prediction, effectively learning to align the visual and code tokens.
Goal-conditioned GFlowNets for Controllable Multi-Objective Molecular Design
In recent years, in-silico molecular design has received much attention from the machine learning community.
Improving Generalization in Task-oriented Dialogues with Workflows and Action Plans
We perform extensive experiments on the Action-Based Conversations Dataset (ABCD) with T5-small, base and large models, and show that such models: a) are able to more readily generalize to unseen workflows by following the provided plan, and b) are able to generalize to executing unseen actions if they are provided in the plan.
Are Diffusion Models Vision-And-Language Reasoners?
We also measure the stereotypical bias in diffusion models, and find that Stable Diffusion 2. 1 is, for the most part, less biased than Stable Diffusion 1. 5.
Conservative objective models are a special kind of contrastive divergence-based energy model
In this work we theoretically show that conservative objective models (COMs) for offline model-based optimisation (MBO) are a special kind of contrastive divergence-based energy model, one where the energy function represents both the unconditional probability of the input and the conditional probability of the reward variable.
Score-based Diffusion Models in Function Space
They consist of a forward process that perturbs input data with Gaussian white noise and a reverse process that learns a score function to generate samples by denoising.
Language Decision Transformers with Exponential Tilt for Interactive Text Environments
Text-based game environments are challenging because agents must deal with long sequences of text, execute compositional actions using text and learn from sparse rewards.
Visual Question Answering From Another Perspective: CLEVR Mental Rotation Tests
Different types of mental rotation tests have been used extensively in psychology to understand human visual reasoning and perception.
Receptive Field Refinement for Convolutional Neural Networks Reliably Improves Predictive Performance
By further developing and formalizing the analysis of receptive field expansion in convolutional neural networks, we can predict unproductive layers in an automated manner before ever training a model.
Exploring validation metrics for offline model-based optimisation with diffusion models
Measuring the mean reward of generated candidates over this approximation is one such `validation metric', whereas we are interested in a more fundamental question which is finding which validation metrics correlate the most with the ground truth.
Attention-based Neural Cellular Automata
We introduce an instance of this class named $\textit{Vision Transformer Cellular Automata}$ (ViTCA).
Bridging the Gap Between Target Networks and Functional Regularization
However, learning the value function via bootstrapping often leads to unstable training due to fast-changing target values.
Score-based Denoising Diffusion with Non-Isotropic Gaussian Noise Models
Generative models based on denoising diffusion techniques have led to an unprecedented increase in the quality and diversity of imagery that is now possible to create with neural generative models.
Using Graph Algorithms to Pretrain Graph Completion Transformers
While using our new path-finding algorithm as a pretraining signal provides 2-3% MRR improvements, we show that pretraining on all signals together gives the best knowledge graph completion results.
SMPL-IK: Learned Morphology-Aware Inverse Kinematics for AI Driven Artistic Workflows
Inverse Kinematics (IK) systems are often rigid with respect to their input character, thus requiring user intervention to be adapted to new skeletons.
Improving Meta-Learning Generalization with Activation-Based Early-Stopping
Specifically, we analyze the evolution, during meta-training, of the neural activations at each hidden layer, on a small set of unlabelled support examples from a single task of the target tasks distribution, as this constitutes a minimal and justifiably accessible information from the target problem.
MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation
We train the model in a manner where we randomly and independently mask all the past frames or all the future frames.
Ranked #4 on
Video Generation
on BAIR Robot Pushing
Overcoming challenges in leveraging GANs for few-shot data augmentation
In this paper, we explore the use of GAN-based few-shot data augmentation as a method to improve few-shot classification performance.
Does Entity Abstraction Help Generative Transformers Reason?
We study the utility of incorporating entity type abstractions into pre-trained Transformers and test these methods on four NLP tasks requiring different forms of logical reasoning: (1) compositional language understanding with text-based relational reasoning (CLUTRR), (2) abductive reasoning (ProofWriter), (3) multi-hop question answering (HotpotQA), and (4) conversational question answering (CoQA).
Direct Behavior Specification via Constrained Reinforcement Learning
The standard formulation of Reinforcement Learning lacks a practical way of specifying what are admissible and forbidden behaviors.
Learning to Guide and to Be Guided in the Architect-Builder Problem
ABIG results in a low-level, high-frequency, guiding communication protocol that not only enables an architect-builder pair to solve the task at hand, but that can also generalize to unseen tasks.
From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains.
Early-Stopping for Meta-Learning: Estimating Generalization from the Activation Dynamics
To this end, we empirically show that as meta-training progresses, a model's generalization to a target distribution of novel tasks can be estimated by analysing the dynamics of its neural activations.
Simple Video Generation using Neural ODEs
Despite having been studied to a great extent, the task of conditional generation of sequences of frames, or videos, remains extremely challenging.
Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
A central goal for AI and causality is thus the joint discovery of abstract representations and causal structure.
Multi-Resolution Continuous Normalizing Flows
In this work we introduce a Multi-Resolution variant of such models (MRCNF), by characterizing the conditional distribution over the additional information required to generate a fine image that is consistent with the coarse image.
Ranked #6 on
Image Generation
on ImageNet 64x64
(Bits per dim metric)
Bridging the Gap Between Target Networks and Functional Regularization
Our findings emphasize that Functional Regularization can be used as a drop-in replacement for Target Networks and result in performance improvement.
Improving Continuous Normalizing Flows using a Multi-Resolution Framework
Recent work has shown that Continuous Normalizing Flows (CNFs) can serve as generative models of images with exact likelihood calculation and invertible generation/density estimation.
Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction
AutoBots can produce either the trajectory of one ego-agent or a distribution over the future trajectories for all agents in the scene.
Robust Motion In-betweening
To quantitatively evaluate performance on transitions and generalizations to longer time horizons, we present well-defined in-betweening benchmarks on a subset of the widely used Human3. 6M dataset and on LaFAN1, a novel high quality motion capture dataset that is more appropriate for transition generation.
Ranked #3 on
Motion Synthesis
on LaFAN1
Visual Imitation with Reinforcement Learning using Recurrent Siamese Networks
It would be desirable for a reinforcement learning (RL) based agent to learn behaviour by merely watching a demonstration.
Dependency Structure Discovery from Interventions
Promising results have driven a recent surge of interest in continuous optimization methods for Bayesian network structure learning from observational data.
Mem2Mem: Learning to Summarize Long Texts with Memory Compression and Transfer
We introduce Mem2Mem, a memory-to-memory mechanism for hierarchical recurrent neural network based encoder decoder architectures and we explore its use for abstractive document summarization.
Bijective-Contrastive Estimation
In this work, we propose Bijective-Contrastive Estimation (BCE), a classification-based learning criterion for energy-based models.
COVI-AgentSim: an Agent-based Model for Evaluating Methods of Digital Contact Tracing
The rapid global spread of COVID-19 has led to an unprecedented demand for effective methods to mitigate the spread of the disease, and various digital contact tracing (DCT) methods have emerged as a component of the solution.
Learning to Summarize Long Texts with Memory Compression and Transfer
We introduce Mem2Mem, a memory-to-memory mechanism for hierarchical recurrent neural network based encoder decoder architectures and we explore its use for abstractive document summarization.
Reinforcement Learning with Random Delays
Action and observation delays commonly occur in many Reinforcement Learning applications, such as remote control scenarios.
Measuring Systematic Generalization in Neural Proof Generation with Transformers
We observe that models that are not trained to generate proofs are better at generalizing to problems based on longer proofs.
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data
Through this construction (a hypernetwork adapter), we achieve more efficient parameter sharing and mitigate forgetting by keeping half of the weights of a pretrained model fixed.
Ranked #1 on
Natural Language Inference
on SciTail
Action-Based Representation Learning for Autonomous Driving
Human drivers produce a vast amount of data which could, in principle, be used to improve autonomous driving systems.
Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization
Adversarial Imitation Learning alternates between learning a discriminator -- which tells apart expert's demonstrations from generated ones -- and a generator's policy to produce trajectories that can fool this discriminator.
Towards an Unsupervised Method for Model Selection in Few-Shot Learning
The study of generalization of neural networks in gradient-based meta-learning has recently great research interest.
AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation
Entropy is ubiquitous in machine learning, but it is in general intractable to compute the entropy of the distribution of an arbitrary continuous random variable.
On the impressive performance of randomly weighted encoders in summarization tasks
In this work, we investigate the performance of untrained randomly initialized encoders in a general class of sequence to sequence models and compare their performance with that of fully-trained encoders on the task of abstractive summarization.
Structural Inductive Biases in Emergent Communication
In order to communicate, humans flatten a complex representation of ideas and their attributes into a single word or a sentence.
Real-Time Reinforcement Learning
Markov Decision Processes (MDPs), the mathematical framework underlying most algorithms in Reinforcement Learning (RL), are often used in a way that wrongfully assumes that the state of an agent's environment does not change during action selection.
Neural Multisensory Scene Inference
For embodied agents to infer representations of the underlying 3D physical world they inhabit, they should efficiently combine multisensory cues from numerous trials, e. g., by looking at and touching objects.
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
We present a method to produce abstractive summaries of long documents that exceed several thousand words via neural abstractive summarization.
Ranked #18 on
Text Summarization
on Pubmed
Interactive Language Learning by Question Answering
In QAit, an agent must interact with a partially observable text-based environment to gather information required to answer questions.
Interactive Machine Comprehension with Information Seeking Agents
Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA).
Promoting Coordination through Policy Regularization in Multi-Agent Deep Reinforcement Learning
Finally, we analyze the effects of our proposed methods on the policies that our agents learn and show that our methods successfully enforce the qualities that we propose as proxies for coordinated behaviors.
Towards Understanding Generalization in Gradient-Based Meta-Learning
We also show that coherence of meta-test gradients, measured by the average inner product between the task-specific gradient vectors evaluated at meta-train solution, is also correlated with generalization.
Supervise Thyself: Examining Self-Supervised Representations in Interactive Environments
Self-supervised methods, wherein an agent learns representations solely by observing the results of its actions, become crucial in environments which do not provide a dense reward signal or have labels.
Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study
Neural generative models have been become increasingly popular when building conversational agents.
CLAREL: Classification via retrieval loss for zero-shot learning
We address the problem of learning fine-grained cross-modal representations.
Adversarial Mixup Resynthesizers
In this paper, we explore new approaches to combining information encoded within the learned representations of autoencoders.
On Adversarial Mixup Resynthesis
In this paper, we explore new approaches to combining information encoded within the learned representations of auto-encoders.
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
We show that causal structures can be parameterized via continuous variables and learned end-to-end.
Towards Learning to Imitate from a Single Video Demonstration
We approach this challenge using contrastive training to learn a reward function comparing an agent's behaviour with a single demonstration.
The Liver Tumor Segmentation Benchmark (LiTS)
In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018.
Dataflow-based Joint Quantization of Weights and Activations for Deep Neural Networks
This paper addresses a challenging problem - how to reduce energy consumption without incurring performance drop when deploying deep neural networks (DNNs) at the inference stage.
Recurrent Transition Networks for Character Locomotion
Manually authoring transition animations for a complete locomotion system can be a tedious and time-consuming task, especially for large games that allow complex and constrained locomotion movements, where the number of transitions grows exponentially with the number of states.
Unsupervised Depth Estimation, 3D Face Rotation and Replacement
We present an unsupervised approach for learning to estimate three dimensional (3D) facial structure from a single image while also predicting 3D viewpoint transformations that match a desired pose and facial geometry.
Improving Landmark Localization with Semi-Supervised Learning
First, we propose the framework of sequential multitasking and explore it here through an architecture for landmark localization where training with class labels acts as an auxiliary signal to guide the landmark localization on unlabeled data.
Ranked #41 on
Face Alignment
on 300W
A step towards procedural terrain generation with GANs
Procedural terrain generation for video games has been traditionally been done with smartly designed but handcrafted algorithms that generate heightmaps.
Adversarial Generation of Natural Language
Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation.
Unimodal probability distributions for deep ordinal classification
Probability distributions produced by the cross-entropy loss for ordinal classification problems can possess undesired properties.
ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events
We present a dataset, ExtremeWeather, to encourage machine learning research in this area and to help facilitate further work in understanding and mitigating the effects of climate change.
A simple squared-error reformulation for ordinal classification
In this paper, we explore ordinal classification (in the context of deep neural networks) through a simple modification of the squared error loss which not only allows it to not only be sensitive to class ordering, but also allows the possibility of having a discrete probability distribution over the classes.
A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering
In addition to presenting statistics and a description of the dataset, we perform a detailed analysis of 5 different models' predictions, and compare these with human performance.
Convolutional Residual Memory Networks
Our experiments and analysis explore the importance of the memory mechanism, network depth, breadth, and predictive performance.
Movie Description
In addition we also collected and aligned movie scripts used in prior work and compare the two sources of descriptions.
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation
Deep neural networks with alternating convolutional, max-pooling and decimation layers are widely used in state of the art architectures for computer vision.
Recurrent Semi-supervised Classification and Constrained Adversarial Generation with Motion Capture Data
We find that using such constraints allow to stabilize the training of recurrent adversarial architectures for animation generation.
EmoNets: Multimodal deep learning approaches for emotion recognition in video
The task of the emotion recognition in the wild (EmotiW) Challenge is to assign one of seven emotions to short video clips extracted from Hollywood style movies.
Using Descriptive Video Services to Create a Large Data Source for Video Annotation Research
DVS is an audio narration describing the visual elements and actions in a movie for the visually impaired.
Describing Videos by Exploiting Temporal Structure
In this context, we propose an approach that successfully takes into account both the local and global temporal structure of videos to produce descriptions.
