Search Results for author:
Found 145 papers, 72 papers with code
SemanticAdv: Generating Adversarial Examples via Attribute-conditioned Image Editing
Deep neural networks (DNNs) have achieved great successes in various vision applications due to their strong expressive power.
Fine-grained Text Style Transfer with Diffusion-Based Language Models
Diffusion probabilistic models have shown great success in generating high-quality images controllably, and researchers have tried to utilize this controllability into text generation domain.
Discriminator-Guided Multi-step Reasoning with Language Models
In the context of multi-step reasoning, language models (LMs) probabilities are often miscalibrated -- solutions with high probabilities are not always correct.
Artificial-intelligence-based molecular classification of diffuse gliomas using rapid, label-free optical imaging
Molecular classification has transformed the management of brain tumors by enabling more accurate prognostication and personalized treatment.
A Picture is Worth a Thousand Words: Language Models Plan from Pixels
Planning is an important capability of artificial agents that perform long-horizon tasks in real-world environments.
Lightweight feature encoder for wake-up word detection based on self-supervised speech representation
Self-supervised learning method that provides generalized speech representations has recently received increasing attention.
Hierarchical discriminative learning improves visual representations of biomedical microscopy
Importantly, sampled patches from WSIs of a patient's tumor are a diverse set of image examples that capture the same underlying cancer diagnosis.
Preference Transformer: Modeling Human Preferences using Transformers for RL
In this paper, we present Preference Transformer, a neural architecture that models human preferences using transformers.
Unsupervised Task Graph Generation from Instructional Video Transcripts
This work explores the problem of generating task graphs of real-world activities.
Multimodal Subtask Graph Generation from Instructional Videos
Real-world tasks consist of multiple inter-dependent subtasks (e. g., a dirty pan needs to be washed before it can be used for cooking).
Composing Task Knowledge with Modular Successor Feature Approximators
Recently, the Successor Features and Generalized Policy Improvement (SF&GPI) framework has been proposed as a method for learning, composing, and transferring predictive knowledge and behavior.
Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers
Since the recent advent of regulations for data protection (e. g., the General Data Protection Regulation), there has been increasing demand in deleting information learned from sensitive data in pre-trained models without retraining from scratch.
Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matching
Despite surprising performance on zero-shot transfer, pre-training a large-scale multimodal model is often prohibitive as it requires a huge amount of data and computing resources.
Neural Shape Compiler: A Unified Framework for Transforming between Text, Point Cloud, and Program
On Text2Shape, ShapeGlot, ABO, Genre, and Program Synthetic datasets, Neural Shape Compiler shows strengths in $\textit{Text}$ $\Longrightarrow$ $\textit{Point Cloud}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Text}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Program}$, and Point Cloud Completion tasks.
Significantly Improving Zero-Shot X-ray Pathology Classification via Fine-tuning Pre-trained Image-Text Encoders
However, large-scale and high-quality data to train powerful neural networks are rare in the medical domain as the labeling must be done by qualified experts.
Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost
The forward and backward cost are thus linear to the number of edges, which each attention head can also choose flexibly based on the input.
UniCLIP: Unified Framework for Contrastive Language-Image Pre-training
Pre-training vision-language models with contrastive objectives has shown promising results that are both scalable to large uncurated datasets and transferable to many downstream applications.
Grouping-matrix based Graph Pooling with Adaptive Number of Clusters
Conventional methods ask users to specify an appropriate number of clusters as a hyperparameter, then assume that all input graphs share the same number of clusters.
Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Tasks
Meta reinforcement learning (meta-RL) aims to learn a policy solving a set of training tasks simultaneously and quickly adapting to new tasks.
Pure Transformers are Powerful Graph Learners
We show that standard Transformers without graph-specific modifications can lead to promising results in graph learning both in theory and practice.
Ranked #7 on
Graph Regression
on PCQM4Mv2-LSC
Towards More Objective Evaluation of Class Incremental Learning: Representation Learning Perspective
While the common method for evaluating CIL algorithms is based on average test accuracy for all learned classes, we argue that maximizing accuracy alone does not necessarily lead to effective CIL algorithms.
OpenSRH: optimizing brain tumor surgery using intraoperative stimulated Raman histology
Here, we present OpenSRH, the first public dataset of clinical SRH images from 300+ brain tumors patients and 1300+ unique whole slide optical images.
Few-shot Subgoal Planning with Language Models
Pre-trained large language models have shown successful progress in many language understanding benchmarks.
Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization
Different from the previous meta-rl methods trying to directly infer the unstructured task embedding, our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks, and use it as a prior to improve the task inference in testing.
Hierarchical Reinforcement Learning
Meta Reinforcement Learning
+2
Few-shot Reranking for Multi-hop QA via Language Model Prompting
To alleviate the need for a large number of labeled question-document pairs for retriever training, we propose PromptRank, which relies on large language models prompting for multi-hop path reranking.
RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects
We test the effectiveness of our representation on the human image harmonization task by predicting shading that is coherent with a given background image.
Simple and Effective Synthesis of Indoor 3D Scenes
We study the problem of synthesizing immersive 3D indoor scenes from one or more images.
Learning Parameterized Task Structure for Generalization to Unseen Entities
These subtasks are defined in terms of entities (e. g., "apple", "pear") that can be recombined to form new subtasks (e. g., "pickup apple", and "pickup pear").
SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning
In order to leverage unlabeled samples for reward learning, we infer pseudo-labels of the unlabeled samples based on the confidence of the preference predictor.
Enriched CNN-Transformer Feature Aggregation Networks for Super-Resolution
Recent transformer-based super-resolution (SR) methods have achieved promising results against conventional CNN-based methods.
Lipschitz-constrained Unsupervised Skill Discovery
To address this issue, we propose Lipschitz-constrained Skill Discovery (LSD), which encourages the agent to discover more diverse, dynamic, and far-reaching skills.
Environment Generation for Zero-Shot Compositional Reinforcement Learning
We learn to generate environments composed of multiple pages or rooms, and train RL agents capable of completing wide-range of complex tasks in those environments.
L-Verse: Bidirectional Generation Between Image and Text
Unlike other models, BiART can distinguish between image (or text) as a conditional reference and a generation target.
Ranked #1 on
Image Reconstruction
on ImageNet 256x256
Improving Transferability of Representations via Augmentation-Aware Self-Supervision
Recent unsupervised representation learning methods have shown to be effective in a range of vision tasks by learning representations invariant to data augmentations such as random cropping and color jittering.
Successor Feature Landmarks for Long-Horizon Goal-Conditioned Reinforcement Learning
SFL leverages the ability of successor features (SF) to capture transition dynamics, using it to drive exploration by estimating state-novelty and to enable high-level planning by abstracting the state-space as a non-parametric landmark-based graph.
Rapid Automated Analysis of Skull Base Tumor Specimens Using Intraoperative Optical Imaging and Artificial Intelligence
Conclusion: SRH with trained artificial intelligence models can provide rapid and accurate intraoperative analysis of skull base tumor specimens to inform surgical decision-making.
Shortest-Path Constrained Reinforcement Learning for Sparse Reward Tasks
We propose the k-Shortest-Path (k-SP) constraint: a novel constraint on the agent's trajectory that improves the sample efficiency in sparse-reward MDPs.
Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learning
Learning to reach goal states and learning diverse skills through mutual information (MI) maximization have been proposed as principled frameworks for self-supervised reinforcement learning, allowing agents to acquire broadly applicable multitask policies with minimal reward engineering.
Pathdreamer: A World Model for Indoor Navigation
People navigating in unfamiliar buildings take advantage of myriad visual, spatial and semantic cues to efficiently achieve their navigation goals.
Revisiting Hierarchical Approach for Persistent Long-Term Video Prediction
Despite the recent advances in the literature, existing approaches are limited to moderately short-term prediction (less than a few seconds), while extrapolating it to a longer future quickly leads to destruction in structure and content.
Adversarial Environment Generation for Learning to Navigate the Web
The regret objective trains the adversary to design a curriculum of environments that are "just-the-right-challenge" for the navigator agents; our results show that over time, the adversary learns to generate increasingly complex web navigation tasks.
State Entropy Maximization with Random Encoders for Efficient Exploration
Recent exploration methods have proven to be a recipe for improving sample-efficiency in deep reinforcement learning (RL).
Cross-Modal Contrastive Learning for Text-to-Image Generation
The quality of XMC-GAN's output is a major step up from previous models, as we show on three challenging datasets.
Ranked #22 on
Text-to-Image Generation
on COCO
(using extra training data)
Evolving Reinforcement Learning Algorithms
Learning from scratch on simple classical control and gridworld tasks, our method rediscovers the temporal-difference (TD) algorithm.
Demystifying Loss Functions for Classification
It is common to use the softmax cross-entropy loss to train neural networks on classification datasets where a single class label is assigned to each example.
Batch Reinforcement Learning Through Continuation Method
Many real-world applications of reinforcement learning (RL) require the agent to learn from a fixed set of trajectories, without collecting new interactions.
Few-shot Sequence Learning with Transformers
In the simplest setting, we append a token to an input sequence which represents the particular task to be undertaken, and show that the embedding of this token can be optimized on the fly given few labeled examples.
Ode to an ODE
We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the orthogonal group O(d).
Text-to-Image Generation Grounded by Fine-Grained User Attention
Localized Narratives is a dataset with detailed natural language descriptions of images paired with mouse traces that provide a sparse, fine-grained visual grounding for phrases.
Why Do Better Loss Functions Lead to Less Transferable Features?
We show that many objectives lead to statistically significant improvements in ImageNet accuracy over vanilla softmax cross-entropy, but the resulting fixed feature extractors transfer substantially worse to downstream tasks, and the choice of loss has little effect when networks are fully fine-tuned on the new tasks.
Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment
In this work, we show that one can learn object-interaction tasks from scratch without supervision by learning an attentive object-model as an auxiliary task during task learning with an object-centric relational RL agent.
Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning
It maximizes the mutual information between imaginary and real trajectories so that the policy improvement learned from imaginary trajectories can be easily generalized to real trajectories.
Model-based Reinforcement Learning
reinforcement-learning
+1
i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning
Contrastive representation learning has shown to be effective to learn representations from unlabeled data.
Text as Neural Operator: Image Manipulation by Text Instruction
In recent years, text-guided image manipulation has gained increasing attention in the multimedia and computer vision community.
Predictive Information Accelerates Learning in RL
The Predictive Information is the mutual information between the past and the future, I(X_past; X_future).
Understanding and Diagnosing Vulnerability under Adversarial Attacks
Moreover, we design the first diagnostic method to quantify the vulnerability contributed by each layer, which can be used to identify vulnerable parts of model architectures.
An Ode to an ODE
We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the orthogonal group O(d).
CompressNet: Generative Compression at Extremely Low Bitrates
Compressing images at extremely low bitrates (< 0. 1 bpp) has always been a challenging task since the quality of reconstruction significantly reduces due to the strong imposed constraint on the number of bits allocated for the compressed data.
Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning
Model-based reinforcement learning (RL) enjoys several benefits, such as data-efficiency and planning, by learning a model of the environment's dynamics.
Model-based Reinforcement Learning
reinforcement-learning
+1
Time Dependence in Non-Autonomous Neural ODEs
Neural Ordinary Differential Equations (ODEs) are elegant reinterpretations of deep networks where continuous time can replace the discrete notion of depth, ODE solvers perform forward propagation, and the adjoint method enables efficient, constant memory backpropagation.
Improved Consistency Regularization for GANs
Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator.
BRPO: Batch Residual Policy Optimization
In batch reinforcement learning (RL), one often constrains a learned policy to be close to the behavior (data-generating) policy, e. g., by constraining the learned action distribution to differ from the behavior policy by some maximum degree that is the same at each state.
High-Fidelity Synthesis with Disentangled Representation
Despite the simplicity, we show that the proposed method is highly effective, achieving comparable image generation quality to the state-of-the-art methods using the disentangled representation.
Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies
We propose and address a novel few-shot RL problem, where a task is characterized by a subtask graph which describes a set of subtasks and their dependencies that are unknown to the agent.
Efficient Adversarial Training with Transferable Adversarial Examples
Adversarial training is an effective defense method to protect classification models against adversarial attacks.
How Should an Agent Practice?
We present a method for learning intrinsic reward functions to drive the learning of an agent during periods of practice in which extrinsic task rewards are not available.
High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks
Predicting future video frames is extremely challenging, as there are many factors of variation that make up the dynamics of how frames change through time.
Consistency Regularization for Generative Adversarial Networks
Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort.
Ranked #5 on
Conditional Image Generation
on ArtBench-10 (32x32)
Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning
Deep reinforcement learning (RL) agents often fail to generalize to unseen environments (yet semantically similar to trained agents), particularly when they are trained on high-dimensional state spaces, such as images.
Distilling Effective Supervision from Severe Label Noise
For instance, on CIFAR100 with a $40\%$ uniform noise ratio and only 10 trusted labeled data per class, our method achieves $80. 2{\pm}0. 3\%$ classification accuracy, where the error rate is only $1. 4\%$ higher than a neural network trained without label noise.
Self-Imitation Learning via Trajectory-Conditioned Policy for Hard-Exploration Tasks
We propose a new method of learning a trajectory-conditioned policy to imitate diverse trajectories from the agent's own past experiences and show that such self-imitation helps avoid myopic behavior and increases the chance of finding a globally optimal solution for hard-exploration tasks, especially when there are misleading rewards.
Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?
Hierarchical reinforcement learning has demonstrated significant success at solving difficult reinforcement learning (RL) tasks.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Memory Based Trajectory-conditioned Policies for Learning from Sparse Rewards
Reinforcement learning with sparse rewards is challenging because an agent can rarely obtain non-zero rewards and hence, gradient-based optimization of parameterized policies can be incremental and slow.
Data-Efficient Learning for Sim-to-Real Robotic Grasping using Deep Point Cloud Prediction Networks
Training a deep network policy for robot manipulation is notoriously costly and time consuming as it depends on collecting a significant amount of real world data.
Unsupervised Learning of Object Structure and Dynamics from Videos
Extracting and predicting object structure and dynamics from videos without supervision is a major challenge in machine learning.
Ranked #11 on
Video Prediction
on KTH
SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing
In this paper, we aim to explore the impact of semantic manipulation on DNNs predictions by manipulating the semantic attributes of images and generate "unrestricted adversarial examples".
Zero-Shot Entity Linking by Reading Entity Descriptions
First, we show that strong reading comprehension models pre-trained on large unlabeled data can be used to generalize to unseen entities.
Robust Determinantal Generative Classifier for Noisy Labels and Adversarial Attacks
For instance, on CIFAR-10 dataset containing 45% noisy training labels, we improve the test accuracy of a deep model optimized by the state-of-the-art noise-handling training method from33. 34% to 43. 02%.
Information Regularized Neural Networks
We formulate an information-based optimization problem for supervised classification.
Similarity of Neural Network Representations Revisited
We introduce a similarity index that measures the relationship between representational similarity matrices and does not suffer from this limitation.
Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild
Lifelong learning with deep neural networks is well-known to suffer from catastrophic forgetting: the performance on previous tasks drastically degrades when learning a new task.
Robust Inference via Generative Classifiers for Handling Noisy Labels
Large-scale datasets may contain significant proportions of noisy (incorrect) class labels, and it is well-known that modern deep neural networks (DNNs) poorly generalize from such noisy training datasets.
Diversity-Sensitive Conditional Generative Adversarial Networks
We propose a simple yet highly effective method that addresses the mode-collapse problem in the Conditional Generative Adversarial Network (cGAN).
Generative Adversarial Self-Imitation Learning
This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversarial imitation learning framework.
Learning Latent Dynamics for Planning from Pixels
Planning has been very successful for control tasks with known environment dynamics.
Ranked #2 on
Continuous Control
on DeepMind Cheetah Run (Images)
Contingency-Aware Exploration in Reinforcement Learning
This paper investigates whether learning contingency-awareness and controllable aspects of an environment can lead to better exploration in reinforcement learning.
Ranked #5 on
Atari Games
on Atari 2600 Montezuma's Revenge
Content preserving text generation with attribute controls
We propose an adversarial loss to enforce generated samples to be attribute compatible and realistic.
Near-Optimal Representation Learning for Hierarchical Reinforcement Learning
We study the problem of representation learning in goal-conditioned hierarchical reinforcement learning.
Learning Hierarchical Semantic Image Manipulation through Structured Representations
In this work, we present a novel hierarchical framework for semantic image manipulation.
MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
Our model jointly learns a feature embedding for motion modes (that the motion sequence can be reconstructed from) and a feature transformation that represents the transition of one motion mode to the next motion mode.
Ranked #7 on
Human Pose Forecasting
on Human3.6M
(ADE metric)
Hierarchical Reinforcement Learning for Zero-shot Generalization with Subtask Dependencies
We introduce a new RL problem where the agent is required to generalize to a previously-unseen environment characterized by a subtask graph which describes a set of subtasks and their dependencies.
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
Detecting test samples drawn sufficiently far away from the training distribution statistically or adversarially is a fundamental requirement for deploying a good classifier in many real-world machine learning applications.
Ranked #2 on
Out-of-Distribution Detection
on MS-1M vs. IJB-C
Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion
Integrating model-free and model-based approaches in reinforcement learning has the potential to achieve the high performance of model-free algorithms with low sample complexity.
Self-Imitation Learning
This paper proposes Self-Imitation Learning (SIL), a simple off-policy actor-critic algorithm that learns to reproduce the agent's past good decisions.
Ranked #3 on
Atari Games
on Atari 2600 Atlantis
Hierarchical Long-term Video Prediction without Supervision
Much of recent research has been devoted to video prediction and generation, yet most of the previous works have demonstrated only limited success in generating videos on short-term horizons.
Data-Efficient Hierarchical Reinforcement Learning
In this paper, we study how we can develop HRL algorithms that are general, in that they do not make onerous additional assumptions beyond standard RL algorithms, and efficient, in the sense that they can be used with modest numbers of interaction samples, making them suitable for real-world problems such as robotic control.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Neural Kinematic Networks for Unsupervised Motion Retargetting
We propose a recurrent neural network architecture with a Forward Kinematics layer and cycle consistency based adversarial training objective for unsupervised motion retargetting.
Unsupervised Discovery of Object Landmarks as Structural Representations
Deep neural networks can model images with rich latent representations, but they cannot naturally conceptualize structures of object categories in a human-perceptible way.
Unsupervised Facial Landmark Detection
Unsupervised Human Pose Estimation
+1
Hierarchical Novelty Detection for Visual Object Recognition
The essential ingredients of our methods are confidence-calibrated classifiers, data relabeling, and the leave-one-out strategy for modeling novel classes under the hierarchical taxonomy.
An efficient framework for learning sentence representations
In this work we propose a simple and efficient framework for learning sentence representations from unlabelled data.
Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis
We propose a novel hierarchical approach for text-to-image synthesis by inferring semantic layout.
Neural Task Graph Execution
Unlike existing approaches which explicitly describe what the agent should do, our problem only describes properties of subtasks and relationships between them, which requires the agent to perform a complex reasoning to find the optimal subtask to execute.
Unsupervised Hierarchical Video Prediction
Much recent research has been devoted to video prediction and generation, but mostly for short-scale time horizons.
Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
The problem of detecting whether a test sample is from in-distribution (i. e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications.
Addressee and Response Selection in Multi-Party Conversations with Speaker Interaction RNNs
In this paper, we study the problem of addressee and response selection in multi-party conversations.
Learning 6-DOF Grasping Interaction via Deep Geometry-aware 3D Representations
Our contributions are fourfold: (1) To best of our knowledge, we are presenting for the first time a method to learn a 6-DOF grasping net from RGBD input; (2) We build a grasping dataset from demonstrations in virtual reality with rich sensory and interaction annotations.
Value Prediction Network
This paper proposes a novel deep reinforcement learning (RL) architecture, called Value Prediction Network (VPN), which integrates model-free and model-based RL methods into a single neural network.
Ranked #9 on
Atari Games
on Atari 2600 Krull
Decomposing Motion and Content for Natural Video Sequence Prediction
To the best of our knowledge, this is the first end-to-end trainable network architecture with motion and content separation to model the spatiotemporal dynamics for pixel-level future prediction in natural videos.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
As a step towards developing zero-shot task generalization capabilities in reinforcement learning (RL), we introduce a new RL problem where the agent should learn to execute sequences of instructions after learning useful skills that solve subtasks.
Towards Understanding the Invertibility of Convolutional Neural Networks
Several recent works have empirically observed that Convolutional Neural Nets (CNNs) are (approximately) invertible.
Exploring the structure of a real-time, arbitrary neural artistic stylization network
In this paper, we present a method which combines the flexibility of the neural algorithm of artistic style with the speed of fast style transfer networks to allow real-time stylization using any content/style image pair.
Learning to Generate Long-term Future via Hierarchical Prediction
To avoid inherent compounding errors in recursive pixel-level prediction, we propose to first estimate high-level structure in the input frames, then predict how that structure evolves in the future, and finally by observing a single frame from the past and the predicted high-level structure, we construct the future frames without having to observe any of the pixel-level predictions.
Discriminative Bimodal Networks for Visual Localization and Detection with Natural Language Queries
Our training objective encourages better localization on single images, incorporates text phrases in a broad range, and properly pairs image regions with text phrases into positive and negative examples.
Weakly Supervised Semantic Segmentation using Web-Crawled Videos
Our goal is to overcome this limitation with no additional human intervention by retrieving videos relevant to target class labels from web repository, and generating segmentation labels from the retrieved videos to simulate strong supervision for semantic segmentation.
Image Classification
Weakly supervised Semantic Segmentation
+1
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
We demonstrate the ability of the model in generating 3D volume from a single 2D image with three sets of experiments: (1) learning from single-class objects; (2) learning from multi-class objects and (3) testing on novel object classes.
Sentence Ordering and Coherence Modeling using Recurrent Neural Networks
Modeling the structure of coherent texts is a key NLP problem.
Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents
Moreover, unlike other CNN-based models that analyze sentences locally by sliding windows, our system captures both the dependency information within each sentence and relationships across sentences in the same document.
Deep Variational Canonical Correlation Analysis
We present deep variational canonical correlation analysis (VCCA), a deep multi-view learning model that extends the latent variable model interpretation of linear CCA to nonlinear observation models parameterized by deep neural networks.
Learning What and Where to Draw
Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers.
Ranked #11 on
Text-to-Image Generation
on CUB
(using extra training data)
Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification
Inspired by the recent trend toward revisiting the importance of unsupervised learning, we investigate joint supervised and unsupervised learning in a large-scale setting by augmenting existing neural networks with decoding pathways for reconstruction.
Control of Memory, Active Perception, and Action in Minecraft
In this paper, we introduce a new set of reinforcement learning (RL) tasks in Minecraft (a flexible 3D world).
Learning Deep Representations of Fine-grained Visual Descriptions
State-of-the-art methods for zero-shot visual recognition formulate learning as a joint embedding problem of images and side information.
Generative Adversarial Text to Image Synthesis
Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal.
Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games
We present an adaptation of PGRD (policy-gradient for reward-design) for learning a reward-bonus function to improve UCT (a MCTS algorithm).
Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units
Recently, convolutional neural networks (CNNs) have been used as a powerful tool to solve many problems of machine learning and computer vision.
Object Contour Detection with a Fully Convolutional Encoder-Decoder Network
We develop a deep learning algorithm for contour detection with a fully convolutional encoder-decoder network.
Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
An important problem for both graphics and vision is to synthesize novel views of a 3D object from a single image.
Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network
We propose a novel weakly-supervised semantic segmentation algorithm based on Deep Convolutional Neural Network (DCNN).
Foreground Segmentation
Weakly supervised Semantic Segmentation
+1
Attribute2Image: Conditional Image Generation from Visual Attributes
This paper investigates a novel problem of generating images from visual attributes.
Deep Visual Analogy-Making
In addition to identifying the content within a single image, relating images and generating related images are critical tasks for image understanding.
Learning Structured Output Representation using Deep Conditional Generative Models
The model is trained efficiently in the framework of stochastic gradient variational Bayes, and allows a fast prediction using stochastic feed-forward inference.
Ranked #1 on
Structured Prediction
on MNIST
Action-Conditional Video Prediction using Deep Networks in Atari Games
Motivated by vision-based reinforcement learning (RL) problems, in particular Atari games from the recent benchmark Aracade Learning Environment (ALE), we consider spatio-temporal prediction problems where future (image-)frames are dependent on control variables or actions as well as previous frames.
Improving Object Detection with Deep Convolutional Networks via Bayesian Optimization and Structured Prediction
Object detection systems based on the deep convolutional neural network (CNN) have recently made ground- breaking advances on several object detection benchmarks.
Training Deep Neural Networks on Noisy Labels with Bootstrapping
On MNIST handwritten digits, we show that our model is robust to label corruption.
Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning
The combination of modern Reinforcement Learning and Deep Learning approaches holds the promise of making significant progress on challenging applications requiring both rich perception and policy-selection.
Improved Multimodal Deep Learning with Variation of Information
Deep learning has been successfully applied to multimodal representation learning problems, with a common strategy to learning joint representations that are shared across multiple modalities on top of layers of modality-specific networks.
Evaluation of Output Embeddings for Fine-Grained Image Classification
Image classification has advanced significantly in recent years with the availability of large-scale image sets.
Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising
Stacked sparse denoising auto-encoders (SSDAs) have recently been shown to be successful at removing noise from corrupted images.
Weakly Supervised Learning of Mid-Level Features with Beta-Bernoulli Process Restricted Boltzmann Machines
In order to address this issue, we propose a weakly supervised approach to learn mid-level features, where only class-level supervision is provided during training.
Augmenting CRFs with Boltzmann Machine Shape Priors for Image Labeling
Although the CRF is a good baseline labeler, we show how an RBM can be added to the architecture to provide a global shape bias that complements the local modeling provided by the CRF.
Deep Learning for Detecting Robotic Grasps
We consider the problem of detecting robotic grasps in an RGB-D view of a scene containing objects.
Ranked #7 on
Robotic Grasping
on Cornell Grasp Dataset
Learning to Align from Scratch
Unsupervised joint alignment of images has been demonstrated to improve performance on recognition tasks such as face verification.
Measuring Invariances in Deep Networks
Our evaluation metrics can also be used to evaluate future work in unsupervised deep learning, and thus help the development of future algorithms.
Unsupervised feature learning for audio classification using convolutional deep belief networks
In this paper, we apply convolutional deep belief networks to audio data and empirically evaluate them on various audio classification tasks.
Sparse deep belief net model for visual area V2
This suggests that our sparse variant of deep belief networks holds promise for modeling more higher-order features.
