Search Results for author:
Found 85 papers, 27 papers with code
Attention as an RNN
Tackling this, we (3) introduce a new efficient method of computing attention's \textit{many-to-many} RNN output based on the parallel prefix scan algorithm.
Ranked #66 on
Time Series Forecasting
on ETTh1 (336) Multivariate
Pretext Training Algorithms for Event Sequence Data
Pretext training followed by task-specific fine-tuning has been a successful approach in vision and language domains.
OPSurv: Orthogonal Polynomials Quadrature Algorithm for Survival Analysis
This paper introduces the Orthogonal Polynomials Quadrature Algorithm for Survival Analysis (OPSurv), a new method providing time-continuous functional outputs for both single and competing risks scenarios in survival analysis.
Rethinking Learning Approaches for Long-Term Action Anticipation
Action anticipation involves predicting future actions having observed the initial portion of a video.
Continuous-time Particle Filtering for Latent Stochastic Differential Equations
Particle filtering is a standard Monte-Carlo approach for a wide range of sequential inference tasks.
RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression
Data imbalance, in which a plurality of the data samples come from a small proportion of labels, poses a challenge in training deep neural networks.
Monotonicity Regularization: Improved Penalties and Novel Applications to Disentangled Representation Learning and Robust Classification
We study settings where gradient penalties are used alongside risk minimization with the goal of obtaining predictors satisfying different notions of monotonicity.
Filtered-CoPhy: Unsupervised Learning of Counterfactual Physics in Pixel Space
Learning causal relationships in high-dimensional data (images, videos) is a hard task, as they are often defined on low dimensional manifolds and must be extracted from complex signals dominated by appearance, lighting, textures and also spurious correlations in the data.
MUSE: Feature Self-Distillation with Mutual Information and Self-Information
Importantly, we argue and empirically demonstrate that MUSE, compared to other feature discrepancy functions, is a more functional proxy to introduce dependency and effectively improve the expressivity of all features in the knowledge distillation framework.
Monotonicity as a requirement and as a regularizer: efficient methods and applications
We study the setting where risk minimization is performed over general classes of models and consider two cases where monotonicity is treated as either a requirement to be satisfied everywhere or a useful property.
D3D-HOI: Dynamic 3D Human-Object Interactions from Videos
We evaluate this approach on our dataset, demonstrating that human-object relations can significantly reduce the ambiguity of articulated object reconstructions from challenging real-world videos.
Continuous Latent Process Flows
Partial observations of continuous time-series dynamics at arbitrary time stamps exist in many disciplines.
TD-GEN: Graph Generation With Tree Decomposition
We propose TD-GEN, a graph generation framework based on tree decomposition, and introduce a reduced upper bound on the maximum number of decisions needed for graph generation.
Hyper-LifelongGAN: Scalable Lifelong Learning for Image Conditioned Generation
Deep neural networks are susceptible to catastrophic forgetting: when encountering a new task, they can only remember the new task and fail to preserve its ability to accomplish previously learned tasks.
Adaptive Appearance Rendering
Hence, we develop an approach based on intermediate representations of poses and appearance: our pose-guided appearance rendering network firstly encodes the targets' poses using an encoder-decoder neural network.
Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generation
In contrast, we propose a parameter efficient framework, Piggyback GAN, which learns the current task by building a set of convolutional and deconvolutional filters that are factorized into filters of the models trained on previous tasks.
Learning Discriminative Prototypes with Dynamic Time Warping
We propose Discriminative Prototype DTW (DP-DTW), a novel method to learn class-specific discriminative prototypes for temporal recognition tasks.
Variational Selective Autoencoder: Learning from Partially-Observed Heterogeneous Data
Learning from heterogeneous data poses challenges such as combining data from various sources and of different types.
Activity Graph Transformer for Temporal Action Localization
Detecting and localizing action instances in untrimmed videos requires reasoning over multiple action instances in a video.
Ranked #3 on
Temporal Action Localization
on THUMOS’14
(mAP IOU@0.1 metric)
Neural fidelity warping for efficient robot morphology design
We consider the problem of optimizing a robot morphology to achieve the best performance for a target task, under computational resource limitations.
MCMI: Multi-Cycle Image Translation with Mutual Information Constraints
We present a mutual information-based framework for unsupervised image-to-image translation.
House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation
This paper proposes a novel graph-constrained generative adversarial network, whose generator and discriminator are built upon relational architecture.
Variational Hyper RNN for Sequence Modeling
In this work, we propose a novel probabilistic sequence model that excels at capturing high variability in time series data, both across sequences and within an individual sequence.
Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows
Normalizing flows transform a simple base distribution into a complex target distribution and have proved to be powerful models for data generation and density estimation.
Adapting Grad-CAM for Embedding Networks
Then, we develop an efficient weight-transfer method to explain decisions for any image without back-propagation.
Generating Videos of Zero-Shot Compositions of Actions and Objects
Human activity videos involve rich, varied interactions between people and objects.
Point Process Flows
Event sequences can be modeled by temporal point processes (TPPs) to capture their asynchronous and probabilistic nature.
Variational Selective Autoencoder
Despite promising progress on unimodal data imputation (e. g. image inpainting), models for multimodal data imputation are far from satisfactory.
Arbitrarily-conditioned Data Imputation
In this paper, we propose an arbitrarily-conditioned data imputation framework built upon variational autoencoders and normalizing flows.
Graph Generation with Variational Recurrent Neural Network
Generating graph structures is a challenging problem due to the diverse representations and complex dependencies among nodes.
Policy Message Passing: A New Algorithm for Probabilistic Graph Inference
A general graph-structured neural network architecture operates on graphs through two core components: (1) complex enough message functions; (2) a fixed information aggregation process.
Relational Graph Learning for Crowd Navigation
We present a relational graph learning approach for robotic crowd navigation using model-based deep reinforcement learning that plans actions by looking into the future.
CoPhy: Counterfactual Learning of Physical Dynamics
Understanding causes and effects in mechanical systems is an essential component of reasoning in the physical world.
Learning from Partially-Observed Multimodal Data with Variational Autoencoders
Learning from only partially-observed data for imputation has been an active research area.
Continuous Graph Flow
In this paper, we propose Continuous Graph Flow, a generative continuous flow based method that aims to model complex distributions of graph-structured data.
LayoutVAE: Stochastic Scene Layout Generation From a Label Set
Recently there is an increasing interest in scene generation within the research community.
Lifelong GAN: Continual Learning for Conditional Image Generation
This makes it possible to perform image-conditioned generation tasks in a lifelong learning setting.
Similarity-Preserving Knowledge Distillation
Knowledge distillation is a widely applicable technique for training a student neural network under the guidance of a trained teacher network.
Variational Autoencoders with Jointly Optimized Latent Dependency Structure
In particular, we express the latent variable space of a variational autoencoder (VAE) in terms of a Bayesian network with a learned, flexible dependency structure.
A Variational Auto-Encoder Model for Stochastic Point Processes
We propose a novel probabilistic generative model for action sequences.
Learning a Deep ConvNet for Multi-label Classification with Partial Labels
Multi-label classification is a more difficult task than single-label classification because both the input images and output label spaces are more complex.
Probabilistic Neural Programmed Networks for Scene Generation
In this paper we address the text to scene image generation problem.
Constraint-Aware Deep Neural Network Compression
Deep neural network compression has the potential to bring modern resource-hungry deep networks to resource-limited devices.
Hierarchical Relational Networks for Group Activity Recognition and Retrieval
Second, we propose a Relational Autoencoder model for unsupervised learning of features for action and scene retrieval.
Time Perception Machine: Temporal Point Processes for the When, Where and What of Activity Prediction
Numerous powerful point process models have been developed to understand temporal patterns in sequential data from fields such as health-care, electronic commerce, social networks, and natural disaster forecasting.
Object Level Visual Reasoning in Videos
Human activity recognition is typically addressed by detecting key concepts like global and local motion, features related to object classes present in the scene, as well as features related to the global context.
Ranked #1 on
Semantic Object Interaction Classification
on VLOG
CLIP-Q: Deep Network Compression Learning by In-Parallel Pruning-Quantization
This allows us to take advantage of the complementary nature of pruning and quantization and to recover from premature pruning errors, which is not possible with current two-stage approaches.
Distribution Aware Active Learning
This strategy does not make good use of the structure of the dataset at hand and is prone to be misguided by outliers.
Probabilistic Video Generation using Holistic Attribute Control
Videos express highly structured spatio-temporal patterns of visual data.
Structured Label Inference for Visual Understanding
In this paper, we exploit this rich structure for performing graph-based inference in label space for a number of tasks: multi-label image and video classification and action detection in untrimmed videos.
Sparsely Aggregated Convolutional Networks
We explore a key architectural aspect of deep convolutional neural networks: the pattern of internal skip connections used to aggregate outputs of earlier layers for consumption by deeper layers.
Learning to Forecast Videos of Human Activity with Multi-granularity Models and Adaptive Rendering
An architecture combining a hierarchical temporal model for predicting human poses and encoder-decoder convolutional neural networks for rendering target appearances is proposed.
Fine-Pruning: Joint Fine-Tuning and Compression of a Convolutional Network with Bayesian Optimization
When approaching a novel visual recognition problem in a specialized image domain, a common strategy is to start with a pre-trained deep neural network and fine-tune it to the specialized domain.
Factorized Variational Autoencoders for Modeling Audience Reactions to Movies
Matrix and tensor factorization methods are often used for finding underlying low-dimensional patterns from noisy data.
Hierarchical Label Inference for Video Classification
Videos are a rich source of high-dimensional structured data, with a wide range of interacting components at varying levels of granularity.
Learning to Learn from Noisy Web Videos
Our method uses Q-learning to learn a data labeling policy on a small labeled training dataset, and then uses this to automatically label noisy web data for new visual concepts.
Active Learning for Structured Prediction from Partially Labeled Data
We propose a general purpose active learning algorithm for structured prediction, gathering labeled data for training a model that outputs a set of related labels for an image or video.
Learning Person Trajectory Representations for Team Activity Analysis
Activity analysis in which multiple people interact across a large space is challenging due to the interplay of individual actions and collective group dynamics.
Generic Tubelet Proposals for Action Localization
Our class-independent TPN outperforms other tubelet generation methods, and our unified temporal deep network achieves state-of-the-art localization results on all three datasets.
LabelBank: Revisiting Global Perspectives for Semantic Segmentation
We advocate that holistic inference of image concepts provides valuable information for detailed pixel labeling.
Recalling Holistic Information for Semantic Segmentation
We advocate that high-recall holistic inference of image concepts provides valuable information for detailed pixel labeling.
Hierarchical Deep Temporal Models for Group Activity Recognition
In order to model both person-level and group-level dynamics, we present a 2-stage deep temporal model for the group activity recognition problem.
Deep Learning of Appearance Models for Online Object Tracking
This paper introduces a novel deep learning based approach for vision based single target tracking.
Learning Ensembles of Potential Functions for Structured Prediction With Latent Variables
This paper presents HCRF-Boost, a novel and general framework for learning HCRFs in functional space.
End-to-end Learning of Action Detection from Frame Glimpses in Videos
In this work we introduce a fully end-to-end approach for action detection in videos that learns to directly predict the temporal bounds of actions.
Ranked #9 on
Temporal Action Localization
on THUMOS’14
(mAP IOU@0.2 metric)
A Hierarchical Deep Temporal Model for Group Activity Recognition
In group activity recognition, the temporal dynamics of the whole activity can be inferred based on the dynamics of the individual people representing the activity.
Learning Structured Inference Neural Networks with Label Relations
Images of scenes have various objects as well as abundant attributes, and diverse levels of visual categorization are possible.
Structure Inference Machines: Recurrent Neural Networks for Analyzing Relations in Group Activity Recognition
As a concrete example, group activity recognition involves the interactions and relative spatial relations of a set of people in a scene.
Ranked #6 on
Group Activity Recognition
on Collective Activity
Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos
Every moment counts in action recognition.
Ranked #7 on
Action Detection
on Multi-THUMOS
Pose Embeddings: A Deep Architecture for Learning to Match Human Poses
We present a method for learning an embedding that places images of humans in similar poses nearby.
Deep Structured Models For Group Activity Recognition
This paper presents a deep neural-network-based hierarchical graphical model for individual and group activity recognition in surveillance scenes.
Learning Temporal Embeddings for Complex Video Analysis
In this paper, we propose to learn temporal embeddings of video frames for complex video analysis.
Discovering Human Interactions in Videos with Limited Data Labeling
We present a novel approach for discovering human interactions in videos.
Visual Recognition by Counting Instances: A Multi-Instance Cardinality Potential Kernel
Many visual recognition problems can be approached by counting instances.
Hierarchical Maximum-Margin Clustering
We present a hierarchical maximum-margin clustering method for unsupervised data analysis.
Action is in the Eye of the Beholder: Eye-gaze Driven Model for Spatio-Temporal Action Localization
We propose a new weakly-supervised structured learning approach for recognition and spatio-temporal localization of actions in video.
Latent Maximum Margin Clustering
We present a maximum margin framework that clusters data using latent variables.
Multiple Instance Learning by Discriminative Training of Markov Networks
We introduce a graphical framework for multiple instance learning (MIL) based on Markov networks.
Learning Class-to-Image Distance with Object Matchings
We conduct image classification by learning a class-toimage distance function that matches objects.
Kernel Latent SVM for Visual Recognition
Latent SVMs (LSVMs) are a class of powerful tools that have been successfully applied to many applications in computer vision.
Beyond Actions: Discriminative Models for Contextual Group Activities
We propose a discriminative model for recognizing group activities.
A Discriminative Latent Model of Image Region and Object Tag Correspondence
In particular, we model the mapping that translates image regions to annotations.
A Rate Distortion Approach for Semi-Supervised Conditional Random Fields
We propose a novel information theoretic approach for semi-supervised learning of conditional random fields.
Learning a discriminative hidden part model for human action recognition
In particular, our experimental results demonstrate that combining large-scale global features and local patch features performs significantly better than directly applying hCRF on local patches alone.
