Search Results for author:
Found 59 papers, 18 papers with code
Laying the Foundations of Deep Long-Term Crowd Flow Prediction
Predicting the crowd behavior in complex environments is a key requirement for crowd and disaster management, architectural design, and urban planning.
CIC-BART-SSA: Controllable Image Captioning with Structured Semantic Augmentation
We use this Structured Semantic Augmentation (SSA) framework to augment existing image-caption datasets with the grounded controlled captions, increasing their spatial and semantic diversity and focal coverage.
Learning from Synthetic Human Group Activities
The study of complex human interactions and group activities has become a focal point in human-centric computer vision.
ALWOD: Active Learning for Weakly-Supervised Object Detection
In this work, we propose ALWOD, a new framework that addresses this problem by fusing active learning (AL) with weakly and semi-supervised object detection paradigms.
NP-SemiSeg: When Neural Processes meet Semi-Supervised Semantic Segmentation
This is useful in a wide range of real-world applications where collecting pixel-wise labels is not feasible in time or cost.
M3Act: Learning from Synthetic Human Group Activities
The study of complex human interactions and group activities has become a focal point in human-centric computer vision.
MSI: Maximize Support-Set Information for Few-Shot Segmentation
To extract information relevant to the target class, a dominant approach in best-performing FSS methods removes background features using a support mask.
D2DF2WOD: Learning Object Proposals for Weakly-Supervised Object Detection via Progressive Domain Adaptation
In its warm-up domain adaptation stage, the model learns a fully-supervised object detector (FSOD) to improve the precision of the object proposals in the target domain, and at the same time learns target-domain-specific and detection-aware proposal features.
An Information-Theoretic Approach for Estimating Scenario Generalization in Crowd Motion Prediction
Learning-based approaches to modeling crowd motion have become increasingly successful but require training and evaluation on large datasets, coupled with complex model selection and parameter tuning.
Visual Semantic Parsing: From Images to Abstract Meaning Representation
The success of scene graphs for visual scene understanding has brought attention to the benefits of abstracting a visual input (e. g., image) into a structured representation, where entities (people and objects) are nodes connected by edges specifying their relations.
NP-Match: When Neural Processes meet Semi-Supervised Learning
Semi-supervised learning (SSL) has been widely explored in recent years, and it is an effective way of leveraging unlabeled data to reduce the reliance on labeled data.
SAViR-T: Spatially Attentive Visual Reasoning with Transformers
We present a novel computational model, "SAViR-T", for the family of visual reasoning problems embodied in the Raven's Progressive Matrices (RPM).
HM: Hybrid Masking for Few-Shot Segmentation
A fundamental limitation of FM is the inability to preserve the fine-grained spatial details that affect the accuracy of segmentation mask, especially for small target objects.
Ranked #5 on
Few-Shot Semantic Segmentation
on FSS-1000 (1-shot)
MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Prediction
Accurate long-term trajectory prediction in complex scenes, where multiple agents (e. g., pedestrians or vehicles) interact with each other and the environment while attempting to accomplish diverse and often unknown goals, is a challenging stochastic forecasting problem.
Multi-attribute Pizza Generator: Cross-domain Attribute Control with Conditional StyleGAN
To synthesize a pizza image with view attributesoutside the range of natural training images, we design a CGI pizza dataset PizzaView using 3D pizza models and employ it to train a view attribute regressor to regularize the generation process, bridging the real and CGI training datasets.
DAReN: A Collaborative Approach Towards Reasoning And Disentangling
Computational learning approaches to solving visual reasoning tests, such as Raven's Progressive Matrices (RPM), critically depend on the ability to identify the visual concepts used in the test (i. e., the representation) as well as the latent rules based on those concepts (i. e., the reasoning).
Cross-Modal Coherence for Text-to-Image Retrieval
Common image-text joint understanding techniques presume that images and the associated text can universally be characterized by a single implicit model.
Reducing the Amortization Gap in Variational Autoencoders: A Bayesian Random Function Approach
In this paper, we address the problem in a completely different way by considering a random inference model, where we model the mean and variance functions of the variational posterior as random Gaussian processes (GP).
CHEF: Cross-modal Hierarchical Embeddings for Food Domain Retrieval
Despite the abundance of multi-modal data, such as image-text pairs, there has been little effort in understanding the individual entities and their different roles in the construction of these data instances.
Private-Shared Disentangled Multimodal VAE for Learning of Hybrid Latent Representations
Multi-modal generative models represent an important family of deep models, whose goal is to facilitate representation learning on data with multiple views or modalities.
MPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs
Because of the complex nature of the multilabel image generation problem, we also regularize synthetic image by predicting the corresponding ingredients as well as encourage the discriminator to distinguish between matched image and mismatched image.
Cross-Modal Retrieval and Synthesis (X-MRS): Closing the Modality Gap in Shared Representation Learning
A key to making CFA possible is multi-modal shared representation learning, which aims to create a joint representation of the multiple views (text and image) of the data.
Ranked #5 on
Cross-Modal Retrieval
on Recipe1M
Learning Disentangled Latent Factors from Paired Data in Cross-Modal Retrieval: An Implicit Identifiable VAE Approach
We deal with the problem of learning the underlying disentangled latent factors that are shared between the paired bi-modal data in cross-modal retrieval.
Recursive Inference for Variational Autoencoders
Using the functional gradient approach, we devise an intuitive learning criteria for selecting a new mixture component: the new component has to improve the data likelihood (lower bound) and, at the same time, be as divergent from the current mixture distribution as possible, thus increasing representational diversity.
Picture-to-Amount (PITA): Predicting Relative Ingredient Amounts from Food Images
Increased awareness of the impact of food consumption on health and lifestyle today has given rise to novel data-driven food analysis systems.
Ordinal-Content VAE: Isolating Ordinal-Valued Content Factors in Deep Latent Variable Models
In deep representational learning, it is often desired to isolate a particular factor (termed {\em content}) from other factors (referred to as {\em style}).
CookGAN: Meal Image Synthesis from Ingredients
In this work we propose a new computational framework, based on generative deep models, for synthesis of photo-realistic food meal images from textual list of its ingredients.
Deep Crowd-Flow Prediction in Built Environments
In this paper, we propose an approach to instantly predict the long-term flow of crowds in arbitrarily large, realistic environments.
Task-Discriminative Domain Alignment for Unsupervised Domain Adaptation
In this paper, we improve the performance of DA by introducing a discriminative discrepancy measure which takes advantage of auxiliary information available in the source and the target domains to better align the source and target distributions.
Fast and Effective Adaptation of Facial Action Unit Detection Deep Model
In this paper, we propose a deep learning approach for facial AU detection that can easily and in a fast manner adapt to a new AU or target subject by leveraging only a few labeled samples from the new task (either an AU or subject).
Deep Cooking: Predicting Relative Food Ingredient Amounts from Images
In this paper, we study the novel problem of not only predicting ingredients from a food image, but also predicting the relative amounts of the detected ingredients.
Bayes-Factor-VAE: Hierarchical Bayesian Deep Auto-Encoder Models for Factor Disentanglement
We propose a family of novel hierarchical Bayesian deep auto-encoder models capable of identifying disentangled factors of variability in data.
Efficient Deep Gaussian Process Models for Variable-Sized Input
The key advantage is that the combination of GP and DRF leads to a tractable model that can both handle a variable-sized input as well as learn deep long-range dependency structures of the data.
The Art of Food: Meal Image Synthesis from Ingredients
In this work we propose a new computational framework, based on generative deep models, for synthesis of photo-realistic food meal images from textual descriptions of its ingredients.
Visibility Constrained Generative Model for Depth-based 3D Facial Pose Tracking
In this paper, we propose a generative framework that unifies depth-based 3D facial pose tracking and face model adaptation on-the-fly, in the unconstrained scenarios with heavy occlusions and arbitrary facial expression variations.
Unsupervised Visual Domain Adaptation: A Deep Max-Margin Gaussian Process Approach
The latter can be achieved by minimizing the maximum discrepancy of predictors (classifiers).
Ranked #3 on
Synthetic-to-Real Translation
on Syn2Real-C
Relevance Factor VAE: Learning and Identifying Disentangled Factors
We propose a novel VAE-based deep auto-encoder model that can learn disentangled latent representations in a fully unsupervised manner, endowed with the ability to identify all meaningful sources of variation and their cardinality.
Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach
Unsupervised domain adaptation (uDA) models focus on pairwise adaptation settings where there is a single, labeled, source and a single target domain.
Generative Adversarial Talking Head: Bringing Portraits to Life with a Weakly Supervised Neural Network
This paper presents Generative Adversarial Talking Head (GATH), a novel deep generative neural network that enables fully automatic facial expression synthesis of an arbitrary portrait with continuous action unit (AU) coefficients.
End-to-end Learning for 3D Facial Animation from Raw Waveforms of Speech
We present a deep learning framework for real-time speech-driven 3D facial animation from just raw waveforms.
PUnDA: Probabilistic Unsupervised Domain Adaptation for Knowledge Transfer Across Visual Categories
This paper introduces a probabilistic latent variable model to address unsupervised domain adaptation problems.
Unsupervised Domain Adaptation with Copula Models
We study the task of unsupervised domain adaptation, where no labeled data from the target domain is provided during training time.
A Generative Model for Depth-Based Robust 3D Facial Pose Tracking
We consider the problem of depth-based robust 3D facial pose tracking under unconstrained scenarios with heavy occlusions and arbitrary facial expression variations.
Probabilistic Temporal Subspace Clustering
In this paper, we propose a unified non-parametric generative framework for temporal subspace clustering to segment data drawn from a sequentially ordered union of subspaces that deals with the missing features in a principled way.
Deep Structured Learning for Facial Action Unit Intensity Estimation
The goal of this paper is to model these structures and estimate complex feature representations simultaneously by combining conditional random field (CRF) encoded AU dependencies with deep learning.
Cartoonish sketch-based face editing in videos using identity deformation transfer
We first recover the facial identity and expressions from the video by fitting a face morphable model for each frame.
Robust Time-Series Retrieval Using Probabilistic Adaptive Segmental Alignment
Traditional pairwise sequence alignment is based on matching individual samples from two sequences, under time monotonicity constraints.
Copula Ordinal Regression for Joint Estimation of Facial Action Unit Intensity
Joint modeling of the intensity of facial action units (AUs) from face images is challenging due to the large number of AUs (30+) and their intensity levels (6).
Variable-state Latent Conditional Random Fields for Facial Expression Recognition and Action Unit Detection
For instance, in the case of AU detection, the goal is to discriminate between the segments of an image sequence in which this AU is active or inactive.
Robust Performance-driven 3D Face Tracking in Long Range Depth Scenes
We introduce a novel robust hybrid 3D face tracking framework from RGBD video streams, which is capable of tracking head pose and facial actions without pre-calibration or intervention from a user.
Intrinsic Non-stationary Covariance Function for Climate Modeling
Designing a covariance function that represents the underlying correlation is a crucial step in modeling complex natural systems, such as climate models.
D-MFVI: Distributed Mean Field Variational Inference using Bregman ADMM
Bayesian models provide a framework for probabilistic modelling of complex datasets.
Fast ADMM Algorithm for Distributed Optimization with Adaptive Penalty
We propose new methods to speed up convergence of the Alternating Direction Method of Multipliers (ADMM), a common optimization tool in the context of large scale and distributed learning.
Gaussian Process for Noisy Inputs with Ordering Constraints
We study the Gaussian Process regression model in the context of training data with noise in both input and output.
Multi-Cue Structure Preserving MRF for Unconstrained Video Segmentation
Video segmentation is a stepping stone to understanding video context.
Discovering Characteristic Landmarks on Ancient Coins using Convolutional Networks
We also propose a new framework to recognize the Roman coins which exploits hierarchical structure of the ancient Roman coins using the state-of-the-art classification power of the CNNs adopted to a new task of coin classification.
Heteroscedastic Conditional Ordinal Random Fields for Pain Intensity Estimation from Facial Images
We propose a novel method for automatic pain intensity estimation from facial images based on the framework of kernel Conditional Ordinal Random Fields (KCORF).
Distributed Probabilistic Learning for Camera Networks with Missing Data
In this work we present an approach to estimation and learning of generative probabilistic models in a distributed context where certain sensor data can be missing.
Scalable Algorithms for String Kernels with Inexact Matching
We present a new family of linear time algorithms based on sufficient statistics for string comparison with mismatches under the string kernels framework.
