Search Results for author:
Found 34 papers, 17 papers with code
I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models
Without complex training and datasets, ThinkDiff effectively unleashes understanding, reasoning, and composing capabilities in diffusion models.
Converting Transformers into DGNNs Form
Recent advances in deep learning have established Transformer architectures as the predominant modeling paradigm.
Ranked #3 on
Long-range modeling
on LRA
Object-level Visual Prompts for Compositional Image Generation
A key challenge in this task is to preserve the identity of the objects depicted in the input visual prompts, while also generating diverse compositions across different images.
Omni-ID: Holistic Identity Representation Designed for Generative Tasks
Our approach uses a few-to-many identity reconstruction training paradigm, where a limited set of input images is used to reconstruct multiple target images of the same individual in various poses and expressions.
Motion Diffusion-Guided 3D Global HMR from a Dynamic Camera
Motion capture technologies have transformed numerous fields, from the film and gaming industries to sports science and healthcare, by providing a tool to capture and analyze human movement in great detail.
Ask, Pose, Unite: Scaling Data Acquisition for Close Interactions with Vision Language Models
Social dynamics in close human interactions pose significant challenges for Human Mesh Estimation (HME), particularly due to the complexity of physical contacts and the scarcity of training data.
Interpreting the Weight Space of Customized Diffusion Models
First, sampling a set of weights from this space results in a new model encoding a novel identity.
MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generation
MoA is designed to retain the original model's prior by fixing its attention layers in the prior branch, while minimally intervening in the generation process with the personalized branch that learns to embed subjects in the layout and context generated by the prior branch.
Iterative Motion Editing with Natural Language
Text-to-motion diffusion models can generate realistic animations from text prompts, but do not support fine-grained motion editing controls.
Open World Object Detection in the Era of Foundation Models
We exploit the inherent connection between classes in application-driven datasets and introduce a novel method, Foundation Object detection Model for the Open world, or FOMO, which identifies unknown objects based on their shared attributes with the base known objects.
Viewpoint Textual Inversion: Discovering Scene Representations and 3D View Control in 2D Diffusion Models
We conclude that since the view token controls the 3D `rendering' viewpoint, there is likely a scene representation embedded in frozen 2D diffusion models.
Generalizable Neural Fields as Partially Observed Neural Processes
Neural fields, which represent signals as a function parameterized by a neural network, are a promising alternative to traditional discrete vector or grid-based representations.
LOVM: Language-Only Vision Model Selection
As the number of open-source VLM variants increases, there is a need for an efficient model selection strategy that does not require access to a curated evaluation dataset.
Diagnosing and Rectifying Vision Models using Language
Our proposed method can discover high-error data slices, identify influential attributes and further rectify undesirable model behaviors, without requiring any visual data.
NeMo: Learning 3D Neural Motion Fields From Multiple Video Instances of the Same Action
Empirically, we show that NeMo can recover 3D motion in sports using videos from the Penn Action dataset, where NeMo outperforms existing HMR methods in terms of 2D keypoint detection.
NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action
Empirically, we show that NeMo can recover 3D motion in sports using videos from the Penn Action dataset, where NeMo outperforms existing HMR methods in terms of 2D keypoint detection.
PROB: Probabilistic Objectness for Open World Object Detection
The resulting Probabilistic Objectness transformer-based open-world detector, PROB, integrates our framework into traditional object detection models, adapting them for the open-world setting.
Domain Adaptive 3D Pose Augmentation for In-the-wild Human Mesh Recovery
The ability to perceive 3D human bodies from a single image has a multitude of applications ranging from entertainment and robotics to neuroscience and healthcare.
Grad2Task: Improved Few-shot Text Classification Using Gradients for Task Representation
Large pretrained language models (LMs) like BERT have improved performance in many disparate natural language processing (NLP) tasks.
Variational Model Inversion Attacks
In this work, we provide a probabilistic interpretation of model inversion attacks, and formulate a variational objective that accounts for both diversity and accuracy.
Disentanglement and Generalization Under Correlation Shifts
Exploiting such correlations may increase predictive performance on noisy data; however, often correlations are not robust (e. g., they may change between domains, datasets, or applications) and models that exploit them do not generalize when correlations shift.
Exploring representation learning for flexible few-shot tasks
In this work, we consider a realistic setting where the relationship between examples can change from episode to episode depending on the task context, which is not given to the learner.
Probing Few-Shot Generalization with Attributes
Despite impressive progress in deep learning, generalizing far beyond the training distribution is an important open challenge.
Understanding and Mitigating Exploding Inverses in Invertible Neural Networks
For problems where global invertibility is necessary, such as applying normalizing flows on OOD data, we show the importance of designing stable INN building blocks.
Learning the Stein Discrepancy for Training and Evaluating Energy-Based Models without Sampling
We estimate the Stein discrepancy between the data density $p(x)$ and the model density $q(x)$ defined by a vector function of the data.
Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
In this setting, the standard class probabilities can be easily computed as well as unnormalized values of p(x) and p(x|y).
On the Invertibility of Invertible Neural Networks
Guarantees in deep learning are hard to achieve due to the interplay of flexible modeling schemes and complex tasks.
Out-of-distribution Detection in Few-shot Classification
In this work, we propose tasks for out-of-distribution detection in the few-shot setting and establish benchmark datasets, based on four popular few-shot classification datasets.
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
Centroid-based deep metric learning for speaker recognition
Speaker embedding models that utilize neural networks to map utterances to a space where distances reflect similarity between speakers have driven recent progress in the speaker recognition task.
Distilling the Posterior in Bayesian Neural Networks
We propose a framework, Adversarial Posterior Distillation, to distill the SGLD samples using a Generative Adversarial Network (GAN).
Adversarial Distillation of Bayesian Neural Network Posteriors
We propose a framework, Adversarial Posterior Distillation, to distill the SGLD samples using a Generative Adversarial Network (GAN).
Neural Relational Inference for Interacting Systems
Interacting systems are prevalent in nature, from dynamical systems in physics to complex societal dynamics.
Dualing GANs
We start from linear discriminators in which case conjugate duality provides a mechanism to reformulate the saddle point objective into a maximization problem, such that both the generator and the discriminator of this 'dualing GAN' act in concert.
