Search Results for author:
Found 41 papers, 5 papers with code
CoCoNO: Attention Contrast-and-Complete for Initial Noise Optimization in Text-to-Image Synthesis
Despite recent advancements in text-to-image models, achieving semantically accurate images in text-to-image diffusion models is a persistent challenge.
TIDE: Training Locally Interpretable Domain Generalization Models Enables Test-time Correction
Our next innovation is TIDE, a novel training scheme with a concept saliency alignment loss that ensures model focus on the right per-concept regions and a local concept contrastive loss that promotes learning domain-invariant concept representations.
Test-time Conditional Text-to-Image Synthesis Using Diffusion Models
To address this gap in the current literature, we propose our method called TINTIN: Test-time Conditional Text-to-Image Synthesis using Diffusion Models which is a new training-free test-time only algorithm to condition text-to-image diffusion model outputs on conditioning factors such as color palettes and edge maps.
Training-free Color-Style Disentanglement for Constrained Text-to-Image Synthesis
We consider the problem of independently, in a disentangled fashion, controlling the outputs of text-to-image diffusion models with color and style attributes of a user-supplied reference image.
SafaRi:Adaptive Sequence Transformer for Weakly Supervised Referring Expression Segmentation
To enable principled training of models in such low-annotation settings, improve image-text region-level alignment, and further enhance spatial localization of the target object in the image, we propose Cross-modal Fusion with Attention Consistency module.
Referring Expression
Weakly Supervised Referring Expression Segmentation
AlignIT: Enhancing Prompt Alignment in Customization of Text-to-Image Models
We propose a new post-processing algorithm, AlignIT, that infuses the keys and values for the concept of interest while ensuring the keys and values for all other tokens in the input prompt are unchanged.
Few Shot Class Incremental Learning using Vision-Language models
Our proposed framework not only empowers the model to embrace novel classes with limited data, but also ensures the preservation of performance on base classes.
class-incremental learning
Few-Shot Class-Incremental Learning
+2
Approximate Caching for Efficiently Serving Diffusion Models
Text-to-image generation using diffusion models has seen explosive popularity owing to their ability in producing high quality images adhering to text prompts.
An Image is Worth Multiple Words: Multi-attribute Inversion for Constrained Text-to-Image Synthesis
Another line of techniques expand the inversion space to learn multiple embeddings but they do this only along the layer dimension (e. g., one per layer of the DDPM model) or the timestep dimension (one for a set of timesteps in the denoising process), leading to suboptimal attribute disentanglement.
Iterative Multi-granular Image Editing using Diffusion Models
We hope our work would attract attention to this newly identified, pragmatic problem setting.
Learning with Multi-modal Gradient Attention for Explainable Composed Image Retrieval
To this end, our key novelty is a new gradient-attention-based learning objective that explicitly forces the model to focus on the local regions of interest being modified in each retrieval step.
CoPL: Contextual Prompt Learning for Vision-Language Understanding
Our key innovations over earlier works include using local image features as part of the prompt learning process, and more crucially, learning to weight these prompts based on local features that are appropriate for the task at hand.
Learning with Difference Attention for Visually Grounded Self-supervised Representations
Recent works in self-supervised learning have shown impressive results on single-object images, but they struggle to perform well on complex multi-object images as evidenced by their poor visual grounding.
A-STAR: Test-time Attention Segregation and Retention for Text-to-image Synthesis
First, our attention segregation loss reduces the cross-attention overlap between attention maps of different concepts in the text prompt, thereby reducing the confusion/conflict among various concepts and the eventual capture of all concepts in the generated output.
Audio Retrieval for Multimodal Design Documents: A New Dataset and Algorithms
We consider and propose a new problem of retrieving audio files relevant to multimodal design document inputs comprising both textual elements and visual imagery, e. g., birthday/greeting cards.
Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation
Federated Learning (FL) is a machine learning paradigm where local nodes collaboratively train a central model while the training data remains decentralized.
Self-supervised Human Mesh Recovery with Cross-Representation Alignment
However, on synthetic dense correspondence maps (i. e., IUV) few have been explored since the domain gap between synthetic training data and real testing data is hard to address for 2D dense representation.
PseudoClick: Interactive Image Segmentation with Click Imitation
The goal of click-based interactive image segmentation is to obtain precise object segmentation masks with limited user interaction, i. e., by a minimal number of user clicks.
SMPL-A: Modeling Person-Specific Deformable Anatomy
In order to obtain accurate target location information, clinicians have to either conduct frequent intraoperative scans, resulting in higher exposition of patients to radiations, or adopt proxy procedures (e. g., creating and using custom molds to keep patients in the exact same pose during both preoperative organ scanning and subsequent treatment.
Learning Hierarchical Attention for Weakly-supervised Chest X-Ray Abnormality Localization and Diagnosis
However, doing this accurately will require a large amount of disease localization annotations by clinical experts, a task that is prohibitively expensive to accomplish for most applications.
Learning Local Recurrent Models for Human Mesh Recovery
We conduct a variety of experiments on standard video mesh recovery benchmark datasets such as Human3. 6M, MPI-INF-3DHP, and 3DPW, demonstrating the efficacy of our design of modeling local dynamics as well as establishing state-of-the-art results based on standard evaluation metrics.
Ranked #54 on
3D Human Pose Estimation
on 3DPW
Spatio-Temporal Representation Factorization for Video-based Person Re-Identification
To alleviate these problems, we propose Spatio-Temporal Representation Factorization (STRF), a flexible new computational unit that can be used in conjunction with most existing 3D convolutional neural network architectures for re-ID.
Ranked #2 on
Person Re-Identification
on DukeMTMC-VideoReID
Everybody Is Unique: Towards Unbiased Human Mesh Recovery
Next, we present a simple baseline to address this problem that is scalable and can be easily used in conjunction with existing algorithms to improve their performance.
Ranked #1 on
3D Human Shape Estimation
on SSP-3D
(PVE-T metric)
A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts
Despite substantial progress in applying neural networks (NN) to a wide variety of areas, they still largely suffer from a lack of transparency and interpretability.
Ensemble Attention Distillation for Privacy-Preserving Federated Learning
Such decentralized training naturally leads to issues of imbalanced or differing data distributions among the local models and challenges in fusing them into a central model.
Towards Visually Explaining Similarity Models
We show that the resulting similarity models perform, and can be visually explained, better than the corresponding baseline models trained without these constraints.
Hierarchical Kinematic Human Mesh Recovery
In this work, we address this gap by proposing a new technique for regression of human parametric model that is explicitly informed by the known hierarchical structure, including joint interdependencies of the model.
Visual Similarity Attention
While there has been substantial progress in learning suitable distance metrics, these techniques in general lack transparency and decision reasoning, i. e., explaining why the input set of images is similar or dissimilar.
Towards Visually Explaining Variational Autoencoders
We present methods to generate visual attention from the learned latent space, and also demonstrate such attention explanations serve more than just explaining VAE predictions.
Incremental Scene Synthesis
We present a method to incrementally generate complete 2D or 3D scenes with the following properties: (a) it is globally consistent at each step according to a learned scene prior, (b) real observations of a scene can be incorporated while observing global consistency, (c) unobserved regions can be hallucinated locally in consistence with previous observations, hallucinations and global priors, and (d) hallucinations are statistical in nature, i. e., different scenes can be generated from the same observations.
Sharpen Focus: Learning with Attention Separability and Consistency
Recent developments in gradient-based attention modeling have seen attention maps emerge as a powerful tool for interpreting convolutional neural networks.
Re-Identification with Consistent Attentive Siamese Networks
We propose a new deep architecture for person re-identification (re-id).
Learning Local RGB-to-CAD Correspondences for Object Pose Estimation
In this paper, we solve this key problem of existing methods requiring expensive 3D pose annotations by proposing a new method that matches RGB images to CAD models for object pose estimation.
Measuring the Temporal Behavior of Real-World Person Re-Identification
Designing real-world person re-identification (re-id) systems requires attention to operational aspects not typically considered in academic research.
End-to-end learning of keypoint detector and descriptor for pose invariant 3D matching
Finding correspondences between images or 3D scans is at the heart of many computer vision and image retrieval applications and is often enabled by matching local keypoint descriptors.
Learning Compositional Visual Concepts with Mutual Consistency
Compositionality of semantic concepts in image synthesis and analysis is appealing as it can help in decomposing known and generatively recomposing unknown data.
Rank Persistence: Assessing the Temporal Performance of Real-World Person Re-Identification
Designing useful person re-identification systems for real-world applications requires attention to operational aspects not typically considered in academic research.
A Systematic Evaluation and Benchmark for Person Re-Identification: Features, Metrics, and Datasets
To ensure a fair comparison, all of the approaches were implemented using a unified code library that includes 11 feature extraction algorithms and 22 metric learning and ranking techniques.
Person Re-Identification With Discriminatively Trained Viewpoint Invariant Dictionaries
This paper introduces a new approach to address the person re-identification problem in cameras with non-overlapping fields of view.
