Search Results for author:
Found 70 papers, 21 papers with code
SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIP
Large-scale vision-language models, such as CLIP, are known to contain harmful societal bias regarding protected attributes (e. g., gender and age).
ASSR-NeRF: Arbitrary-Scale Super-Resolution on Voxel Grid for High-Quality Radiance Fields Reconstruction
While NeRF-based methods can perform novel view synthesis (NVS) at arbitrary scale, the performance in high-resolution novel view synthesis (HRNVS) with low-resolution (LR) optimization often results in oversmoothing.
ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos
This form of reasoning, requiring advanced understanding of cause-and-effect relationships across video segments, poses significant challenges to even the frontier multimodal large language models.
DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment
Recent speech language models (SLMs) typically incorporate pre-trained speech models to extend the capabilities from large language models (LLMs).
GroPrompt: Efficient Grounded Prompting and Adaptation for Referring Video Object Segmentation
With the proposed TAP-CL, our GroPrompt framework can generate temporal-consistent yet text-aware position prompts describing locations and movements for the referred object from the video.
Diffusion-Reward Adversarial Imitation Learning
Inspired by the recent dominance of diffusion models in generative modeling, this work proposes Diffusion-Reward Adversarial Imitation Learning (DRAIL), which integrates a diffusion model into GAIL, aiming to yield more precise and smoother rewards for policy learning.
Data-Efficient 3D Visual Grounding via Order-Aware Referring
Vigor leverages LLM to produce a desirable referential order from the input description for 3D visual grounding.
Select and Distill: Selective Dual-Teacher Knowledge Transfer for Continual Learning on Vision-Language Models
Large-scale vision-language models (VLMs) have shown a strong zero-shot generalization capability on unseen-domain data.
GSNeRF: Generalizable Semantic Neural Radiance Fields with Enhanced 3D Scene Understanding
Utilizing multi-view inputs to synthesize novel-view images, Neural Radiance Fields (NeRF) have emerged as a popular research topic in 3D vision.
Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech
To improve the robustness of the encoder for SE, a novel self-distillation mechanism combined with adversarial training is introduced.
DoRA: Weight-Decomposed Low-Rank Adaptation
By employing \ours, we enhance both the learning capacity and training stability of LoRA while avoiding any additional inference overhead.
Ranked #2 on
parameter-efficient fine-tuning
on WinoGrande
(using extra training data)
SemPLeS: Semantic Prompt Learning for Weakly-Supervised Semantic Segmentation
In this way, SemPLeS can perform better semantic alignment between object regions and the associated class labels, resulting in desired pseudo masks for training the segmentation model.
Language-Guided Transformer for Federated Multi-Label Classification
Nevertheless, it is still challenging for FL to deal with user heterogeneity in their local data distribution in the real-world FL scenario, and this issue becomes even more severe in multi-label image classification.
TPA3D: Triplane Attention for Fast Text-to-3D Generation
With only 3D shape data and their rendered 2D images observed during training, our TPA3D is designed to retrieve detailed visual descriptions for synthesizing the corresponding 3D mesh data.
Seg2Reg: Differentiable 2D Segmentation to 1D Regression Rendering for 360 Room Layout Reconstruction
State-of-the-art single-view 360-degree room layout reconstruction methods formulate the problem as a high-level 1D (per-column) regression task.
Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers
Concept erasure in text-to-image diffusion models aims to disable pre-trained diffusion models from generating images related to a target concept.
LACMA: Language-Aligning Contrastive Learning with Meta-Actions for Embodied Instruction Following
Additional analysis shows that the contrastive objective and meta-actions are complementary in achieving the best results, and the resulting agent better aligns its states with corresponding instructions, making it more suitable for real-world embodied agents.
Frequency-Aware Self-Supervised Long-Tailed Learning
Data collected from the real world typically exhibit long-tailed distributions, where frequent classes contain abundant data while rare ones have only a limited number of samples.
Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generation
To leverage robust representations from large-scale models while enabling efficient model personalization for heterogeneous clients, we propose a novel personalized FL framework of client-specific Prompt Generation (pFedPG), which learns to deploy a personalized prompt generator at the server for producing client-specific visual prompts that efficiently adapts frozen backbones to local data distributions.
FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learning
In this paper, we propose FedBug (Federated Learning with Bottom-Up Gradual Unfreezing), a novel FL framework designed to effectively mitigate client drift.
QuAVF: Quality-aware Audio-Visual Fusion for Ego4D Talking to Me Challenge
This technical report describes our QuAVF@NTU-NVIDIA submission to the Ego4D Talking to Me (TTM) Challenge 2023.
TAX: Tendency-and-Assignment Explainer for Semantic Segmentation with Multi-Annotators
In this paper, we present a learning framework of Tendency-and-Assignment Explainer (TAX), designed to offer interpretability at the annotator and assignment levels.
Bias-Eliminating Augmentation Learning for Debiased Federated Learning
Learning models trained on biased datasets tend to observe correlations between categorical and undesirable features, which result in degraded performances.
Target-Free Text-guided Image Manipulation
We tackle the problem of target-free text-guided image manipulation, which requires one to modify the input reference image based on the given text instruction, while no ground truth target image is observed during training.
Paraphrasing Is All You Need for Novel Object Captioning
Since no ground truth captions are available for novel object images during training, our P2C leverages cross-modality (image-text) association modules to ensure the above caption characteristics can be properly preserved.
Self-Supervised Pyramid Representation Learning for Multi-Label Visual Analysis and Beyond
In particular, we present a cross-scale patch-level correlation learning in SS-PRL, which allows the model to aggregate and associate information learned across patch scales.
Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis
Diffusion models (DMs) have shown great potential for high-quality image synthesis.
Learning Facial Liveness Representation for Domain Generalized Face Anti-spoofing
Face anti-spoofing (FAS) aims at distinguishing face spoof attacks from the authentic ones, which is typically approached by learning proper models for performing the associated classification task.
Scene Graph Expansion for Semantics-Guided Image Outpainting
In particular, we propose a novel network of Scene Graph Transformer (SGT), which is designed to take node and edge features as inputs for modeling the associated structural information.
NeurMiPs: Neural Mixture of Planar Experts for View Synthesis
We present Neural Mixtures of Planar Experts (NeurMiPs), a novel planar-based scene representation for modeling geometry and appearance.
Domain-Generalized Textured Surface Anomaly Detection
By observing normal and abnormal surface data across multiple source domains, our model is expected to be generalized to an unseen textured surface of interest, in which only a small number of normal data can be observed during testing.
Few-Shot Classification in Unseen Domains by Episodic Meta-Learning Across Visual Domains
Few-shot classification aims to carry out classification given only few labeled examples for the categories of interest.
Meta-Learned Feature Critics for Domain Generalized Semantic Segmentation
How to handle domain shifts when recognizing or segmenting visual data across domains has been studied by learning and vision communities.
Adversarial Teacher-Student Representation Learning for Domain Generalization
Domain generalization (DG) aims to transfer the learning task from a single or multiple source domains to unseen target domains.
A Pixel-Level Meta-Learner for Weakly Supervised Few-Shot Semantic Segmentation
Few-shot semantic segmentation addresses the learning task in which only few images with ground truth pixel-level labels are available for the novel classes of interest.
Learning Visual-Linguistic Adequacy, Fidelity, and Fluency for Novel Object Captioning
Novel object captioning (NOC) learns image captioning models for describing objects or visual concepts which are unseen (i. e., novel) in the training captions.
LayoutTransformer: Scene Layout Generation With Conceptual and Spatial Diversity
To better exploit the text input, so that implicit objects or relationships can be properly inferred during layout generation, we propose a LayoutTransformer Network (LT-Net) in this paper.
Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generation
Human perceives rich auditory experience with distinct sound heard by ears.
Dual-MTGAN: Stochastic and Deterministic Motion Transfer for Image-to-Video Synthesis
Generating videos with content and motion variations is a challenging task in computer vision.
LayoutTransformer: Relation-Aware Scene Layout Generation
In the areas of machine learning and computer vision, text-to-image synthesis aims at producing image outputs given the input text.
Representation Decomposition for Image Manipulation and Beyond
Representation disentanglement aims at learning interpretable features, so that the output can be recovered or manipulated accordingly.
Semantics-Guided Representation Learning with Applications to Visual Synthesis
Learning interpretable and interpolatable latent representations has been an emerging research direction, allowing researchers to understand and utilize the derived latent space for further applications such as visual synthesis or recognition.
Domain Generalized Person Re-Identification via Cross-Domain Episodic Learning
That is, while a number of labeled source-domain datasets are available, we do not have access to any target-domain training data.
Domain Generalization
Generalizable Person Re-identification
+1
Semantics-Guided Clustering with Deep Progressive Learning for Semi-Supervised Person Re-identification
As a result, our approach is able to augment the labeled training data in the semi-supervised setting.
Learning to Learn in a Semi-Supervised Fashion
To address semi-supervised learning from both labeled and unlabeled data, we present a novel meta-learning scheme.
Wavelet Channel Attention Module with a Fusion Network for Single Image Deraining
Wavelet transform and the inverse wavelet transform are substituted for down-sampling and up-sampling so feature maps from the wavelet transform and convolutions contain different frequencies and scales.
Transforming Multi-Concept Attention into Video Summarization
Video summarization is among challenging tasks in computer vision, which aims at identifying highlight frames or shots over a lengthy video input.
Cross-Resolution Adversarial Dual Network for Person Re-Identification and Beyond
Person re-identification (re-ID) aims at matching images of the same person across camera views.
Cross-Dataset Person Re-Identification via Unsupervised Pose Disentanglement and Adaptation
Person re-identification (re-ID) aims at recognizing the same person from images taken across different cameras.
Ranked #17 on
Unsupervised Domain Adaptation
on Market to Duke
Recover and Identify: A Generative Dual Model for Cross-Resolution Person Re-Identification
Person re-identification (re-ID) aims at matching images of the same identity across camera views.
Spatially and Temporally Efficient Non-local Attention Network for Video-based Person Re-Identification
Video-based person re-identification (Re-ID) aims at matching video sequences of pedestrians across non-overlapping cameras.
Ranked #11 on
Person Re-Identification
on MARS
Learning Resolution-Invariant Deep Representations for Person Re-Identification
Moreover, the extension of our model for semi-supervised re-ID further confirms the scalability of our proposed method for real-world scenarios and applications.
A Closer Look at Few-shot Classification
Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples.
Dual-modality seq2seq network for audio-visual event localization
Audio-visual event localization requires one to identify theevent which is both visible and audible in a video (eitherat a frame or video level).
3D Shape Reconstruction from a Single 2D Image via 2D-3D Self-Consistency
Aiming at inferring 3D shapes from 2D images, 3D shape reconstruction has drawn huge attention from researchers in computer vision and deep learning communities.
3D Reconstruction
3D Shape Reconstruction From A Single 2D Image
+1
A Unified Feature Disentangler for Multi-Domain Image Translation and Manipulation
We present a novel and unified deep learning framework which is capable of learning domain-invariant representation from data across multiple domains.
Summarizing First-Person Videos from Third Persons' Points of View
Video highlight or summarization is among interesting topics in computer vision, which benefits a variety of applications like viewing, searching, or storage.
Deep Generative Models for Weakly-Supervised Multi-Label Classification
In order to train learning models for multi-label classification (MLC), it is typically desirable to have a large amount of fully annotated multi-label data.
Deep Reinforcement Learning for Playing 2.5D Fighting Games
Deep reinforcement learning has shown its success in game playing.
Adaptation and Re-Identification Network: An Unsupervised Deep Transfer Learning Approach to Person Re-Identification
Person re-identification (Re-ID) aims at recognizing the same person from images taken across different cameras.
Ranked #20 on
Unsupervised Domain Adaptation
on Duke to Market
Summarizing First-Person Videos from Third Persons' Points of Views
Video highlight or summarization is among interesting topics in computer vision, which benefits a variety of applications like viewing, searching, or storage.
Multi-Label Zero-Shot Learning with Structured Knowledge Graphs
In this paper, we propose a novel deep learning architecture for multi-label zero-shot learning (ML-ZSL), which is able to predict multiple unseen class labels for each input instance.
Order-Free RNN with Visual Attention for Multi-Label Classification
In this paper, we propose the joint learning attention and recurrent neural network (RNN) models for multi-label classification.
Learning Deep Latent Spaces for Multi-Label Classification
Multi-label classification is a practical yet challenging task in machine learning related fields, since it requires the prediction of more than one label category for each input instance.
Generative-Discriminative Variational Model for Visual Recognition
In other words, our GDVM casts the supervised learning task as a generative learning process, with data discrimination to be jointly exploited for improved classification.
Detach and Adapt: Learning Cross-Domain Disentangled Deep Representation
While representation learning aims to derive interpretable features for describing visual data, representation disentanglement further results in such features so that particular image attributes can be identified and manipulated.
No More Discrimination: Cross City Adaptation of Road Scene Segmenters
Despite the recent success of deep-learning based semantic segmentation, deploying a pre-trained road scene segmenter to a city whose images are not presented in the training set would not achieve satisfactory performance due to dataset biases.
Learning Cross-Domain Landmarks for Heterogeneous Domain Adaptation
With the goal of deriving a domain-invariant feature subspace for HDA, our CDLS is able to identify representative cross-domain data, including the unlabeled ones in the target domain, for performing adaptation.
Unsupervised Domain Adaptation With Imbalanced Cross-Domain Data
For standard unsupervised domain adaptation, one typically obtains labeled data in the source domain and only observes unlabeled data in the target domain.
Propagated Image Filtering
In this paper, we propose the propagation filter as a novel image filtering operator, with the goal of smoothing over neighboring image pixels while preserving image context like edges or textural regions.
