Search Results for author:
Found 48 papers, 35 papers with code
Probabilistic Language-Image Pre-Training
Vision-language models (VLMs) embed aligned image-text pairs into a joint space but often rely on deterministic embeddings, assuming a one-to-one correspondence between images and texts.
Read, Watch and Scream! Sound Generation from Text and Video
Despite the impressive progress of multimodal generative models, video-to-audio generation still suffers from limited performance and limits the flexibility to prioritize sound synthesis for specific objects within the scene.
Ranked #7 on
Video-to-Sound Generation
on VGG-Sound
Reducing Task Discrepancy of Text Encoders for Zero-Shot Composed Image Retrieval
To that end, we introduce the Reducing Task Discrepancy of text encoders for Composed Image Retrieval (RTD), a plug-and-play training scheme for the text encoder that enhances its capability using a novel target-anchored text contrastive learning.
HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts
In an era where the volume of data drives the effectiveness of self-supervised learning, the specificity and clarity of data semantics play a crucial role in model training.
Toward Interactive Regional Understanding in Vision-Large Language Models
Recent Vision-Language Pre-training (VLP) models have demonstrated significant advancements.
Language-only Training of Zero-shot Composed Image Retrieval
Our LinCIR (Language-only training for CIR) can be trained only with text datasets by a novel self-supervision named self-masking projection (SMP).
Language-only Efficient Training of Zero-shot Composed Image Retrieval
Our LinCIR (Language-only training for CIR) can be trained only with text datasets by a novel self-supervision named self-masking projection (SMP).
Improved Probabilistic Image-Text Representations
Image-Text Matching (ITM) task, a fundamental vision-language (VL) task, suffers from the inherent ambiguity arising from multiplicity and imperfect annotations.
RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Models
With the extensive use of vision-language models in various downstream tasks, evaluating their robustness is crucial.
Three Recipes for Better 3D Pseudo-GTs of 3D Human Mesh Estimation in the Wild
Recovering 3D human mesh in the wild is greatly challenging as in-the-wild (ITW) datasets provide only 2D pose ground truths (GTs).
Ranked #6 on
3D Multi-Person Pose Estimation
on MuPoTS-3D
CompoDiff: Versatile Composed Image Retrieval With Latent Diffusion
This paper proposes a novel diffusion-based model, CompoDiff, for solving zero-shot Composed Image Retrieval (ZS-CIR) with latent diffusion.
SeiT: Storage-Efficient Vision Training with Tokens Using 1% of Pixel Storage
We need billion-scale images to achieve more generalizable and ground-breaking vision models, as well as massive dataset storage to ship the images (e. g., the LAION-4B dataset needs 240TB storage space).
Re-weighting Based Group Fairness Regularization via Classwise Robust Optimization
Many existing group fairness-aware training methods aim to achieve the group fairness by either re-weighting underrepresented groups based on certain rules or using weakly approximated surrogates for the fairness metrics in the objective as regularization terms.
Group Generalized Mean Pooling for Vision Transformer
As ViT groups the channels via a multi-head attention mechanism, grouping the channels by GGeM leads to lower head-wise dependence while amplifying important channels on the activation maps.
Similarity of Neural Architectures using Adversarial Attack Transferability
In recent years, many deep neural architectures have been developed for image classification.
A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspective
Our theoretical results show that regardless of the choice of the mixing strategy, MSDA behaves as a pixel-level regularization of the underlying training loss and a regularization of the first layer parameters.
An Extendable, Efficient and Effective Transformer-based Object Detector
Transformers have been widely used in numerous vision problems especially for visual recognition and detection.
ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-verified Image-Caption Associations for MS-COCO
Image-Text matching (ITM) is a common task for evaluating the quality of Vision and Language (VL) models.
Domain Generalization by Mutual-Information Regularization with Pre-trained Models
Domain generalization (DG) aims to learn a generalized model to an unseen target domain using only limited source domains.
Ranked #3 on
Domain Generalization
on TerraIncognita
Dataset Condensation with Contrastive Signals
However, in this study, we prove that the existing DC methods can perform worse than the random selection method when task-irrelevant information forms a significant part of the training dataset.
Few-shot Font Generation with Weakly Supervised Localized Representations
Existing methods learn to disentangle style and content elements by developing a universal style representation for each font style.
Learning Fair Classifiers with Partially Annotated Group Labels
To address this problem, we propose a simple Confidence-based Group Label assignment (CGL) strategy that is readily applicable to any fairness-aware learning method.
ViDT: An Efficient and Effective Fully Transformer-based Object Detector
Transformers are transforming the landscape of computer vision, especially for recognition tasks.
Ranked #16 on
Object Detection
on COCO 2017 val
Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective
This phenomenon, also known as shortcut learning, is emerging as a key limitation of the current generation of machine learning models.
Biased Multi-Domain Adversarial Training
Several recent studies have shown that the use of extra in-distribution data can lead to a high level of adversarial robustness.
StyleAugment: Learning Texture De-biased Representations by Style Augmentation without Pre-defined Textures
Hence, StyleAugment let the model observe abundant confounding cues for each image by on-the-fly the augmentation strategy, while the augmented images are more realistic than artistic style transferred images.
Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions
Effective control and prediction of dynamical systems often require appropriate handling of continuous-time and discrete, event-triggered processes.
Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts
MX-Font extracts multiple style features not explicitly conditioned on component labels, but automatically by multiple experts to represent different local concepts, e. g., left-side sub-glyph.
Rethinking Spatial Dimensions of Vision Transformers
We empirically show that such a spatial dimension reduction is beneficial to a transformer architecture as well, and propose a novel Pooling-based Vision Transformer (PiT) upon the original ViT model.
Ranked #368 on
Image Classification
on ImageNet
SWAD: Domain Generalization by Seeking Flat Minima
Domain generalization (DG) methods aim to achieve generalizability to an unseen target domain by using only training data from the source domains.
Ranked #25 on
Domain Generalization
on TerraIncognita
Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels
However, they have not fixed the training set, presumably because of a formidable annotation cost.
Ranked #20 on
Image Classification
on OmniBenchmark
Probabilistic Embeddings for Cross-Modal Retrieval
Instead, we propose to use Probabilistic Cross-Modal Embedding (PCME), where samples from the different modalities are represented as probabilistic distributions in the common embedding space.
Few-shot Font Generation with Localized Style Representations and Factorization
However, learning component-wise styles solely from reference glyphs is infeasible in the few-shot font generation scenario, when a target script has a large number of components, e. g., over 200 for Chinese.
Evaluation for Weakly Supervised Object Localization: Protocol, Metrics, and Datasets
In this paper, we argue that WSOL task is ill-posed with only image-level labels, and propose a new evaluation protocol where full supervision is limited to only a small held-out set not overlapping with the test set.
AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights
Because of the scale invariance, this modification only alters the effective step sizes without changing the effective update directions, thus enjoying the original convergence properties of GD optimizers.
Few-shot Compositional Font Generation with Dual Memory
By utilizing the compositionality of compositional scripts, we propose a novel font generation framework, named Dual Memory-augmented Font Generation Network (DM-Font), which enables us to generate a high-quality font library with only a few samples.
An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods
Despite apparent human-level performances of deep neural networks (DNN), they behave fundamentally differently from humans.
Evaluating Weakly Supervised Object Localization Methods Right
In this paper, we argue that WSOL task is ill-posed with only image-level labels, and propose a new evaluation protocol where full supervision is limited to only a small held-out set not overlapping with the test set.
Visualizing and Understanding Self-attention based Music Tagging
Recently, we proposed a self-attention based music tagging model.
Sound Audio and Speech Processing
Neural Approximation of an Auto-Regressive Process through Confidence Guided Sampling
We first assume that the priors of future samples can be generated in an independently and identically distributed (i. i. d.)
Learning De-biased Representations with Biased Representations
This tactic is feasible in many scenarios where it is much easier to define a set of biased representations than to define and quantify bias.
Toward Interpretable Music Tagging with Self-Attention
In addition, we demonstrate the interpretability of the proposed architecture with a heat map visualization.
Sound Audio and Speech Processing
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
Regional dropout strategies have been proposed to enhance the performance of convolutional neural network classifiers.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
ACE: Artificial Checkerboard Enhancer to Induce and Evade Adversarial Attacks
The checkerboard phenomenon is one of the well-known visual artifacts in the computer vision field.
Photorealistic Style Transfer via Wavelet Transforms
The key ingredient of our method is wavelet transforms that naturally fits in deep networks.
Multi-Domain Processing via Hybrid Denoising Networks for Speech Enhancement
We present a hybrid framework that leverages the trade-off between temporal and frequency precision in audio representations to improve the performance of speech enhancement task.
Audio and Speech Processing Sound
Scalable Iterative Algorithm for Robust Subspace Clustering
Subspace clustering (SC) is a popular method for dimensionality reduction of high-dimensional data, where it generalizes Principal Component Analysis (PCA).
