Search Results for author:
Found 30 papers, 22 papers with code
HyperCLOVA X Technical Report
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding.
Lipsum-FT: Robust Fine-Tuning of Zero-Shot Models Using Random Text Guidance
Large-scale contrastive vision-language pre-trained models provide the zero-shot model achieving competitive performance across a range of image classification tasks without requiring training on downstream data.
DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTs
This paper revives Densely Connected Convolutional Networks (DenseNets) and reveals the underrated effectiveness over predominant ResNet-style architectures.
Rotary Position Embedding for Vision Transformer
This study provides a comprehensive analysis of RoPE when applied to ViTs, utilizing practical implementations of RoPE for 2D vision data.
Masked Image Modeling via Dynamic Token Morphing
Masked Image Modeling (MIM) arises as a promising option for Vision Transformers among various self-supervised learning (SSL) methods.
SeiT++: Masked Token Modeling Improves Storage-efficient Training
A recent breakthrough by SeiT proposed the use of Vector-Quantized (VQ) feature vectors (i. e., tokens) as network inputs for vision classification.
Match me if you can: Semantic Correspondence Learning with Unpaired Images
Recent approaches for semantic correspondence have focused on obtaining high-quality correspondences using a complicated network, refining the ambiguous or noisy matching points.
Learning with Unmasked Tokens Drives Stronger Vision Learners
MIMs such as Masked Autoencoder (MAE) learn strong representations by randomly masking input tokens for the encoder to process, with the decoder reconstructing the masked tokens to the input.
Masking Augmentation for Supervised Learning
In this paper, we propose a novel way to involve masking augmentations dubbed Masked Sub-model (MaskSub).
What Do Self-Supervised Vision Transformers Learn?
We present a comparative study on how and why contrastive learning (CL) and masked image modeling (MIM) differ in their representations and in their performance of downstream tasks.
SeiT: Storage-Efficient Vision Training with Tokens Using 1% of Pixel Storage
We need billion-scale images to achieve more generalizable and ground-breaking vision models, as well as massive dataset storage to ship the images (e. g., the LAION-4B dataset needs 240TB storage space).
Group Generalized Mean Pooling for Vision Transformer
As ViT groups the channels via a multi-head attention mechanism, grouping the channels by GGeM leads to lower head-wise dependence while amplifying important channels on the activation maps.
Similarity of Neural Architectures using Adversarial Attack Transferability
In recent years, many deep neural architectures have been developed for image classification.
Scratching Visual Transformer's Back with Uniform Attention
We observe that the inclusion of CB reduces the degree of density in the original attention maps and increases both the capacity and generalizability of the ViT models.
Improving Ensemble Distillation With Weight Averaging and Diversifying Perturbation
Ensembles of deep neural networks have demonstrated superior performance, but their heavy computational cost hinders applying them for resource-limited environments.
An Extendable, Efficient and Effective Transformer-based Object Detector
Transformers have been widely used in numerous vision problems especially for visual recognition and detection.
Learning Features with Parameter-Free Layers
We aim to break the stereotype of organizing the spatial operations of building blocks into trainable layers.
Joint Global and Local Hierarchical Priors for Learned Image Compression
Recently, learned image compression methods have outperformed traditional hand-crafted ones including BPG.
The Majority Can Help The Minority: Context-rich Minority Oversampling for Long-tailed Classification
The problem of class imbalanced data is that the generalization performance of the classifier deteriorates due to the lack of data from minority classes.
Ranked #20 on
Long-tail Learning
on ImageNet-LT
ViDT: An Efficient and Effective Fully Transformer-based Object Detector
Transformers are transforming the landscape of computer vision, especially for recognition tasks.
Ranked #12 on
Object Detection
on COCO 2017 val
Rethinking Spatial Dimensions of Vision Transformers
We empirically show that such a spatial dimension reduction is beneficial to a transformer architecture as well, and propose a novel Pooling-based Vision Transformer (PiT) upon the original ViT model.
Ranked #336 on
Image Classification
on ImageNet
Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matching
Knowledge distillation extracts general knowledge from a pre-trained teacher network and provides guidance to a target student network.
Ranked #36 on
Knowledge Distillation
on ImageNet
Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels
However, they have not fixed the training set, presumably because of a formidable annotation cost.
Ranked #20 on
Image Classification
on OmniBenchmark
VideoMix: Rethinking Data Augmentation for Video Classification
Recent data augmentation strategies have been reported to address the overfitting problems in static image classifiers.
Rethinking Channel Dimensions for Efficient Model Design
We then investigate the channel configuration of a model by searching network architectures concerning the channel configuration under the computational cost restriction.
Ranked #293 on
Image Classification
on ImageNet
(using extra training data)
AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights
Because of the scale invariance, this modification only alters the effective step sizes without changing the effective update directions, thus enjoying the original convergence properties of GD optimizers.
A Comprehensive Overhaul of Feature Distillation
We investigate the design aspects of feature distillation methods achieving network compression and propose a novel feature distillation method in which the distillation loss is designed to make a synergy among various aspects: teacher transform, student transform, distillation feature position and distance function.
Ranked #38 on
Knowledge Distillation
on ImageNet
Backbone Can Not be Trained at Once: Rolling Back to Pre-trained Network for Person Re-Identification
Our strategy alleviates the problem of gradient vanishing in low-level layers and robustly trains the low-level layers to fit the ReID dataset, thereby increasing the performance of ReID tasks.
Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons
In this paper, we propose a knowledge transfer method via distillation of activation boundaries formed by hidden neurons.
Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
In this paper, we provide a new perspective based on a decision boundary, which is one of the most important component of a classifier.
