Search Results for author:
Found 7 papers, 5 papers with code
BCN: Batch Channel Normalization for Image Classification
Normalization techniques have been widely used in the field of deep learning due to their capability of enabling higher learning rates and are less careful in initialization.
FP8-LM: Training FP8 Large Language Models
In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs).
GMConv: Modulating Effective Receptive Fields for Convolutional Kernels
Inspired by the property that ERFs typically exhibit a Gaussian distribution, we propose a Gaussian Mask convolutional kernel (GMConv) in this work.
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token
With these new techniques and other designs, we show that the proposed general-purpose task-solver can perform both instance segmentation and depth estimation well.
Ranked #14 on
Monocular Depth Estimation
on NYU-Depth V2
Enhancing the Robustness, Efficiency, and Diversity of Differentiable Architecture Search
Differentiable architecture search (DARTS) has attracted much attention due to its simplicity and significant improvement in efficiency.
Swin Transformer V2: Scaling Up Capacity and Resolution
Three main techniques are proposed: 1) a residual-post-norm method combined with cosine attention to improve training stability; 2) A log-spaced continuous position bias method to effectively transfer models pre-trained using low-resolution images to downstream tasks with high-resolution inputs; 3) A self-supervised pre-training method, SimMIM, to reduce the needs of vast labeled images.
Ranked #4 on
Image Classification
on ImageNet V2
(using extra training data)
Video Swin Transformer
The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchmarks.
Ranked #28 on
Action Classification
on Kinetics-600
(using extra training data)
