Search Results for author:
Found 30 papers, 4 papers with code
Weight subcloning: direct initialization of transformers using larger pretrained ones
The usual practice of transfer learning overcomes this challenge by initializing the model with weights of a pretrained model of the same size and specification to increase the convergence and training speed.
Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models
Motivated by this, we ask the following important question, "How can we leverage the knowledge from a large VFM to train a small task-specific model for a new target task with limited labeled training data?
Probabilistic Speech-Driven 3D Facial Motion Synthesis: New Benchmarks, Methods, and Applications
While these models can achieve high-quality lip articulation for speakers in the training set, they are unable to capture the full and diverse distribution of 3D facial motions that accompany speech in the real world.
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training
We further demonstrate the effectiveness of our multi-modal reinforced training by training a CLIP model based on ViT-B/16 image backbone and achieving +2. 9% average performance improvement on 38 evaluation benchmarks compared to the previous best.
TiC-CLIP: Continual Training of CLIP Models
We introduce the first set of web-scale Time-Continual (TiC) benchmarks for training vision-language models: TiC-DataComp, TiC-YFCC, and TiC-Redcaps.
SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding
By applying our method to SAM and CLIP, we obtain SAM-CLIP: a unified model that combines the capabilities of SAM and CLIP into a single vision transformer.
CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement
While CLIP is scalable, promptable, and robust to distribution shifts on image classification tasks, it lacks object localization capabilities.
Corpus Synthesis for Zero-shot ASR domain Adaptation using Large Language Models
In this paper, we propose a new strategy for adapting ASR models to new target domains without any text or speech from those domains.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Towards Federated Learning Under Resource Constraints via Layer-wise Training and Depth Dropout
To mitigate this issue and facilitate training of large models on edge devices, we introduce a simple yet effective strategy, Federated Layer-wise Learning, to simultaneously reduce per-client memory, computation, and communication costs.
Federated Training of Dual Encoding Models on Small Non-IID Client Datasets
In this work, we focus on federated training of dual encoding models on decentralized data composed of many small, non-IID (independent and identically distributed) client datasets.
Less data is more: Selecting informative and diverse subsets with balancing constraints
Deep learning has yielded extraordinary results in vision and natural language processing, but this achievement comes at a cost.
Less is more: Selecting informative and diverse subsets with balancing constraints
Deep learning has yielded extraordinary results in vision and natural language processing, but this achievement comes at a cost.
Camera View Adjustment Prediction for Improving Image Composition
To address this issue, we propose a deep learning-based approach that provides suggestions to the photographer on how to adjust the camera view before capturing.
Contrastive Learning for Label Efficient Semantic Segmentation
While recent Convolutional Neural Network (CNN) based semantic segmentation approaches have achieved impressive results by using large amounts of labeled training data, their performance drops significantly as the amount of labeled data decreases.
Global Self-Attention Networks
Based on the proposed GSA module, we introduce new standalone global attention-based deep networks that use GSA modules instead of convolutions to model pixel interactions.
Contrastive Learning for Label-Efficient Semantic Segmentation
While recent Convolutional Neural Network (CNN) based semantic segmentation approaches have achieved impressive results by using large amounts of labeled training data, their performance drops significantly as the amount of labeled data decreases.
Boosting Image-based Mutual Gaze Detection using Pseudo 3D Gaze
In this work, we focus on the task of image-based mutual gaze detection, and propose a simple and effective approach to boost the performance by using an auxiliary 3D gaze estimation task during the training phase.
Global Self-Attention Networks for Image Recognition
Based on the proposed GSA module, we introduce new standalone global attention-based deep networks that use GSA modules instead of convolutions to model pixel interactions.
Search to Distill: Pearls are Everywhere but not the Eyes
Standard Knowledge Distillation (KD) approaches distill the knowledge of a cumbersome teacher model into the parameters of a student model with a pre-defined architecture.
A Compact Embedding for Facial Expression Similarity
Most of the existing work on automatic facial expression analysis focuses on discrete emotion recognition, or facial action unit detection.
Frame-Recurrent Video Super-Resolution
Recent advances in video super-resolution have shown that convolutional neural networks combined with motion compensation are able to merge information from multiple low-resolution (LR) frames to generate high-quality images.
Ranked #6 on
Video Super-Resolution
on MSU Video Upscalers: Quality Enhancement
(VMAF metric)
Designing Deep Convolutional Neural Networks for Continuous Object Orientation Estimation
The third method works by first converting the continuous orientation estimation task into a set of discrete orientation estimation tasks and then converting the discrete orientation outputs back to the continuous orientation using a mean-shift algorithm.
Rolling Rotations for Recognizing Human Actions from 3D Skeletal Data
Then, using this representation, we model human actions as curves in this Lie group.
Ranked #4 on
Skeleton Based Action Recognition
on Gaming 3D (G3D)
Gaussian Conditional Random Field Network for Semantic Segmentation
In contrast to the existing approaches that use discrete Conditional Random Field (CRF) models, we propose to use a Gaussian CRF model for the task of semantic segmentation.
Unsupervised Cross-Modal Synthesis of Subject-Specific Scans
Our experiments on generating T1-MRI brain scans from T2-MRI and vice versa demonstrate that the synthesis capability of the proposed unsupervised approach is comparable to various state-of-the-art supervised approaches in the literature.
Deep Gaussian Conditional Random Field Network: A Model-based Deep Network for Discriminative Denoising
We propose a novel deep network architecture for image\\ denoising based on a Gaussian Conditional Random Field (GCRF) model.
Riemannian Metric Learning for Symmetric Positive Definite Matrices
In this work, we focus on the log-Euclidean Riemannian geometry and propose a data-driven approach for learning Riemannian metrics/geodesic distances for SPD matrices.
MKL-RT: Multiple Kernel Learning for Ratio-trace Problems via Convex Optimization
We experimentally demonstrate that the proposed MKL approach, which we refer to as MKL-RT, can be successfully used to select features for discriminative dimensionality reduction and cross-modal retrieval.
Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group
Recently introduced cost-effective depth sensors coupled with the real-time skeleton estimation algorithm of Shotton et al. have generated a renewed interest in skeleton-based human action recognition.
Ranked #6 on
Skeleton Based Action Recognition
on UT-Kinect
Kernel Learning for Extrinsic Classification of Manifold Features
In this paper, we address the issue of kernelselection for the classification of features that lie on Riemannian manifolds using the kernel learning approach.
