Search Results for author:
Found 24 papers, 11 papers with code
Contrastive Localized Language-Image Pre-Training
Contrastive Language-Image Pre-training (CLIP) has been a celebrated method for training vision encoders to generate image/text representations facilitating various applications.
Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models
Our findings reveal that a hybrid approach that keeps both synthetic captions and AltTexts can outperform the use of synthetic captions alone, improving both alignment and performance, with each model demonstrating preferences for particular caption formats.
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning
We present MM1. 5, a new family of multimodal large language models (MLLMs) designed to enhance capabilities in text-rich image understanding, visual referring and grounding, and multi-image reasoning.
Ranked #55 on
Visual Question Answering
on MM-Vet
Lessons Learned from a Unifying Empirical Study of Parameter-Efficient Transfer Learning (PETL) in Visual Recognition
Last but not least, we investigate PETL's ability to preserve a pre-trained model's robustness to distribution shifts (e. g., a CLIP backbone).
Fine-Tuning is Fine, if Calibrated
For example, fine-tuning a pre-trained classifier capable of recognizing a large number of classes to master a subset of classes at hand is shown to drastically degrade the model's accuracy in the other classes it had previously learned.
FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reduction
Federated learning (FL) has rapidly evolved as a promising paradigm that enables collaborative model training across distributed participants without exchanging their local data.
SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models
As a result, this design allows us to adequately capture both spatial and temporal features that are beneficial for detailed video understanding.
Jigsaw Game: Federated Clustering
Federated learning has recently garnered significant attention, especially within the domain of supervised learning.
Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models
While Ferret seamlessly integrates regional understanding into the Large Language Model (LLM) to facilitate its referring and grounding capability, it poses certain limitations: constrained by the pre-trained fixed visual encoder and failed to perform well on broader tasks.
Ranked #150 on
Visual Question Answering
on MM-Vet
Reviving the Context: Camera Trap Species Classification as Link Prediction on Multimodal Knowledge Graphs
In this work, we exploit the structured context linked to camera trap images to boost out-of-distribution generalization for species classification tasks in camera traps.
Ranked #1 on
Image Classification
on iWildCam2020-WILDS
(using extra training data)
Federated Learning of Shareable Bases for Personalization-Friendly Image Classification
FedBasis learns a set of few shareable ``basis'' models, which can be linearly combined to form personalized models for clients.
Learning Fractals by Gradient Descent
Fractals are geometric shapes that can display complex and self-similar patterns found in nature (e. g., clouds and plants).
Making Batch Normalization Great in Federated Deep Learning
We reinvestigate factors that are believed to cause this problem, including the mismatch of BN statistics across clients and the deviation of gradients during local training.
Train-Once-for-All Personalization
We study the problem of how to train a "personalization-friendly" model such that given only the task descriptions, the model can be adapted to different end-users' needs, e. g., for accurately classifying different subsets of objects.
Gradual Domain Adaptation without Indexed Intermediate Domains
This coarse domain sequence then undergoes a fine indexing step via a novel cycle-consistency loss, which encourages the next intermediate domain to preserve sufficient discriminative knowledge of the current intermediate domain.
On the Importance and Applicability of Pre-Training for Federated Learning
To make our findings applicable to situations where pre-trained models are not directly available, we explore pre-training with synthetic data or even with clients' data in a decentralized manner, and found that they can already improve FL notably.
On Bridging Generic and Personalized Federated Learning for Image Classification
On the one hand, we introduce a family of losses that are robust to non-identical class distributions, enabling clients to train a generic predictor with a consistent objective across them.
FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning
Federated learning aims to collaboratively train a strong global model by accessing users' locally trained models but not their own data.
Glyph2Vec: Learning Chinese Out-of-Vocabulary Word Embedding from Glyphs
Chinese NLP applications that rely on large text often contain huge amounts of vocabulary which are sparse in corpus.
Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning
Classifiers trained with class-imbalanced data are known to perform poorly on test data of the "minor" classes, of which we have insufficient training data.
Multiple Text Style Transfer by using Word-level Conditional Generative Adversarial Network with Two-Phase Training
The objective of non-parallel text style transfer, or controllable text generation, is to alter specific attributes (e. g. sentiment, mood, tense, politeness, etc) of a given text while preserving its remaining attributes and content.
Self-Discriminative Learning for Unsupervised Document Embedding
Unsupervised document representation learning is an important task providing pre-trained features for NLP applications.
DEEP-TRIM: REVISITING L1 REGULARIZATION FOR CONNECTION PRUNING OF DEEP NETWORK
State-of-the-art deep neural networks (DNNs) typically have tens of millions of parameters, which might not fit into the upper levels of the memory hierarchy, thus increasing the inference time and energy consumption significantly, and prohibiting their use on edge devices such as mobile phones.
Word Relation Autoencoder for Unseen Hypernym Extraction Using Word Embeddings
Lexicon relation extraction given distributional representation of words is an important topic in NLP.
