Search Results for author:
Found 19 papers, 3 papers with code
Asynchronous Convergence in Multi-Task Learning via Knowledge Distillation from Converged Tasks
However, one issue that often arises in MTL is the convergence speed between tasks varies due to differences in task difficulty, so it can be a challenge to simultaneously achieve the best performance on all tasks with a single model checkpoint.
M-LLM Based Video Frame Selection for Efficient Video Understanding
Empirical results show that the proposed M-LLM video frame selector improves the performances various downstream video Large Language Model (video-LLM) across medium (ActivityNet, NExT-QA) and long (EgoSchema, LongVideoBench) context video question answering benchmarks.
DreamBlend: Advancing Personalized Fine-tuning of Text-to-Image Diffusion Models
As the pre-trained model is fine-tuned, earlier checkpoints synthesize images with low subject fidelity but high prompt fidelity and diversity.
Evolutionary Contrastive Distillation for Language Model Alignment
The ability of large language models (LLMs) to execute complex instructions is essential for their real-world applications.
X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs
In this work, we focus on enhancing the visual representations for MLLMs by combining high-frequency and detailed visual representations, obtained through masked image modeling (MIM), with semantically-enriched low-frequency representations captured by CL.
Open Vocabulary Multi-Label Video Classification
Pre-trained vision-language models (VLMs) have enabled significant progress in open vocabulary computer vision tasks such as image classification, object detection and image segmentation.
VidLA: Video-Language Alignment at Scale
To effectively address this limitation, we instead keep the network architecture simple and use a set of data tokens that operate at different temporal resolutions in a hierarchical manner, accounting for the temporally hierarchical nature of videos.
Robust Multi-Task Learning with Excess Risks
To overcome this limitation, we propose Multi-Task Learning with Excess Risks (ExcessMTL), an excess risk-based task balancing method that updates the task weights by their distances to convergence instead.
Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications
Model pre-training on large text corpora has been demonstrated effective for various downstream applications in the NLP domain.
Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning
Hence we advocate that the key of better performance lies in meaningful latent modality structures instead of perfect modality alignment.
SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing
The mixture of Expert (MoE) parallelism is a recent advancement that scales up the model size with constant computational cost.
MICO: Selective Search with Mutual Information Co-training
In contrast to traditional exhaustive search, selective search first clusters documents into several groups before all the documents are searched exhaustively by a query, to limit the search executed within one group or only a few groups.
Efficient and effective training of language and graph neural network models
The effectiveness in our framework is achieved by applying stage-wise fine-tuning of the BERT model first with heterogenous graph information and then with a GNN model.
DynaMaR: Dynamic Prompt with Mask Token Representation
In this paper, we propose an improvement to prompt-based fine-tuning that addresses these two issues.
Multi-modal Alignment using Representation Codebook
Aligning signals from different modalities is an important step in vision-language representation learning as it affects the performance of later stages such as cross-modality fusion.
Vision-Language Pre-Training with Triple Contrastive Learning
Besides CMA, TCL introduces an intra-modal contrastive objective to provide complementary benefits in representation learning.
Ranked #3 on
Zero-Shot Cross-Modal Retrieval
on COCO 2014
Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning
To achieve this, we propose a novel idea, Magic Pyramid (MP), to reduce both width-wise and depth-wise computation via token pruning and early exiting for Transformer-based models, particularly BERT.
MLIM: Vision-and-Language Model Pre-training with Masked Language and Image Modeling
Vision-and-Language Pre-training (VLP) improves model performance for downstream tasks that require image and text inputs.
Tiering as a Stochastic Submodular Optimization Problem
Tiering is an essential technique for building large-scale information retrieval systems.
