Search Results for author:
Found 15 papers, 2 papers with code
Asynchronous Convergence in Multi-Task Learning via Knowledge Distillation from Converged Tasks
However, one issue that often arises in MTL is the convergence speed between tasks varies due to differences in task difficulty, so it can be a challenge to simultaneously achieve the best performance on all tasks with a single model checkpoint.
Semantic Aligned Multi-modal Transformer for Vision-LanguageUnderstanding: A Preliminary Study on Visual QA
Recent vision-language understanding approaches adopt a multi-modal transformer pre-training and finetuning paradigm.
VidLA: Video-Language Alignment at Scale
To effectively address this limitation, we instead keep the network architecture simple and use a set of data tokens that operate at different temporal resolutions in a hierarchical manner, accounting for the temporally hierarchical nature of videos.
Robust Multi-Task Learning with Excess Risks
To overcome this limitation, we propose Multi-Task Learning with Excess Risks (ExcessMTL), an excess risk-based task balancing method that updates the task weights by their distances to convergence instead.
Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective
Pre-training is known to generate universal representations for downstream tasks in large-scale deep learning such as large language models.
ForeSeer: Product Aspect Forecasting Using Temporal Graph Embedding
ForeSeer transfers reviews from similar products on a large product graph and exploits these reviews to predict aspects that might emerge in future reviews.
Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications
Model pre-training on large text corpora has been demonstrated effective for various downstream applications in the NLP domain.
Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning
Hence we advocate that the key of better performance lies in meaningful latent modality structures instead of perfect modality alignment.
Few-Shot Image Classification
Open-Ended Question Answering
+6
Efficient and effective training of language and graph neural network models
The effectiveness in our framework is achieved by applying stage-wise fine-tuning of the BERT model first with heterogenous graph information and then with a GNN model.
DynaMaR: Dynamic Prompt with Mask Token Representation
In this paper, we propose an improvement to prompt-based fine-tuning that addresses these two issues.
Multi-modal Alignment using Representation Codebook
Aligning signals from different modalities is an important step in vision-language representation learning as it affects the performance of later stages such as cross-modality fusion.
Vision-Language Pre-Training with Triple Contrastive Learning
Besides CMA, TCL introduces an intra-modal contrastive objective to provide complementary benefits in representation learning.
Ranked #3 on
Zero-Shot Cross-Modal Retrieval
on COCO 2014
Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning
To achieve this, we propose a novel idea, Magic Pyramid (MP), to reduce both width-wise and depth-wise computation via token pruning and early exiting for Transformer-based models, particularly BERT.
MLIM: Vision-and-Language Model Pre-training with Masked Language and Image Modeling
Vision-and-Language Pre-training (VLP) improves model performance for downstream tasks that require image and text inputs.
Simpler, Faster, Stronger: Breaking The log-K Curse On Contrastive Learners With FlatNCE
InfoNCE-based contrastive representation learners, such as SimCLR, have been tremendously successful in recent years.
