Search Results for author:
Found 15 papers, 10 papers with code
How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning?
This effectiveness, however, hinges on the appropriate selection of in-context examples, a process that is currently biased towards visual data, overlooking textual information.
DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers
Scaling large models with long sequences across applications like language generation, video generation and multimodal tasks requires efficient sequence parallelism.
Helen: Optimizing CTR Prediction Models with Frequency-wise Hessian Eigenvalue Regularization
We explore the typical data characteristics and optimization statistics of CTR prediction, revealing a strong positive correlation between the top hessian eigenvalue and feature frequency.
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
To help the open-source community have a better understanding of Mixture-of-Experts (MoE) based large language models (LLMs), we train and release OpenMoE, a series of fully open-sourced and reproducible decoder-only MoE LLMs, ranging from 650M to 34B parameters and trained on up to over 1T tokens.
CAME: Confidence-guided Adaptive Memory Efficient Optimization
Adaptive gradient methods, such as Adam and LAMB, have demonstrated excellent performance in the training of large language models.
Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline
By leveraging this information, we introduce an efficient sequence scheduling technique that groups queries with similar response lengths into micro-batches.
Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models
To address this challenge, we propose a novel method ZSCL to prevent zero-shot transfer degradation in the continual learning of vision-language models in both feature and parameter space.
InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning
To solve this problem, we propose \textbf{InfoBatch}, a novel framework aiming to achieve lossless training acceleration by unbiased dynamic data pruning.
Prompt Vision Transformer for Domain Generalization
In this paper, we propose a novel approach DoPrompt based on prompt learning to embed the knowledge of source domains in domain prompts for target domain prediction.
A Study on Transformer Configuration and Training Objective
In this paper, we revisit these conventional configurations.
Ranked #103 on
Image Classification
on ImageNet
CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPU
Our experiments show that previous scaling rules fail in the training of CTR prediction neural networks.
Multi-source Few-shot Domain Adaptation
Multi-source Domain Adaptation (MDA) aims to transfer predictive models from multiple, fully-labeled source domains to an unlabeled target domain.
Cross-token Modeling with Conditional Computation
Mixture-of-Experts (MoE), a conditional computation architecture, achieved promising performance by scaling local module (i. e. feed-forward network) of transformer.
Scene-aware Learning Network for Radar Object Detection
In this paper, we propose a scene-aware radar learning framework for accurate and robust object detection.
Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation
In this paper, we propose an end-to-end Prototypical Cross-domain Self-Supervised Learning (PCS) framework for Few-shot Unsupervised Domain Adaptation (FUDA).
Ranked #6 on
Semantic Segmentation
on DensePASS
