Search Results for author:
Found 23 papers, 11 papers with code
QLIP: Text-Aligned Visual Tokenization Unifies Auto-Regressive Multimodal Understanding and Generation
We validate the effectiveness of QLIP for multimodal understanding and text-conditioned image generation with a single model.
Boosting LLM via Learning from Data Iteratively and Selectively
On the other hand, the diversity score is defined on top of the samples' responses under the consideration of their informativeness.
MixEval-X: Any-to-Any Evaluations from Real-World Data Mixtures
Perceiving and generating diverse modalities are crucial for AI models to effectively learn from and engage with real-world signals, necessitating reliable evaluations for their development.
LongVILA: Scaling Long-Context Visual Language Models for Long Videos
We introduce the long-context Multi-Modal Sequence Parallelism (MM-SP) system that efficiently parallelizes long video training and inference, enabling 2M context length training on 256 GPUs without any gradient checkpointing.
Ranked #9 on
Video Question Answering
on NExT-QA
Wolf: Captioning Everything with a World Summarization Framework
Finally, we establish a benchmark for video captioning and introduce a leaderboard, aiming to accelerate advancements in video understanding, captioning, and data alignment.
MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures
Our benchmarks' advantages lie in (1) a 0. 96 model ranking correlation with Chatbot Arena arising from the highly impartial query distribution and grading mechanism, (2) fast, cheap, and reproducible execution (6% of the time and cost of MMLU), and (3) dynamic evaluation enabled by the rapid and stable data update pipeline.
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
To help the open-source community have a better understanding of Mixture-of-Experts (MoE) based large language models (LLMs), we train and release OpenMoE, a series of fully open-sourced and reproducible decoder-only MoE LLMs, ranging from 650M to 34B parameters and trained on up to over 1T tokens.
Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline
By leveraging this information, we introduce an efficient sequence scheduling technique that groups queries with similar response lengths into micro-batches.
Hierarchical Dialogue Understanding with Special Tokens and Turn-level Attention
We evaluate our model on various dialogue understanding tasks including dialogue relation extraction, dialogue emotion recognition, and dialogue act classification.
Ranked #1 on
Dialog Relation Extraction
on DialogRE
Adaptive Computation with Elastic Input Sequence
However, most standard neural networks have a fixed function type and computation budget regardless of the sample's nature or difficulty.
A Study on Transformer Configuration and Training Objective
In this paper, we revisit these conventional configurations.
Ranked #102 on
Image Classification
on ImageNet
CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPU
Our experiments show that previous scaling rules fail in the training of CTR prediction neural networks.
Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation
Text-based video segmentation aims to segment the target object in a video based on a describing sentence.
Ranked #10 on
Referring Expression Segmentation
on A2D Sentences
Optical Flow Estimation
Referring Expression Segmentation
+4
One Student Knows All Experts Know: From Sparse to Dense
Mixture-of-experts (MoE) is a powerful sparse architecture including multiple experts.
Large-Scale Deep Learning Optimizations: A Comprehensive Survey
Deep learning have achieved promising results on a wide spectrum of AI applications.
Cross-token Modeling with Conditional Computation
Mixture-of-Experts (MoE), a conditional computation architecture, achieved promising performance by scaling local module (i. e. feed-forward network) of transformer.
Automated Audio Captioning using Transfer Learning and Reconstruction Latent Space Similarity Regularization
In this paper, we examine the use of Transfer Learning using Pretrained Audio Neural Networks (PANNs), and propose an architecture that is able to better leverage the acoustic features provided by PANNs for the Automated Audio Captioning Task.
Go Wider Instead of Deeper
To achieve better performance with fewer trainable parameters, recent methods are proposed to go shallower by parameter sharing or model compressing along with the depth.
Ranked #723 on
Image Classification
on ImageNet
Sequence Parallelism: Long Sequence Training from System Perspective
That is, with sparse attention, our sequence parallelism enables us to train transformer with infinite long sequence.
Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey
To the best of our knowledge, this survey is the most comprehensive and up-to-date one at present for deep learning based dialogue systems, extensively covering the popular techniques.
An Embarrassingly Simple Model for Dialogue Relation Extraction
Dialogue relation extraction (RE) is to predict the relation type of two entities mentioned in a dialogue.
Ranked #9 on
Dialog Relation Extraction
on DialogRE
GDPNet: Refining Latent Multi-View Graph for Relation Extraction
Recent advances on RE task are from BERT-based sequence modeling and graph-based modeling of relationships among the tokens in the sequence.
Ranked #4 on
Dialog Relation Extraction
on DialogRE
(F1c (v1) metric)
Deep Graph Random Process for Relational-Thinking-Based Speech Recognition
Lying at the core of human intelligence, relational thinking is characterized by initially relying on innumerable unconscious percepts pertaining to relations between new sensory signals and prior knowledge, consequently becoming a recognizable concept or object through coupling and transformation of these percepts.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
