Search Results for author:
Found 22 papers, 17 papers with code
ModelGrow: Continual Text-to-Video Pre-training with Model Expansion and Language Understanding Enhancement
There is a lack of extensive study of the continual pre-training techniques in T2V generation.
Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
We introduce InternVL 2. 5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2. 0, maintaining its core model architecture while introducing significant enhancements in training and testing strategies as well as data quality.
Ranked #7 on
Visual Question Answering
on MM-Vet
ONION: Physics-Informed Deep Learning Model for Line Integral Diagnostics Across Fusion Devices
This paper introduces a Physics-Informed model architecture that can be adapted to various backbone networks.
What is Wrong with Perplexity for Long-context Language Modeling?
To address this, we propose \textbf{LongPPL}, a novel metric that focuses on key tokens by employing a long-short context contrastive method to identify them.
MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptions
To fulfill this gap, we present MMTrail, a large-scale multi-modality video-language dataset incorporating more than 20M trailer clips with visual captions, and 2M high-quality clips with multimodal captions.
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks
It not only allows flexible transmission of task information and gradient feedback between the MLLM and multiple downstream decoders but also effectively resolves training conflicts in multi-tasking scenarios.
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling
In this work, we systematically study music generation conditioned solely on the video.
LLMs Meet Multimodal Generation and Editing: A Survey
With the recent advancement in large language models (LLMs), there is a growing interest in combining LLMs with multimodal learning.
Paths of A Million People: Extracting Life Trajectories from Wikipedia
The life trajectories of notable people have been studied to pinpoint the times and places of significant events such as birth, death, education, marriage, competition, work, speeches, scientific discoveries, artistic achievements, and battles.
Linear Gaussian Bounding Box Representation and Ring-Shaped Rotated Convolution for Oriented Object Detection
We propose linear GBB (LGBB), a novel OBB representation.
ControlLLM: Augment Language Models with Tools by Searching on Graphs
We present ControlLLM, a novel framework that enables large language models (LLMs) to utilize multi-modal tools for solving complex real-world tasks.
Data-Juicer: A One-Stop Data Processing System for Large Language Models
A data recipe is a mixture of data from different sources for training LLMs, which plays a vital role in LLMs' performance.
InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language
Different from existing interactive systems that rely on pure language, by incorporating pointing instructions, the proposed iGPT significantly improves the efficiency of communication between users and chatbots, as well as the accuracy of chatbots in vision-centric tasks, especially in complicated visual scenarios where the number of objects is greater than 2.
VLG: General Video Recognition with Web Textual Knowledge
Our VLG is first pre-trained on video and language datasets to learn a shared feature space, and then devises a flexible bi-modal attention head to collaborate high-level semantic concepts under different settings.
MotionBERT: A Unified Perspective on Learning Human Motion Representations
We present a unified perspective on tackling various human-centric video tasks by learning human motion representations from large-scale and heterogeneous data resources.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(using extra training data)
Submission to Generic Event Boundary Detection Challenge@CVPR 2022: Local Context Modeling and Global Boundary Decoding Approach
Local context modeling sub-network is proposed to perceive diverse patterns of generic event boundaries, and it generates powerful video representations and reliable boundary confidence.
Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsing
This paper focuses on the weakly-supervised audio-visual video parsing task, which aims to recognize all events belonging to each modality and localize their temporal boundaries.
Progressive Attention on Multi-Level Dense Difference Maps for Generic Event Boundary Detection
Generic event boundary detection is an important yet challenging task in video understanding, which aims at detecting the moments where humans naturally perceive event boundaries.
CausCF: Causal Collaborative Filtering for RecommendationEffect Estimation
It is based on the idea that similar users not only have a similar taste on items, but also have similar treatment effect under recommendations.
DeeperForensics Challenge 2020 on Real-World Face Forgery Detection: Methods and Results
This paper reports methods and results in the DeeperForensics Challenge 2020 on real-world face forgery detection.
PURE: An Uncertainty-aware Recommendation Framework for Maximizing Expected Posterior Utility of Platform
In this paper, we propose a novel recommendation framework which effectively utilizes the information of user uncertainty over different item dimensions and explicitly takes into consideration the impact of display policy on user in order to achieve maximal expected posterior utility for the platform.
Contrastive Learning for Sequential Recommendation
Sequential recommendation methods play a crucial role in modern recommender systems because of their ability to capture a user's dynamic interest from her/his historical interactions.
