Search Results for author:
Found 81 papers, 38 papers with code
VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models
Visual reasoning is a core component of human intelligence and a critical capability for advanced multimodal models.
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm.
Any2Caption:Interpreting Any Condition to Caption for Controllable Video Generation
To address the bottleneck of accurate user intent interpretation within the current video generation community, we present Any2Caption, a novel framework for controllable video generation under any condition.
Griffin: Aerial-Ground Cooperative Detection and Tracking Dataset and Benchmark
Despite significant advancements, autonomous driving systems continue to struggle with occluded objects and long-range detection due to the inherent limitations of single-perspective sensing.
DynamicID: Zero-Shot Multi-ID Image Personalization with Flexible Facial Editability
We present DynamicID, a tuning-free framework supported by a dual-stage training paradigm that inherently facilitates both single-ID and multi-ID personalized generation with high fidelity and flexible facial editability.
FuzzyLight: A Robust Two-Stage Fuzzy Approach for Traffic Signal Control Works in Real Cities
FuzzyLight offers several key contributions: (1) It employs fuzzy logic and compressed sensing to address sensor noise and enhances the efficiency of TSP decisions.
LiT: Delving into a Simplified Linear Diffusion Transformer for Image Generation
(3) Hybrid knowledge distillation objective: using a pre-trained diffusion Transformer to help the training of the student linear Transformer, supervising not only the predicted noise but also the variance of the reverse diffusion process.
GenEx: Generating an Explorable World
In this work, we take a step toward this goal by introducing GenEx, a system capable of planning complex embodied world exploration, guided by its generative imagination that forms priors (expectations) about the surrounding environments.
PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models
To this end, we extend each image into a "static" video and introduce a unified token compression strategy called Progressive Visual Token Compression (PVC), where the tokens of each frame are progressively encoded and adaptively compressed to supplement the information not extracted from previous frames.
VP-MEL: Visual Prompts Guided Multimodal Entity Linking
Multimodal entity linking (MEL), a task aimed at linking mentions within multimodal contexts to their corresponding entities in a knowledge base (KB), has attracted much attention due to its wide applications in recent years.
Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
We introduce InternVL 2. 5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2. 0, maintaining its core model architecture while introducing significant enhancements in training and testing strategies as well as data quality.
Ranked #1 on
Video Question Answering
on NExT-QA
Towards Precise Scaling Laws for Video Diffusion Transformers
Achieving optimal performance of video diffusion transformers within given data and compute budget is crucial due to their high training costs.
TableTime: Reformulating Time Series Classification as Zero-Shot Table Understanding via Large Language Models
Despite their effectiveness, we reveal that these methods conceal three inherent bottlenecks: (1) they struggle to encode temporal and channel-specific information in a lossless manner, both of which are critical components of multivariate time series; (2) it is much difficult to align the learned representation space with the semantic space of the LLMs; (3) they require task-specific retraining, which is both computationally expensive and labor-intensive.
The Oxford Spires Dataset: Benchmarking Large-Scale LiDAR-Visual Localisation, Reconstruction and Radiance Field Methods
Our evaluation demonstrates a key limitation of state-of-the-art radiance field methods: we show that they tend to overfit to the training poses/images and do not generalise well to out-of-sequence poses.
LLM-Neo: Parameter Efficient Knowledge Distillation for Large Language Models
Inspired by this observation, we explore the strategy that combines LoRA and KD to enhance the efficiency of knowledge transfer.
Research on gesture recognition method based on SEDCNN-SVM
The DCNN can automatically extract and learn the feature information of sEMG through the convolution operation of the convolutional layer, so that it can capture the complex and high-level features in the data.
Schedule Your Edit: A Simple yet Effective Diffusion Noise Schedule for Image Editing
To resolve this, we introduce the Logistic Schedule, a novel noise schedule designed to eliminate singularities, improve inversion stability, and provide a better noise space for image editing.
Transit Pulse: Utilizing Social Media as a Source for Customer Feedback and Information Extraction with Large Language Model
Users of the transit system flood social networks daily with messages that contain valuable insights crucial for improving service quality.
DST-TransitNet: A Dynamic Spatio-Temporal Deep Learning Model for Scalable and Efficient Network-Wide Prediction of Station-Level Transit Ridership
This paper introduces DST-TransitNet, a hybrid DL model for system-wide station-level ridership prediction.
Leveraging Large Language Models for Enhancing Public Transit Services
With the help of these three LLM transit applications, transit system media personnel can provide system updates more efficiently, and customers can access travel information and policy answers in a more user-friendly manner.
Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content
As visual generation technologies continue to advance, the scale of video datasets has expanded rapidly, and the quality of these datasets is critical to the performance of video generation models.
PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization
In this work, we propose PrefixQuant, a novel quantization method that achieves state-of-the-art performance across various precision levels (W4A4KV4 and W4A8KV4) and granularities (dynamic and static quantization) by effectively isolating token-wise outliers.
Accelerating Non-Maximum Suppression: A Graph Theory Perspective
The optimal eQSI-NMS, with only a $0. 3\%$ mAP decrease, achieves $10. 7\times$ speed.
Flipped Classroom: Aligning Teacher Attention with Student in Generalized Category Discovery
Recent advancements have shown promise in applying traditional Semi-Supervised Learning strategies to the task of Generalized Category Discovery (GCD).
SpotActor: Training-Free Layout-Controlled Consistent Image Generation
For these issues, we pioneer a novel task, Layout-to-Consistent-Image (L2CI) generation, which produces consistent and compositional images in accordance with the given layout conditions and text prompts.
SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs
Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable perceptual and reasoning abilities, typically comprising a Vision Encoder, an Adapter, and a Large Language Model (LLM).
Ranked #71 on
Visual Question Answering
on MM-Vet
HBot: A Chatbot for Healthcare Applications in Traditional Chinese Medicine Based on Human Body 3D Visualization
However, concepts such as acupuncture points (acupoints) and meridians involved in TCM always appear in the consultation, which cannot be displayed intuitively.
Power-LLaVA: Large Language and Vision Assistant for Power Transmission Line Inspection
Moreover, we also construct a large-scale and high-quality dataset specialized for the inspection task.
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
To the best of our knowledge, Block-AP is the first method to enable direct training of all parameters in a block-wise manner, reducing accuracy loss in low-bit scenarios by enhancing the solution space during optimization.
Fast and Continual Knowledge Graph Embedding via Incremental LoRA
To address this issue, we propose a fast CKGE framework (\model), incorporating an incremental low-rank adapter (\mec) mechanism to efficiently acquire new knowledge while preserving old knowledge.
Mixture-of-Subspaces in Low-Rank Adaptation
In this paper, we introduce a subspace-inspired Low-Rank Adaptation (LoRA) method, which is computationally efficient, easy to implement, and readily applicable to large language, multimodal, and diffusion models.
Ranked #1 on
Visual Question Answering
on MMBench
LOGO: Video Text Spotting with Language Collaboration and Glyph Perception Model
In this paper, we propose a Language Collaboration and Glyph Perception Model, termed LOGO, an innovative framework designed to enhance the performance of conventional text spotters.
Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast
In MoE, each token in the input sequence activates a different subset of experts determined by a routing mechanism.
Mamba-R: Vision Mamba ALSO Needs Registers
Similar to Vision Transformers, this paper identifies artifacts also present within the feature maps of Vision Mamba.
Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots
Furthermore, we propose three automatic evaluation metrics, including code pass rate, text-match ratio, and GPT-4V overall rating, for a fine-grained assessment of the output code and rendered images.
OneActor: Consistent Character Generation via Cluster-Conditioned Guidance
To mitigate the overfitting challenge shared by one-shot tuning pipelines, we augment the tuning with auxiliary samples and devise two inference strategies: semantic interpolation and cluster guidance.
Adapting LLaMA Decoder to Vision Transformer
We first "LLaMAfy" a standard ViT step-by-step to align with LLaMA's architecture, and find that directly applying a causal mask to the self-attention brings an attention collapse issue, resulting in the failure to the network training.
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
Kullback-Leiber divergence has been widely used in Knowledge Distillation (KD) to compress Large Language Models (LLMs).
Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
We categorize Mamba into four roles for modeling videos, deriving a Video Mamba Suite composed of 14 models/modules, and evaluating them on 12 video understanding tasks.
Ranked #2 on
Temporal Action Localization
on FineAction
Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis
X-ray is widely applied for transmission imaging due to its stronger penetration than natural light.
A Poisson-Gamma Dynamic Factor Model with Time-Varying Transition Dynamics
Probabilistic approaches for handling count-valued time sequences have attracted amounts of research attentions because their ability to infer explainable latent structures and to estimate uncertainties, and thus are especially suitable for dealing with \emph{noisy} and \emph{incomplete} count data.
LLMs as Bridges: Reformulating Grounded Multimodal Named Entity Recognition
Grounded Multimodal Named Entity Recognition (GMNER) is a nascent multimodal task that aims to identify named entities, entity types and their corresponding visual regions.
Ranked #1 on
Grounded Multimodal Named Entity Recognition
on Twitter-GMNER
(using extra training data)
Grounded Multimodal Named Entity Recognition
Multi-modal Named Entity Recognition
+8
Less is more: Ensemble Learning for Retinal Disease Recognition Under Limited Resources
This approach establishes a robust model even when confronted with limited labeled data, eliminating the need for an extensive array of parameters, as required in learning from scratch.
A Survey on Data Selection for LLM Instruction Tuning
Instruction tuning is a vital step of training large language models (LLM), so how to enhance the effect of instruction tuning has received increased attention.
LLaMA Pro: Progressive LLaMA with Block Expansion
Humans generally acquire new skills without compromising the old; however, the opposite holds for Large Language Models (LLMs), e. g., from LLaMA to CodeLLaMA.
CRA-PCN: Point Cloud Completion with Intra- and Inter-level Cross-Resolution Transformers
Point cloud completion is an indispensable task for recovering complete point clouds due to incompleteness caused by occlusion, limited sensor resolution, etc.
Free Lunch for Federated Remote Sensing Target Fine-Grained Classification: A Parameter-Efficient Framework
Remote Sensing Target Fine-grained Classification (TFGC) is of great significance in both military and civilian fields.
RepKPU: Point Cloud Upsampling with Kernel Point Representation and Deformation
Moreover we propose a novel paradigm namely Kernel-to-Displacement generation for point generation where point cloud upsampling is reformulated as the deformation of kernel points.
Universal Segmentation at Arbitrary Granularity with Language Instruction
This paper aims to achieve universal segmentation of arbitrary semantic level.
Ranked #1 on
Referring Expression Segmentation
on RefCOCOg-test
(using extra training data)
Structure-Aware Sparse-View X-ray 3D Reconstruction
In this paper, we propose a framework, Structure-Aware X-ray Neural Radiodensity Fields (SAX-NeRF), for sparse-view X-ray 3D reconstruction.
Ranked #1 on
Low-Dose X-Ray Ct Reconstruction
on X3D
Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape
Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model.
Ranked #1 on
Animal Pose Estimation
on Animal3D
Memory-and-Anticipation Transformer for Online Action Understanding
Based on this idea, we present Memory-and-Anticipation Transformer (MAT), a memory-anticipation-based approach, to address the online action detection and anticipation tasks.
Ranked #1 on
Action Detection
on THUMOS' 14
Generating Images with 3D Annotations Using Diffusion Models
With explicit 3D geometry control, we can easily change the 3D structures of the objects in the generated images and obtain ground-truth 3D annotations automatically.
4D Millimeter-Wave Radar in Autonomous Driving: A Survey
In an effort to bridge this gap and stimulate future research, this paper presents an exhaustive survey on the utilization of 4D mmWave radar in autonomous driving.
VideoLLM: Modeling Video Sequence with Large Language Models
Building upon this insight, we propose a novel framework called VideoLLM that leverages the sequence reasoning capabilities of pre-trained LLMs from natural language processing (NLP) for video sequence understanding.
Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge
However, these methods either neglect the necessity of providing the model with external knowledge, or encounter issues of high redundancy in the retrieved knowledge.
Ranked #1 on
Multi-modal Named Entity Recognition
on Twitter-2017
(using extra training data)
Multi-modal Named Entity Recognition
named-entity-recognition
+1
OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images
Enhancing the robustness of vision algorithms in real-world scenarios is challenging.
RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer
Extensive experiments and ablative analysis also demonstrate that the inductive bias of network architecture, can be incorporated into simple network structure with appropriate optimization strategy.
Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
Specifically, we first employ the class prototypes to analyze the impact of graph structures on GNN teachers, and then design two losses to distill such information from GNNs to MLPs.
RIFormer: Keep Your Vision Backbone Effective but Removing Token Mixer
Extensive experiments and ablative analysis also demonstrate that the inductive bias of network architecture, can be incorporated into simple network structure with appropriate optimization strategy.
InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges
In this report, we present our champion solutions to five tracks at Ego4D challenge.
Ranked #1 on
State Change Object Detection
on Ego4D
Exploring State Change Capture of Heterogeneous Backbones @ Ego4D Hands and Objects Challenge 2022
Our method achieves an accuracy of 0. 796 on OSCC while achieving an absolute temporal localization error of 0. 516 on PNR.
Uncertainty Guided Policy for Active Robotic 3D Reconstruction using Neural Radiance Fields
In this paper, we tackle the problem of active robotic 3D reconstruction of an object.
Towards Real-World Video Deblurring by Exploring Blur Formation Process
We believe the novel realistic synthesis pipeline and the corresponding RAW video dataset can help the community to easily construct customized blur datasets to improve real-world video deblurring performance largely, instead of laboriously collecting real data pairs.
Learning Adaptive Warping for Real-World Rolling Shutter Correction
This paper proposes the first real-world rolling shutter (RS) correction dataset, BS-RSC, and a corresponding model to correct the RS frames in a distorted video.
Ranked #5 on
Rolling Shutter Correction
on BS-RSC
Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network
Image quality assessment (IQA) algorithm aims to quantify the human perception of image quality.
Ranked #1 on
Image Quality Assessment
on MSU FR VQA Database
MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment
No-Reference Image Quality Assessment (NR-IQA) aims to assess the perceptual quality of images in accordance with human subjective perception.
Ranked #8 on
Video Quality Assessment
on MSU SR-QA Dataset
Accelerating Neural Network Optimization Through an Automated Control Theory Lens
We conduct experiments both from a control theory lens through a phase locus verification and from a network training lens on several models, including CNNs, Transformers, MLPs, and on benchmark datasets.
SAGA: Stochastic Whole-Body Grasping with Contact
Given an initial pose and the generated whole-body grasping pose as the start and end of the motion respectively, we design a novel contact-aware generative motion infilling module to generate a diverse set of grasp-oriented motions.
Adder Attention for Vision Transformer
With the new operation, vision transformers constructed using additions can also provide powerful feature representations.
Will You Ever Become Popular? Learning to Predict Virality of Dance Clips
Therefore, virality prediction from dance challenges is of great commercial value and has a wide range of applications, such as smart recommendation and popularity promotion.
FAST: Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation
We propose an accurate and efficient scene text detection framework, termed FAST (i. e., faster arbitrarily-shaped text detector).
Ranked #2 on
Scene Text Detection
on MSRA-TD500
LUAI Challenge 2021 on Learning to Understand Aerial Images
This report summarizes the results of Learning to Understand Aerial Images (LUAI) 2021 challenge held on ICCV 2021, which focuses on object detection and semantic segmentation in aerial images.
The Multi-Modal Video Reasoning and Analyzing Competition
In this paper, we introduce the Multi-Modal Video Reasoning and Analyzing Competition (MMVRAC) workshop in conjunction with ICCV 2021.
Few-Shot Fine-Grained Action Recognition via Bidirectional Attention and Contrastive Meta-Learning
Fine-grained action recognition is attracting increasing attention due to the emerging demand of specific action understanding in real-world applications, whereas the data of rare fine-grained categories is very limited.
Quantitatively Nonblocking Supervisory Control of Discrete-Event Systems
Finally, combining with the algorithm of computing the supremal controllable sublanguage, we design algorithms to compute the maximally permissive solutions to the formulated (heterogeneously) quantitatively nonblocking supervisory control problems.
NTIRE 2021 Challenge on Perceptual Image Quality Assessment
This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021.
Region-Adaptive Deformable Network for Image Quality Assessment
Image quality assessment (IQA) aims to assess the perceptual quality of images.
Coarse-to-Fine Searching for Efficient Generative Adversarial Networks
This paper studies the neural architecture search (NAS) problem for developing efficient generator networks.
