Search Results for author:
Found 55 papers, 32 papers with code
VITA: Towards Open-Source Interactive Omni Multimodal LLM
The remarkable multimodal capabilities and interactive experience of GPT-4o underscore their necessity in practical applications, yet open-source models rarely excel in both areas.
Multi-branch Collaborative Learning Network for 3D Visual Grounding
3D referring expression comprehension (3DREC) and segmentation (3DRES) have overlapping objectives, indicating their potential for collaboration.
Efficient Event Stream Super-Resolution with Recursive Multi-Branch Fusion
FFM is designed for the fusion of contextual information within neighboring event streams, leveraging the coupling relationship between positive and negative events to alleviate the misleading of noises in the respective branches.
Local Manifold Learning for No-Reference Image Quality Assessment
This crop is then used to cluster other crops from the same image as the positive class, while crops from different images are treated as negative classes to increase inter-class distance.
Depth-Guided Semi-Supervised Instance Segmentation
Additionally, to manage the variability of depth images during training, we introduce the Depth Controller.
VEGA: Learning Interleaved Image-Text Comprehension in Vision-Language Large Models
In support of this task, we further craft a new VEGA dataset, tailored for the IITC task on scientific content, and devised a subtask, Image-Text Association (ITA), to refine image-text correlation skills.
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
With Video-MME, we extensively evaluate various state-of-the-art MLLMs, including GPT-4 series and Gemini 1. 5 Pro, as well as open-source image models like InternVL-Chat-V1. 5 and video models like LLaVA-NeXT-Video.
Bilateral Event Mining and Complementary for Event Stream Super-Resolution
In this paper, we propose a bilateral event mining and complementary network (BMCNet) to fully leverage the potential of each event and capture the shared information to complement each other simultaneously.
GraCo: Granularity-Controllable Interactive Segmentation
In this work, we introduce Granularity-Controllable Interactive Segmentation (GraCo), a novel approach that allows precise control of prediction granularity by introducing additional parameters to input.
Cantor: Inspiring Multimodal Chain-of-Thought of MLLM
This paper delves into the realm of multimodal CoT to solve intricate visual reasoning tasks with multimodal large language models(MLLMs) and their cognitive capability.
Multi-Modal Prompt Learning on Blind Image Quality Assessment
Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly.
Motion-aware Latent Diffusion Models for Video Frame Interpolation
With the advancement of AIGC, video frame interpolation (VFI) has become a crucial component in existing video generation frameworks, attracting widespread research interest.
Rethinking 3D Dense Caption and Visual Grounding in A Unified Framework through Prompt-based Localization
3D Visual Grounding (3DVG) and 3D Dense Captioning (3DDC) are two crucial tasks in various 3D applications, which require both shared and complementary information in localization and visual-language relationships.
AffineQuant: Affine Transformation Quantization for Large Language Models
Among these techniques, Post-Training Quantization (PTQ) has emerged as a subject of considerable interest due to its noteworthy compression efficiency and cost-effectiveness in the context of training.
Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models
Contrary to previous works, we study this problem from the perspective of image resolution, and reveal that a combination of low- and high-resolution visual features can effectively mitigate this shortcoming.
Ranked #85 on
Visual Question Answering
on MM-Vet
Data Interpreter: An LLM Agent For Data Science
Large Language Model (LLM)-based agents have demonstrated remarkable effectiveness.
EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs
For instance, on the Wikitext2 dataset with LlamaV1-7B at 70% sparsity, our proposed EBFT achieves a perplexity of 16. 88, surpassing the state-of-the-art DSnoT with a perplexity of 75. 14.
Feature Denoising Diffusion Model for Blind Image Quality Assessment
Blind Image Quality Assessment (BIQA) aims to evaluate image quality in line with human perception, without reference benchmarks.
Instance Brownian Bridge as Texts for Open-vocabulary Video Instance Segmentation
To mold instance queries to follow Brownian bridge and accomplish alignment with class texts, we design Bridge-Text Alignment (BTA) to learn discriminative bridge-level representations of instances via contrastive objectives.
Binding-Adaptive Diffusion Models for Structure-Based Drug Design
Then the selected protein-ligand subcomplex is processed with SE(3)-equivariant neural networks, and transmitted back to each atom of the complex for augmenting the target-aware 3D molecule diffusion generation with binding interaction information.
RepAn: Enhanced Annealing through Re-parameterization
The simulated annealing algorithm aims to improve model convergence through multiple restarts of training.
Solving the Catastrophic Forgetting Problem in Generalized Category Discovery
The LER optimizes the distribution of potential known class samples in unlabeled data thus ensuring the preservation of knowledge related to known categories while learning novel classes.
Adaptive Feature Selection for No-Reference Image Quality Assessment by Mitigating Semantic Noise Sensitivity
Specifically, QFM-IQM enhances the semantic noise distinguish capabilities by matching image pairs with similar quality scores but varying semantic features as adversarial semantic noise and adaptively adjusting the upstream task's features by reducing sensitivity to adversarial noise perturbation.
Less is More: Learning Reference Knowledge Using No-Reference Image Quality Assessment
Concretely, by innovatively introducing a novel feature distillation method in IQA, we propose a new framework to learn comparative knowledge from non-aligned reference images.
AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Acceleration
Therefore, we propose to search the optimal time steps sequence and compressed model architecture in a unified framework to achieve effective image generation for diffusion models without any further training.
DLIP: Distilling Language-Image Pre-training
Furthermore, DLIP succeeds in retaining more than 95% of the performance with 22. 4% parameters and 24. 8% FLOPs compared to the teacher model and accelerates inference speed by 2. 7x.
A Unified Framework for 3D Point Cloud Visual Grounding
This elaborate design enables 3DRefTR to achieve both well-performing 3DRES and 3DREC capacities with only a 6% additional latency compared to the original 3DREC model.
Knowledge Prompt-tuning for Sequential Recommendation
Pre-trained language models (PLMs) have demonstrated strong performance in sequential recommendation (SR), which are utilized to extract general knowledge.
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Remarkable progress has been made on automated problem solving through societies of agents based on large language models (LLMs).
Ranked #14 on
Code Generation
on HumanEval
Learning Sparse Neural Networks with Identity Layers
Applying such theory, we propose a plug-and-play CKA-based Sparsity Regularization for sparse network training, dubbed CKA-SR, which utilizes CKA to reduce feature similarity between layers and increase network sparsity.
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image.
Out-of-Distributed Semantic Pruning for Robust Semi-Supervised Learning
We argue that an overlooked problem of robust SSL is its corrupted information on semantic level, practically limiting the development of the field.
Probabilistic Modeling: Proving the Lottery Ticket Hypothesis in Spiking Neural Network
LTH opens up a new path for network pruning.
Data-Efficient Image Quality Assessment with Attention-Panel Decoder
Blind Image Quality Assessment (BIQA) is a fundamental task in computer vision, which however remains unresolved due to the complex distortion conditions and diversified image contents.
Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective
In particular, we first formulate the oscillation in PTQ and prove the problem is caused by the difference in module capacity.
A Unified Framework for Soft Threshold Pruning
In this work, we reformulate soft threshold pruning as an implicit optimization problem solved using the Iterative Shrinkage-Thresholding Algorithm (ISTA), a classic method from the fields of sparse recovery and compressed sensing.
Automatic Network Pruning via Hilbert-Schmidt Independence Criterion Lasso under Information Bottleneck Principle
In this paper, we try to solve this problem by introducing a principled and unified framework based on Information Bottleneck (IB) theory, which further guides us to an automatic pruning approach.
Meta Architecture for Point Cloud Analysis
However, the lack of a unified framework to interpret those networks makes any systematic comparison, contrast, or analysis challenging, and practically limits healthy development of the field.
Ranked #2 on
3D Semantic Segmentation
on OpenTrench3D
Iterative Data Refinement for Self-Supervised MR Image Reconstruction
Then, we design an effective self-supervised training data refinement method to reduce this data bias.
DIGEST: Deeply supervIsed knowledGE tranSfer neTwork learning for brain tumor segmentation with incomplete multi-modal MRI scans
In this work, we propose a Deeply supervIsed knowledGE tranSfer neTwork (DIGEST), which achieves accurate brain tumor segmentation under different modality-missing scenarios.
Adaptive PromptNet For Auxiliary Glioma Diagnosis without Contrast-Enhanced MRI
Multi-contrast magnetic resonance imaging (MRI)-based automatic auxiliary glioma diagnosis plays an important role in the clinic.
Training-free Transformer Architecture Search
Recently, Vision Transformer (ViT) has achieved remarkable success in several computer vision tasks.
Uncovering the Over-smoothing Challenge in Image Super-Resolution: Entropy-based Quantification and Contrastive Optimization
PSNR-oriented models are a critical class of super-resolution models with applications across various fields.
Neural Architecture Search With Representation Mutual Information
Building upon RMI, we further propose a new search algorithm termed RMI-NAS, facilitating with a theorem to guarantee the global optimal of the searched architecture.
OMPQ: Orthogonal Mixed Precision Quantization
Instead of solving a problem of the original integer programming, we propose to optimize a proxy metric, the concept of network orthogonality, which is highly correlated with the loss of the integer programming but also easy to optimize with linear programming.
An Information Theory-inspired Strategy for Automatic Network Pruning
This practically limits the application of model compression when the model needs to be deployed on a wide range of devices.
You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient
Motivated by the necessity of efficient inference across various constraints on BERT, we propose a novel approach, YOCO-BERT, to achieve compress once and deploy everywhere.
On Evolving Attention Towards Domain Adaptation
However, considering that the configuration of attention, i. e., the type and the position of attention module, affects the performance significantly, it is more generalized to optimize the attention configuration automatically to be specialized for arbitrary UDA scenario.
Ranked #1 on
Partial Domain Adaptation
on Office-Home
EC-DARTS: Inducing Equalized and Consistent Optimization Into DARTS
EC-DARTS decouples different operations based on their categories to optimize the operation weights so that the operation gap between them is shrinked.
PAMS: Quantized Super-Resolution via Parameterized Max Scale
Specifically, most state-of-the-art SR models without batch normalization have a large dynamic quantization range, which also serves as another cause of performance drop.
Binarized Neural Architecture Search for Efficient Object Recognition
In this paper, binarized neural architecture search (BNAS), with a search space of binarized convolutions, is introduced to produce extremely compressed models to reduce huge computational cost on embedded devices for edge computing.
Rethinking Performance Estimation in Neural Architecture Search
In this paper, we provide a novel yet systematic rethinking of PE in a resource constrained regime, termed budgeted PE (BPE), which precisely and effectively estimates the performance of an architecture sampled from an architecture space.
Binarized Neural Architecture Search
A variant, binarized neural architecture search (BNAS), with a search space of binarized convolutions, can produce extremely compressed models.
DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning
With the proposed efficient network generation method, we directly obtain the optimal neural architectures on given constraints, which is practical for on-device models across diverse search spaces and constraints.
Multinomial Distribution Learning for Effective Neural Architecture Search
Therefore, NAS can be transformed to a multinomial distribution learning problem, i. e., the distribution is optimized to have a high expectation of the performance.
