Search Results for author:
Found 40 papers, 20 papers with code
mPLUG-PaperOwl: Scientific Diagram Analysis with the Multimodal Large Language Model
In this work, towards a more versatile copilot for academic paper writing, we mainly focus on strengthening the multi-modal diagram analysis ability of Multimodal LLMs.
Stable Cluster Discrimination for Deep Clustering
Meanwhile, one-stage methods are developed mainly for representation learning rather than clustering, where various constraints for cluster assignments are designed to avoid collapsing explicitly.
Ranked #1 on
Unsupervised Image Classification
on CIFAR-10
mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration
Multi-modal Large Language Models (MLLMs) have demonstrated impressive instruction abilities across various open-ended tasks.
Ranked #11 on
Visual Question Answering (VQA)
on InfiMM-Eval
UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model
Text is ubiquitous in our visual world, conveying crucial information, such as in documents, websites, and everyday photographs.
Graph Convolution Based Efficient Re-Ranking for Visual Retrieval
In particular, the plain GCR is extended for cross-camera retrieval and an improved feature propagation formulation is presented to leverage affinity relationships across different cameras.
Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Dataset for Pre-training and Benchmarks
In addition, to facilitate a comprehensive evaluation of video-language models, we carefully build the largest human-annotated Chinese benchmarks covering three popular video-language tasks of cross-modal retrieval, video captioning, and video category classification.
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality
Our code, pre-trained model, instruction-tuned models, and evaluation set are available at https://github. com/X-PLUG/mPLUG-Owl.
Ranked #3 on
Visual Question Answering (VQA)
on HallusionBench
Visual Question Answering (VQA)
Zero-Shot Video Question Answer
ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human
In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format.
Improved Visual Fine-tuning with Natural Language Supervision
Fine-tuning a visual pre-trained model can leverage the semantic information from large-scale pre-training data and mitigate the over-fitting problem on downstream vision tasks with limited training examples.
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
In contrast to predominant paradigms of solely relying on sequence-to-sequence generation or encoder-based instance discrimination, mPLUG-2 introduces a multi-module composition network by sharing common universal modules for modality collaboration and disentangling different modality modules to deal with modality entanglement.
Ranked #1 on
Video Captioning
on MSR-VTT
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
We achieve state-of-the-art results on 15 well-established video-language understanding and generation tasks, especially on temporal-oriented datasets (e. g., SSv2-Template and SSv2-Label) with 8. 6% and 11. 1% improvement respectively.
Ranked #1 on
Visual Question Answering (VQA)
on TGIF-QA
Semantic Data Augmentation based Distance Metric Learning for Domain Generalization
Further, we provide an in-depth analysis of the mechanism and rational behind our approach, which gives us a better understanding of why leverage logits in lieu of features can help domain generalization.
An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation
With our empirical result obtained from 1, 330 models, we provide the following main observations: 1) ERM combined with data augmentation can achieve state-of-the-art performance if we choose a proper pre-trained model respecting the data property; 2) specialized algorithms further improve the robustness on top of ERM when handling a specific type of distribution shift, e. g., GroupDRO for spurious correlation and CORAL for large-scale out-of-distribution data; 3) Comparing different pre-training modes, architectures and data sizes, we provide novel observations about pre-training on distribution shift, which sheds light on designing or selecting pre-training strategy for different kinds of distribution shifts.
RBGNet: Ray-based Grouping for 3D Object Detection
In order to learn better representations of object shape to enhance cluster features for predicting 3D boxes, we propose a ray-based feature grouping module, which aggregates the point-wise features on object surfaces using a group of determined rays uniformly emitted from cluster centers.
Ranked #13 on
3D Object Detection
on ScanNetV2
Reconstruction Task Finds Universal Winning Tickets
Pruning well-trained neural networks is effective to achieve a promising accuracy-efficiency trade-off in computer vision regimes.
Improved Fine-Tuning by Better Leveraging Pre-Training Data
The generalization result of using pre-training data shows that the excess risk bound on a target task can be improved when the appropriate pre-training data is included in fine-tuning.
Dash: Semi-Supervised Learning with Dynamic Thresholding
In this work we develop a simple yet powerful framework, whose key idea is to select a subset of training examples from the unlabeled data when performing existing SSL methods so that only the unlabeled examples with pseudo labels related to the labeled data will be used to train models.
Unsupervised Visual Representation Learning by Online Constrained K-Means
Clustering is to assign each instance a pseudo label that will be used to learn representations in discrimination.
Ranked #5 on
Unsupervised Image Classification
on CIFAR-10
Why Does Multi-Epoch Training Help?
Stochastic gradient descent (SGD) has become the most attractive optimization method in training large-scale deep neural networks due to its simplicity, low computational cost in each updating step, and good performance.
A Theoretical Analysis of Learning with Noisily Labeled Data
Noisy labels are very common in deep supervised learning.
Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework
To alleviate the confirmation bias problem and improve the quality of pseudo annotations, we further propose a co-rectify scheme based on Instant-Teaching, denoted as Instant-Teaching$^*$.
Ranked #12 on
Semi-Supervised Object Detection
on COCO 100% labeled data
(using extra training data)
Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition
Comparing with previous NAS methods, the proposed Zen-NAS is magnitude times faster on multiple server-side and mobile-side GPU platforms with state-of-the-art accuracy on ImageNet.
Ranked #2 on
Neural Architecture Search
on ImageNet
Zen-NAS: A Zero-Shot NAS for High-Performance Image Recognition
To address this issue, instead of using an accuracy predictor, we propose a novel zero-shot index dubbed Zen-Score to rank the architectures.
Neural Architecture Search
Vocal Bursts Intensity Prediction
WeMix: How to Better Utilize Data Augmentation
To this end, we develop two novel algorithms, termed "AugDrop" and "MixLoss", to correct the data bias in the data augmentation.
Improved Knowledge Distillation via Full Kernel Matrix Transfer
Recently, a number of works propose to transfer the pairwise similarity between examples to distill relative information.
Semi-Anchored Detector for One-Stage Object Detection
The anchor-free strategy benefits the classification task but can lead to sup-optimum for the regression task due to the lack of prior bounding boxes.
Neural Architecture Design for GPU-Efficient Networks
To address this issue, we propose a general principle for designing GPU-efficient networks based on extensive empirical studies.
Towards Understanding Label Smoothing
Label smoothing regularization (LSR) has a great success in training deep neural networks by stochastic algorithms such as stochastic gradient descent and its variants.
Weakly Supervised Representation Learning with Coarse Labels
To mitigate this challenge, we propose an algorithm to learn the fine-grained patterns for the target task, when only its coarse-class labels are available.
Hierarchically Robust Representation Learning
Experiments on benchmark data sets demonstrate the effectiveness of the robust deep representations.
SoftTriple Loss: Deep Metric Learning Without Triplet Sampling
The set of triplet constraints has to be sampled within the mini-batch.
Ranked #21 on
Metric Learning
on CUB-200-2011
(using extra training data)
DR Loss: Improving Object Detection by Distributional Ranking
This architecture is efficient but can suffer from the imbalance issue with respect to two aspects: the inter-class imbalance between the number of candidates from foreground and background classes and the intra-class imbalance in the hardness of background candidates, where only a few candidates are hard to be identified.
Robust Gaussian Process Regression for Real-Time High Precision GPS Signal Enhancement
We validate the superiority of the proposed method in our real-time high precision positioning system against several popular state-of-the-art robust regression methods.
Which Factorization Machine Modeling is Better: A Theoretical Answer with Optimal Guarantee
This bound is sub-optimal comparing to the information theoretical lower bound $\mathcal{O}(kd)$.
Large-scale Distance Metric Learning with Uncertainty
Furthermore, we can show that the metric is learned from latent examples only, but it can preserve the large margin property even for the original data.
Robust Optimization over Multiple Domains
Hence, we propose to learn the model and the adversarial distribution simultaneously with the stochastic algorithm for efficiency.
Similarity Learning via Adaptive Regression and Its Application to Image Retrieval
The similarity between pairs of images can be measured by the distances between their high dimensional representations, and the problem of learning the appropriate similarity is often addressed by distance metric learning.
Towards Making High Dimensional Distance Metric Learning Practical
In this work, we present a dual random projection frame for DML with high dimensional data that explicitly addresses the limitation of dimensionality reduction for DML.
Fine-Grained Visual Categorization via Multi-stage Metric Learning
To this end, we proposed a multi-stage metric learning framework that divides the large-scale high dimensional learning problem to a series of simple subproblems, achieving $\mathcal{O}(d)$ computational complexity.
Efficient Distance Metric Learning by Adaptive Sampling and Mini-Batch Stochastic Gradient Descent (SGD)
Although stochastic gradient descent (SGD) has been successfully applied to improve the efficiency of DML, it can still be computationally expensive because in order to ensure that the solution is a PSD matrix, it has to, at every iteration, project the updated distance metric onto the PSD cone, an expensive operation.
