Search Results for author:
Found 43 papers, 26 papers with code
$\texttt{Complex-Edit}$: CoT-Like Instruction Generation for Complexity-Controllable Image Editing Benchmark
We introduce $\texttt{Complex-Edit}$, a comprehensive benchmark designed to systematically evaluate instruction-based image editing models across instructions of varying complexity.
"Principal Components" Enable A New Language of Images
We introduce a novel visual tokenization framework that embeds a provable PCA-like structure into the latent token space.
Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis
Our method features a delicate quality control method which ensures the data quality.
A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning
Pre-trained vision models (PVMs) are fundamental to modern robotics, yet their optimal configuration remains unclear.
EENet: Frequency-Aware and Spatially Multiscale Network for Single Image Dehazing
EENet comprises three primary modules: the frequency processing module, the spatial processing module, and the dual-domain interaction module.
Ranked #1 on
Image Dehazing
on RB-Dust
Humanity's Last Exam
However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities.
Ranked #4 on
Humanity's Last Exam
on Humanity's Last Exam
Libra-Leaderboard: Towards Responsible AI through a Balanced Leaderboard of Safety and Capability
To address this gap, we introduce Libra-Leaderboard, a comprehensive framework designed to rank LLMs through a balanced evaluation of performance and safety.
CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions
Previous works show that noisy, web-crawled image-text pairs may limit vision-language pretraining like CLIP and propose learning with synthetic captions as a promising alternative.
AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation
This paper studies the vulnerabilities of transformer-based Large Language Models (LLMs) to jailbreaking attacks, focusing specifically on the optimization-based Greedy Coordinate Gradient (GCG) strategy.
A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor?
Large language models (LLMs) have exhibited remarkable capabilities across various domains and tasks, pushing the boundaries of our knowledge in learning and cognition.
Contextuality Helps Representation Learning for Generalized Category Discovery
This paper introduces a novel approach to Generalized Category Discovery (GCD) by leveraging the concept of contextuality to enhance the identification and classification of categories in unlabeled datasets.
PromptCCD: Learning Gaussian Mixture Prompt Pool for Continual Category Discovery
We tackle the problem of Continual Category Discovery (CCD), which aims to automatically discover novel categories in a continuous stream of unlabeled data while mitigating the challenge of catastrophic forgetting -- an open problem that persists even in conventional, fully supervised continual learning.
Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning
The advancement of large language models (LLMs) has significantly broadened the scope of applications in natural language processing, with multi-modal LLMs extending these capabilities to integrate and interpret visual data.
What If We Recaption Billions of Web Images with LLaMA-3?
For discriminative models like CLIP, we observe enhanced zero-shot performance in cross-modal retrieval tasks.
Ranked #127 on
Visual Question Answering
on MM-Vet
Labeled Data Selection for Category Discovery
The labeled data provides guidance during training by indicating what types of visual properties and features are relevant for performing discovery in the unlabeled data.
What Makes CLIP More Robust to Long-Tailed Pre-Training Data? A Controlled Study for Transferable Insights
Severe data imbalance naturally exists among web-scale vision-language datasets.
HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing
This study introduces HQ-Edit, a high-quality instruction-based image editing dataset with around 200, 000 edits.
Beyond Known Clusters: Probe New Prototypes for Efficient Generalized Class Discovery
To counteract this inefficiency, we opt to cluster only the unlabelled instances and subsequently expand the cluster prototypes with our introduced potential prototypes to fast explore novel classes.
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
We present Eagle (RWKV-5) and Finch (RWKV-6), sequence models improving upon the RWKV (RWKV-4) architecture.
Beyond the Known: Novel Class Discovery for Open-world Graph Learning
Inter-class correlations are subsequently eliminated by the prototypical attention network, leading to distinctive representations for different classes.
AQA-Bench: An Interactive Benchmark for Evaluating LLMs' Sequential Reasoning Ability
This paper introduces AQA-Bench, a novel benchmark to assess the sequential reasoning capabilities of large language models (LLMs) in algorithmic contexts, such as depth-first search (DFS).
Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning
This paper introduces an efficient strategy to transform Large Language Models (LLMs) into Multi-Modal Large Language Models (MLLMs).
How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs
Different from prior studies, we shift our focus from evaluating standard performance to introducing a comprehensive safety evaluation suite, covering both out-of-distribution (OOD) generalization and adversarial robustness.
What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Models
In light of the advancements in current multi-modal large language models, we explore their effectiveness in counterfactual reasoning.
Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations
Large language and vision-language models are rapidly being deployed in practice thanks to their impressive capabilities in instruction following, in-context learning, and so on.
Sight Beyond Text: Multi-Modal Training Enhances LLMs in Truthfulness and Ethics
Multi-modal large language models (MLLMs) are trained based on large language models (LLM), with an enhanced capability to comprehend multi-modal inputs and generate textual responses.
Learning Semi-supervised Gaussian Mixture Models for Generalized Category Discovery
In this paper, we address the problem of generalized category discovery (GCD), \ie, given a set of images where part of them are labelled and the rest are not, the task is to automatically cluster the images in the unlabelled data, leveraging the information from the labelled data, while the unlabelled data contain images from the labelled classes and also new ones.
Incremental Generalized Category Discovery
We explore the problem of Incremental Generalized Category Discovery (IGCD).
OOD-CV-v2: An extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images
Enhancing the robustness of vision algorithms in real-world scenarios is challenging.
Vision Learners Meet Web Image-Text Pairs
In this work, given the excellent scalability of web data, we consider self-supervised pre-training on noisy web sourced image-text paired data.
Parametric Classification for Generalized Category Discovery: A Baseline Study
Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples.
One Venue, Two Conferences: The Separation of Chinese and American Citation Networks
At NeurIPS, American and Chinese institutions cite papers from each other's regions substantially less than they cite endogamously.
XCon: Learning with Experts for Fine-grained Category Discovery
We address the problem of generalized category discovery (GCD) in this paper, i. e. clustering the unlabeled images leveraging the information from a set of seen classes, where the unlabeled images could contain both seen classes and unseen classes.
Self-Supervised Visual Representation Learning with Semantic Grouping
The semantic grouping is performed by assigning pixels to a set of learnable prototypes, which can adapt to each sample by attentive pooling over the feature and form new slots.
Ranked #22 on
Unsupervised Semantic Segmentation
on COCO-Stuff-27
(Clustering [Accuracy] metric)
Discriminability-Transferability Trade-Off: An Information-Theoretic Perspective
This work simultaneously considers the discriminability and transferability properties of deep representations in the typical supervised learning task, i. e., image classification.
OOD-CV: A Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images
One reason is that existing robustness benchmarks are limited, as they either rely on synthetic data or ignore the effects of individual nuisance factors.
Improving Contrastive Learning by Visualizing Feature Transformation
To our knowledge, this is the first attempt of its kind.
Novel Visual Category Discovery with Dual Ranking Statistics and Mutual Knowledge Distillation
In this paper, we tackle the problem of novel visual category discovery, i. e., grouping unlabelled images from new classes into different semantic partitions by leveraging a labelled dataset that contains images from other different but relevant categories.
Rail-5k: a Real-World Dataset for Rail Surface Defects Detection
This paper presents the Rail-5k dataset for benchmarking the performance of visual algorithms in a real-world application scenario, namely the rail surface defects detection task.
Reducing the feature divergence of RGB and near-infrared images using Switchable Normalization
Visual pattern recognition over agricultural areas is an important application of aerial image processing.
Temporal Context Aggregation for Video Retrieval with Contrastive Learning
The current research focus on Content-Based Video Retrieval requires higher-level video representation describing the long-range semantic dependencies of relevant incidents, events, etc.
Ranked #6 on
Video Retrieval
on FIVR-200K
Distilling Visual Priors from Self-Supervised Learning
Convolutional Neural Networks (CNNs) are prone to overfit small training datasets.
The 1st Agriculture-Vision Challenge: Methods and Results
The first Agriculture-Vision Challenge aims to encourage research in developing novel and effective algorithms for agricultural pattern recognition from aerial images, especially for the semantic segmentation task associated with our challenge dataset.
