Search Results for author:
Found 192 papers, 112 papers with code
Interpretable Visual Reasoning via Probabilistic Formulation under Natural Supervision
In this paper, we rethink implicit reasoning process in VQA, and propose a new formulation which maximizes the log-likelihood of joint distribution for the observed question and predicted answer.
Weakly-Supervised Crowd Counting Learns from Sorting rather than Locations
We propose a soft-label sorting network along with the counting network, which sorts the given images by their crowd numbers.
Exploring Non-contrastive Self-supervised Representation Learning for Image-based Profiling
Image-based cell profiling aims to create informative representations of cell images.
Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025
Multimodal Large Language Models (MLLMs) have enabled transformative advancements across diverse applications but remain susceptible to safety threats, especially jailbreak attacks that induce harmful outputs.
One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Framework
This approach effectively bypasses the knowledge gap between text and image, significantly enhancing erasure efficacy.
MixBridge: Heterogeneous Image-to-Image Backdoor Attack through Mixture of Schrödinger Bridges
This paper focuses on implanting multiple heterogeneous backdoor triggers in bridge-based diffusion models designed for complex and arbitrary input distributions.
OpenworldAUC: Towards Unified Evaluation and Optimization for Open-world Prompt Tuning
This practical challenge has spurred the development of open-world prompt tuning, which demands a unified evaluation of two stages: 1) detecting whether an input belongs to the base or new domain (P1), and 2) classifying the sample into its correct class (P2).
ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via $α$-$β$-Divergence
In contrast, both are too strong in RKLD, causing the student to overly emphasize the target class while ignoring the broader distributional information from the teacher.
Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs
Despite the remarkable performance of Large Language Models (LLMs), they remain vulnerable to jailbreak attacks, which can compromise their safety mechanisms.
Focal-SAM: Focal Sharpness-Aware Minimization for Long-Tailed Classification
Recent methods resorted to long-tail variants of Sharpness-Aware Minimization (SAM), such as ImbSAM and CC-SAM, to improve generalization by flattening the loss landscape.
Diffusion-based Adversarial Purification from the Perspective of the Frequency Domain
Specifically, at each time step during the reverse process, for the amplitude spectrum, we replace the low-frequency components of the estimated image's amplitude spectrum with the corresponding parts of the adversarial image.
FlowDubber: Movie Dubbing with LLM-based Semantic-aware Learning and Flow Matching based Voice Enhancing
Movie Dubbing aims to convert scripts into speeches that align with the given movie clip in both temporal and emotional aspects while preserving the vocal timbre of a given brief reference audio.
The Devil is in the Distributions: Explicit Modeling of Scene Content is Key in Zero-Shot Video Captioning
Zero-shot video captioning requires that a model generate high-quality captions without human-annotated video-text pairs for training.
Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation
The hallucination of large multimodal models (LMMs), providing responses that appear correct but are actually incorrect, limits their reliability and applicability.
Collaborative Temporal Consistency Learning for Point-supervised Natural Language Video Localization
To address this problem, we propose a new COllaborative Temporal consistEncy Learning (COTEL) framework that leverages the synergy between saliency detection and moment localization to strengthen the video-language alignment.
When the Future Becomes the Past: Taming Temporal Correspondence for Self-supervised Video Representation Learning
For challenge 1), we propose a sandwich sampling strategy that selects two auxiliary frames to reduce reconstruction uncertainty in a two-side-squeezing manner.
Limb-Aware Virtual Try-On Network with Progressive Clothing Warping
Finally, we introduce Limb-aware Texture Fusion (LTF) that focuses on generating realistic details in limb regions, where a coarse try-on result is first generated by fusing the warped clothing image with the person image, then limb textures are further fused with the coarse result under limb-aware guidance to refine limb details.
Uncertainty-aware Long-tailed Weights Model the Utility of Pseudo-labels for Semi-supervised Learning
We further model the utility of pseudo-labels as long-tailed weights to avoid the open problem of setting the threshold.
Stereo Image Coding for Machines with Joint Visual Feature Compression
2D image coding for machines (ICM) has achieved great success in coding efficiency, while less effort has been devoted to stereo image fields.
CANeRV: Content Adaptive Neural Representation for Video Compression
Furthermore, to enhance the capture of dynamics between frames within a sequence, we implement a dynamic frame-level adjustment (DFA).
Enhancing Sample Utilization in Noise-Robust Deep Metric Learning With Subgroup-Based Positive-Pair Selection
Existing noisy label learning methods designed for DML mainly discard suspicious noisy samples, resulting in a waste of the training data.
Separation of Powers: On Segregating Knowledge from Observation in LLM-enabled Knowledge-based Visual Question Answering
To train such a model, we utilize GPT-4 to build a corresponding high-quality question-aware caption dataset on top of existing KBVQA datasets.
Divide and Conquer: Heterogeneous Noise Integration for Diffusion-based Adversarial Purification
Existing diffusion-based purification methods aim to disrupt adversarial perturbations by introducing a certain amount of noise through a forward diffusion process, followed by a reverse process to recover clean examples.
Video Language Model Pretraining with Spatio-temporal Masking
The development of self-supervised video-language models based on mask learning has significantly advanced downstream video tasks.
SafeCFG: Redirecting Harmful Classifier-Free Guidance for Safe Generation
Experimental results show that by incorporating HGR, images generated by diffusion models achieve both high quality and strong safety, and safe DMs trained through unsupervised methods according to the harmfulness detected by HGR also exhibit good safety performance.
Query-centric Audio-Visual Cognition Network for Moment Retrieval, Segmentation and Step-Captioning
In this paper, guided by the shallow-to-deep principle, we propose a query-centric audio-visual cognition (QUAG) network to construct a reliable multi-modal representation for moment retrieval, segmentation and step-captioning.
SSE-SAM: Balancing Head and Tail Classes Gradually through Stage-Wise SAM
Our experiments confirm that SSE-SAM has better ability in escaping saddles both on head and tail classes, and shows performance improvements.
Bidirectional Logits Tree: Pursuing Granularity Reconcilement in Fine-Grained Classification
This paper addresses the challenge of Granularity Competition in fine-grained classification tasks, which arises due to the semantic gap between multi-granularity labels.
Expanding Sparse Tuning for Low Memory Usage
In this paper, we propose a method named SNELL (Sparse tuning with kerNELized LoRA) for sparse tuning with low memory usage.
Towards Dynamic Message Passing on Graphs
With nodes moving in the space, their evolving relations facilitate flexible pathway construction for a dynamic message-passing process.
Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answering
Long-term Video Question Answering (VideoQA) is a challenging vision-and-language bridging task focusing on semantic understanding of untrimmed long-term videos and diverse free-form questions, simultaneously emphasizing comprehensive cross-modal reasoning to yield precise answers.
Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering
Video Question Answering (VideoQA) represents a crucial intersection between video understanding and language processing, requiring both discriminative unimodal comprehension and sophisticated cross-modal interaction for accurate inference.
Suppress Content Shift: Better Diffusion Features via Off-the-Shelf Generation Techniques
We locate the cause of content shift as one inherent characteristic of diffusion models, which suggests the broad existence of this phenomenon in diffusion feature.
Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features
To this end, the early study of this field performs a large-scale quantitative comparison of the discriminative ability of the activations.
AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation
On one hand, AUC optimization in a pixel-level task involves complex coupling across loss terms, with structured inner-image and pairwise inter-image dependencies, complicating theoretical analysis.
Bundle Fragments into a Whole: Mining More Complete Clusters via Submodular Selection of Interesting webpages for Web Topic Detection
A state-of-the-art solution is firstly to organize webpages into a large volume of multi-granularity topic candidates; hot topics are further identified by estimating their interestingness.
Improved Diversity-Promoting Collaborative Metric Learning for Recommendation
Under this setting, the unique user representation might induce preference bias, especially when the item category distribution is imbalanced.
Decorrelating Structure via Adapters Makes Ensemble Learning Practical for Semi-supervised Learning
This allows DSA to be easily extensible to architecture-agnostic networks for a range of computer vision tasks.
Regularized Contrastive Partial Multi-view Outlier Detection
In recent years, multi-view outlier detection (MVOD) methods have advanced significantly, aiming to identify outliers within multi-view datasets.
Towards Scalable Topic Detection on Web via Simulating Levy Walks Nature of Topics in Similarity Space
Discovering popular topics from web faces a sea of noise webpages which never evolve into popular topics.
Scalable Graph Compressed Convolutions
Previous methods of designing optimal pathways are limited with information loss on the input features.
Downstream-Pretext Domain Knowledge Traceback for Active Learning
DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator.
Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning
However, most existing methods directly capture the difference between them, which risk obtaining error-prone difference features.
Top-K Pairwise Ranking: Bridging the Gap Among Ranking-Based Measures for Multi-Label Classification
On the other hand, we establish a sharp generalization bound for the proposed framework based on a novel technique named data-dependent contraction.
Sequential Manipulation Against Rank Aggregation: Theory and Algorithm
Since it is independent of rank aggregation and lacks effective protection mechanisms, we disrupt the data collection process by fabricating pairwise comparisons without knowledge of the future data or the true distribution.
Context-aware Difference Distilling for Multi-change Captioning
Given an image pair, CARD first decouples context features that aggregate all similar/dissimilar semantics, termed common/difference context features.
Size-invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detection
This paper explores the size-invariance of evaluation metrics in Salient Object Detection (SOD), especially when multiple targets of diverse sizes co-exist in the same image.
ReconBoost: Boosting Can Achieve Modality Reconcilement
This paper explores a novel multi-modal alternating learning paradigm pursuing a reconciliation between the exploitation of uni-modal features and the exploration of cross-modal interactions.
Harnessing Hierarchical Label Distribution Variations in Test Agnostic Long-tail Recognition
Traditional methods predominantly use a Mixture-of-Expert (MoE) approach, targeting a few fixed test label distributions that exhibit substantial global variations.
Ranked #1 on
Test Agnostic Long-Tailed Learning
on CIFAR-10-LT
RETTA: Retrieval-Enhanced Test-Time Adaptation for Zero-Shot Video Captioning
Specifically, we bridge video and text using four key models: a general video-text retrieval model XCLIP, a general image-text matching model CLIP, a text alignment model AnglE, and a text generation model GPT-2, due to their source-code availability.
Uncertainty-boosted Robust Video Activity Anticipation
Video activity anticipation aims to predict what will happen in the future, embracing a broad application prospect ranging from robot vision and autonomous driving.
A Channel-ensemble Approach: Unbiased and Low-variance Pseudo-labels is Critical for Semi-supervised Classification
Semi-supervised learning (SSL) is a practical challenge in computer vision.
A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes
Three-Dimensional (3D) dense captioning is an emerging vision-language bridging task that aims to generate multiple detailed and accurate descriptions for 3D scenes.
Query-guided Prototype Evolution Network for Few-Shot Segmentation
To address this, we present the Query-guided Prototype Evolution Network (QPENet), a new method that integrates query features into the generation process of foreground and background prototypes, thereby yielding customized prototypes attuned to specific queries.
StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing
Given a script, the challenge in Movie Dubbing (Visual Voice Cloning, V2C) is to generate speech that aligns well with the video in both time and emotion, based on the tone of a reference audio track.
Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization
To this end, we propose Pick-and-Draw, a training-free semantic guidance approach to boost identity consistency and generative diversity for personalization methods.
Bias-Conflict Sample Synthesis and Adversarial Removal Debias Strategy for Temporal Sentence Grounding in Video
Temporal Sentence Grounding in Video (TSGV) is troubled by dataset bias issue, which is caused by the uneven temporal distribution of the target moments for samples with similar semantic components in input videos or query texts.
Prompt-Enhanced Multiple Instance Learning for Weakly Supervised Video Anomaly Detection
The detector can utilize the semantic-rich features to capture diverse abnormal patterns.
Weakly Supervised Video Individual Counting
To provide a more realistic reflection of the underlying practical challenge we introduce a weakly supervised VIC task wherein trajectory labels are not provided.
ADA-GAD: Anomaly-Denoised Autoencoders for Graph Anomaly Detection
We pretrain graph autoencoders on these augmented graphs at multiple levels, which enables the graph autoencoders to capture normal patterns.
SOVC: Subject-Oriented Video Captioning
To address this problem, we propose a new video captioning task, Subject-Oriented Video Captioning (SOVC), which aims to allow users to specify the describing target via a bounding box.
Weakly Supervised Video Individual CountingWeakly Supervised Video Individual Counting
% To provide a more realistic reflection of the underlying practical challenge, we introduce a weakly supervised VIC task, wherein trajectory labels are not provided.
Dynamic Erasing Network Based on Multi-Scale Temporal Features for Weakly Supervised Video Anomaly Detection
To address these limitations, we propose a Dynamic Erasing Network (DE-Net) for weakly supervised video anomaly detection, which learns multi-scale temporal features.
DRAUC: An Instance-wise Distributionally Robust AUC Optimization Framework
To tackle this challenge, methodically we propose an instance-wise surrogate loss of Distributionally Robust AUC (DRAUC) and build our optimization framework on top of it.
Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression
Both semi-supervised classification and regression are practically challenging tasks for computer vision.
Modeling the Uncertainty with Maximum Discrepant Students for Semi-supervised 2D Pose Estimation
Semi-supervised pose estimation is a practically challenging task for computer vision.
R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation
Recent text-to-image (T2I) diffusion models have achieved remarkable progress in generating high-quality images given text-prompts as input.
Towards Demystifying the Generalization Behaviors When Neural Collapse Emerges
While there have been extensive studies on optimization characteristics showing the global optimality of neural collapse, little research has been done on the generalization behaviors during the occurrence of NC.
Open-Set Knowledge-Based Visual Question Answering with Inference Paths
Given an image and an associated textual question, the purpose of Knowledge-Based Visual Question Answering (KB-VQA) is to provide a correct answer to the question with the aid of external knowledge bases.
A Unified Generalization Analysis of Re-Weighting and Logit-Adjustment for Imbalanced Learning
However, existing generalization analysis of such losses is still coarse-grained and fragmented, failing to explain some empirical results.
A Unified Generalization Analysis of Re-Weighting and Logit-Adjustment for Imbalanced Learning. paper with code
However, existing generalization analysis of such losses is still coarse-grained and fragmented, failing to explain some empirical results.
Ranked #6 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
Learning node representation via Motif Coarsening
For the motif-based node representation learning process, we propose a Motif Coarsening strategy for incorporating motif structure into the graph representation learning process.
Self-supervised Cross-view Representation Reconstruction for Change Captioning
Change captioning aims to describe the difference between a pair of similar images.
AUC-Oriented Domain Adaptation: From Theory to Algorithm
We propose a new result that not only addresses the interdependency issue but also brings a much sharper bound with weaker assumptions about the loss function.
Revisiting AUC-oriented Adversarial Training with Loss-Agnostic Perturbations
On top of this, we can show that: 1) Under mild conditions, AdAUC can be optimized equivalently with score-based or instance-wise-loss-based perturbations, which is compatible with most of the popular adversarial example generation methods.
When Measures are Unreliable: Imperceptible Adversarial Perturbations toward Top-$k$ Multi-Label Learning
However, existing adversarial attacks toward multi-label learning only pursue the traditional visual imperceptibility but ignore the new perceptible problem coming from measures such as Precision@$k$ and mAP@$k$.
PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN with Dual-Discriminators
Among existing UIE methods, Generative Adversarial Networks (GANs) based methods perform well in visual aesthetics, while the physical model-based methods have better scene adaptability.
Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-training
To address this limitation, we propose a unified Med-VLP framework based on Multi-task Paired Masking with Alignment (MPMA) to integrate the cross-modal alignment task into the joint image-text reconstruction framework to achieve more comprehensive cross-modal interaction, while a Global and Local Alignment (GLA) module is designed to assist self-supervised paradigm in obtaining semantic representations with rich domain knowledge.
A Study of Neural Collapse Phenomenon: Grassmannian Frame, Symmetry and Generalization
Out of curiosity about the symmetry of Grassmannian Frame, we conduct experiments to explore if models with different Grassmannian Frames have different performance.
Neighborhood Contrastive Transformer for Change Captioning
Change captioning is to describe the semantic change between a pair of similar images in natural language.
Stable Attribute Group Editing for Reliable Few-shot Image Generation
SAGE takes use of all given few-shot images and estimates a class center embedding based on the category-relevant attribute dictionary.
Building Bridge Across the Time: Disruption and Restoration of Murals In the Wild
Finally, experimental results support the effectiveness of the proposed framework in terms of both mural synthesis and restoration.
Text-Driven Generative Domain Adaptation with Spectral Consistency Regularization
The experiments demonstrate the effectiveness of our method to preserve the diversity of source domain and generate high fidelity target images.
Multi-Projection Fusion and Refinement Network for Salient Object Detection in 360° Omnidirectional Image
Salient object detection (SOD) aims to determine the most visually attractive objects in an image.
Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detection
Weakly supervised video anomaly detection aims to identify abnormal events in videos using only video-level labels.
Progressive Multi-resolution Loss for Crowd Counting
Crowd counting is usually handled in a density map regression fashion, which is supervised via a L2 loss between the predicted density map and ground truth.
Consistency-Aware Anchor Pyramid Network for Crowd Localization
Crowd localization aims to predict the spatial position of humans in a crowd scenario.
Learning to Dub Movies via Hierarchical Prosody Models
Given a piece of text, a video clip and a reference audio, the movie dubbing (also known as visual voice clone V2C) task aims to generate speeches that match the speaker's emotion presented in the video using the desired speaker voice as reference.
Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective
Positive-Unlabeled (PU) learning tries to learn binary classifiers from a few labeled positive examples with many unlabeled ones.
OpenAUC: Towards AUC-Oriented Open-Set Recognition
In this paper, a systematic analysis reveals that most existing metrics are essentially inconsistent with the aforementioned goal of OSR: (1) For metrics extended from close-set classification, such as Open-set F-score, Youden's index, and Normalized Accuracy, a poor open-set prediction can escape from a low performance score with a superior close-set prediction.
Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspective
We conduct comprehensive transferable attacks against multiple DNNs to demonstrate the effectiveness of the proposed method.
Does Thermal Really Always Matter for RGB-T Salient Object Detection?
In addition, considering the role of thermal modality, we set up different cross-modality interaction mechanisms in the encoding phase and the decoding phase.
Asymptotically Unbiased Instance-wise Regularized Partial AUC Optimization: Theory and Algorithm
sample size and a slow convergence rate, especially for TPAUC.
CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection
Focusing on the issue of how to effectively capture and utilize cross-modality information in RGB-D salient object detection (SOD) task, we present a convolutional neural network (CNN) model, named CIR-Net, based on the novel cross-modality interaction and refinement.
A Unified Framework against Topology and Class Imbalance
We develop a multi-class AUC optimization work to deal with the class imbalance problem.
Recurrent Meta-Learning against Generalized Cold-start Problem in CTR Prediction
To attack this problem, we propose a recursive meta-learning model with the user's behavior sequence prediction as a separate training task.
The Minority Matters: A Diversity-Promoting Collaborative Metric Learning Algorithm
Collaborative Metric Learning (CML) has recently emerged as a popular method in recommendation systems (RS), closing the gap between metric learning and Collaborative Filtering.
Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability
Stochastic optimization of the Area Under the Precision-Recall Curve (AUPRC) is a crucial problem for machine learning.
MaxMatch: Semi-Supervised Learning with Worst-Case Consistency
In recent years, great progress has been made to incorporate unlabeled data to overcome the inefficiently supervised problem via semi-supervised learning (SSL).
A Tale of HodgeRank and Spectral Method: Target Attack Against Rank Aggregation Is the Fixed Point of Adversarial Game
From the perspective of the dynamical system, the attack behavior with a target ranking list is a fixed point belonging to the composition of the adversary and the victim.
Optimizing Partial Area Under the Top-k Curve: Theory and Practice
Finally, the experimental results on four benchmark datasets validate the effectiveness of our proposed framework.
Multi-Attention Network for Compressed Video Referring Object Segmentation
To address this problem, we propose a multi-attention network which consists of dual-path dual-attention module and a query-based cross-modal Transformer module.
Ranked #5 on
Referring Expression Segmentation
on A2D Sentences
Entity-enhanced Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding
Second, most previous weakly supervised REG methods ignore the discriminative location and context of the referent, causing difficulties in distinguishing the target from other same-category objects.
Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction
In this paper, we regard user interest modeling as a feature selection problem, which we call user interest selection.
Geometry Interaction Knowledge Graph Embeddings
Knowledge graph (KG) embeddings have shown great power in learning representations of entities and relations for link prediction tasks.
ER: Equivariance Regularizer for Knowledge Graph Completion
To address this issue, we propose a new regularizer, namely, Equivariance Regularizer (ER), which can suppress overfitting by leveraging the implicit semantic information.
AdAUC: End-to-end Adversarial AUC Optimization Against Long-tail Problems
Our analysis differs from the existing studies since the algorithm is asked to generate adversarial examples by calculating the gradient of a min-max problem.
Rethinking Collaborative Metric Learning: Toward an Efficient Alternative without Negative Sampling
However, in this work, by taking a theoretical analysis, we find that negative sampling would lead to a biased estimation of the generalization error.
Optimizing Two-way Partial AUC with an End-to-end Framework
The critical challenge along this course lies in the difficulty of performing gradient-based optimization with end-to-end stochastic training, even with a proper choice of surrogate loss.
Automatic Relation-aware Graph Network Proliferation
However, the currently used graph search space overemphasizes learning node features and neglects mining hierarchical relational information.
Ranked #2 on
Link Prediction
on TSP/HCP Benchmark set
Global-and-Local Collaborative Learning for Co-Salient Object Detection
In this paper, we propose a global-and-local collaborative learning architecture, which includes a global correspondence modeling (GCM) and a local correspondence modeling (LCM) to capture comprehensive inter-image corresponding relationship among different images from the global and local perspectives.
CenterNet++ for Object Detection
Our approach, named CenterNet, detects each object as a triplet keypoints (top-left and bottom-right corners and the center keypoint).
Ranked #38 on
Object Detection
on COCO test-dev
IR-GAN: Image Manipulation with Linguistic Instruction by Increment Reasoning
Conditional image generation is an active research topic including text2image and image translation.
Conditional Image Generation
Generative Adversarial Network
+1
Attribute Group Editing for Reliable Few-shot Image Generation
Few-shot image generation is a challenging task even using the state-of-the-art Generative Adversarial Networks (GANs).
Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment
A feasible solution is to start with a GAN well-trained on a large scale source domain and adapt it to the target domain with a few samples, termed as few shot generative model adaption.
General Greedy De-bias Learning
Existing de-bias learning frameworks try to capture specific dataset bias by annotations but they fail to handle complicated OOD scenarios.
When False Positive is Intolerant: End-to-End Optimization with Low FPR for Multipartite Ranking
To leverage high performance under low FPRs, we consider an alternative metric for multipartite ranking evaluating the True Positive Rate (TPR) at a given FPR, denoted as TPR@FPR.
Hierarchical Modular Network for Video Captioning
(II) Predicate level, which learns the actions conditioned on highlighted objects and is supervised by the predicate in captions.
Modeling Temporal Concept Receptive Field Dynamically for Untrimmed Video Analysis
Based on TDC, we propose the temporal dynamic concept modeling network (TDCMN) to learn an accurate and complete concept representation for efficient untrimmed video analysis.
Self-Regulated Learning for Egocentric Video Activity Anticipation
Future activity anticipation is a challenging problem in egocentric vision.
DVCFlow: Modeling Information Flow Towards Human-like Video Captioning
Dense video captioning (DVC) aims to generate multi-sentence descriptions to elucidate the multiple events in the video, which is challenging and demands visual consistency, discoursal coherence, and linguistic diversity.
Implicit Feedbacks are Not Always Favorable: Iterative Relabeled One-Class Collaborative Filtering against Noisy Interactions
As the core of the framework, the iterative relabeling module exploits the self-training principle to dynamically generate pseudo labels for user preferences.
Learning Meta-path-aware Embeddings for Recommender Systems
Heterogeneous information networks (HINs) have become a popular tool to capture complicated user-item relationships in recommendation problems in recent years.
Semi-Autoregressive Image Captioning
Non-autoregressive image captioning with continuous iterative refinement, which eliminates the sequential dependence in a sentence generation, can achieve comparable performance to the autoregressive counterparts with a considerable acceleration.
Pareto Optimality for Fairness-constrained Collaborative Filtering
To balance overall recommendation performance and fairness, prevalent solutions apply fairness constraints or regularizations to enforce equality of certain performance across different subgroups.
Edge-featured Graph Neural Architecture Search
Recently, researchers study neural architecture search (NAS) to reduce the dependence of human expertise and explore better GNN architectures, but they over-emphasize entity features and ignore latent relation information concealed in the edges.
Learning with Multiclass AUC: Theory and Algorithms
Our foundation is based on the M metric, which is a well-known multiclass extension of AUC.
Greedy Gradient Ensemble for Robust Visual Question Answering
Language bias is a critical issue in Visual Question Answering (VQA), where models often exploit dataset biases for the final decision without considering the image information.
Ranked #2 on
Visual Question Answering (VQA)
on VQA-CP
When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC.
The critical challenge along this course lies in the difficulty of performing gradient-based optimization with end-to-end stochastic training, even with a proper choice of surrogate loss.
Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation
Due to the domain discrepancy in visual domain adaptation, the performance of source model degrades when bumping into the high data density near decision boundary in target domain.
Poisoning Attack against Estimating from Pairwise Comparisons
In this paper, to the best of our knowledge, we initiate the first systematic investigation of data poisoning attacks on pairwise ranking algorithms, which can be formalized as the dynamic and static games between the ranker and the attacker and can be modeled as certain kinds of integer programming problems.
When False Positive is Intolerant: End-to-End Optimization with Low FPR for Multipartite Ranking
To leverage high performance under low FPRs, we consider an alternative metric for multipartite ranking evaluating the True Positive Rate (TPR) at a given FPR, denoted as TPR@FPR.
Location-Sensitive Visual Recognition with Cross-IOU Loss
Object detection, instance segmentation, and pose estimation are popular visual recognition tasks which require localizing the object by internal or boundary landmarks.
Ranked #60 on
Object Detection
on COCO test-dev
Rethinking Graph Neural Architecture Search from Message-passing
Inspired by the strong searching capability of neural architecture search (NAS) in CNN, this paper proposes Graph Neural Architecture Search (GNAS) with novel-designed search space.
Seeking the Shape of Sound: An Adaptive Framework for Learning Voice-Face Association
Targeting at (a), we propose a two-level modality alignment loss where both global and local information are considered.
Viewpoint and Scale Consistency Reinforcement for UAV Vehicle Re-Identification
Moreover, our method also achieves competitive performance compared with recent works on existing vehicle ReID datasets including VehicleID, VeRi-776 and VERI-Wild.
Exploiting Sample Correlation for Crowd Counting With Multi-Expert Network
Specifically, we propose a task-driven similarity metric based on sample's mutual enhancement, referred as co-fine-tune similarity, which can find a more efficient subset of data for training the expert network.
Heuristic Domain Adaptation
In visual domain adaptation (DA), separating the domain-specific characteristics from the domain-invariant representations is an ill-posed problem.
Semantic Editing On Segmentation Map Via Multi-Expansion Loss
Each MEx area has the mask area of the generation as the majority and the boundary of original context as the minority.
Label Decoupling Framework for Salient Object Detection
Though remarkable progress has been achieved, we observe that the closer the pixel is to the edge, the more difficult it is to be predicted, because edge pixels have a very imbalance distribution.
Ranked #1 on
Saliency Detection
on HKU-IS
Corner Proposal Network for Anchor-free, Two-stage Object Detection
On the MS-COCO dataset, CPN achieves an AP of 49. 2% which is competitive among state-of-the-art object detection methods.
Ranked #98 on
Object Detection
on COCO test-dev
Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction
To this end, we propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL).
Parsing-based View-aware Embedding Network for Vehicle Re-Identification
Vehicle Re-Identification is to find images of the same vehicle from various views in the cross-camera scenario.
State-Relabeling Adversarial Active Learning
In this paper, we propose a state relabeling adversarial active learning model (SRAAL), that leverages both the annotation and the labeled/unlabeled state information for deriving the most informative unlabeled samples.
Gradually Vanishing Bridge for Adversarial Domain Adaptation
On the discriminator, GVB contributes to enhance the discriminating ability, and balance the adversarial training process.
Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations
We find by theoretical analysis that the prediction discriminability and diversity could be separately measured by the Frobenius-norm and rank of the batch output matrix.
DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection
There are two main issues in RGB-D salient object detection: (1) how to effectively integrate the complementarity from the cross-modal RGB-D data; (2) how to prevent the contamination effect from the unreliable depth map.
Ranked #22 on
Thermal Image Segmentation
on RGB-T-Glass-Segmentation
Global Context-Aware Progressive Aggregation Network for Salient Object Detection
Deep convolutional neural networks have achieved competitive performance in salient object detection, in which how to learn effective and comprehensive features plays a critical role.
Ranked #21 on
Dichotomous Image Segmentation
on DIS-TE1
DM2C: Deep Mixed-Modal Clustering
Instead of transforming all the samples into a joint modality-independent space, our framework learns the mappings across individual modal spaces by virtue of cycle-consistency.
Generalized Block-Diagonal Structure Pursuit: Learning Soft Latent Task Assignment against Negative Transfer
Different from most of the previous work, pursuing the Block-Diagonal structure of LTAM (assigning latent tasks to output tasks) alleviates negative transfer via collaboratively grouping latent tasks and output tasks such that inter-group knowledge transfer and sharing is suppressed.
F3Net: Fusion, Feedback and Focus for Salient Object Detection
Furthermore, different from binary cross entropy, the proposed PPA loss doesn't treat pixels equally, which can synthesize the local structure information of a pixel to guide the network to focus more on local details.
Ranked #6 on
Salient Object Detection
on DUT-OMRON
Camouflaged Object Segmentation
Dichotomous Image Segmentation
+3
Collaborative Preference Embedding against Sparse Labels
From the margin theory point-of-view, we then propose a generalization enhancement scheme for sparse and insufficient labels via optimizing the margin distribution.
iSplit LBI: Individualized Partial Ranking with Ties via Split LBI
In this paper, instead of learning a global ranking which is agreed with the consensus, we pursue the tie-aware partial ranking from an individualized perspective.
Learning fragment self-attention embeddings for image-text matching
In this paper, we propose Self-Attention Embeddings (SAEM) to exploit fragment relations in images or texts by self-attention mechanism, and aggregate fragment information into visual and textual embeddings.
Knowledge-guided Pairwise Reconstruction Network for Weakly Supervised Referring Expression Grounding
Weakly supervised referring expression grounding (REG) aims at localizing the referential entity in an image according to linguistic query, where the mapping between the image region (proposal) and the query is unknown in the training stage.
Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding
It builds the correspondence between image region proposal and query in an adaptive manner: adaptive grounding and collaborative reconstruction.
Harmonized Multimodal Learning with Gaussian Process Latent Variable Models
Multimodal learning aims to discover the relationship between multiple modalities.
Learning Personalized Attribute Preference via Multi-task AUC Optimization
Traditionally, most of the existing attribute learning methods are trained based on the consensus of annotations aggregated from a limited number of annotators.
Multimodal Transformer with Multi-View Visual Representation for Image Captioning
Despite the success of existing studies, current methods only model the co-attention that characterizes the inter-modal interactions while neglecting the self-attention that characterizes the intra-modal interactions.
Unsupervised Open Domain Recognition by Semantic Discrepancy Minimization
We address the unsupervised open domain recognition (UODR) problem, where categories in labeled source domain S is only a subset of those in unlabeled target domain T. The task is to correctly classify all samples in T including known and unknown categories.
Cascaded Partial Decoder for Fast and Accurate Salient Object Detection
In this paper, we propose a novel Cascaded Partial Decoder (CPD) framework for fast and accurate salient object detection.
Ranked #1 on
RGB Salient Object Detection
on ISTD
CenterNet: Keypoint Triplets for Object Detection
In object detection, keypoint-based approaches often suffer a large number of incorrect object bounding boxes, arguably due to the lack of an additional look into the cropped regions.
Ranked #121 on
Object Detection
on COCO test-dev
Spatiotemporal CNN for Video Object Segmentation
Specifically, the temporal coherence branch pretrained in an adversarial fashion from unlabeled video data, is designed to capture the dynamic appearance and motion cues of video sequences to guide object segmentation.
Ranked #1 on
Semi-Supervised Video Object Segmentation
on YouTube
Deep Robust Subjective Visual Property Prediction in Crowdsourcing
The problem of estimating subjective visual properties (SVP) of images (e. g., Shoes A is more comfortable than B) is gaining rising attention.
See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification
Specifically, for each training image, we first generate attention maps to represent the object's discriminative parts by weakly supervised learning.
Ranked #16 on
Fine-Grained Image Classification
on CUB-200-2011
HSCS: Hierarchical Sparsity Based Co-saliency Detection for RGBD Images
In this paper, we propose a novel co-saliency detection method for RGBD images based on hierarchical sparsity reconstruction and energy function refinement.
Person Re-Identification by Semantic Region Representation and Topology Constraint
In this paper, we propose a novel person re-identification method, which consists of a reliable representation called Semantic Region Representation (SRR), and an effective metric learning with Mapping Space Topology Constraint (MSTC).
Weakly Supervised Bilinear Attention Network for Fine-Grained Visual Classification
Besides, we propose attention regularization and attention dropout to weakly supervise the generating process of attention maps.
A Margin-based MLE for Crowdsourced Partial Ranking
A preference order or ranking aggregated from pairwise comparison data is commonly understood as a strict total order.
RAM: A Region-Aware Deep Model for Vehicle Re-Identification
Specifically, in addition to extracting global features, RAM also extracts features from a series of local regions.
The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking
Selected from 10 hours raw videos, about 80, 000 representative frames are fully annotated with bounding boxes as well as up to 14 kinds of attributes (e. g., weather condition, flying altitude, camera view, vehicle category, and occlusion) for three fundamental computer vision tasks: object detection, single object tracking, and multiple object tracking.
Ranked #5 on
Object Detection
on UAVDT
Facial Landmarks Detection by Self-Iterative Regression based Landmarks-Attention Network
Only one self-iterative regressor is trained to learn the descent directions for samples from coarse stages to fine stages, and parameters are iteratively updated by the same regressor.
Ranked #16 on
Face Alignment
on 300W
(NME_inter-pupil (%, Common) metric)
Review of Visual Saliency Detection with Comprehensive Information
With the acquisition technology development, more comprehensive information, such as depth cue, inter-image correspondence, or temporal relationship, is available to extend image saliency detection to RGBD saliency detection, co-saliency detection, or video saliency detection.
From Social to Individuals: a Parsimonious Path of Multi-level Models for Crowdsourced Preference Aggregation
In crowdsourced preference aggregation, it is often assumed that all the annotators are subject to a common preference or social utility function which generates their comparison behaviors in experiments.
Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs
Experimental results show that our method is capable of generating both visually comfortable and identity-preserving face sketches/photos over a wide range of challenging data.
Ranked #1 on
Face Sketch Synthesis
on CUFS
(FID metric)
From Common to Special: When Multi-Attribute Learning Meets Personalized Opinions
However, both categories ignore the joint effect of the two mentioned factors: the personal diversity with respect to the global consensus; and the intrinsic correlation among multiple attributes.
HodgeRank with Information Maximization for Crowdsourced Pairwise Ranking Aggregation
Recently, crowdsourcing has emerged as an effective paradigm for human-powered large scale problem solving in various domains.
An Iterative Co-Saliency Framework for RGBD Images
In this paper, we propose an iterative RGBD co-saliency framework, which utilizes the existing single saliency maps as the initialization, and generates the final RGBD cosaliency map by using a refinement-cycle model.
Saliency Detection for Stereoscopic Images Based on Depth Confidence Analysis and Multiple Cues Fusion
Stereoscopic perception is an important part of human visual system that allows the brain to perceive depth.
Co-saliency Detection for RGBD Images Based on Multi-constraint Feature Matching and Cross Label Propagation
Different from the most existing co-saliency methods focusing on RGB images, this paper proposes a novel co-saliency detection model for RGBD images, which utilizes the depth information to enhance identification of co-saliency.
Multimodal Gaussian Process Latent Variable Models With Harmonization
We incorporate the harmonization mechanism into the learning process of multimodal GPLVMs.
Exploring Outliers in Crowdsourced Ranking for QoE
Outlier detection is a crucial part of robust evaluation for crowdsourceable assessment of Quality of Experience (QoE) and has attracted much attention in recent years.
A Graph Regularized Deep Neural Network for Unsupervised Image Representation Learning
Deep Auto-Encoder (DAE) has shown its promising power in high-level representation learning.
Online Asymmetric Similarity Learning for Cross-Modal Retrieval
In this paper, we propose an online learning method to learn the similarity function between heterogeneous modalities by preserving the relative similarity in the training data, which is modeled as a set of bi-directional hinge loss constraints on the cross-modal training triplets.
Hedged Deep Tracking
In recent years, several methods have been developed to utilize hierarchical features learned from a deep convolutional neural network (CNN) for visual tracking.
Geometric Hypergraph Learning for Visual Tracking
Graph based representation is widely used in visual tracking field by finding correct correspondences between target parts in consecutive frames.
Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks
Learning deeper convolutional neural networks becomes a tendency in recent years.
Ranked #8 on
Long-tail Learning
on VOC-MLT
Similarity Gaussian Process Latent Variable Model for Multi-Modal Data Analysis
Data from real applications involve multiple modalities representing content with the same semantics and deliver rich information from complementary aspects.
Evaluating Visual Properties via Robust HodgeRank
In this paper we study the problem of how to estimate such visual properties from a ranking perspective with the help of the annotators from online crowdsourcing platforms.
Multi-level Discriminative Dictionary Learning towards Hierarchical Visual Categorization
For each internode of the hierarchical category structure, a discriminative dictionary and a set of classification models are learnt for visual categorization, and the dictionaries in different layers are learnt to exploit the discriminative visual properties of different granularity.
