Search Results for author:
Found 102 papers, 35 papers with code
Robust Computer Vision in an Ever-Changing World: A Survey of Techniques for Tackling Distribution Shifts
As a result, it raises concerns about the overall robustness of the machine learning techniques for computer vision applications that are deployed publicly for consumers.
Multilevel Saliency-Guided Self-Supervised Learning for Image Anomaly Detection
To fully exploit saliency guidance, on each map, we select a pixel pair from the cluster with the highest centroid saliency to form a patch pair.
Image-Pointcloud Fusion based Anomaly Detection using PD-REAL Dataset
We present PD-REAL, a novel large-scale dataset for unsupervised anomaly detection (AD) in the 3D domain.
FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning
Therefore, the label guidance on labeled data is hard to be propagated to unlabeled data.
Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization
By fully exploring both variant and invariant parameters, our EVIL can effectively identify a robust subnetwork to improve OOD generalization.
On the Onset of Robust Overfitting in Adversarial Training
Adversarial Training (AT) is a widely-used algorithm for building robust neural networks, but it suffers from the issue of robust overfitting, the fundamental mechanism of which remains unclear.
A Multi-In and Multi-Out Dendritic Neuron Model and its Optimization
Recently, a novel mathematical ANN model, known as the dendritic neuron model (DNM), has been proposed to address nonlinear problems by more accurately reflecting the structure of real neurons.
Regularly Truncated M-estimators for Learning with Noisy Labels
However, such a procedure is arguably debatable from two folds: (a) it does not consider the bad influence of noisy labels in selected small-loss examples; (b) it does not make good use of the discarded large-loss examples, which may be clean or have meaningful information for generalization.
The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation
In this paper, we summarize the winning solutions from the RoboDepth Challenge -- an academic competition designed to facilitate and advance robust OoD depth estimation.
A Two-stage Fine-tuning Strategy for Generalizable Manipulation Skill of Embodied AI
The advent of Chat-GPT has led to a surge of interest in Embodied AI.
Unleashing the Potential of Regularization Strategies in Learning with Noisy Labels
In recent years, research on learning with noisy labels has focused on devising novel algorithms that can achieve robustness to noisy training labels while generalizing to clean data.
Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervision
Training a classifier exploiting a huge amount of supervised data is expensive or even prohibited in a situation, where the labeling cost is high.
Importance Sparsification for Sinkhorn Algorithm
Sinkhorn algorithm has been used pervasively to approximate the solution to optimal transport (OT) and unbalanced optimal transport (UOT) problems.
SAGE-NDVI: A Stereotype-Breaking Evaluation Metric for Remote Sensing Image Dehazing Using Satellite-to-Ground NDVI Knowledge
Image dehazing is a meaningful low-level computer vision task and can be applied to a variety of contexts.
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks
In this paper, we introduce a unified and generalist Biomedical Generative Pre-trained Transformer (BiomedGPT) model, which leverages self-supervision on large and diverse datasets to accept multi-modal inputs and perform a range of downstream tasks.
Ranked #1 on
Text Summarization
on MeQSum
Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-training
To address this limitation, we propose a unified Med-VLP framework based on Multi-task Paired Masking with Alignment (MPMA) to integrate the cross-modal alignment task into the joint image-text reconstruction framework to achieve more comprehensive cross-modal interaction, while a Global and Local Alignment (GLA) module is designed to assist self-supervised paradigm in obtaining semantic representations with rich domain knowledge.
ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos
A recent benchmark AGQA poses a promising paradigm to generate QA pairs automatically from pre-annotated scene graphs, enabling it to measure diverse reasoning abilities with granular control.
A Comparison of Image Denoising Methods
In this paper, to investigate the applicability of existing denoising techniques, we compare a variety of denoising methods on both synthetic and real-world datasets for different applications.
The Second Monocular Depth Estimation Challenge
This paper discusses the results for the second edition of the Monocular Depth Estimation Challenge (MDEC).
SAR2EO: A High-resolution Image Translation Framework with Denoising Enhancement
Synthetic Aperture Radar (SAR) to electro-optical (EO) image translation is a fundamental task in remote sensing that can enrich the dataset by fusing information from different sources.
Robust Generalization against Photon-Limited Corruptions via Worst-Case Sharpness Minimization
Experimentally, we simulate photon-limited corruptions using CIFAR10/100 and ImageNet30 datasets and show that SharpDRO exhibits a strong generalization ability against severe corruptions and exceeds well-known baseline methods with large performance gains.
A Dual Branch Network for Emotional Reaction Intensity Estimation
Emotional Reaction Intensity(ERI) estimation is an important task in multimodal scenarios, and has fundamental applications in medicine, safe driving and other fields.
Local Region Perception and Relationship Learning Combined with Feature Fusion for Facial Action Unit Detection
Human affective behavior analysis plays a vital role in human-computer interaction (HCI) systems.
Exploring Large-scale Unlabeled Faces to Enhance Facial Expression Recognition
Facial Expression Recognition (FER) is an important task in computer vision and has wide applications in human-computer interaction, intelligent security, emotion analysis, and other fields.
Prompting Large Language Models with Answer Heuristics for Knowledge-based Visual Question Answering
Knowledge-based visual question answering (VQA) requires external knowledge beyond the image to answer the question.
Ranked #2 on
Visual Question Answering (VQA)
on A-OKVQA
GLOW: Global Layout Aware Attacks on Object Detection
More desired attacks, to this end, should be able to fool defenses with such consistency checks.
A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
This study provides a comprehensive review of recent research advancements, challenges, and opportunities for PFMs in text, image, graph, as well as other data modalities.
ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectories
In this paper, we propose a novel and flexible conditional diffusion model by introducing conditions into the forward process.
Combating Noisy Labels with Sample Selection by Mining High-Discrepancy Examples
As selected data have high discrepancies in probabilities, the divergence of two networks can be maintained by training on such data.
Graph Matching with Bi-level Noisy Correspondence
In this paper, we study a novel and widely existing problem in graph matching (GM), namely, Bi-level Noisy Correspondence (BNC), which refers to node-level noisy correspondence (NNC) and edge-level noisy correspondence (ENC).
Ranked #1 on
Graph Matching
on Willow Object Class
Teacher-Student Network for 3D Point Cloud Anomaly Detection with Few Normal Samples
Moreover, our method only requires very few normal samples to train the student network due to the teacher-student distillation mechanism.
Facial Expression Spotting Based on Optical Flow Features
The purpose of micro expression (ME) and macro expression (MaE) spotting task is to locate the onset and offset frames of MaE and ME clips.
Micro Expression Generation with Thin-plate Spline Motion Model and Face Parsing
We (Team: USTC-IAT-United) also compare our method with other competitors' in MEGC2022, and the expert evaluation results show that our method performs best, which verifies the effectiveness of our method.
Strength-Adaptive Adversarial Training
Firstly, applying a pre-specified perturbation budget on networks of various model capacities will yield divergent degree of robustness disparity between natural and robust accuracies, which deviates from robust network's desideratum.
Pseudo-Label Generation and Various Data Augmentation for Semi-Supervised Hyperspectral Object Detection
To overcome these difficulties, this paper first select fewer but suitable data augmentation methods to improve the accuracy of the supervised model based on the labeled training set, which is suitable for the characteristics of hyperspectral images.
Tensor-Based Multi-Modality Feature Selection and Regression for Alzheimer's Disease Diagnosis
The assessment of Alzheimer's Disease (AD) and Mild Cognitive Impairment (MCI) associated with brain changes remains a challenging task.
Bag of Tricks and a Strong Baseline for FGVC
This motivates us to explore the impact of different methods and components in fine-grained classification on FungiCLEF 2022.
High-Performance Transmission Mechanism Design of Multi-Stream Carrier Aggregation for 5G Non-Standalone Network
Multi-stream carrier aggregation is a key technology to expand bandwidth and improve the throughput of the fifth-generation wireless communication systems.
A viable framework for semi-supervised learning on realistic dataset
Semi-supervised Fine-Grained Recognition is a challenging task due to the difficulty of data imbalance, high inter-class similarity and domain mismatch.
Learning Disentangled Representations for Controllable Human Motion Prediction
Specifically, the inductive bias imposed by the extra CVAE path encourages two latent variables in two paths to respectively govern separate representations for each partial-body motion.
Understanding Robust Overfitting of Adversarial Training and Beyond
Here, we explore the causes of robust overfitting by comparing the data distribution of \emph{non-overfit} (weak adversary) and \emph{overfitted} (strong adversary) adversarial training, and observe that the distribution of the adversarial data generated by weak adversary mainly contain small-loss data.
An optimal transport approach for selecting a representative subsample with application in efficient kernel density estimation
Theoretically, we show the selected subsample can be used for efficient density estimation by deriving the convergence rate for the proposed subsample kernel density estimator.
Hilbert Curve Projection Distance for Distribution Comparison
Distribution comparison plays a central role in many machine learning tasks like data classification and generative modeling.
Efficient Approximation of Gromov-Wasserstein Distance Using Importance Sparsification
As a valid metric of metric-measure spaces, Gromov-Wasserstein (GW) distance has shown the potential for matching problems of structured data like point clouds and graphs.
Scene Clustering Based Pseudo-labeling Strategy for Multi-modal Aerial View Object Classification
Multi-modal aerial view object classification (MAVOC) in Automatic target recognition (ATR), although an important and challenging problem, has been under studied.
Multi-model Ensemble Learning Method for Human Expression Recognition
Moreover, we introduce the multi-fold ensemble method to train and ensemble several models with the same architecture but different data distributions to enhance the performance of our solution.
Bilaterally Slimmable Transformer for Elastic and Efficient Visual Question Answering
Recent advances in Transformer architectures [1] have brought remarkable improvements to visual question answering (VQA).
Hyper-relationship Learning Network for Scene Graph Generation
As a result, HLN significantly improves the performance of scene graph generation by integrating and reasoning from object interactions, relationship interactions, and transitive inference of hyper-relationships.
Do We Need to Penalize Variance of Losses for Learning with Label Noise?
Algorithms which minimize the averaged loss have been widely designed for dealing with noisy labels.
Wnet: Audio-Guided Video Object Segmentation via Wavelet-Based Cross-Modal Denoising Networks
Audio-Guided video semantic segmentation is a challenging problem in visual analysis and editing, which automatically separates foreground objects from background in a video sequence according to the referring audio expressions.
ESCNet: Gaze Target Detection With the Understanding of 3D Scenes
This paper aims to address the single image gaze target detection problem.
Mining limited data for more robust and generalized ML models
It’s more efficient to look for ways improving the data based a fixed neural network architecture.
Structure-Preserving Graph Kernel for Brain Network Classification
This paper presents a novel graph-based kernel learning approach for connectome analysis.
Uncertainty Set Prediction of Aggregated Wind Power Generation based on Bayesian LSTM and Spatio-Temporal Analysis
This paper focuses on the uncertainty set prediction of the aggregated generation of geographically distributed wind farms.
Co-variance: Tackling Noisy Labels with Sample Selection by Emphasizing High-variance Examples
The sample selection approach is popular in learning with noisy labels, which tends to select potentially clean data out of noisy data for robust training.
ROSITA: Enhancing Vision-and-Language Semantic Alignments via Cross- and Intra-modal Knowledge Integration
Nevertheless, most existing VLP approaches have not fully utilized the intrinsic knowledge within the image-text pairs, which limits the effectiveness of the learned alignments and further restricts the performance of their models.
BiSTF: Bilateral-Branch Self-Training Framework for Semi-Supervised Large-scale Fine-Grained Recognition
Semi-supervised Fine-Grained Recognition is a challenge task due to the difficulty of data imbalance, high inter-class similarity and domain mismatch.
A Weakly-Supervised Depth Estimation Network Using Attention Mechanism
It consists of a dense residual network structure, an adaptive weight channel attention (AWCA) module, a patch second non-local (PSNL) module and a soft label generation method.
The Story in Your Eyes: An Individual-difference-aware Model for Cross-person Gaze Estimation
We propose a novel method on refining cross-person gaze prediction task with eye/face images only by explicitly modelling the person-specific differences.
Improving White-box Robustness of Pre-processing Defenses via Joint Adversarial Training
However, pre-processing methods may suffer from the robustness degradation effect, in which the defense reduces rather than improving the adversarial robustness of a target model in a white-box setting.
Sample Selection with Uncertainty of Losses for Learning with Noisy Labels
In this way, we also give large-loss but less selected data a try; then, we can better distinguish between the cases (a) and (b) by seeing if the losses effectively decrease with the uncertainty after the try.
Ranked #26 on
Image Classification
on mini WebVision 1.0
Instance Correction for Learning with Open-set Noisy Labels
Lots of approaches, e. g., loss correction and label correction, cannot handle such open-set noisy labels well, since they need training data and test data to share the same label space, which does not hold for learning with open-set noisy labels.
Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching
This paper proposes an approach to Dense Video Captioning (DVC) without pairwise event-sentence annotation.
Removing Adversarial Noise in Class Activation Feature Space
Then, we train a denoising model to minimize the distances between the adversarial examples and the natural examples in the class activation feature space.
Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashing
Essentially, DGCPN addresses the inaccurate similarity problem by exploring and exploiting the data's intrinsic relationships in a graph.
Sufficient dimension reduction for classification using principal optimal transport direction
The proposed method, named principal optimal transport direction (POTD), estimates the basis of the SDR subspace using the principal directions of the optimal transport coupling between the data respecting different response categories.
Learning Domain-invariant Graph for Adaptive Semi-supervised Domain Adaptation with Few Labeled Source Samples
Traditional domain adaptation algorithms assume that enough labeled data, which are treated as the prior knowledge are available in the source domain.
Retrieval of Family Members Using Siamese Neural Network
While in inference procedure, we try another similarity computing method by dropping the followed several fully connected layers and directly computing the cosine similarity of the two feature vectors.
Deep Fusion Siamese Network for Automatic Kinship Verification
Recognizing Families In the Wild (RFIW) is a challenging kinship recognition task with multiple tracks, which is based on Families in the Wild (FIW), a large-scale and comprehensive image database for automatic kinship recognition.
Optimal Distributed Subsampling for Maximum Quasi-Likelihood Estimators with Massive Data
We first derive optimal Poisson subsampling probabilities in the context of quasi-likelihood estimation under the A- and L-optimality criteria.
Deep Multimodal Neural Architecture Search
Most existing works focus on a single task and design neural architectures manually, which are highly task-specific and hard to generalize to different tasks.
Ranked #19 on
Visual Question Answering (VQA)
on VQA v2 test-std
Repulsive Mixture Models of Exponential Family PCA for Clustering
To alleviate this problem, in this paper, a repulsiveness-encouraging prior is introduced among mixing components and a diversified EPCA mixture (DEPCAM) model is developed in the Bayesian framework.
Detecting Communities in Heterogeneous Multi-Relational Networks:A Message Passing based Approach
Specifically, an HMRNet is reorganized into a hierarchical structure with homogeneous networks as its layers and heterogeneous links connecting them.
Weakly-Supervised Multi-Level Attentional Reconstruction Network for Grounding Textual Queries in Videos
The task of temporally grounding textual queries in videos is to localize one video segment that semantically corresponds to the given query.
Multimodal Unified Attention Networks for Vision-and-Language Interactions
Learning an effective attention mechanism for multimodal data is important in many vision-and-language tasks that require a synergic understanding of both the visual and textual contents.
Deep Modular Co-Attention Networks for Visual Question Answering
In this paper, we propose a deep Modular Co-Attention Network (MCAN) that consists of Modular Co-Attention (MCA) layers cascaded in depth.
Ranked #26 on
Visual Question Answering (VQA)
on VQA v2 test-std
Effective degrees of freedom for surface finish defect detection and classification
One of the primary concerns of product quality control in the automotive industry is an automated detection of defects of small sizes on specular car body surfaces.
ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering
It is both crucial and natural to extend this research direction to the video domain for video question answering (VideoQA).
Ranked #15 on
Video Question Answering
on ActivityNet-QA
Visual Question Answering (VQA)
Zero-Shot Video Question Answer
Multimodal Transformer with Multi-View Visual Representation for Image Captioning
Despite the success of existing studies, current methods only model the co-attention that characterizes the inter-modal interactions while neglecting the self-attention that characterizes the intra-modal interactions.
Grand Challenge of 106-Point Facial Landmark Localization
However, previous competitions on facial landmark localization (i. e., the 300-W, 300-VW and Menpo challenges) aim to predict 68-point landmarks, which are incompetent to depict the structure of facial components.
On Exploring Undetermined Relationships for Visual Relationship Detection
In this paper, we explore the beneficial effect of undetermined relationships on visual relationship detection.
Local Deep-Feature Alignment for Unsupervised Dimension Reduction
Next, we exploit an affine transformation to align the local deep features of each neighbourhood with the global features.
Single Pixel Reconstruction for One-stage Instance Segmentation
In this paper, we propose a one-stage framework, SPRNet, which performs efficient instance segmentation by introducing a single pixel reconstruction (SPR) branch to off-the-shelf one-stage detectors.
Adapting Stochastic Block Models to Power-Law Degree Distributions
Stochastic block models (SBMs) have been playing an important role in modeling clusters or community structures of network data.
Cross-modal Subspace Learning via Kernel Correlation Maximization and Discriminative Structure Preserving
Our model not only considers the inter-modality correlation by maximizing the kernel correlation but also preserves the semantically structural information within each modality.
Unsupervised Multi-modal Hashing for Cross-modal retrieval
With the advantage of low storage cost and high efficiency, hashing learning has received much attention in the domain of Big Data.
Discriminative Supervised Hashing for Cross-Modal similarity Search
With the advantage of low storage cost and high retrieval efficiency, hashing techniques have recently been an emerging topic in cross-modal similarity search.
Textually Guided Ranking Network for Attentional Image Retweet Modeling
In this paper, we study the problem of image retweet prediction in social media, which predicts the image sharing behavior that the user reposts the image tweets from their followees.
Learning Discriminative Hashing Codes for Cross-Modal Retrieval based on Multi-view Features
Many hashing methods based on a single view have been extensively studied for information retrieval.
Semi-supervised Hashing for Semi-Paired Cross-View Retrieval
Recently, hashing techniques have gained importance in large-scale retrieval tasks because of their retrieval speed.
Rethinking Diversified and Discriminative Proposal Generation for Visual Grounding
Visual grounding aims to localize an object in an image referred to by a textual query phrase.
Ranked #9 on
Phrase Grounding
on Flickr30k Entities Test
FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification
Images captured by fisheye lenses violate the pinhole camera assumption and suffer from distortions.
Deep Boosting of Diverse Experts
In this paper, a deep boosting algorithm is developed to learn more discriminative ensemble classifier by seamlessly combining a set of base deep CNNs (base experts) with diverse capabilities, e. g., these base deep CNNs are sequentially trained to recognize a set of object classes in an easy-to-hard way according to their learning complexities.
Multi-modal Face Pose Estimation with Multi-task Manifold Deep Learning
In the proposed deep learning based framework, Manifold Regularized Convolutional Layers (MRCL) improve traditional convolutional layers by learning the relationship among outputs of neurons.
Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs
Experimental results show that our method is capable of generating both visually comfortable and identity-preserving face sketches/photos over a wide range of challenging data.
Ranked #1 on
Face Sketch Synthesis
on CUFS
(FID metric)
Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answering
For fine-grained image and question representations, a `co-attention' mechanism is developed by using a deep neural network architecture to jointly learn the attentions for both the image and the question, which can allow us to reduce the irrelevant features effectively and obtain more discriminative features for image and question representations.
Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering
For multi-modal feature fusion, here we develop a Multi-modal Factorized Bilinear (MFB) pooling approach to efficiently and effectively combine multi-modal features, which results in superior performance for VQA compared with other bilinear pooling approaches.
Embedding Visual Hierarchy with Deep Networks for Large-Scale Visual Recognition
Our LMM model can provide an end-to-end approach for jointly learning: (a) the deep networks to extract more discriminative deep features for image and object class representation; (b) the tree classifier for recognizing large numbers of object classes hierarchically; and (c) the visual hierarchy adaptation for achieving more accurate indexing of large numbers of object classes hierarchically.
Deep Mixture of Diverse Experts for Large-Scale Visual Recognition
In this paper, a deep mixture of diverse experts algorithm is developed for seamlessly combining a set of base deep CNNs (convolutional neural networks) with diverse outputs (task spaces), e. g., such base deep CNNs are trained to recognize different subsets of tens of thousands of atomic object classes.
Constrained Low-Rank Learning Using Least Squares-Based Regularization
To achieve this goal, we cast the problem into a constrained rank minimization framework by adopting the least squares regularization.
Whole-brain substitute CT generation using Markov random field mixture models
The amount of data needed to train a model for s-CT generation is of the order of 100 million voxels.
Bayesian and empirical Bayesian forests
We derive ensembles of decision trees through a nonparametric Bayesian model, allowing us to view random forests as samples from a posterior distribution.
Applications
