Search Results for author:
Found 105 papers, 49 papers with code
Hierarchical Sparse Representation Clustering for High-Dimensional Data Streams
Second, these algorithms are highly sensitive to the noise contained in high-dimensional data streams.
DEAL: Disentangle and Localize Concept-level Explanations for VLMs
Large pre-trained Vision-Language Models (VLMs) have become ubiquitous foundational components of other models and downstream tasks.
Adaptive Cascading Network for Continual Test-Time Adaptation
In light of these challenges, we propose a cascading paradigm that simultaneously updates the feature extractor and classifier at test time, mitigating the mismatch between them and enabling long-term model adaptation.
Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients
To achieve OOF-resiliency in a scalable manner, we propose Topology-aware Federated Learning (TFL) that leverages client topology - a graph representing client relationships - to effectively train robust models against OOF data.
A Survey on Deep Clustering: From the Prior Perspective
Facilitated by the powerful feature extraction ability of neural networks, deep clustering has achieved great success in analyzing high-dimensional and complex real-world data.
Multimodal Fusion on Low-quality Data: A Comprehensive Survey
Multimodal fusion focuses on integrating information from multiple modalities with the goal of more accurate prediction, which has achieved remarkable progress in a wide range of scenarios, including autonomous driving and medical diagnosis.
Learning with Noisy Correspondence
This paper studies a new learning paradigm for noisy labels, i. e., noisy correspondence (NC).
Cross-modal retrieval with noisy correspondence
Text Retrieval
+1
PointCloud-Text Matching: Benchmark Datasets and a Baseline
We observe that the data is challenging and with noisy correspondence due to the sparsity, noise, or disorder of point clouds and the ambiguity, vagueness, or incompleteness of texts, which make existing cross-modal matching methods ineffective for PTM.
Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining
Thanks to the consistency refining and mining strategy of CREAM, the overfitting on the false positives could be prevented and the consistency rooted in the false negatives could be exploited, thus leading to a robust CMR method.
Ranked #1 on
Graph Matching
on SPair-71k
Cross-modal retrieval with noisy correspondence
Graph Matching
+1
An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train Model
A natural expectation for PEFTs is that the performance of various PEFTs is positively related to the data size and fine-tunable parameter size.
Multi-granularity Correspondence Learning from Long-term Noisy Videos
Existing video-language studies mainly focus on learning short video clips, leaving long-term temporal dependencies rarely explored due to over-high computational cost of modeling long videos.
Ranked #1 on
Zero-Shot Video Retrieval
on YouCook2
Action Segmentation
Long Video Retrieval (Background Removed)
+2
Consistency Enhancement-Based Deep Multiview Clustering via Contrastive Learning
Furthermore, the representation process for clustering is enhanced through spectral clustering, and the consistency across multiple views is improved.
Test-Time Degradation Adaptation for Open-Set Image Restoration
In contrast to close-set scenarios that restore images from a predefined set of degradations, open-set image restoration aims to handle the unknown degradations that were unforeseen during the pretraining phase, which is less-touched as far as we know.
Cross-View Graph Consistency Learning for Invariant Graph Representations
In this paper, we propose a cross-view graph consistency learning (CGCL) method that learns invariant graph representations for link prediction.
Cross-modal Active Complementary Learning with Self-refining Correspondence
Recently, image-text matching has attracted more and more attention from academia and industry, which is fundamental to understanding the latent correspondence across visual and textual modalities.
Cross-modal retrieval with noisy correspondence
Image-text matching
+1
Image Clustering with External Guidance
The core of clustering is incorporating prior knowledge to construct supervision signals.
Learning from Semantic Alignment between Unpaired Multiviews for Egocentric Video Recognition
To facilitate the data efficiency of multiview learning, we further perform video-text alignment for first-person and third-person videos, to fully leverage the semantic knowledge to improve video representations.
Decoupled Contrastive Multi-View Clustering with High-Order Random Walks
In recent, some robust contrastive multi-view clustering (MvC) methods have been proposed, which construct data pairs from neighborhoods to alleviate the false negative issue, i. e., some intra-cluster samples are wrongly treated as negative pairs.
Noisy-Correspondence Learning for Text-to-Image Person Re-identification
Text-to-image person re-identification (TIReID) is a compelling topic in the cross-modal community, which aims to retrieve the target person based on a textual query.
Ranked #1 on
Text-based Person Retrieval with Noisy Correspondence
on ICFG-PEDES
(using extra training data)
Topology-aware Robust Optimization for Out-of-distribution Generalization
Out-of-distribution (OOD) generalization is a challenging machine learning problem yet highly desirable in many high-stake applications.
Deep learning-based estimation of whole-body kinematics from multi-view images
It is necessary to analyze the whole-body kinematics (including joint locations and joint angles) to assess risks of fatal and musculoskeletal injuries in occupational tasks.
Non-Hierarchical Transformers for Pedestrian Segmentation
We propose a methodology to address the challenge of instance segmentation in autonomous systems, specifically targeting accessibility and inclusivity.
High-Dimensional MR Reconstruction Integrating Subspace and Adaptive Generative Models
We present a novel method that integrates subspace modeling with an adaptive generative image prior for high-dimensional MR image reconstruction.
Provable Dynamic Fusion for Low-Quality Multimodal Data
The inherent challenge of multimodal fusion is to precisely capture the cross-modal correlation and flexibly conduct cross-modal interaction.
dugMatting: Decomposed-Uncertainty-Guided Matting
Cutting out an object and estimating its opacity mask, known as image matting, is a key task in image and video editing.
Semantic Invariant Multi-view Clustering with Fully Incomplete Information
To address this problem, we present a novel framework called SeMantic Invariance LEarning (SMILE) for multi-view clustering with incomplete information that does not require any paired samples.
Deep Multiview Clustering by Contrasting Cluster Assignments
Then, a cluster-level CVCL strategy is presented to explore consistent semantic label information among the multiple views in the fine-tuning stage.
Dual Contrastive Prediction for Incomplete Multi-view Representation Learning
In this article, we propose a unified framework to solve the following two challenging problems in incomplete multi-view representation learning: i) how to learn a consistent representation unifying different views, and ii) how to recover the missing views.
Are Data-driven Explanations Robust against Out-of-distribution Data?
How to develop robust explanations against out-of-distribution data?
GLOW: Global Layout Aware Attacks on Object Detection
More desired attacks, to this end, should be able to fool defenses with such consistency checks.
Cross-Modal Retrieval with Partially Mismatched Pairs
On the one hand, our method only utilizes the negative information which is much less likely false compared with the positive information, thus avoiding the overfitting issue to PMPs.
Contrastive Learning
Cross-modal retrieval with noisy correspondence
+2
Incomplete Multi-view Clustering via Prototype-based Imputation
Thanks to our dual-stream model, both cluster- and view-specific information could be captured, and thus the instance commonality and view versatility could be preserved to facilitate IMvC.
Rethinking Image Super Resolution From Long-Tailed Distribution Learning Perspective
Existing studies have empirically observed that the resolution of the low-frequency region is easier to enhance than that of the high-frequency one.
RONO: Robust Discriminative Learning With Noisy Labels for 2D-3D Cross-Modal Retrieval
Recently, with the advent of Metaverse and AI Generated Content, cross-modal retrieval becomes popular with a burst of 2D and 3D data.
Comprehensive and Delicate: An Efficient Transformer for Image Restoration
Vision Transformers have shown promising performance in image restoration, which usually conduct window- or channel-based attention to avoid intensive computations.
Graph Matching with Bi-level Noisy Correspondence
In this paper, we study a novel and widely existing problem in graph matching (GM), namely, Bi-level Noisy Correspondence (BNC), which refers to node-level noisy correspondence (NNC) and edge-level noisy correspondence (ENC).
Ranked #1 on
Graph Matching
on Willow Object Class
Relationship Quantification of Image Degradations
To tackle the first challenge, we proposed a Degradation Relationship Index (DRI) which is defined as the mean drop rate difference in the validation loss between two models which are respectively trained using the anchor degradation and the mixture of the anchor and the auxiliary degradations.
Twin Contrastive Learning for Online Clustering
Specifically, we find that when the data is projected into a feature space with a dimensionality of the target cluster number, the rows and columns of its feature matrix correspond to the instance and cluster representation, respectively.
Ranked #1 on
Short Text Clustering
on Biomedical
Deep Evidential Learning with Noisy Correspondence for Cross-Modal Retrieval
However, it will unavoidably introduce noise (i. e., mismatched pairs) into training data, dubbed noisy correspondence.
Ranked #2 on
Text-based Person Retrieval with Noisy Correspondence
on RSTPReid
(using extra training data)
Cross-modal retrieval with noisy correspondence
Retrieval
+1
Deep Fair Clustering via Maximizing and Minimizing Mutual Information: Theory, Algorithm and Metric
Fair clustering aims to divide data into distinct clusters while preventing sensitive attributes (\textit{e. g.}, gender, race, RNA sequencing technique) from dominating the clustering.
Ranked #1 on
Image Clustering
on HAR
Robust Domain Adaptation for Machine Reading Comprehension
Such a process will inevitably introduce mismatched pairs (i. e., noisy correspondence) due to i) the unavailable QA pairs in target documents, and ii) the domain shift during applying the QA construction model to the target domain.
FALCON: Scalable Reasoning over Inconsistent ALC Ontologies
Ontologies are one of the richest sources of knowledge.
TAR: Neural Logical Reasoning across TBox and ABox
Neural logical reasoning (NLR) is a fundamental task to explore such knowledge bases, which aims at answering multi-hop queries with logical operations based on distributed representations of queries and answers.
OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization
Numerous network compression methods such as pruning and quantization are proposed to reduce the model size significantly, of which the key is to find suitable compression allocation (e. g., pruning sparsity and quantization codebook) of each layer.
Are Multimodal Transformers Robust to Missing Modality?
Based on these findings, we propose a principle method to improve the robustness of Transformer models by automatically searching for an optimal fusion strategy regarding input data.
Multi-Scale Adaptive Network for Single Image Denoising
AFuB devotes to adaptively sampling and transferring the features from one scale to another scale, which fuses the multi-scale features with varying characteristics from coarse to fine.
Description Logic EL++ Embeddings with Intersectional Closure
Since the intersection of boxes remains as a box, the intersectional closure is satisfied.
Learning With Twin Noisy Labels for Visible-Infrared Person Re-Identification
To solve the TNL problem, we propose a novel method for robust VI-ReID, termed DuAlly Robust Training (DART).
All-in-One Image Restoration for Unknown Corruption
In this paper, we study a challenging problem in image restoration, namely, how to develop an all-in-one method that could recover images from a variety of unknown corruption types and levels.
Deep Learning for Spatiotemporal Modeling of Urbanization
Here we explore the capacity of deep spatial learning for the predictive modeling of urbanization.
Learning with Noisy Correspondence for Cross-modal Matching
Based on this observation, we reveal and study a latent and challenging direction in cross-modal matching, named noisy correspondence, which could be regarded as a new paradigm of noisy labels.
Cross-modal retrieval with noisy correspondence
Image-text matching
+3
Calibrating Probabilistic Embeddings for Cross-Modal Retrieval
The key idea is to estimate the density ratio between the distributions of the two modalities, and use it to calibrate the similarity measurement in the embedding space.
Understanding the factors driving the opioid epidemic using machine learning
Furthermore, we discovered that, as the epidemic has shifted from legal (i. e., prescription opioids) to illegal (e. g., heroin and fentanyl) drugs in recent years, the correlation of environment, crime and health related variables with the opioid risk has increased significantly while the correlation of economic and socio-demographic variables has decreased.
Out-of-Domain Generalization from a Single Source: An Uncertainty Quantification Approach
We are concerned with a worst-case scenario in model generalization, in the sense that a model aims to perform well on many unseen domains while there is only one single domain available for training.
Accelerated MRI Reconstruction with Separable and Enhanced Low-Rank Hankel Regularization
The combination of the sparse sampling and the low-rank structured matrix reconstruction has shown promising performance, enabling a significant reduction of the magnetic resonance imaging data acquisition time.
Unsupervised Neural Rendering for Image Hazing
To generate plausible haze, we study two less-touched but challenging problems in hazy image rendering, namely, i) how to estimate the transmission map from a single image without auxiliary information, and ii) how to adaptively learn the airlight from exemplars, i. e., unpaired real hazy images.
Dizygotic Conditional Variational AutoEncoder for Multi-Modal and Partial Modality Absent Few-Shot Learning
DCVAE conducts feature synthesis via pairing two Conditional Variational AutoEncoders (CVAEs) with the same seed but different modality conditions in a dizygotic symbiosis manner.
Learning Cross-Modal Retrieval With Noisy Labels
Recently, cross-modal retrieval is emerging with the help of deep multimodal learning.
A Good Image Generator Is What You Need for High-Resolution Video Synthesis
We introduce a motion generator that discovers the desired trajectory, in which content and motion are disentangled.
Ranked #34 on
Video Generation
on UCF-101
COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction
In this paper, we study two challenging problems in incomplete multi-view clustering analysis, namely, i) how to learn an informative and consistent representation among different views without the help of labels and ii) how to recover the missing views from data.
Ranked #1 on
Incomplete multi-view clustering
on n-MNIST
Uncertainty-guided Model Generalization to Unseen Domains
To the best of our knowledge, this is the first work to (1) access the generalization uncertainty from a single source and (2) leverage it to guide both input and label augmentation for robust generalization.
SMIL: Multimodal Learning with Severely Missing Modality
A common assumption in multimodal learning is the completeness of training data, i. e., full modalities are available in all training examples.
Partially View-aligned Representation Learning with Noise-robust Contrastive Loss
To solve such a less-touched problem without the help of labels, we propose simultaneously learning representation and aligning data using a noise-robust contrastive loss.
Uncertain Out-of-Domain Generalization
To the best of our knowledge, this is the first work to (1) access the generalization uncertainty from a single source and (2) leverage it to guide both input and label augmentation for robust generalization.
Learning View-Disentangled Human Pose Representation by Contrastive Cross-View Mutual Information Maximization
To evaluate the power of the learned representations, in addition to the conventional fully-supervised action recognition settings, we introduce a novel task called single-shot cross-view action recognition.
Partially View-aligned Clustering
To solve this practical and challenging problem, we propose a novel multi-view clustering method termed partially view-aligned clustering (PVC).
CLEARER: Multi-Scale Neural Architecture Search for Image Restoration
Different from the existing labor-intensive handcrafted architecture design paradigms, we present a novel method, termed as multi-sCaLe nEural ARchitecture sEarch for image Restoration (CLEARER), which is a specifically designed neural architecture search (NAS) for image restoration.
Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
In this paper, we propose a novel and effective regularization term for adversarial data augmentation.
Contrastive Clustering
In this paper, we propose a one-stage online clustering method called Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning.
Ranked #5 on
Image Clustering
on Tiny-ImageNet
Structured Graph Learning for Clustering and Semi-supervised Classification
Furthermore, most existing graph-based methods conduct clustering and semi-supervised classification on the graph learned from the original data matrix, which doesn't have explicit cluster structure, thus they might not achieve the optimal performance.
You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural Network
In this paper, we study two challenging and less-touched problems in single image dehazing, namely, how to make deep learning achieve image dehazing without training on the ground-truth clean image (unsupervised) and a image collection (untrained).
Heterogeneous Representation Learning: A Review
The real-world data usually exhibits heterogeneous properties such as modalities, views, or resources, which brings some unique challenges wherein the key is Heterogeneous Representation Learning (HRL) termed in this paper.
Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge
Our key idea is to generalize the distilled cross-modal knowledge learned from a Source dataset, which contains paired examples from both modalities, to the Target dataset by modeling knowledge as priors on parameters of the Student.
Learning to Learn Single Domain Generalization
We are concerned with a worst-case scenario in model generalization, in the sense that a model aims to perform well on many unseen domains while there is only one single domain available for training.
Learning by Sampling and Compressing: Efficient Graph Representation Learning with Extremely Limited Annotations
Graph convolution network (GCN) attracts intensive research interest with broad applications.
Scalable Global Alignment Graph Kernel Using Random Features: From Node Embedding to Graph Embedding
In particular, RGE is shown to achieve \emph{(quasi-)linear scalability} with respect to the number and the size of the graphs.
Improving Distant Supervised Relation Extraction by Dynamic Neural Network
To account for this style shift, the model should adjust its parameters in accordance with entity types.
Rethinking Kernel Methods for Node Representation Learning on Graphs
Graph kernels are kernel methods measuring graph similarity and serve as a standard tool for graph classification.
Ranked #8 on
Link Prediction
on Cora
Construct Dynamic Graphs for Hand Gesture Recognition via Spatial-Temporal Attention
We propose a Dynamic Graph-Based Spatial-Temporal Attention (DG-STA) method for hand gesture recognition.
Ranked #4 on
Hand Gesture Recognition
on SHREC 2017
Automatic Health Problem Detection from Gait Videos Using Deep Neural Networks
This study presents a starting point toward a powerful tool for automatic classification of gait disorders and can be used as a basis for future applications of Deep Learning in clinical gait analysis.
Semantic Graph Convolutional Networks for 3D Human Pose Regression
In this paper, we study the problem of learning Graph Convolutional Networks (GCNs) for regression.
Ranked #26 on
Monocular 3D Human Pose Estimation
on Human3.6M
Semantic-Guided Multi-Attention Localization for Zero-Shot Learning
Zero-shot learning extends the conventional object classification to the unseen class recognition by introducing semantic representations of classes.
XAI Beyond Classification: Interpretable Neural Clustering
The second is implementing discrete $k$-means with a differentiable neural network that embraces the advantages of parallel computing, online clustering, and clustering-favorable representation learning.
CU-Net: Coupled U-Nets
We design a new connectivity pattern for the U-Net architecture.
Ranked #30 on
Pose Estimation
on MPII Human Pose
Quantized Densely Connected U-Nets for Efficient Landmark Localization
Finally, to reduce the memory consumption and high precision operations both in training and testing, we further quantize weights, inputs, and gradients of our localization network to low bit-width numbers.
Ranked #19 on
Pose Estimation
on MPII Human Pose
Learning to Forecast and Refine Residual Motion for Image-to-Video Generation
We consider the problem of image-to-video translation, where an input image is translated into an output video containing motions of a single object.
CR-GAN: Learning Complete Representations for Multi-view Generation
Generating multi-view images from a single-view input is an essential yet challenging problem.
Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation
Random data augmentation is a critical technique to avoid overfitting in training deep neural network models.
Ranked #3 on
Pose Estimation
on Leeds Sports Poses
Toward Marker-free 3D Pose Estimation in Lifting: A Deep Multi-view Solution
The results are also compared with state-of-the-art methods on HumanEva-I dataset, which demonstrates the superior performance of our approach.
RED-Net: A Recurrent Encoder-Decoder Network for Video-based Face Alignment
We propose a novel method for real-time face alignment in videos based on a recurrent encoder-decoder network model.
A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts
Most existing zero-shot learning methods consider the problem as a visual semantic embedding one.
Deep Sparse Subspace Clustering
In this paper, we present a deep extension of Sparse Subspace Clustering, termed Deep Sparse Subspace Clustering (DSSC).
Cartoonish sketch-based face editing in videos using identity deformation transfer
We first recover the facial identity and expressions from the video by fitting a face morphable model for each frame.
Reconstruction-Based Disentanglement for Pose-invariant Face Recognition
Finally, we propose a new feature reconstruction metric learning to explicitly disentangle identity and pose, by demanding alignment between the feature reconstructions through various combinations of identity and pose features, which is obtained from two images of the same subject.
Track Facial Points in Unconstrained Videos
Our approach takes advantage of part-based representation and cascade regression for robust and efficient alignment on each frame.
A Recurrent Encoder-Decoder Network for Sequential Face Alignment
We propose a novel recurrent encoder-decoder network model for real-time video-based face alignment.
PIEFA: Personalized Incremental and Ensemble Face Alignment
Face alignment, especially on real-time or large-scale sequential images, is a challenging task with broad applications.
Connections Between Nuclear Norm and Frobenius Norm Based Representations
A lot of works have shown that frobenius-norm based representation (FNR) is competitive to sparse representation and nuclear-norm based representation (NNR) in numerous tasks such as subspace clustering.
Automatic Subspace Learning via Principal Coefficients Embedding
In this paper, we address two challenging problems in unsupervised subspace learning: 1) how to automatically identify the feature dimension of the learned subspace (i. e., automatic subspace learning), and 2) how to learn the underlying subspace in the presence of Gaussian noise (i. e., robust subspace learning).
Fast Low-rank Representation based Spatial Pyramid Matching for Image Classification
Although the methods achieve a higher recognition rate than the traditional SPM, they consume more time to encode the local descriptors extracted from the image.
A Unified Framework for Representation-based Subspace Clustering of Out-of-sample and Large-scale Data
In this paper, we propose a unified framework which makes representation-based subspace clustering algorithms feasible to cluster both out-of-sample and large-scale data.
Scalable Sparse Subspace Clustering
To address the problems, this paper proposes out-of-sample extension of SSC, named as Scalable Sparse Subspace Clustering (SSSC), which makes SSC feasible to cluster large scale data sets.
Locally linear representation for image clustering
There are two popular schemes to construct a similarity graph, i. e., pairwise distance based scheme and linear representation based scheme.
Learning Locality-Constrained Collaborative Representation for Face Recognition
The model of low-dimensional manifold and sparse representation are two well-known concise models that suggest each data can be described by a few characteristics.
Constructing the L2-Graph for Robust Subspace Learning and Subspace Clustering
Under the framework of graph-based learning, the key to robust subspace clustering and subspace learning is to obtain a good similarity graph that eliminates the effects of errors and retains only connections between the data points from the same subspace (i. e., intra-subspace data points).
