Search Results for author:
Found 143 papers, 58 papers with code
Diffusion Models Learn Low-Dimensional Distributions via Subspace Clustering
Remarkably, these models can achieve this even with a small number of training samples despite a large image dimension, circumventing the curse of dimensionality.
Lessons from Learning to Spin "Pens"
This serves two purposes: 1) pre-training a sensorimotor policy in simulation; 2) conducting open-loop trajectory replay in the real world.
Semantic-Aware Power Allocation for Generative Semantic Communications with Foundation Models
Recent advancements in diffusion models have made a significant breakthrough in generative modeling.
Multi-Satellite MIMO Systems for Direct User-Satellite Communications: A Survey
These aspects encompass satellite orbits, the structure of satellite systems, SatCom links, including the inter-satellite links (ISL) which facilitate satellite cooperation, satellite frequency bands, satellite antenna design, and satellite channel models, which should be known or estimated for effective data transmission to and from multiple satellites.
CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Making
By revisiting the roles of DMs in the decision-making domain, we identify a set of essential sub-modules that constitute the core of CleanDiffuser, allowing for the implementation of various DM algorithms with simple and flexible building blocks.
How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?
Recent work shows that countermeasures (CMs) trained on partially spoofed audio can effectively detect such spoofing.
A Global Geometric Analysis of Maximal Coding Rate Reduction
The maximal coding rate reduction (MCR$^2$) objective for learning structured and compact deep representations is drawing increasing attention, especially after its recent usage in the derivation of fully explainable and highly effective deep network architectures.
Scaling White-Box Transformers for Vision
CRATE, a white-box transformer architecture designed to learn compressed and sparse representations, offers an intriguing alternative to standard vision transformers (ViTs) due to its inherent mathematical interpretability.
Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
Finally, our framework uses these task rewards to fine-tune the entire VLM with RL.
Masked Completion via Structured Diffusion with White-Box Transformers
We do this by exploiting a fundamental connection between diffusion, compression, and (masked) completion, deriving a deep transformer-like masked autoencoder architecture, called CRATE-MAE, in which the role of each layer is mathematically fully interpretable: they transform the data distribution to and from a structured representation.
Ctrl123: Consistent Novel View Synthesis via Closed-Loop Transcription
This consequently limits the performance of downstream tasks, such as image-to-multiview generation and 3D reconstruction.
Differentially Private Representation Learning via Image Captioning
In this work, we show that effective DP representation learning can be done via image captioning and scaling up to internet-scale multimodal datasets.
Bit Rate Matching Algorithm Optimization in JPEG-AI Verification Model
To generate reconstructed images with the desired bits per pixel and assess the BD-rate performance of both the JPEG-AI verification model and VVC intra, bit rate matching is employed.
Gen4Gen: Generative Data Pipeline for Generative Multi-Concept Composition
First, current personalization techniques fail to reliably extend to multiple concepts -- we hypothesize this to be due to the mismatch between complex scenes and simple text descriptions in the pre-training dataset (e. g., LAION).
Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback
It is crucial to consider diverse human feedback types and various learning methods in different environments.
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
To understand the roots of these errors, we explore the gap between the visual embedding space of CLIP and vision-only self-supervised learning.
On Chernoff Lower-Bound of Outage Threshold for Non-Central $χ^2$-Distributed Beamforming Gain in URLLC Systems
The cumulative distribution function (CDF) of a non-central $\chi^2$-distributed random variable (RV) is often used when measuring the outage probability of communication systems.
Learning a Diffusion Model Policy from Rewards via Q-Score Matching
However, previous works fail to exploit the score-based structure of diffusion models, and instead utilize a simple behavior cloning term to train the actor, limiting their ability in the actor-critic setting.
EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms with Real-captured Hybrid Dataset
Event cameras are emerging imaging technology that offers advantages over conventional frame-based imaging sensors in dynamic range and sensing speed.
White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
This leads to a family of white-box transformer-like deep network architectures, named CRATE, which are mathematically fully interpretable.
RLIF: Interactive Imitation Learning as Reinforcement Learning
We also provide a unified framework to analyze our RL method and DAgger; for which we present the asymptotic analysis of the suboptimal gap for both methods as well as the non-asymptotic sample complexity bound of our method.
Rethinking Decision Transformer via Hierarchical Reinforcement Learning
Decision Transformer (DT) is an innovative algorithm leveraging recent advances of the transformer architecture in reinforcement learning (RL).
Near and Far Field Model Mismatch: Implications on 6G Communications, Localization, and Sensing
The upcoming 6G technology is expected to operate in near-field (NF) radiating conditions thanks to high-frequency and electrically large antenna arrays.
Investigating the Catastrophic Forgetting in Multimodal Large Language Models
However, catastrophic forgetting, a notorious phenomenon where the fine-tuned model fails to retain similar performance compared to the pre-trained model, still remains an inherent problem in multimodal LLMs (MLLM).
General In-Hand Object Rotation with Vision and Touch
We introduce RotateIt, a system that enables fingertip-based object rotation along multiple axes by leveraging multimodal sensory inputs.
Emergence of Segmentation with Minimalistic White-Box Transformers
Transformer-like models for vision tasks have recently proven effective for a wide range of downstream applications such as segmentation and detection.
Canonical Factors for Hybrid Neural Fields
Factored feature volumes offer a simple way to build more compact, efficient, and intepretable neural fields, but also introduce biases that are not necessarily beneficial for real-world data.
On Chernoff Lower-Bound of Outage Threshold for Non-Central $χ^2$-Distributed MIMO Beamforming Gain
The cumulative distribution function (CDF) of a non-central $\chi^2$-distributed random variable (RV) is often used when measuring the outage probability of communication systems.
Scaff-PD: Communication Efficient Fair and Robust Federated Learning
We present Scaff-PD, a fast and communication-efficient algorithm for distributionally robust federated learning.
Pixel-wise Graph Attention Networks for Person Re-identification
Based on these two modules, we consulted the ResNet and design a pixel-wise graph attention network (PGANet).
Prioritized Trajectory Replay: A Replay Memory for Data-driven Reinforcement Learning
In summary, our research emphasizes the significance of trajectory-based data sampling techniques in enhancing the efficiency and performance of offline RL algorithms.
ViP: A Differentially Private Foundation Model for Computer Vision
In this work, we propose as a mitigation measure a recipe to train foundation vision models with differential privacy (DP) guarantee.
ENOTO: Improving Offline-to-Online Reinforcement Learning with Q-Ensembles
Offline reinforcement learning (RL) is a learning paradigm where an agent learns from a fixed dataset of experience.
HIPODE: Enhancing Offline Reinforcement Learning with High-Quality Synthetic Data from a Policy-Decoupled Approach
Offline reinforcement learning (ORL) has gained attention as a means of training reinforcement learning models using pre-collected static data.
Iteratively Refined Behavior Regularization for Offline Reinforcement Learning
One of the fundamental challenges for offline reinforcement learning (RL) is ensuring robustness to data distribution.
Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models
In this paper, we propose a novel image clustering pipeline that leverages the powerful feature representation of large pre-trained models such as CLIP and cluster images effectively and efficiently at scale.
White-Box Transformers via Sparse Rate Reduction
Particularly, we show that the standard transformer block can be derived from alternating optimization on complementary parts of this objective: the multi-head self-attention operator can be viewed as a gradient descent step to compress the token sets by minimizing their lossy coding rate, and the subsequent multi-layer perceptron can be viewed as attempting to sparsify the representation of the tokens.
Representation Learning via Manifold Flattening and Reconstruction
This work proposes an algorithm for explicitly constructing a pair of neural networks that linearize and reconstruct an embedded submanifold, from finite samples of this manifold.
EMP-SSL: Towards Self-Supervised Learning in One Training Epoch
Recently, self-supervised learning (SSL) has achieved tremendous success in learning image representation.
Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
Our approach, calibrated Q-learning (Cal-QL), accomplishes this by learning a conservative value function initialization that underestimates the value of the learned policy from offline data, while also being calibrated, in the sense that the learned Q-values are at a reasonable scale.
Closed-Loop Transcription via Convolutional Sparse Coding
Our method is arguably the first to demonstrate that a concatenation of multiple convolution sparse coding/decoding layers leads to an interpretable and effective autoencoder for modeling the distribution of large-scale natural image datasets.
Sherman-Morrison Regularization for ELAA Iterative Linear Precoding
The design of iterative linear precoding is recently challenged by extremely large aperture array (ELAA) systems, where conventional preconditioning techniques could hardly improve the channel condition.
Robot Subset Selection for Swarm Lifetime Maximization in Computation Offloading with Correlated Data Sources
When the number of subgraphs is maximized, the proposed subset selection approach is shown to be optimum in the AWGN channel.
Unsupervised Manifold Linearizing and Clustering
We consider the problem of simultaneously clustering and learning a linear representation of data lying close to a union of low-dimensional manifolds, a fundamental task in machine learning and computer vision.
Understanding the Complexity Gains of Single-Task RL with a Curriculum
Under mild regularity conditions on the curriculum, we show that sequentially solving each task in the multi-task RL problem is more computationally efficient than solving the original single-task problem, without any explicit exploration bonuses or other exploration strategies.
State-Aware Proximal Pessimistic Algorithms for Offline Reinforcement Learning
The key idea of SA-PP is leveraging discounted stationary state distribution ratios between the learning policy and the offline dataset to modulate the degree of behavior regularization in a state-wise manner, so that pessimism can be implemented in a more appropriate way.
Unsupervised Learning of Structured Representations via Closed-Loop Transcription
This paper proposes an unsupervised method for learning a unified representation that serves both discriminative and generative purposes.
Revisiting Sparse Convolutional Model for Visual Recognition
We show that such models have equally strong empirical performance on CIFAR-10, CIFAR-100, and ImageNet datasets when compared to conventional neural networks.
In-Hand Object Rotation via Rapid Motor Adaptation
Generalized in-hand manipulation has long been an unsolved challenge of robotics.
Minimalistic Unsupervised Learning with the Sparse Manifold Transform
Though there remains a small performance gap between our simple constructive model and SOTA methods, the evidence points to this as a promising direction for achieving a principled and white-box approach to unsupervised learning.
Ranked #1 on
Unsupervised MNIST
on MNIST
Self-Supervised Learning
Sparse Representation-based Classification
+3
Convergence Acceleration in Wireless Federated Learning: A Stackelberg Game Approach
At the leader-level, we derive an upper bound of convergence rate and subsequently reformulate the global loss minimization problem and propose a new age-of-update (AoU) based device selection algorithm.
Constellation-Oriented Perturbation for Scalable-Complexity MIMO Nonlinear Precoding
The basic concept of COP is to apply vector perturbation (VP) in the constellation domain instead of symbol domain; as often used in conventional techniques.
Power Allocation for FDMA-URLLC Downlink with Random Channel Assignment
With imperfect CSIT, the proposed approach can still provide remarkable user capacity at limited cost of transmit-power efficiency.
TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels
Leveraging this observation, we propose a Train-Convexify-Train (TCT) procedure to sidestep this issue: first, learn features using off-the-shelf methods (e. g., FedAvg); then, optimize a convexified problem obtained from the network's empirical neural tangent kernel approximation.
On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligence
Ten years into the revival of deep networks and artificial intelligence, we propose a theoretical framework that sheds light on understanding deep networks within a bigger picture of Intelligence in general.
Pursuit of a Discriminative Representation for Multiple Subspaces via Sequential Games
We consider the problem of learning discriminative representations for data in a high-dimensional space with distribution supported on or around multiple low-dimensional linear subspaces.
Robust Calibration with Multi-domain Temperature Scaling
Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains.
PAnDR: Fast Adaptation to New Environments from Offline Experiences via Decoupling Policy and Environment Representations
In online adaptation phase, with the environment context inferred from few experiences collected in new environments, the policy is optimized by gradient ascent with respect to the PDVF.
Efficient Maximal Coding Rate Reduction by Variational Forms
The principle of Maximal Coding Rate Reduction (MCR$^2$) has recently been proposed as a training objective for learning discriminative low-dimensional structures intrinsic to high-dimensional data to allow for more robust training than standard approaches, such as cross-entropy minimization.
Predicting Out-of-Distribution Error with the Projection Norm
Projection Norm first uses model predictions to pseudo-label test samples and then trains a new model on the pseudo-labels.
Incremental Learning of Structured Memory via Closed-Loop Transcription
Our method is simpler than existing approaches for incremental learning, and more efficient in terms of model size, storage, and computation: it requires only a single, fixed-capacity autoencoding network with a feature space that is used for both discriminative and generative purposes.
Network-ELAA Beamforming and Coverage Analysis for eMBB/URLLC in Spatially Non-Stationary Rician Channels
Finally, it is shown that the network-ELAA can offer significant coverage extension (50% or more in most of cases) when comparing with the single-AP scenario.
DeepFGS: Fine-Grained Scalable Coding for Learned Image Compression
In this paper, we propose the first learned fine-grained scalable image compression model (DeepFGS) to overcome the above two shortcomings.
EvUnroll: Neuromorphic Events Based Rolling Shutter Image Correction
This paper proposes to use neuromorphic events for correcting rolling shutter (RS) images as consecutive global shutter (GS) frames.
A Hierarchical Reinforcement Learning Based Optimization Framework for Large-scale Dynamic Pickup and Delivery Problems
To address this problem, existing methods partition the overall DPDP into fixed-size sub-problems by caching online generated orders and solve each sub-problem, or on this basis to utilize the predicted future orders to optimize each sub-problem further.
Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reduction
In particular, we propose to learn a closed-loop transcription between a multi-class multi-dimensional data distribution and a linear discriminative representation (LDR) in the feature space that consists of multiple independent multi-dimensional linear subspaces.
LungAttn: advanced lung sound classification using attention mechanism with dual TQWT and triple STFT spectrogram
We propose a deep learning architecture, LungAttn, which incorporates augmented attention convolution into ResNet block to improve the classification accuracy of lung sound.
Ranked #17 on
Audio Classification
on ICBHI Respiratory Sound Database
Massive-MIMO MF Beamforming with or without Grouped STBC for Ultra-Reliable Single-Shot Transmission Using Aged CSIT
In addition, a combinatorial approach of the MF beamforming and grouped space-time block code (G-STBC) is proposed to further mitigate the detrimental impact of the CSIT uncertainty.
PL-EESR: Perceptual Loss Based END-TO-END Robust Speaker Representation Extraction
Speech enhancement aims to improve the perceptual quality of the speech signal by suppression of the background noise.
An Empirical Study of Pre-trained Models on Out-of-distribution Generalization
Concretely, we show that larger models and larger datasets need to be simultaneously leveraged to improve OOD performance.
Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learning
In both OWSP and OWMP settings, we demonstrate that adding {\em intermediate rewards} to subgoals is more computationally efficient than only rewarding the agent once it completes the goal of reaching a terminal state.
On the Convergence of Stochastic Extragradient for Bilinear Games using Restarted Iteration Averaging
We study the stochastic bilinear minimax optimization problem, presenting an analysis of the same-sample Stochastic ExtraGradient (SEG) method with constant step size, and presenting variations of the method that yield favorable convergence.
EventZoom: Learning To Denoise and Super Resolve Neuromorphic Events
EventZoom is trained in a noise-to-noise fashion where the two ends of the network are unfiltered noisy events, enforcing noise-free event restoration.
Learning and Meshing from Deep Implicit Surface Networks Using an Efficient Implementation of Analytic Marching
More specifically, we identify from the linear regions, partitioned by an MLP based implicit function, the analytic cells and analytic faces that are associated with the function's zero-level isosurface.
ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction
This work attempts to provide a plausible theoretical framework that aims to interpret modern deep (convolutional) networks from the principles of data compression and discriminative representation.
End-to-End Learning for Uplink MU-SIMO Joint Transmitter and Non-Coherent Receiver Design in Fading Channels
In this paper, a novel end-to-end learning approach, namely JTRD-Net, is proposed for uplink multiuser single-input multiple-output (MU-SIMO) joint transmitter and non-coherent receiver design (JTRD) in fading channels.
Fully Convolutional Line Parsing
We conduct extensive experiments and show that our method achieves a significantly better trade-off between efficiency and accuracy, resulting in a real-time line detector at up to 73 FPS on a single GPU.
Ranked #1 on
Line Segment Detection
on York Urban Dataset
NeRD: Neural 3D Reflection Symmetry Detector
Recent advances have shown that symmetry, a structural prior that most objects exhibit, can support a variety of single-view 3D understanding tasks.
Learning to Reconstruct 3D Non-Cuboid Room Layout from a Single RGB Image
Single-image room layout reconstruction aims to reconstruct the enclosed 3D structure of a room from a single image.
Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
To investigate this gap, we decompose the test risk into its bias and variance components and study their behavior as a function of adversarial training perturbation radii ($\varepsilon$).
Measuring GAN Training in Real Time
Generative Adversarial Networks (GAN) are popular generative models of images.
Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalities
The optimization problems associated with training generative adversarial neural networks can be largely reduced to certain {\em non-monotone} variational inequality problems (VIPs), whereas existing convergence results are mostly based on monotone or strongly monotone assumptions.
Incremental Learning via Rate Reduction
Current deep learning architectures suffer from catastrophic forgetting, a failure to retain knowledge of previously learned classes when incrementally trained on new classes.
Deep Networks from the Principle of Rate Reduction
The layered architectures, linear and nonlinear operators, and even parameters of the network are all explicitly constructed layer-by-layer in a forward propagation fashion by emulating the gradient scheme.
Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradients
Even replacing only the first layer of a ResNet by such a ODE block can exhibit further improvement in robustness, e. g., under PGD-20 ($\ell_\infty=0. 031$) attack on CIFAR-10 dataset, it achieves 91. 57\% and natural accuracy and 62. 35\% robust accuracy, while a counterpart architecture of ResNet trained with TRADES achieves natural and robust accuracy 76. 29\% and 45. 24\%, respectively.
HoliCity: A City-Scale Data Platform for Learning Holistic 3D Structures
We present HoliCity, a city-scale 3D dataset with rich structural information.
Learning Long-term Visual Dynamics with Region Proposal Interaction Networks
Learning long-term dynamics models is the key to understanding physical common sense.
Ranked #1 on
Visual Reasoning
on PHYRE-1B-Within
LungRN+NL: An Improved Adventitious Lung Sound Classification Using Non-Local Block ResNet Neural Network with Mixup Data Augmentation
An adventitious lung sound classification model, LungRN+NL, is proposed in this work, which has demonstrated a drastic improvement compared to our previous work and the state-of-the-art models.
Ranked #18 on
Audio Classification
on ICBHI Respiratory Sound Database
Learning to Parse Wireframes in Images of Man-Made Environments
To this end, we have built a very large new dataset of over 5, 000 images with wireframes thoroughly labelled by humans.
Deep Isometric Learning for Visual Recognition
Initialization, normalization, and skip connections are believed to be three indispensable techniques for training very deep convolutional neural networks and obtaining state-of-the-art performance.
Dynamic Knapsack Optimization Towards Efficient Multi-Channel Sequential Advertising
In E-commerce, advertising is essential for merchants to reach their target users.
Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization
Meanwhile, VRADA matches the lower bound of the general convex setting up to a $\log\log n$ factor and matches the lower bounds in both regimes $n\le \Theta(\kappa)$ and $n\gg \kappa$ of the strongly convex setting, where $\kappa$ denotes the condition number.
Learning to Detect 3D Reflection Symmetry for Single-View Reconstruction
In this work, we focus on object-level 3D reconstruction and present a geometry-based end-to-end deep learning framework that first detects the mirror plane of reflection symmetry that commonly exists in man-made objects and then predicts depth maps by finding the intra-image pixel-wise correspondence of the symmetry.
Robust Recovery via Implicit Bias of Discrepant Learning Rates for Double Over-parameterization
This paper shows that with a double over-parameterization for both the low-rank matrix and sparse corruption, gradient descent with discrepant learning rates provably recovers the underlying matrix even without prior knowledge on neither rank of the matrix nor sparsity of the corruption.
Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction
To learn intrinsic low-dimensional structures from high-dimensional data that most discriminate between classes, we propose the principle of Maximal Coding Rate Reduction ($\text{MCR}^2$), an information-theoretic measure that maximizes the coding rate difference between the whole dataset and the sum of each individual class.
Ranked #21 on
Image Clustering
on STL-10
Learning to Accelerate Heuristic Searching for Large-Scale Maximum Weighted b-Matching Problems in Online Advertising
Bipartite b-matching is fundamental in algorithm design, and has been widely applied into economic markets, labor markets, etc.
Understanding l4-based Dictionary Learning: Interpretation, Stability, and Robustness
Recently, the $\ell^4$-norm maximization has been proposed to solve the sparse dictionary learning (SDL) problem.
On Deep Learning Solutions for Joint Transmitter and Noncoherent Receiver Design in MU-MIMO Systems
Otherwise, it is called non-systematic waveform, where no artificial design is involved.
A Modular Neural Network Based Deep Learning Approach for MIMO Signal Detection
Motivated by this finding, we propose a novel modular neural network based approach, termed MNNet, where the whole network is formed by a set of pre-defined ANN modules.
Rethinking Bias-Variance Trade-off for Generalization of Neural Networks
We provide a simple explanation for this by measuring the bias and variance of neural networks: while the bias is monotonically decreasing as in the classical theory, the variance is unimodal or bell-shaped: it increases then decreases with the width of the network.
KoGuN: Accelerating Deep Reinforcement Learning via Integrating Human Suboptimal Knowledge
Our framework consists of a fuzzy rule controller to represent human knowledge and a refine module to fine-tune suboptimal prior knowledge.
LungBRN: A Smart Digital Stethoscope for Detecting Respiratory Disease Using bi-ResNet Deep Learning Algorithm
Improving access to health care services for the medically under-served population is vital to ensure that critical illness can be addressed immediately.
Ranked #19 on
Audio Classification
on ICBHI Respiratory Sound Database
NeurVPS: Neural Vanishing Point Scanning via Conic Convolution
We present a simple yet effective end-to-end trainable deep network with geometry-inspired convolutional operators for detecting vanishing points in images.
Spectral-based Graph Convolutional Network for Directed Graphs
Our approach can work directly on directed graph data in semi-supervised nodes classification tasks.
Complete Dictionary Learning via $\ell^4$-Norm Maximization over the Orthogonal Group
Most existing methods solve the dictionary (and sparse representations) based on heuristic algorithms, usually without theoretical guarantees for either optimality or complexity.
Unified Acceleration of High-Order Algorithms under Hölder Continuity and Uniform Convexity
In this general convex setting, we propose a concise unified acceleration framework (UAF), which reconciles the two different high-order acceleration approaches, one by Nesterov and Baes [29, 3, 33] and one by Monteiro and Svaiter [25].
Learning to Reconstruct 3D Manhattan Wireframes from a Single Image
In this paper, we propose a method to obtain a compact and accurate 3D wireframe representation from a single image by effectively exploiting global structural regularities.
End-to-End Wireframe Parsing
We conduct extensive experiments and show that our method significantly outperforms the previous state-of-the-art wireframe and line extraction algorithms.
Ranked #5 on
Line Segment Detection
on wireframe dataset
PizzaPal: Conversational Pizza Ordering using a High-Density Conversational AI Platform
This paper describes PizzaPal, a voice-only agent for ordering pizza, as well as the Conversational AI architecture built at b4. ai.
Fine-grained Video Categorization with Redundancy Reduction Attention
Specifically, it firstly summarizes the video by weight-summing all feature vectors in the feature maps of selected frames with a spatio-temporal soft attention, and then predicts which channels to suppress or to enhance according to this summary with a learned non-linear transform.
Ranked #12 on
Action Recognition
on ActivityNet
Structured Attentions for Visual Question Answering
In this paper, we demonstrate the importance of encoding such relations by showing the limited effective receptive field of ResNet on two datasets, and propose to model the visual attention as a multivariate distribution over a grid-structured Conditional Random Field on image regions.
Graph Construction with Label Information for Semi-Supervised Learning
In the literature, most existing graph-based semi-supervised learning (SSL) methods only use the label information of observed samples in the label propagation stage, while ignoring such valuable information when learning the graph.
Single-Image Crowd Counting via Multi-Column Convolutional Neural Network
To this end, we have proposed a simple but effective Multi-column Convolutional Neural Network (MCNN) architecture to map the image to its crowd density map.
Ranked #5 on
Crowd Counting
on Venice
Low-Rank Tensor Approximation With Laplacian Scale Mixture Modeling for Multiframe Image Denoising
Patch-based low-rank models have shown effective in exploiting spatial redundancy of natural images especially for the application of image denoising.
Automatic Layer Separation using Light Field Imaging
We propose a novel approach that jointly removes reflection or translucent layer from a scene and estimates scene depth.
Generalized Tensor Total Variation Minimization for Visual Data Recovery
In this paper, we propose a definition of Generalized Tensor Total Variation norm (GTV) that considers both the inhomogeneity and the multi-directionality of responses to derivative-like filters.
Robust Face Recognition by Constrained Part-based Alignment
By assuming a human face as piece-wise planar surfaces, where each surface corresponds to a facial part, we develop in this paper a Constrained Part-based Alignment (CPA) algorithm for face recognition across pose and/or expression.
Constructing a Non-Negative Low Rank and Sparse Graph with Data-Adaptive Features
This paper aims at constructing a good graph for discovering intrinsic data structures in a semi-supervised learning setting.
Robust Separation of Reflection from Multiple Images
When one records a video/image sequence through a transparent medium (e. g. glass), the image is often a superposition of a transmitted layer (scene behind the medium) and a reflected layer.
Partial Occlusion Handling for Visual Tracking via Robust Part Matching
The proposed part matching tracker (PMT) has a number of attractive properties.
PCANet: A Simple Deep Learning Baseline for Image Classification?
In this work, we propose a very simple deep learning network for image classification which comprises only the very basic data processing components: cascaded principal component analysis (PCA), binary hashing, and block-wise histograms.
Ranked #40 on
Image Classification
on MNIST
ROML: A Robust Feature Correspondence Approach for Matching Objects in A Set of Images
The task is to identify the inlier features and establish their consistent correspondences across the image set.
Sparse Illumination Learning and Transfer for Single-Sample Face Recognition with Image Corruption and Misalignment
In particular, the single-sample face alignment accuracy is comparable to that of the well-known Deformable SRC algorithm using multiple gallery images per class.
Simultaneous Rectification and Alignment via Robust Recovery of Low-rank Tensors
In this context, the state-of-the-art algorithms RASL'' and "TILT'' can be viewed as two special cases of our work, and yet each only performs part of the function of our method."
Video Editing with Temporal, Spatial and Appearance Consistency
Given an area of interest in a video sequence, one may want to manipulate or edit the area, e. g. remove occlusions from or replace with an advertisement on it.
Single-Sample Face Recognition with Image Corruption and Misalignment via Sparse Illumination Transfer
To compensate the missing illumination information typically provided by multiple training images, a sparse illumination transfer (SIT) technique is introduced.
Plane-Based Content Preserving Warps for Video Stabilization
Recently, a new image deformation technique called content-preserving warping (CPW) has been successfully employed to produce the state-of-the-art video stabilization results in many challenging cases.
Learning by Associating Ambiguously Labeled Images
Our framework is motivated by the observation that samples from the same class repetitively appear in the collection of ambiguously labeled training images, while they are just ambiguously labeled in each image.
Blind Image Deblurring by Spectral Properties of Convolution Operators
We show that the minimizer of this regularizer guarantees to give good approximation to the blur kernel if the original image is sharp enough.
Collaborative Representation based Classification for Face Recognition
It is widely believed that the l1- norm sparsity constraint on coding coefficients plays a key role in the success of SRC, while its use of all training samples to collaboratively represent the query sample is rather ignored.
Compressive Principal Component Pursuit
We consider the problem of recovering a target matrix that is a superposition of low-rank and sparse components, from a small set of linear measurements.
Information Theory Information Theory
Sparsity and Robustness in Face Recognition
This report concerns the use of techniques for sparse signal representation and sparse error correction for automatic face recognition.
Robust Recovery of Subspace Structures by Low-Rank Representation
In this work we address the subspace recovery problem.
The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices
This paper proposes scalable and fast algorithms for solving the Robust PCA problem, namely recovering a low-rank matrix with an unknown fraction of its entries being arbitrarily corrupted.
Optimization and Control Numerical Analysis Systems and Control
Fast L1-Minimization Algorithms For Robust Face Recognition
L1-minimization refers to finding the minimum L1-norm solution to an underdetermined linear system b=Ax.
Fast L1-Minimization Algorithms For Robust Face Recognition
L1-minimization refers to finding the minimum L1-norm solution to an underdetermined linear system b=Ax.
Stable Principal Component Pursuit
We further prove that the solution to a related convex program (a relaxed PCP) gives an estimate of the low-rank matrix that is simultaneously stable to small entrywise noise and robust to gross sparse errors.
Information Theory Information Theory
Robust Principal Component Analysis?
This suggests the possibility of a principled approach to robust principal component analysis since our methodology and results assert that one can recover the principal components of a data matrix even though a positive fraction of its entries are arbitrarily corrupted.
Information Theory Information Theory
Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization
Principal component analysis is a fundamental operation in computational data analysis, with myriad applications ranging from web search to bioinformatics to computer vision and image analysis.
