Search Results for author:
Found 357 papers, 221 papers with code
Learning Simple Thresholded Features with Sparse Support Recovery
The thresholded feature has recently emerged as an extremely efficient, yet rough empirical approximation, of the time-consuming sparse coding inference process.
Enhance Visual Recognition under Adverse Conditions via Deep Networks
Visual recognition under adverse conditions is a very important and challenging problem of high practical value, due to the ubiquitous existence of quality distortions during image acquisition, transmission, or storage.
Predicting Depression Severity by Multi-Modal Feature Engineering and Fusion
We present our preliminary work to determine if patient's vocal acoustic, linguistic, and facial patterns could predict clinical ratings of depression severity, namely Patient Health Questionnaire depression scale (PHQ-8).
End-to-End United Video Dehazing and Detection
Furthermore, we build an End-to-End United Video Dehazing and Detection Network(EVDD-Net), which concatenates and jointly trains EVD-Net with a video object detection model.
Robust Emotion Recognition from Low Quality and Low Bit Rate Video: A Deep Learning Approach
Emotion recognition from facial expressions is tremendously useful, especially when coupled with smart devices and wireless multimedia applications.
Deep Double Sparsity Encoder: Learning to Sparsify Not Only Features But Also Parameters
This paper emphasizes the significance to jointly exploit the problem structure and the parameter structure, in the context of deep modeling.
Stacked Approximated Regression Machine: A Simple Deep Learning Approach
With the agreement of my coauthors, I Zhangyang Wang would like to withdraw the manuscript "Stacked Approximated Regression Machine: A Simple Deep Learning Approach".
UnitBox: An Advanced Object Detection Network
In present object detection systems, the deep convolutional neural networks (CNNs) are utilized to predict bounding boxes of object candidates, and have gained performance advantages over the traditional region proposal methods.
$\mathbf{D^3}$: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images
In this paper, we design a Deep Dual-Domain ($\mathbf{D^3}$) based fast restoration model to remove artifacts of JPEG compressed images.
Learning A Deep $\ell_\infty$ Encoder for Hashing
We investigate the $\ell_\infty$-constrained representation which demonstrates robustness to quantization errors, utilizing the tool of deep learning.
Studying Very Low Resolution Recognition Using Deep Networks
Visual recognition research often assumes a sufficient resolution of the region of interest (ROI).
Brain-Inspired Deep Networks for Image Aesthetics Assessment
Image aesthetics assessment has been challenging due to its subjective nature.
Learning Deep $\ell_0$ Encoders
We study the $\ell_0$ sparse approximation problem with the tool of deep learning, by proposing Deep $\ell_0$ Encoders.
Learning A Task-Specific Deep Architecture For Clustering
In this paper, we propose to emulate the sparse coding-based clustering pipeline in the context of deep learning, leading to a carefully crafted deep model benefiting from both.
Designing A Composite Dictionary Adaptively From Joint Examples
We study the complementary behaviors of external and internal examples in image restoration, and are motivated to formulate a composite dictionary design framework.
Learning Super-Resolution Jointly from External and Internal Examples
Single image super-resolution (SR) aims to estimate a high-resolution (HR) image from a lowresolution (LR) input.
Self-Tuned Deep Super Resolution
Deep learning has been successfully applied to image super resolution (SR).
Decomposition-Based Domain Adaptation for Real-World Font Recognition
We present a domain adaption framework to address a domain mismatch between synthetic training and real-world testing data.
Real-World Font Recognition Using Deep Network and Domain Adaptation
We address a challenging fine-grain classification problem: recognizing a font style from an image of text.
On a Theory of Nonparametric Pairwise Similarity for Clustering: Connecting Clustering to Classification
Modeling the underlying data distribution by nonparametric kernel density estimation, the generalization error bounds for both unsupervised nonparametric classifiers are the sum of nonparametric pairwise similarity terms between the data points for the purpose of clustering.
ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA
In this work, we propose Analytic LISTA (ALISTA), where the weight matrix in LISTA is computed as the solution to a data-free optimization problem, leaving only the stepsize and threshold parameters to data-driven learning.
Adaptive Activity Monitoring with Uncertainty Quantification in Switching Gaussian Process Models
This power-efficient sensing scheme can be achieved by deciding which group of sensors to use at a given time, requiring an accurate characterization of the trade-off between sensor energy usage and the uncertainty in ignoring certain sensor signals while monitoring.
Bridging the Gap Between Computational Photography and Visual Recognition
From the observed results, it is evident that we are in the early days of building a bridge between computational photography and visual recognition, leaving many opportunities for innovation in this area.
D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images
In this paper, we design a Deep Dual-Domain (D3) based fast restoration model to remove artifacts of JPEG compressed images.
Robust Video Super-Resolution With Learned Temporal Dynamics
Second, we reduce the complexity of motion between neighboring frames using a spatial alignment network that is much more robust and efficient than competing alignment methods and can be jointly trained with the temporal adaptive network in an end-to-end manner.
Convolutional Sparse Coding for Compressed Sensing CT Reconstruction
Over the past few years, dictionary learning (DL)-based methods have been successfully used in various image reconstruction problems.
UG$^{2+}$ Track 2: A Collective Benchmark Effort for Evaluating and Advancing Image Understanding in Poor Visibility Environments
The UG$^{2+}$ challenge in IEEE CVPR 2019 aims to evoke a comprehensive discussion and exploration about how low-level vision techniques can benefit the high-level automatic visual recognition in various scenarios.
Dual Dynamic Inference: Enabling More Efficient, Adaptive and Controllable Deep Inference
State-of-the-art convolutional neural networks (CNNs) yield record-breaking predictive performance, yet at the cost of high-energy-consumption inference, that prohibits their widely deployments in resource-constrained Internet of Things (IoT) applications.
Report on UG^2+ Challenge Track 1: Assessing Algorithms to Improve Video Object Detection and Classification from Unconstrained Mobility Platforms
How can we effectively engineer a computer vision system that is able to interpret videos from unconstrained mobility platforms like UAVs?
E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savings
Extensive simulations and ablation studies, with real energy measurements from an FPGA board, confirm the superiority of our proposed strategies and demonstrate remarkable energy savings for training.
DAVID: Dual-Attentional Video Deblurring
To train and evaluate on more diverse blur severity levels, we propose a Challenging DVD dataset generated from the raw DVD video set by pooling frames with different temporal windows.
Practical Solutions for Machine Learning Safety in Autonomous Vehicles
Autonomous vehicles rely on machine learning to solve challenging tasks in perception and motion planning.
Explainable Deep Relational Networks for Predicting Compound-Protein Affinities and Contacts
DeepRelations shows superior interpretability to the state-of-the-art: without compromising affinity prediction, it boosts the AUPRC of contact prediction 9. 5, 16. 9, 19. 3 and 5. 7-fold for the test, compound-unique, protein-unique, and both-unique sets, respectively.
VGAI: End-to-End Learning of Vision-Based Decentralized Controllers for Robot Swarms
More specifically, we consider that each robot has access to a visual perception of the immediate surroundings, and communication capabilities to transmit and receive messages from other neighboring robots.
Safeguarded Learned Convex Optimization
Our numerical examples show convergence of Safe-L2O algorithms, even when the provided data is not from the distribution of training data.
Uncertainty Quantification for Deep Context-Aware Mobile Activity Recognition and Unknown Context Discovery
Activity recognition in wearable computing faces two key challenges: i) activity characteristics may be context-dependent and change under different contexts or situations; ii) unknown contexts and activities may occur from time to time, requiring flexibility and adaptability of the algorithm.
SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation
We present SmartExchange, an algorithm-hardware co-design framework to trade higher-cost memory storage/access for lower-cost computation, for energy-efficient inference of deep neural networks (DNNs).
NADS: Neural Architecture Distribution Search for Uncertainty Awareness
Machine learning (ML) systems often encounter Out-of-Distribution (OoD) errors when dealing with testing data coming from a distribution different from training data.
AutoPose: Searching Multi-Scale Branch Aggregation for Pose Estimation
Such simplification limits the fusion of information at different scales and fails to maintain high-resolution representations.
Eliminating the Invariance on the Loss Landscape of Linear Autoencoders
For this new loss, we characterize the full structure of the loss landscape in the following sense: we establish analytical expression for the set of all critical points, show that it is a subset of critical points of MSE, and that all local minima are still global.
Dynamic Feature Selection for Efficient and Interpretable Human Activity Recognition
In many machine learning tasks, input features with varying degrees of predictive capability are usually acquired at some cost.
Robust Overfitting may be mitigated by properly learned smoothening
A recent study (Rice et al., 2020) revealed overfitting to be a dominant phenomenon in adversarially robust training of deep networks, and that appropriate early-stopping of adversarial training (AT) could match the performance gains of most recent algorithmic improvements.
Bayesian Learning to Optimize: Quantifying the Optimizer Uncertainty
Optimizing an objective function with uncertainty awareness is well-known to improve the accuracy and confidence of optimization solutions.
Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning
In view of those, we introduce two pruning options, e. g., top-down and bottom-up, for finding lifelong tickets.
Learning A Minimax Optimizer: A Pilot Study
We first present Twin L2O, the first dedicated minimax L2O framework consisting of two LSTMs for updating min and max variables, respectively.
ALFA: Adversarial Feature Augmentation for Enhanced Image Recognition
Adversarial training is an effective method to combat adversarial attacks in order to create robust neural networks.
On Dynamic Noise Influence in Differential Private Learning
In this paper, we provide comprehensive analysis of noise influence in dynamic privacy schedules to answer these critical questions.
UMEC: Unified model and embedding compression for efficient recommendation systems
The recommendation system (RS) plays an important role in the content recommendation and retrieval scenarios.
Efficiently Troubleshooting Image Segmentation Models with Human-In-The-Loop
Image segmentation lays the foundation for many high-stakes vision applications such as autonomous driving and medical image analysis.
Weak NAS Predictor Is All You Need
Rather than expecting a single strong predictor to model the whole space, we seek a progressive line of weak predictors that can connect a path to the best architecture, thus greatly simplifying the learning task of each predictor.
PCAL: A Privacy-preserving Intelligent Credit Risk Modeling Framework Based on Adversarial Learning
While machine learning is increasingly used in this field, the resulting large-scale collection of user private information has reinvigorated the privacy debate, considering dozens of data breach incidents every year caused by unauthorized hackers, and (potentially even more) information misuse/abuse by authorized parties.
What Does CNN Shift Invariance Look Like? A Visualization Study
We present the results of three experiments comparing representations of millions of images with exhaustively shifted objects, examining both local invariance (within a few pixels) and global invariance (across the image frame).
Uncertainty-Aware Physically-Guided Proxy Tasks for Unseen Domain Face Anti-spoofing
Face anti-spoofing (FAS) seeks to discriminate genuine faces from fake ones arising from any type of spoofing attack.
Growing Deep Forests Efficiently with Soft Routing and Learned Connectivity
For example, our model with only 1 layer of 15 trees can perform comparably with the model in [3] with 2 layers of 2000 trees each.
Dynamic Privacy Budget Allocation Improves Data Efficiency of Differentially Private Gradient Descent
In this paper, we provide comprehensive analysis of noise influence in dynamic privacy schedules to answer these critical questions.
Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data
Electronic Health Record (EHR) data has been of tremendous utility in Artificial Intelligence (AI) for healthcare such as predicting future clinical events.
A Design Space Study for LISTA and Beyond
Unrolling is believed to incorporate the model-based prior with the learning capacity of deep learning.
Knowledge-Augmented Contrastive Learning for Abnormality Classification and Localization in Chest X-rays with Radiomics using a Feedback Loop
The key knob of our framework is a unique positive sampling approach tailored for the medical images, by seamlessly integrating radiomic features as a knowledge augmentation.
Troubleshooting Blind Image Quality Models in the Wild
Recently, the group maximum differentiation competition (gMAD) has been used to improve blind image quality assessment (BIQA) models, with the help of full-reference metrics.
Taxonomy of Machine Learning Safety: A Survey and Primer
The open-world deployment of Machine Learning (ML) algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities such as interpretability, verifiability, and performance limitations.
IA-RED$^2$: Interpretability-Aware Redundancy Reduction for Vision Transformers
The self-attention-based model, transformer, is recently becoming the leading backbone in the field of computer vision.
Ranked #29 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference
Semantic segmentation for scene understanding is nowadays widely demanded, raising significant challenges for the algorithm efficiency, especially its applications on resource-limited platforms.
Searching for Two-Stream Models in Multivariate Space for Video Recognition
We design a multivariate search space, including 6 search variables to capture a wide variety of choices in designing two-stream models.
GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization
During the training process, the polarization effect will drive a subset of gates to smoothly decrease to exact zero, while other gates gradually stay away from zero by a large margin.
Universality of Winning Tickets: A Renormalization Group Perspective
Foundational work on the Lottery Ticket Hypothesis has suggested an exciting corollary: winning tickets found in the context of one task can be transferred to similar tasks, possibly even across different architectures.
Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently
In this work, we define an extended class of subnetworks in randomly initialized NNs called disguised subnetworks, which are not only "hidden" in the random networks but also "disguised" -- hence can only be "unmasked" with certain transformations on weights.
Equalized Robustness: Towards Sustainable Fairness Under Distributional Shifts
In this paper, we first propose a new fairness goal, termed Equalized Robustness (ER), to impose fair model robustness against unseen distribution shifts across majority and minority groups.
Lottery Image Prior
Drawing inspirations from the recently prosperous research on lottery ticket hypothesis (LTH), we conjecture and study a novel “lottery image prior” (LIP), stated as: given an (untrained or trained) DNN-based image prior, it will have a sparse subnetwork that can be training in isolation, to match the original DNN’s performance when being applied as a prior to various image inverse problems.
AutoCoG: A Unified Data-Modal Co-Search Framework for Graph Neural Networks
Despite the preliminary success, we argue that for GNNs, NAS has to be customized further, due to the topological complicacy of GNN input data (graph) as well as the notorious training instability.
Scaling the Depth of Vision Transformers via the Fourier Domain Analysis
The first technique, termed AttnScale, decomposes a self-attention block into low-pass and high-pass components, then rescales and combines these two filters to produce an all-pass self-attention matrix.
Lottery Tickets can have Structural Sparsity
The lottery ticket hypothesis (LTH) has shown that dense models contain highly sparse subnetworks (i. e., $\textit{winning tickets}$) that can be trained in isolation to match full accuracy.
Reasoning With Hierarchical Symbols: Reclaiming Symbolic Policies For Visual Reinforcement Learning
Deep vision models are nowadays widely integrated into visual reinforcement learning (RL) to parameterize the policy networks.
Bayesian Modeling and Uncertainty Quantification for Learning to Optimize: What, Why, and How
Optimizing an objective function with uncertainty awareness is well-known to improve the accuracy and confidence of optimization solutions.
Stingy Teacher: Sparse Logits Suffice to Fail Knowledge Distillation
However, it is unclear why the distorted distribution of the logits is catastrophic to the student model.
CheXT: Knowledge-Guided Cross-Attention Transformer for Abnormality Classification and Localization in Chest X-rays
During training, the image branch leverages its learned attention to estimate pathology localization, which is then utilized to extract radiomic features from images in the radiomics branch.
Generalizable Learning to Optimize into Wide Valleys
Learning to optimize (L2O) has gained increasing popularity in various optimization tasks, since classical optimizers usually require laborious, problem-specific design and hyperparameter tuning.
Audio Lottery: Speech Recognition Made Ultra-Lightweight, Noise-Robust, and Transferable
In this paper, we investigate the tantalizing possibility of using lottery ticket hypothesis to discover lightweight speech recognition models, that are (1) robust to various noise existing in speech; (2) transferable to fit the open-world personalization; and 3) compatible with structured sparsity.
Universality of Deep Neural Network Lottery Tickets: A Renormalization Group Perspective
Foundational work on the Lottery Ticket Hypothesis has suggested an exciting corollary: winning tickets found in the context of one task can be transferred to similar tasks, possibly even across different architectures.
Neural Networks for Principal Component Analysis: A New Loss Function Provably Yields Ordered Exact Eigenvectors
In this paper, we propose a new loss function for performing principal component analysis (PCA) using linear autoencoders (LAEs).
Is There Mode Collapse? A Case Study on Face Generation and Its Black-box Calibration
Generative adversarial networks (GANs) nowadays are capable of producing im-ages of incredible realism.
Universal Safeguarded Learned Convex Optimization with Guaranteed Convergence
Inferences by each network form solution estimates, and networks are trained to optimize these estimates for a particular distribution of data.
EnergyNet: Energy-Efficient Dynamic Inference
The prohibitive energy cost of running high-performance Convolutional Neural Networks (CNNs) has been limiting their deployment on resource-constrained platforms including mobile and wearable devices.
Auto-X3D: Ultra-Efficient Video Understanding via Finer-Grained Neural Architecture Search
Efficient video architecture is the key to deploying video recognition systems on devices with limited computing resources.
Fast and High-Quality Image Denoising via Malleable Convolutions
These spatially-varying kernels are produced by an efficient predictor network running on a downsampled input, making them much more efficient to compute than per-pixel kernels produced by a full-resolution image, and also enlarging the network's receptive field compared with static kernels.
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer
To the best of our knowledge, this is the first time quantization has been incorporated into ViT acceleration on FPGAs with the help of a fully automatic framework to guide the quantization strategy on the software side and the accelerator implementations on the hardware side given the target frame rate.
On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks
It is shown that given a pruning probability, for fully-connected neural networks with the weights randomly pruned at the initialization, as the width of each layer grows to infinity sequentially, the NTK of the pruned neural network converges to the limiting NTK of the original network with some extra scaling.
VFDS: Variational Foresight Dynamic Selection in Bayesian Neural Networks for Efficient Human Activity Recognition
At its core is an implicit variational distribution on binary gates that are dependent on previous observations, which will select the next subset of features to observe.
EI-CLIP: Entity-Aware Interventional Contrastive Learning for E-Commerce Cross-Modal Retrieval
recommendation, and marketing services.
A Multi-purpose Realistic Haze Benchmark with Quantifiable Haze Levels and Ground Truth
The full dataset presented in this paper, including the ground truth object classification bounding boxes and haze density measurements, is provided for the community to evaluate their algorithms at: https://a2i2-archangel. vision.
Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning
Inspired by the recent success of learning robust models with unlabeled data, we explore a new robustness-aware CIL setting, where the learned adversarial robustness has to resist forgetting and be transferred as new tasks come in continually.
How Robust is Your Fairness? Evaluating and Sustaining Fairness under Unseen Distribution Shifts
Increasing concerns have been raised on deep learning fairness in recent years.
Density-Aware Personalized Training for Risk Prediction in Imbalanced Medical Data
Medical events of interest, such as mortality, often happen at a low rate in electronic medical records, as most admitted patients survive.
Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization
Compared with state-of-the-art ViT quantization work (algorithmic approach only without hardware acceleration), our quantization achieves 0. 47% to 1. 36% higher Top-1 accuracy under the same bit-width.
DynImp: Dynamic Imputation for Wearable Sensing Data Through Sensory and Temporal Relatedness
We argue that traditional methods have rarely made use of both times-series dynamics of the data as well as the relatedness of the features from different sensors.
Signal Processing for Implicit Neural Representations
We answer this question by proposing an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.
Data-Model-Circuit Tri-Design for Ultra-Light Video Intelligence on Edge Devices
In this paper, we propose a data-model-hardware tri-design framework for high-throughput, low-cost, and high-accuracy multi-object tracking (MOT) on High-Definition (HD) video stream.
RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging
Our extensive experiments on popular medical imaging classification tasks (cardiopulmonary disease and lesion classification) using real-world datasets, show the performance benefit of RoS-KD, its ability to distill knowledge from many popular large networks (ResNet-50, DenseNet-121, MobileNet-V2) in a comparatively small network, and its robustness to adversarial attacks (PGD, FSGM).
Scaling Multimodal Pre-Training via Cross-Modality Gradient Harmonization
For example, even in the commonly adopted instructional videos, a speaker can sometimes refer to something that is not visually present in the current frame; and the semantic misalignment would only be more unpredictable for the raw videos from the internet.
QuanGCN: Noise-Adaptive Training for Robust Quantum Graph Convolutional Networks
Quantum neural networks (QNNs), an interdisciplinary field of quantum computing and machine learning, have attracted tremendous research interests due to the specific quantum advantages.
AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training
Neural Radiance Fields (NeRFs) are a powerful representation for modeling a 3D scene as a continuous function.
StegaNeRF: Embedding Invisible Information within Neural Radiance Fields
Recent advances in neural rendering imply a future of widespread visual data distributions through sharing NeRF model weights.
Attend Who is Weak: Pruning-assisted Medical Image Localization under Sophisticated and Implicit Imbalances
However, DNNs are notoriously fragile to the class imbalance in image classification.
Convergence and Generalization of Wide Neural Networks with Large Bias
This work studies training one-hidden-layer overparameterized ReLU networks via gradient descent in the neural tangent kernel (NTK) regime, where the networks' biases are initialized to some constant rather than zero.
Pruning Before Training May Improve Generalization, Provably
It is shown that as long as the pruning fraction is below a certain threshold, gradient descent can drive the training loss toward zero and the network exhibits good generalization performance.
Evaluate underdiagnosis and overdiagnosis bias of deep learning model on primary open-angle glaucoma diagnosis in under-served patient populations
In the United States, primary open-angle glaucoma (POAG) is the leading cause of blindness, especially among African American and Hispanic individuals.
Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers
In response, we summarize ten Q\&As of SNNs from many key aspects, including dense vs. sparse, unstructured sparse vs. structured sparse, pruning vs. sparse training, dense-to-sparse training vs. sparse-to-sparse training, static sparsity vs. dynamic sparsity, before-training/during-training vs. post-training sparsity, and many more.
Learning to Grow Pretrained Models for Efficient Transformer Training
Scaling transformers has led to significant breakthroughs in many domains, leading to a paradigm in which larger versions of existing models are trained and released on a periodic basis.
MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognition
In this paper, we study a novel problem in egocentric action recognition, which we term as "Multimodal Generalization" (MMG).
NeuralLift-360: Lifting an In-the-Wild 2D Photo to a 3D Object With 360deg Views
In this work, we study the challenging task of lifting a single image to a 3D object and, for the first time, demonstrate the ability to generate a plausible 3D object with 360deg views that corresponds well with the given reference image.
Specialist Diffusion: Plug-and-Play Sample-Efficient Fine-Tuning of Text-to-Image Diffusion Models To Learn Any Unseen Style
Diffusion models have demonstrated impressive capability of text-conditioned image synthesis, and broader application horizons are emerging by personalizing those pretrained diffusion models toward generating some specialized target object or style.
MMG-Ego4D: Multimodal Generalization in Egocentric Action Recognition
In this paper, we study a novel problem in egocentric action recognition, which we term as "Multimodal Generalization" (MMG).
FarSight: A Physics-Driven Whole-Body Biometric System at Large Distance and Altitude
Whole-body biometric recognition is an important area of research due to its vast applications in law enforcement, border security, and surveillance.
Polynomial Width is Sufficient for Set Representation with High-dimensional Features
To investigate the minimal value of $L$ that achieves sufficient expressive power, we present two set-element embedding layers: (a) linear + power activation (LP) and (b) linear + exponential activations (LE).
Reference-based Painterly Inpainting via Diffusion: Crossing the Wild Reference Domain Gap
We propose Reference-based Painterly Inpainting, a novel task that crosses the wild reference domain gap and implants novel objects into artworks.
Doubly Robust Instance-Reweighted Adversarial Training
Experiments on standard classification datasets demonstrate that our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance, and at the same time improves the robustness against attacks on the weakest data points.
Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications
We start by proposing two conjectures on the nature of the damage: one is certain knowledge being forgotten (or erased) after LLM compression, hence necessitating the compressed model to (re)learn from data with additional parameters; the other presumes that knowledge is internally displaced and hence one requires merely "inference re-direction" with input-side augmentation such as prompting, to recover the knowledge-related performance.
Efficient-3DiM: Learning a Generalizable Single-image Novel-view Synthesizer in One Day
Although the fidelity and generalizability are greatly improved, training such a powerful diffusion model requires a vast volume of training data and model parameters, resulting in a notoriously long time and high computational costs.
Pose-Free Generalizable Rendering Transformer
To address this challenge, we introduce PF-GRT, a new Pose-Free framework for Generalizable Rendering Transformer, eliminating the need for pre-computed camera poses and instead leveraging feature-matching learned directly from data.
Data Distillation Can Be Like Vodka: Distilling More Times For Better Quality
Dataset distillation aims to minimize the time and memory needed for training deep networks on large datasets, by creating a small set of synthetic images that has a similar generalization performance to that of the full dataset.
Multi-Concept T2I-Zero: Tweaking Only The Text Embeddings and Nothing Else
To achieve this goal, we identify the limitations in the text embeddings used for the pre-trained text-to-image diffusion models.
Towards long-tailed, multi-label disease classification from chest X-ray: Overview of the CXR-LT challenge
Many real-world image recognition problems, such as diagnostic medical imaging exams, are "long-tailed" $\unicode{x2013}$ there are a few common findings followed by many more relatively rare conditions.
Fine-Tuning Language Models Using Formal Methods Feedback
Although pre-trained language models encode generic knowledge beneficial for planning and control, they may fail to generate appropriate control policies for domain-specific tasks.
FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting
Novel view synthesis from limited observations remains an important and persistent task.
Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields
In this work, we go one step further: in addition to radiance field rendering, we enable 3D Gaussian splatting on arbitrary-dimension semantic features via 2D foundation model distillation.
The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need
Therefore, it remains unclear whether the self-attention operation is crucial for the recent advances in SSL - or CNNs can deliver the same excellence with more advanced designs, too?
4DGen: Grounded 4D Content Generation with Spatial-temporal Consistency
Our pipeline facilitates conditional 4D generation, enabling users to specify geometry (3D assets) and motion (monocular videos), thus offering superior control over content creation.
SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity
In this paper, we reveal that the gradient estimation in score distillation is inherent to high variance.
Taming Mode Collapse in Score Distillation for Text-to-3D Generation
In this paper, we reveal that the existing score distillation-based text-to-3D generation frameworks degenerate to maximal likelihood seeking on each view independently and thus suffer from the mode collapse problem, manifesting as the Janus artifact in practice.
VASE: Object-Centric Appearance and Shape Manipulation of Real Videos
Recently, several works tackled the video editing task fostered by the success of large-scale text-to-image generative models.
AGG: Amortized Generative 3D Gaussians for Single Image to 3D
To overcome these challenges, we introduce an Amortized Generative 3D Gaussian framework (AGG) that instantly produces 3D Gaussians from a single image, eliminating the need for per-instance optimization.
Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression
While state-of-the-art (SoTA) compression methods boast impressive advancements in preserving benign task performance, the potential risks of compression in terms of safety and trustworthiness have been largely neglected.
Comp4D: LLM-Guided Compositional 4D Scene Generation
Recent advancements in diffusion models for 2D and 3D content creation have sparked a surge of interest in generating 4D content.
Generalization Error Analysis for Sparse Mixture-of-Experts: A Preliminary Study
Specifically, we investigate the impact of the number of data samples, the total number of experts, the sparsity in expert selection, the complexity of the routing mechanism, and the complexity of individual experts.
Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3D
In recent years, there has been an explosion of 2D vision models for numerous tasks such as semantic segmentation, style transfer or scene editing, enabled by large-scale 2D image datasets.
InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40 Seconds
This pre-processing is usually conducted via a Structure-from-Motion (SfM) pipeline, a procedure that can be slow and unreliable, particularly in sparse-view scenarios with insufficient matched features for accurate reconstruction.
MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements
We show for the first time that using 3D Gaussians for map representation with unposed camera images and inertial measurements can enable accurate SLAM.
DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting
This point cloud serves as the initial state for the centroids of 3D Gaussians.
Black-Box Diagnosis and Calibration on GAN Intra-Mode Collapse: A Pilot Study
Existing diversity tests of samples from GANs are usually conducted qualitatively on a small scale, and/or depends on the access to original training data as well as the trained model parameters.
INR-Arch: A Dataflow Architecture and Compiler for Arbitrary-Order Gradient Computations in Implicit Neural Representation Processing
In this work, we introduce INR-Arch, a framework that transforms the computation graph of an nth-order gradient into a hardware-optimized dataflow architecture.
Rethinking PGD Attack: Is Sign Function Necessary?
Based on that, we further propose a new raw gradient descent (RGD) algorithm that eliminates the use of sign.
SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training
Results show that: 1) applied to inference, SD achieves up to 2. 44x energy efficiency as evaluated via real hardware implementations; 2) applied to training, SD leads to 10. 56x and 4. 48x reduction in the storage and training energy, with negligible accuracy loss compared to state-of-the-art training baselines.
Safe and Robust Watermark Injection with a Single OoD Image
Training a high-performance deep neural network requires large amounts of data and computational resources.
QuantumSEA: In-Time Sparse Exploration for Noise Adaptive Quantum Circuits
To address these two pain points, we propose QuantumSEA, an in-time sparse exploration for noise-adaptive quantum circuits, aiming to achieve two key objectives: (1) implicit circuits capacity during training - by dynamically exploring the circuit's sparse connectivity and sticking a fixed small number of quantum gates throughout the training which satisfies the coherence time and enjoy light noises, enabling feasible executions on real quantum devices; (2) noise robustness - by jointly optimizing the topology and parameters of quantum circuits under real device noise models.
Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Risk
While diffusion models have recently demonstrated remarkable progress in generating realistic images, privacy risks also arise: published models or APIs could generate training images and thus leak privacy-sensitive training information.
Predicting Model Failure using Saliency Maps in Autonomous Driving Systems
While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations.
InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks
The promise of Deep Neural Network (DNN) powered Internet of Thing (IoT) devices has motivated a tremendous demand for automated solutions to enable fast development and deployment of efficient (1) DNNs equipped with instantaneous accuracy-efficiency trade-off capability to accommodate the time-varying resources at IoT devices and (2) dataflows to optimize DNNs' execution efficiency on different devices.
E^2VTS: Energy-Efficient Video Text Spotting from Unmanned Aerial Vehicles
Unmanned Aerial Vehicles (UAVs) based video text spotting has been extensively used in civil and military domains.
Can pruning improve certified robustness of neural networks?
Given the fact that neural networks are often over-parameterized, one effective way to reduce such computational overhead is neural network pruning, by removing redundant parameters from trained neural networks.
Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization
Our study starts from an empirical strategy for the light continual training of LLMs using their original pre-training data sets, with a specific focus on selective retention of samples that incur moderately high losses.
OpenBias: Open-set Bias Detection in Text-to-Image Generative Models
In this paper, we tackle the challenge of open-set bias detection in text-to-image generative models presenting OpenBias, a new pipeline that identifies and quantifies the severity of biases agnostically, without access to any precompiled set.
U-Finger: Multi-Scale Dilated Convolutional Network for Fingerprint Image Denoising and Inpainting
This paper studies the challenging problem of fingerprint image denoising and inpainting.
Optimizer Amalgamation
Selecting an appropriate optimizer for a given problem is of major interest for researchers and practitioners.
How Does Pruning Impact Long-Tailed Multi-Label Medical Image Classifiers?
This work represents a first step toward understanding the impact of pruning on model behavior in deep long-tailed, multi-label medical image classification.
Principled Architecture-aware Scaling of Hyperparameters
However, most designs or optimization methods are agnostic to the choice of network structures, and thus largely ignore the impact of neural architectures on hyperparameters.
Towards Lifelong Learning of Multilingual Text-To-Speech Synthesis
This work presents a lifelong learning approach to train a multilingual Text-To-Speech (TTS) system, where each language was seen as an individual task and was learned sequentially and continually.
Self-supervised contrastive learning of echocardiogram videos enables label-efficient cardiac disease diagnosis
Advances in self-supervised learning (SSL) have shown that self-supervised pretraining on medical imaging data can provide a strong initialization for downstream supervised classification and segmentation.
Spending Your Winning Lottery Better After Drawing It
In this paper, we demonstrate that it is unnecessary for spare retraining to strictly inherit those properties from the dense network.
Learning Transferable 3D Adversarial Cloaks for Deep Trained Detectors
This paper presents a novel patch-based adversarial attack pipeline that trains adversarial patches on 3D human meshes.
Font Completion and Manipulation by Cycling Between Multi-Modality Representations
The novel graph constructor maps a glyph's latent code to its graph representation that matches expert knowledge, which is trained to help the translation task.
Delayed Propagation Transformer: A Universal Computation Engine towards Practical Control in Cyber-Physical Systems
This paper presents the Delayed Propagation Transformer (DePT), a new transformer-based model that specializes in the global modeling of CPS while taking into account the immutable constraints from the physical world.
Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis
Advanced deep neural networks (DNNs), designed by either human or AutoML algorithms, are growing increasingly complex.
M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation
This data-driven procedure yields L2O that can efficiently solve problems similar to those seen in training, that is, drawn from the same ``task distribution".
Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding
We propose an interpretable deep structure namely Frank-Wolfe Network (F-W Net), whose architecture is inspired by unrolling and truncating the Frank-Wolfe algorithm for solving an $L_p$-norm constrained problem with $p\geq 1$.
Sparse Winning Tickets are Data-Efficient Image Recognizers
Improving the performance of deep networks in data-limited regimes has warranted much attention.
Social Reward: Evaluating and Enhancing Generative AI through Million-User Feedback from an Online Creative Community
These findings highlight the relevance and effectiveness of Social Reward in assessing community appreciation for AI-generated artworks, establishing a closer alignment with users' creative goals: creating popular visual art.
MATE: Plugging in Model Awareness to Task Embedding for Meta Learning
Meta-learning improves generalization of machine learning models when faced with previously unseen tasks by leveraging experiences from different, yet related prior tasks.
Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again
In this paper, firstly, we provide a new perspective of gradient flow to understand the substandard performance of deep GCNs and hypothesize that by facilitating healthy gradient flow, we can significantly improve their trainability, as well as achieve state-of-the-art (SOTA) level performance from vanilla-GCNs.
Towards Constituting Mathematical Structures for Learning to Optimize
Learning to Optimize (L2O), a technique that utilizes machine learning to learn an optimization algorithm automatically from data, has gained arising attention in recent years.
Compressing LLMs: The Truth is Rarely Pure and Never Simple
Recently, several works have shown significant success in training-free and data-free compression (pruning and quantization) of LLMs that achieve 50 - 60% sparsity and reduce the bit width to 3 or 4 bits per weight, with negligible degradation of perplexity over the uncompressed baseline.
Subspace Network: Deep Multi-Task Censored Regression for Modeling Neurodegenerative Diseases
Over the past decade a wide spectrum of machine learning models have been developed to model the neurodegenerative diseases, associating biomarkers, especially non-intrusive neuroimaging markers, with key clinical scores measuring the cognitive status of patients.
An All-in-One Network for Dehazing and Beyond
This paper proposes an image dehazing model built with a convolutional neural network (CNN), called All-in-One Dehazing Network (AOD-Net).
ArcticNet: A Deep Learning Solution to Classify Arctic Wetlands
Arctic environments are rapidly changing under the warming climate.
HALO: Hardware-Aware Learning to Optimize
There has been an explosive demand for bringing machine learning (ML) powered intelligence into numerous Internet-of-Things (IoT) devices.
Sparse and Imperceptible Adversarial Attack via a Homotopy Algorithm
Compared to state-of-the-art methods, our homotopy attack leads to significantly fewer perturbations, e. g., reducing 42. 91% on CIFAR-10 and 75. 03% on ImageNet (average case, targeted attack), at similar maximal perturbation magnitudes, when still achieving 100% attack success rates.
Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks
Our framework is implemented by a cascade of a convolutional and a graph neural network (CNN / GNN), addressing agent-level visual perception and feature learning, as well as swarm-level communication, local information aggregation and agent action inference, respectively.
You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership
The lottery ticket hypothesis (LTH) emerges as a promising framework to leverage a special sparse subnetwork (i. e., winning ticket) instead of a full model for both training and inference, that can lower both costs without sacrificing the performance.
DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models
To address these pain points, we propose a framework for resource- and parameter-efficient fine-tuning by leveraging the sparsity prior in both weight updates and the final model weights.
Data-Efficient Double-Win Lottery Tickets from Robust Pre-training
For example, on downstream CIFAR-10/100 datasets, we identify double-win matching subnetworks with the standard, fast adversarial, and adversarial pre-training from ImageNet, at 89. 26%/73. 79%, 89. 26%/79. 03%, and 91. 41%/83. 22% sparsity, respectively.
The Elastic Lottery Ticket Hypothesis
Based on these results, we articulate the Elastic Lottery Ticket Hypothesis (E-LTH): by mindfully replicating (or dropping) and re-ordering layers for one network, its corresponding winning ticket could be stretched (or squeezed) into a subnetwork for another deeper (or shallower) network from the same family, whose performance is nearly the same competitive as the latter's winning ticket directly found by IMP.
Symbolic Learning to Optimize: Towards Interpretability and Scalability
Recent studies on Learning to Optimize (L2O) suggest a promising path to automating and accelerating the optimization procedure for complicated tasks.
Search Behavior Prediction: A Hypergraph Perspective
With these hyperedges, we augment the original bipartite graph into a new \textit{hypergraph}.
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
Recent works have impressively demonstrated that there exists a subnetwork in randomly initialized convolutional neural networks (CNNs) that can match the performance of the fully trained dense networks at initialization, without any optimization of the weights of the network (i. e., untrained networks).
Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models
Motivated by the recent observations of model soups, which suggest that fine-tuned weights of multiple models can be merged to a better minima, we propose Instant Soup Pruning (ISP) to generate lottery ticket quality subnetworks, using a fraction of the original IMP cost by replacing the expensive intermediate pruning stages of IMP with computationally efficient weak mask generation and aggregation routine.
Calibrated Domain-Invariant Learning for Highly Generalizable Large Scale Re-Identification
Many real-world applications, such as city-scale traffic monitoring and control, requires large-scale re-identification.
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training
Recent breakthroughs in deep neural networks (DNNs) have fueled a tremendous demand for intelligent edge devices featuring on-site learning, while the practical realization of such systems remains a challenge due to the limited resources available at the edge and the required massive training costs for state-of-the-art (SOTA) DNNs.
Symbolic Distillation for Learned TCP Congestion Control
At the core of our proposal is a novel symbolic branching algorithm that enables the rule to be aware of the context in terms of various network conditions, eventually converting the NN policy into a symbolic tree.
Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication
By dividing giant graph data, we build multiple independently and parallelly trained weaker GNNs (soup ingredient) without any intermediate communication, and combine their strength using a greedy interpolation soup procedure to achieve state-of-the-art performance.
Exposing Semantic Segmentation Failures via Maximum Discrepancy Competition
A natural question then arises: Does the superior performance on the closed (and frequently re-used) test sets transfer to the open visual world with unconstrained variations?
Trap and Replace: Defending Backdoor Attacks by Trapping Them into an Easy-to-Replace Subnetwork
As a result, both the stem and the classification head in the final network are hardly affected by backdoor training samples.
DiSparse: Disentangled Sparsification for Multitask Model Compression
We analyzed the pruning masks generated with DiSparse and observed strikingly similar sparse network architecture identified by each task even before the training starts.
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training and inference time limit their generalization.
Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Search
Learning efficient and interpretable policies has been a challenging task in reinforcement learning (RL), particularly in the visual RL setting with complex scenes.
Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?
Given a robust model trained to be resilient to one or multiple types of distribution shifts (e. g., natural image corruptions), how is that "robustness" encoded in the model weights, and how easily can it be disentangled and/or "zero-shot" transferred to some other models?
Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs
Contrary to this belief, this paper presents a counter-argument: small-magnitude weights of pre-trained model weights encode vital knowledge essential for tackling difficult downstream tasks - manifested as the monotonic relationship between the performance drop of downstream tasks across the difficulty spectrum, as we prune more pre-trained weights by magnitude.
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
To address this problem, this paper introduces Multi-scale Positional Encoding (Ms-PoE) which is a simple yet effective plug-and-play approach to enhance the capacity of LLMs to handle the relevant information located in the middle of the context, without fine-tuning or introducing any additional overhead.
CERL: A Unified Optimization Framework for Light Enhancement with Realistic Noise
We present \underline{C}oordinated \underline{E}nhancement for \underline{R}eal-world \underline{L}ow-light Noisy Images (CERL), that seamlessly integrates light enhancement and noise suppression parts into a unified and physics-grounded optimization framework.
E^2TAD: An Energy-Efficient Tracking-based Action Detector
Video action detection (spatio-temporal action localization) is usually the starting point for human-centric intelligent analysis of videos nowadays.
Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images
The accurate estimation of six degrees-of-freedom (6DoF) object poses is essential for many applications in robotics and augmented reality.
Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities
Recently, zero-shot (or training-free) Neural Architecture Search (NAS) approaches have been proposed to liberate NAS from the expensive training process.
Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference
The core idea of DFS is to hypothesize layer-wise quantization (to different bitwidths) as intermediate "soft" choices to be made between fully utilizing and skipping a layer.
