Search Results for author:
Found 49 papers, 20 papers with code
RENO: Real-Time Neural Compression for 3D LiDAR Point Clouds
Despite the substantial advancements demonstrated by learning-based neural models in the LiDAR Point Cloud Compression (LPCC) task, realizing real-time compression - an indispensable criterion for numerous industrial applications - remains a formidable challenge.
U2A: Unified Unimodal Adaptation for Robust and Efficient Multimodal Learning
Our method significantly reduces the number of learnable parameters and eliminates the need for complex training strategies, such as alternating training, gradient modifications, or unimodal fine-tuning.
MMP: Towards Robust Multi-Modal Learning with Masked Modality Projection
We achieve this by randomly masking a subset of modalities during training and learning to project available input modalities to estimate the tokens for the masked modalities.
Gaussian is All You Need: A Unified Framework for Solving Inverse Problems via Diffusion Posterior Sampling
Furthermore, we present an efficient way to factorize and invert the covariance matrix of the likelihood function for several inverse problems.
Unlearning Targeted Information via Single Layer Unlearning Gradient
Unauthorized privacy-related and copyrighted content generation using generative-AI is becoming a significant concern for human society, raising ethical, legal, and privacy issues that demand urgent attention.
Transformation-Dependent Adversarial Attacks
We introduce transformation-dependent adversarial attacks, a new class of threats where a single additive perturbation can trigger diverse, controllable mis-predictions by systematically transforming the input (e. g., scaling, blurring, compression).
Cross-Modal Safety Alignment: Is textual unlearning all you need?
Recent studies reveal that integrating new modalities into Large Language Models (LLMs), such as Vision-Language Models (VLMs), creates a new attack surface that bypasses existing safety training techniques like Supervised Fine-tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF).
PNeRV: Enhancing Spatial Consistency via Pyramidal Neural Representation for Videos
To address this issue, we introduce the Pyramidal Neural Representation for Videos (PNeRV), which is built on a multi-scale information connection and comprises a lightweight rescaling operator, Kronecker Fully-connected layer (KFc), and a Benign Selective Memory (BSM) mechanism.
Overcoming Distribution Shifts in Plug-and-Play Methods with Test-Time Training
A common issue with the learned models is that of a performance drop when there is a distribution shift between the training and testing data.
STRIDE: Single-video based Temporally Continuous Occlusion-Robust 3D Pose Estimation
This challenge arises because these models struggle to generalize beyond their training datasets, and the variety of occlusions is hard to capture in the training data.
Domain Expansion via Network Adaptation for Solving Inverse Problems
Deep learning-based methods deliver state-of-the-art performance for solving inverse problems that arise in computational imaging.
Factorized Tensor Networks for Multi-Task and Multi-Domain Learning
In this paper, we propose a factorized tensor network (FTN) that can achieve accuracy comparable to independent single-task/domain networks with a small number of additional parameters.
Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation
We conduct a series of experiments to highlight the missing modality robustness of our proposed method on five different multimodal tasks across seven datasets.
Prior Mismatch and Adaptation in PnP-ADMM with a Nonconvex Convergence Analysis
Our second set of numerical results considers a simple and effective domain adaption strategy that closes the performance gap due to the use of mismatched denoisers.
MMSFormer: Multimodal Transformer for Material and Semantic Segmentation
Furthermore, our ablation studies also highlight the capacity of different input modalities to improve performance in the identification of different types of materials.
Ranked #1 on
Semantic Segmentation
on MCubeS (P)
DNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videos
DNeRV achieves competitive results against the state-of-the-art neural compression approaches and outperforms existing implicit methods on downstream inpainting and interpolation for $960 \times 1920$ videos.
Ensemble-based Blackbox Attacks on Dense Prediction
We performed a number of experiments for object detectors and segmentation to highlight the significance of the our proposed methods.
Disguise without Disruption: Utility-Preserving Face De-Identification
We extensively evaluate our method using multiple datasets, demonstrating a higher de-identification rate and superior consistency compared to prior approaches in various downstream tasks.
Compressive Sensing with Tensorized Autoencoder
Furthermore, in many cases, parts of images can be corrupted by noise or missing entries.
Coded Illumination for 3D Lensless Imaging
With such baseline distance between the lensless camera and illumination source, the camera observes a slice of the 3D volume, and the PSF of each depth plane becomes more resolvable from each other.
Efficient Visual Computing with Camera RAW Snapshots
Conventional cameras capture image irradiance on a sensor and convert it to RGB images using an image signal processor (ISP).
GAMA: Generative Adversarial Multi-Object Scene Attacks
Using the joint image-text features to train the generator, we show that GAMA can craft potent transferable perturbations in order to fool victim classifiers in various attack settings.
Leveraging Local Patch Differences in Multi-Object Scenes for Generative Adversarial Attacks
Our goal is to design an attack strategy that can learn from such natural scenes by leveraging the local patch differences that occur inherently in such images (e. g. difference between the local patch on the object `person' and the object `bike' in a traffic scene).
Blackbox Attacks via Surrogate Ensemble Search
We also show that the perturbations generated by our proposed method are highly transferable and can be adopted for hard-label blackbox attacks.
H2-Stereo: High-Speed, High-Resolution Stereoscopic Video System
We utilize a disparity network to transfer spatiotemporal information across views even in large disparity scenes, based on which, we propose disparity-guided flow-based warping for LSR-HFR view and complementary warping for HSR-LFR view.
Incremental Task Learning with Incremental Rank Updates
Our method also offers better memory efficiency compared to episodic memory- and mask-based approaches.
Event Transformer
The event camera's low power consumption and ability to capture microsecond brightness changes make it attractive for various computer vision tasks.
Zero-Query Transfer Attacks on Context-Aware Object Detectors
We compare our zero-query attack against a few-query scheme that repeatedly checks if the victim system is fooled.
Provable and Efficient Continual Representation Learning
In continual learning (CL), the goal is to design models that can learn a sequence of tasks without catastrophic forgetting.
Context-Aware Transfer Attacks for Object Detection
In this paper, we present a new approach to generate context-aware attacks for object detectors.
Coded Illumination for Improved Lensless Imaging
In this paper, we propose to use coded illumination to improve the quality of images reconstructed with lensless cameras.
ADC: Adversarial attacks against object Detection that evade Context consistency checks
A very recent defense strategy for detecting adversarial examples, that has been shown to be robust to current attacks, is to check for intrinsic context consistencies in the input data, where context refers to various relationships (e. g., object-to-object co-occurrence relationships) in images.
Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations
When compared to the image classification models, black-box adversarial attacks against video classification models have been largely understudied.
Exploiting Multi-Object Relationships for Detecting Adversarial Attacks in Complex Scenes
Vision systems that deploy Deep Neural Networks (DNNs) are known to be vulnerable to adversarial examples.
A Simple Framework for 3D Lensless Imaging with Programmable Masks
Existing methods for lensless imaging can recover the depth and intensity of the scene, but they require solving computationally-expensive inverse problems.
Recovery Analysis for Plug-and-Play Priors using the Restricted Eigenvalue Condition
The plug-and-play priors (PnP) and regularization by denoising (RED) methods have become widely used for solving inverse problems by leveraging pre-trained deep denoisers as image priors.
Provably Convergent Algorithms for Solving Inverse Problems Using Generative Models
The traditional approach of hand-crafting priors (such as sparsity) for solving inverse problems is slowly being replaced by the use of richer learned priors (such as those modeled by deep generative networks).
Solving Phase Retrieval with a Learned Reference
We performed a number of simulations on a variety of datasets under different conditions and found that our proposed method for phase retrieval via unrolled network and learned reference provides near-perfect recovery at fixed (small) computational cost.
Non-Adversarial Video Synthesis with Learned Priors
Different from these methods, we focus on the problem of generating videos from latent noise vectors, without any reference input frames.
Joint Image and Depth Estimation with Mask-Based Lensless Cameras
We built a prototype lensless camera and present experimental results for reconstruction of intensity and depth maps of different real objects.
A Dual Camera System for High Spatiotemporal Resolution Video Acquisition
This paper presents a dual camera system for high spatiotemporal resolution (HSTR) video acquisition, where one camera shoots a video with high spatial resolution and low frame rate (HSR-LFR) and another one captures a low spatial resolution and high frame rate (LSR-HFR) video.
Generative Models for Low-Dimensional Video Representation and Compressive Sensing
In the context of compressive sensing, if the unknown image belongs to the range of a pretrained generative network, then we can recover the image by estimating the underlying compact latent code from the available measurements.
Alternating Phase Projected Gradient Descent with Generative Priors for Solving Compressive Phase Retrieval
We empirically show that the performance of our method with projected gradient descent is superior to the existing approach for solving phase retrieval under generative priors.
Generative Models for Low-Rank Video Representation and Reconstruction
Finding compact representation of videos is an essential component in almost every problem related to video processing or understanding.
Toward Depth Estimation Using Mask-Based Lensless Cameras
Recently, coded masks have been used to demonstrate a thin form-factor lensless camera, FlatCam, in which a mask is placed immediately on top of a bare image sensor.
Toward Long Distance, Sub-diffraction Imaging Using Coherent Camera Arrays
Recent advances in ptychography have demonstrated that one can image beyond the diffraction limit of the objective lens in a microscope.
FlatCam: Thin, Bare-Sensor Cameras using Coded Aperture and Computation
FlatCam is a thin form-factor lensless camera that consists of a coded mask placed on top of a bare, conventional sensor array.
FPA-CS: Focal Plane Array-based Compressive Imaging in Short-wave Infrared
Unfortunately, the measurement rate of a SPC is insufficient to enable imaging at high spatial and temporal resolutions.
Sparse Recovery of Streaming Signals Using L1-Homotopy
In this paper, we discuss two such streaming systems and a homotopy-based algorithm for quickly solving the associated L1-norm minimization programs: 1) Recovery of a smooth, time-varying signal for which, instead of using block transforms, we use lapped orthogonal transforms for sparse representation.
