Search Results for author:
Found 38 papers, 18 papers with code
SpacTor-T5: Pre-training T5 Models with Span Corruption and Replaced Token Detection
In this paper, we present SpacTor, a new training procedure consisting of (1) a hybrid objective combining span corruption (SC) and token replacement detection (RTD), and (2) a two-stage curriculum that optimizes the hybrid objective over the initial $\tau$ iterations, then transitions to standard SC loss.
Rethinking FID: Towards a Better Evaluation Metric for Image Generation
It is an unbiased estimator that does not make any assumptions on the probability distribution of the embeddings and is sample efficient.
MarkovGen: Structured Prediction for Efficient Text-to-Image Generation
Modern text-to-image generation models produce high-quality images that are both photorealistic and faithful to the text prompts.
Substance or Style: What Does Your Image Embedding Know?
We find that image-text models (CLIP and ALIGN) are better at recognizing new examples of style transfer than masking-based models (CAN and MAE).
Benchmarking Robustness to Adversarial Image Obfuscations
We evaluate 33 pretrained models on the benchmark and train models with different augmentations, architectures and training methods on subsets of the obfuscations to measure generalization.
Adaptive Edge Offloading for Image Classification Under Rate Limit
Because of limited computing capacity, embedded devices rely on a parsimonious classification model with uneven accuracy.
PROVES: Establishing Image Provenance using Semantic Signatures
Modern AI tools, such as generative adversarial networks, have transformed our ability to create and modify visual data with photorealistic results.
Leveraging redundancy in attention with Reuse Transformers
Pairwise dot product-based attention allows Transformers to exchange information between tokens in an input-dependent way, and is key to their success across diverse applications in language and vision.
Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation
State-of-the-art transformer models use pairwise dot-product based self-attention, which comes at a computational cost quadratic in the input sequence length.
Understanding Robustness of Transformers for Image Classification
We find that when pre-trained with a sufficient amount of data, ViT models are at least as robust as the ResNet counterparts on a broad range of perturbations.
Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments
We introduce a neural network-based method to denoise pairs of images taken in quick succession, with and without a flash, in low-light environments.
Real-Time Edge Classification: Optimal Offloading under Token Bucket Constraints
To deploy machine learning-based algorithms for real-time applications with strict latency constraints, we consider an edge-computing setting where a subset of inputs are offloaded to the edge for processing by an accurate but resource-intensive model, and the rest are processed only by a less-accurate model on the device itself.
Finding Physical Adversarial Examples for Autonomous Driving with Fast and Differentiable Image Compositing
We propose a scalable approach for finding adversarial modifications of a simulated autonomous driving environment using a differentiable approximation for the mapping from environmental modifications (rectangles on the road) to the corresponding video inputs to the controller neural network.
Adversarial Robustness of Deep Sensor Fusion Models
First, we find that the fusion model is usually both more accurate, and more robust against single-source attacks than single-sensor deep neural networks.
Towards a MEMS-based Adaptive LIDAR
We present a proof-of-concept LIDAR design that allows adaptive real-time measurements according to dynamically specified measurement patterns.
Basis Prediction Networks for Effective Burst Denoising with Large Kernels
Bursts of images exhibit significant self-similarity across both time and space.
Protecting Geolocation Privacy of Photo Collections
We study this problem formally as a combinatorial optimization problem in the context of geolocation prediction facilitated by deep learning.
Training Image Estimators without Image Ground Truth
Deep neural networks have been very successful in compressive-sensing and image restoration applications, as a means to estimate images from partial, blurry, or otherwise degraded measurements.
Neural Network-Inspired Analog-to-Digital Conversion to Achieve Super-Resolution with Low-Precision RRAM Devices
Recent works propose neural network- (NN-) inspired analog-to-digital converters (NNADCs) and demonstrate their great potentials in many emerging applications.
Training Image Estimators without Image Ground-Truth
We evaluate our method for training networks for compressive-sensing and blind deconvolution, considering both non-blind and blind training for the latter.
Generating and Exploiting Probabilistic Monocular Depth Estimates
Beyond depth estimation from a single image, the monocular cue is useful in a broader range of depth inference applications and settings---such as when one can leverage other available depth cues for improved accuracy.
Ranked #64 on
Monocular Depth Estimation
on NYU-Depth V2
(RMSE metric)
Fast Deep Stereo with 2D Convolutional Processing of Cost Signatures
Modern neural network-based algorithms are able to produce highly accurate depth estimates from stereo image pairs, nearly matching the reliability of measurements from more expensive depth sensors.
Backprop with Approximate Activations for Memory-efficient Network Training
Training convolutional neural network models is memory intensive since back-propagation requires storing activations of all intermediate layers.
Learning to Separate Multiple Illuminants in a Single Image
We present a method to separate a single image captured under two illuminants, with different spectra, into the two images corresponding to the appearance of the scene under each individual illuminant.
Identifying Recurring Patterns with Deep Neural Networks for Natural Image Denoising
In this work, we propose a new method for natural image denoising that trains a deep neural network to determine whether patches in a noisy image input share common underlying patterns.
Learning Privacy Preserving Encodings through Adversarial Training
We present a framework to learn privacy-preserving encodings of images that inhibit inference of chosen private attributes, while allowing recovery of other desirable information.
Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning
The physical design of a robot and the policy that controls its motion are inherently coupled, and should be determined according to the task and environment.
Stabilizing GAN Training with Multiple Random Projections
Training generative adversarial networks is unstable in high-dimensions as the true data distribution tends to be concentrated in a small fraction of the ambient space.
Jointly Optimizing Placement and Inference for Beacon-based Localization
The accuracy of such a beacon-based localization system depends both on how beacons are distributed in the environment, and how the robot's location is inferred based on noisy and potentially ambiguous measurements.
Examining the Impact of Blur on Recognition by Convolutional Networks
We investigate the extent to which this degradation is due to the mismatch between training and input image statistics.
Single-image RGB Photometric Stereo With Spatially-varying Albedo
We present a single-shot system to recover surface geometry of objects with spatially-varying albedos, from images captured under a calibrated RGB photometric stereo setup---with three light directions multiplexed across different color channels in the observed RGB image.
Depth from a Single Image by Harmonizing Overcomplete Local Network Predictions
A single color image can contain many cues informative towards different aspects of local geometric structure.
Learning Sensor Multiplexing Design through Back-propagation
In this paper, we specifically consider the design and inference problems in a typical color camera---where the sensor is able to measure only one color channel at each pixel location, and computational inference is required to reconstruct a full color image.
A Neural Approach to Blind Motion Deblurring
We present a new method for blind motion deblurring that uses a neural network trained to compute estimates of sharp image patches from observations that are blurred by an unknown motion kernel.
Color Constancy by Learning to Predict Chromaticity from Luminance
Specifically, we describe an illuminant estimation method that is built around a "classifier" for identifying the true chromaticity of a pixel given its luminance (absolute brightness across color channels).
Low-level Vision by Consensus in a Spatial Hierarchy of Regions
We introduce a multi-scale framework for low-level vision, where the goal is estimating physical scene values from image data---such as depth from stereo image pairs.
Modeling Radiometric Uncertainty for Vision with Tone-mapped Color Images
To produce images that are suitable for display, tone-mapping is widely used in digital cameras to map linear color measurements into narrow gamuts with limited dynamic range.
From Shading to Local Shape
We develop a framework for extracting a concise representation of the shape information available from diffuse shading in a small image patch.
