Search Results for author:
Found 110 papers, 57 papers with code
Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Models
Vision and Language (VL) models offer an effective method for aligning representation spaces of images and text, leading to numerous applications such as cross-modal retrieval, visual question answering, captioning, and more.
Pruning at Initialization -- A Sketching Perspective
We show that finding a sparse mask at initialization is equivalent to the sketching problem introduced for efficient matrix multiplication.
Neural (Tangent Kernel) Collapse
This work bridges two important concepts: the Neural Tangent Kernel (NTK), which captures the evolution of deep neural networks (DNNs) during training, and the Neural Collapse (NC) phenomenon, which refers to the emergence of symmetry and structure in the last-layer features of well-trained classification DNNs.
UDPM: Upsampling Diffusion Probabilistic Models
In this work, we propose to generalize the denoising diffusion process into an Upsampling Diffusion Probabilistic Model (UDPM), in which we reduce the latent variable dimension in addition to the traditional noise level addition.
VisDA 2022 Challenge: Domain Adaptation for Industrial Waste Sorting
To test the abilities of computer vision models on this task, we present the VisDA 2022 Challenge on Domain Adaptation for Industrial Waste Sorting.
Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenes
We show that using proxies allows a wide variety of editing options, such as adjusting the placement of each independent object, removing objects from a scene, or refining an object.
3D Masked Autoencoders with Application to Anomaly Detection in Non-Contrast Enhanced Breast MRI
This new model, coined masked autoencoder for medical imaging (MAEMI) is trained on two non-contrast enhanced MRI sequences, aiming at lesion detection without the need for intravenous injection of contrast media and temporal image acquisition.
TEXTure: Text-Guided Texturing of 3D Shapes
In this paper, we present TEXTure, a novel method for text-guided generation, editing, and transfer of textures for 3D shapes.
Teaching Structured Vision & Language Concepts to Vision & Language Models
Vision and Language (VL) models have demonstrated remarkable zero-shot performance in a variety of tasks.
ADIR: Adaptive Diffusion for Image Reconstruction
In recent years, denoising diffusion models have demonstrated outstanding image generation performance.
MAEDAY: MAE for few and zero shot AnomalY-Detection
We show that MAEDAY performs surprisingly well at this task.
PIP: Positional-encoding Image Prior
In Deep Image Prior (DIP), a Convolutional Neural Network (CNN) is fitted to map a latent space to a degraded (e. g. noisy) image but in the process learns to reconstruct the clean image.
Teaching Structured Vision&Language Concepts to Vision&Language Models
Vision and Language (VL) models have demonstrated remarkable zero-shot performance in a variety of tasks.
Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures
This unique combination of text and shape guidance allows for increased control over the generation process.
Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets
Overparameterization in deep learning typically refers to settings where a trained neural network (NN) has representational capacity to fit the training data in many ways, some of which generalize well, while others do not.
A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images
Diffusion models are a special type of generative model, capable of synthesising new data from a learnt distribution.
Utilizing Excess Resources in Training Neural Networks
In this work, we suggest Kernel Filtering Linear Overparameterization (KFLO), where a linear cascade of filtering layers is used during training to improve network performance in test time.
Membership Inference Attack Using Self Influence Functions
Member inference (MI) attacks aim to determine if a specific data sample was used to train a machine learning model.
ProCST: Boosting Semantic Segmentation Using Progressive Cyclic Style-Transfer
This new data has a reduced domain gap from the desired target domain, which facilitates the applied UDA approach to close the gap further.
Ranked #5 on
Unsupervised Domain Adaptation
on SYNTHIA-to-Cityscapes
(using extra training data)
Stress-Testing Point Cloud Registration on Automotive LiDAR
Rigid Point Cloud Registration (PCR) algorithms aim to estimate the 6-DOF relative motion between two point clouds, which is important in various fields, including autonomous driving.
Denoiser-based projections for 2-D super-resolution multi-reference alignment
We study the 2-D super-resolution multi-reference alignment (SR-MRA) problem: estimating an image from its down-sampled, circularly-translated, and noisy copies.
Generative Adversarial Networks
Generative Adversarial Networks (GANs) are very popular frameworks for generating high-quality data, and are immensely used in both the academia and industry in many domains.
SPAGHETTI: Editing Implicit Shapes Through Part Aware Generation
Neural implicit fields are quickly emerging as an attractive representation for learning based techniques.
Extending the Vocabulary of Fictional Languages using Neural Networks
Fictional languages have become increasingly popular over the recent years appearing in novels, movies, TV shows, comics, and video games.
NeuralMLS: Geometry-Aware Control Point Deformation
In this work, we opt to learn the weighting function, by training a neural network on the control points from a single input shape, and exploit the innate smoothness of neural networks.
Video Reconstruction from a Single Motion Blurred Image using Learned Dynamic Phase Coding
Video reconstruction from a single motion-blurred image is a challenging problem, which can enhance the capabilities of existing cameras.
Unsupervised Domain Generalization by Learning a Bridge Across Domains
The ability to generalize learned representations across significantly different visual domains, such as between real photos, clipart, paintings, and sketches, is a fundamental capacity of the human visual system.
Mesh Draping: Parametrization-Free Neural Mesh Transfer
The method drapes the source mesh over the target geometry and at the same time seeks to preserve the carefully designed characteristics of the source mesh.
DeepBBS: Deep Best Buddies for Point Cloud Registration
The Best Buddies criterion is a strong indication for correct matches that, in turn, leads to accurate registration.
Simple Post-Training Robustness Using Test Time Augmentations and Random Forest
A leading defense against such attacks is adversarial training, a technique in which a DNN is trained to be robust to adversarial attacks by introducing adversarial noise to its input.
MISS GAN: A Multi-IlluStrator Style Generative Adversarial Network for image to illustration translation
MISS GAN is both input image specific and uses the information of other images using only one trained model.
Z2P: Instant Visualization of Point Clouds
We present a technique for visualizing point clouds using a neural network.
FLEX: Extrinsic Parameters-free Multi-view 3D Human Motion Reconstruction
We compare our model to state-of-the-art methods that are not ep-free and show that in the absence of camera parameters, we outperform them by a large margin while obtaining comparable results when camera parameters are available.
Ranked #14 on
3D Human Pose Estimation
on Human3.6M
Orienting Point Clouds with Dipole Propagation
In the global phase, we propagate the orientation across all coherent patches using a dipole propagation.
Detector-Free Weakly Supervised Grounding by Separation
In this work, we focus on the task of Detector-Free WSG (DF-WSG) to solve WSG without relying on a pre-trained detector.
Ranked #1 on
Phrase Grounding
on Visual Genome
SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimization
Multilayer-perceptrons (MLP) are known to struggle with learning functions of high-frequencies, and in particular cases with wide frequency bands.
Multiplicative Reweighting for Robust Neural Network Optimization
Yet, their performance degrades in the presence of noisy labels at train time.
Separable Joint Blind Deconvolution and Demixing
Blind deconvolution and demixing is the problem of reconstructing convolved signals and kernels from the sum of their convolutions.
Image Restoration by Deep Projected GSURE
In the first one, where no explicit prior is used, we show that the proposed approach outperforms other internal learning methods, such as DIP.
An Interpretation of Regularization by Denoising and its Application with the Back-Projected Fidelity Term
The vast majority of image recovery tasks are ill-posed problems.
ISP Distillation
We then train a model that is applied directly to the RAW data by using knowledge distillation such that the model predictions for RAW images will be aligned with the predictions of an off-the-shelf pre-trained model for processed RGB images.
A Spectral Perspective of Neural Networks Robustness to Label Noise
Deep networks usually require a massive amount of labeled data for their training.
GMM-Based Generative Adversarial Encoder Learning
We model the output of the encoder latent space via a GMM, which leads to both good clustering using this latent space and improved image generation by the GAN.
Fine-grained Angular Contrastive Learning with Coarse Labels
A very practical example of C2FS is when the target classes are sub-classes of the training classes.
Best Buddies Registration for Point Clouds
We propose new, and robust, loss functions for the point cloud registration problem.
Kernel-Based Smoothness Analysis of Residual Networks
A major factor in the success of deep neural networks is the use of sophisticated architectures rather than the classical multilayer perceptron (MLP).
Deep Sparse Light Field Refocusing
Light field photography enables to record 4D images, containing angular information alongside spatial information of the scene.
Self-Sampling for Neural Point Cloud Consolidation
We introduce a novel technique for neural point cloud consolidation which learns from only the input point cloud.
Self-supervised Neural Architecture Search
Neural Architecture Search (NAS) has been used recently to achieve improved performance in various tasks and most prominently in image classification.
Deep Geometric Texture Synthesis
Learning and synthesizing on local geometric patches enables a genus-oblivious framework, facilitating texture transfer between shapes of different genus.
Face Authentication from Grayscale Coded Light Field
Face verification is a fast-growing authentication tool for everyday systems, such as smartphones.
Point2Mesh: A Self-Prior for Deformable Meshes
We optimize the network weights to deform an initial mesh to shrink-wrap a single input point cloud.
On the Convergence Rate of Projected Gradient Descent for a Back-Projection based Objective
Recently, several works have considered a back-projection (BP) based fidelity term as an alternative to the common least squares (LS), and demonstrated excellent results for popular inverse problems.
A function space analysis of finite neural networks with insights from sampling theory
This work suggests using sampling theory to analyze the function space represented by neural networks.
PointGMM: a Neural GMM Network for Point Clouds
We present PointGMM, a neural network that learns to generate hGMMs which are characteristic of the shape class, and also coincide with the input point cloud.
StarNet: towards Weakly Supervised Few-Shot Object Detection
Few-shot detection and classification have advanced significantly in recent years.
TAFSSL: Task-Adaptive Feature Sub-Space Learning for few-shot classification
The field of Few-Shot Learning (FSL), or learning from very few (typically $1$ or $5$) examples per novel class (unseen during training), has received a lot of attention and significant performance advances in the recent literature.
Autoencoders
An autoencoder is a specific type of a neural network, which is mainly designed to encode the input into a compressed and meaningful representation, and then decode it back such that the reconstructed input is similar as possible to the original one.
BP-DIP: A Backprojection based Deep Image Prior
Deep neural networks are a very powerful tool for many computer vision tasks, including image restoration, exhibiting state-of-the-art results.
Introduction to deep learning
Deep Learning (DL) has made a major impact on data science in the last decade.
Motion Deblurring using Spatiotemporal Phase Aperture Coding
In this work, a computational imaging approach for motion deblurring is proposed and demonstrated.
Supervised and Unsupervised Learning of Parameterized Color Enhancement
We treat the problem of color enhancement as an image translation task, which we tackle using both supervised and unsupervised learning.
DEGAS: Differentiable Efficient Generator Search
In this work, we propose an alternative strategy for GAN search by using a method called DEGAS (Differentiable Efficient GenerAtor Search), which focuses on efficiently finding the generator in the GAN.
Ranked #14 on
Image Generation
on STL-10
MetAdapt: Meta-Learned Task-Adaptive Architecture for Few-Shot Classification
In this work, we propose to employ tools inspired by the Differentiable Neural Architecture Search (D-NAS) literature in order to optimize the architecture for FSL without over-fitting.
Correction Filter for Single Image Super-Resolution: Robustifying Off-the-Shelf Deep Super-Resolvers
For a known kernel, we design a closed-form correction filter that modifies the low-resolution image to match one which is obtained by another kernel (e. g. bicubic), and thus improves the results of existing pre-trained DNNs.
MonSter: Awakening the Mono in Stereo
The most common methods for passive depth estimation are either a stereo or a monocular system.
Detecting Adversarial Samples Using Influence Functions and Nearest Neighbors
We use influence functions to measure the impact of every training sample on the validation set data.
Deep Radar Detector
While camera and LiDAR processing have been revolutionized since the introduction of deep learning, radar processing still relies on classical tools.
Back-Projection based Fidelity Term for Ill-Posed Linear Inverse Problems
This term encourages agreement between the projection of the optimization variable onto the row space of the linear operator and the pseudo-inverse of the linear operator ("back-projection") applied on the observations.
Image-Adaptive GAN based Reconstruction
In the recent years, there has been a significant improvement in the quality of samples produced by (deep) generative models such as variational auto-encoders and generative adversarial networks.
Baby steps towards few-shot learning with multiple semantics
Learning from one or few visual examples is one of the key capabilities of humans since early infancy, but is still a significant challenge for modern AI systems.
Ranked #9 on
Few-Shot Image Classification
on Mini-ImageNet - 1-Shot Learning
(using extra training data)
ASAP: Architecture Search, Anneal and Prune
In this paper, we propose a differentiable search space that allows the annealing of architecture weights, while gradually pruning inferior operations.
Taco-VC: A Single Speaker Tacotron based Voice Conversion with Limited Data
Taco-VC is implemented using a single speaker Tacotron synthesizer based on Phonetic PosteriorGrams (PPGs) and a single speaker WaveNet vocoder conditioned on mel spectrograms.
Blind Visual Motif Removal from a Single Image
Many images shared over the web include overlaid objects, or visual motifs, such as text, symbols or drawings, which add a description or decoration to the image.
Lautum Regularization for Semi-supervised Transfer Learning
We focus on the task of semi-supervised transfer learning, in which unlabeled samples from the target dataset are available during the network training on the source dataset.
LaSO: Label-Set Operations networks for multi-label few-shot learning
We conduct numerous experiments showing promising results for the label-set manipulation capabilities of the proposed approach, both directly (using the classification and retrieval metrics), and in the context of performing data augmentation for multi-label few-shot learning.
A Greedy Approach to $\ell_{0,\infty}$ Based Convolutional Sparse Coding
This has the disadvantage that the reconstructed image no longer obeys the sparsity prior used in the processing.
TOP-GAN: Label-Free Cancer Cell Classification Using Deep Learning with a Small Training Set
After this preliminary training, and after transforming the last layer of the network with new ones, we have designed an automatic classifier for the correct cell type (healthy/primary cancer/metastatic cancer) with 90-99% accuracy, although small training sets of down to several images have been used.
Super-Resolution via Image-Adapted Denoising CNNs: Incorporating External and Internal Learning
While deep neural networks exhibit state-of-the-art results in the task of image super-resolution (SR) with a fixed known acquisition process (e. g., a bicubic downscaling kernel), they experience a huge performance loss when the real observation model mismatches the one used in training.
Efficient non-uniform quantizer for quantized neural network targeting reconfigurable hardware
While most works use uniform quantizers for both parameters and activations, it is not always the optimal one, and a non-uniform quantizer need to be considered.
An ETF view of Dropout regularization
Dropout is a popular regularization technique in deep learning.
NICE: Noise Injection and Clamping Estimation for Neural Network Quantization
Convolutional Neural Networks (CNN) are very popular in many fields including computer vision, speech recognition, natural language processing, to name a few.
MeshCNN: A Network with an Edge
In this paper, we utilize the unique properties of the mesh for a direct analysis of 3D shapes using MeshCNN, a convolutional neural network designed specifically for triangular meshes.
Class-Aware Fully-Convolutional Gaussian and Poisson Denoising
We propose a fully-convolutional neural-network architecture for image denoising which is simple yet powerful.
Generalization Error in Deep Learning
Deep learning models have lately shown great performance in various fields such as computer vision, speech recognition, speech translation, and natural language processing.
Ranking Recovery from Limited Comparisons using Low-Rank Matrix Completion
The partial and noisy data of pairwise comparisons is transformed into a matrix form.
Delta-encoder: an effective sample synthesis method for few-shot object recognition
Our approach is based on a modified auto-encoder, denoted Delta-encoder, that learns to synthesize new samples for an unseen category just by seeing few examples from it.
RepMet: Representative-based metric learning for classification and one-shot object detection
Distance metric learning (DML) has been successfully applied to object classification, both in the standard regime of rich training data and in the few-shot scenario, where each category is represented by only a few examples.
DNN or k-NN: That is the Generalize vs. Memorize Question
Moreover, the behavior of DNNs compared to the linear classifiers SVM and LR is quite the same on the training and test data regardless of whether the network generalizes.
UNIQ: Uniform Noise Injection for Non-Uniform Quantization of Neural Networks
We present a novel method for neural network quantization that emulates a non-uniform $k$-quantile quantizer, which adapts to the distribution of the quantized parameters.
ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning
The process of aligning a pair of shapes is a fundamental operation in computer graphics.
The Resistance to Label Noise in K-NN and DNN Depends on its Concentration
We investigate the classification performance of K-nearest neighbors (K-NN) and deep neural networks (DNNs) in the presence of label noise.
Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization
We demonstrate empirically that it leads to enhanced robustness results with a minimal change in the original network's accuracy.
Fast and Accurate Reconstruction of Compressed Color Light Field
One of its drawbacks is the need for multi-lens in the imaging.
DeepISP: Towards Learning an End-to-End Image Processing Pipeline
We present DeepISP, a full end-to-end deep neural model of the camera image signal processing (ISP) pipeline.
Mathematics of Deep Learning
Recently there has been a dramatic increase in the performance of recognition systems due to the introduction of deep architectures for representation learning and classification.
Image Restoration by Iterative Denoising and Backward Projections
In this work, we propose an alternative method for solving inverse problems using off-the-shelf denoisers, which requires less parameter tuning.
Deep Class Aware Denoising
We further show that a significant boost in performance of up to $0. 4$ dB PSNR can be achieved by making our network class-aware, namely, by fine-tuning it for images belonging to a specific semantic class.
Deep Convolutional Denoising of Low-Light Images
Poisson distribution is used for modeling noise in photon-limited imaging.
Generalization Error of Invariant Classifiers
We show that whereas the generalization error of a non-invariant classifier is proportional to the complexity of the input space, the generalization error of an invariant classifier is proportional to the complexity of the base space.
Tradeoffs between Convergence Speed and Reconstruction Accuracy in Inverse Problems
Solving inverse problems with iterative algorithms is popular, especially for large data.
Robust Large Margin Deep Neural Networks
The generalization error of deep neural networks via their classification margin is studied in this work.
Poisson Inverse Problems by the Plug-and-Play scheme
In this work we suggest a novel method for coupling Gaussian denoising algorithms to Poisson noisy inverse problems, which is based on a general approach termed "Plug-and-Play".
Postprocessing of Compressed Images via Sequential Denoising
In this work we propose a novel postprocessing technique for compression-artifact reduction.
Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?
Three important properties of a classification machinery are: (i) the system preserves the core information of the input data; (ii) the training examples convey information about unseen data; and (iii) the system is able to treat differently points from different classes.
On the Stability of Deep Networks
In particular, we formally prove in the longer version that DNN with random Gaussian weights perform a distance-preserving embedding of the data, with a special treatment for in-class and out-of-class data.
Sparsity Based Methods for Overparameterized Variational Problems
Two complementary approaches have been extensively used in signal and image processing leading to novel results, the sparse representation methodology and the variational strategy.
Sparsity Based Poisson Denoising with Dictionary Learning
In cases of high SNR, several transformations exist so as to convert the Poisson noise into an additive i. i. d.
