Search Results for author:
Found 230 papers, 107 papers with code
IlluSign: Illustrating Sign Language Videos by Leveraging the Attention Mechanism
Our approach focuses on transferring a sketch like illustration style to video footage of sign language, combining the start and end frames of a sign into a single illustration, and using arrows to highlight the hand's direction and motion.
Adapting to the Unknown: Training-Free Audio-Visual Event Perception with Dynamic Thresholds
In the domain of audio-visual event perception, which focuses on the temporal localization and classification of events across distinct modalities (audio and visual), existing approaches are constrained by the vocabulary available in their training data.
VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models
During training, we extend the objective to predict both the generated pixels and their corresponding motion from a single learned representation.
On the Expressivity of Selective State-Space Layers: A Multivariate Polynomial Approach
Recent advances in efficient sequence modeling have introduced selective state-space layers, a key component of the Mamba architecture, which have demonstrated remarkable success in a wide range of NLP and vision tasks.
Deep Active Speech Cancellation with Multi-Band Mamba Network
We present a novel deep learning network for Active Speech Cancellation (ASC), advancing beyond Active Noise Cancellation (ANC) methods by effectively canceling both noise and speech signals.
Classifier-Guided Captioning Across Modalities
Most current captioning systems use language models trained on data from specific settings, such as image-based captioning via Amazon Mechanical Turk, limiting their ability to generalize to other modality distributions and contexts.
Segment-Based Attention Masking for GPTs
Each of these is treated as a unit for the purpose of masking, such that the first tokens in each block can access the subsequent tokens in a non-causal manner.
Reversed Attention: On The Gradient Descent Of Attention Layers In GPT
The success of Transformer-based Language Models (LMs) stems from their attention mechanism.
Diffusion-Based Attention Warping for Consistent 3D Scene Editing
We present a novel method for 3D scene editing using diffusion models, designed to ensure view consistency and realism across perspectives.
SphereUFormer: A U-Shaped Transformer for Spherical 360 Perception
This paper proposes a novel method for omnidirectional 360$\degree$ perception.
Add-it: Training-Free Object Insertion in Images With Pretrained Diffusion Models
Adding Object into images based on text instructions is a challenging task in semantic image editing, requiring a balance between preserving the original scene and seamlessly integrating the new object in a fitting location.
Accelerating Error Correction Code Transformers
Error correction codes (ECC) are crucial for ensuring reliable information transmission in communication systems.
Mitigating Copy Bias in In-Context Learning through Neuron Pruning
Large language models (LLMs) have demonstrated impressive few-shot in-context learning (ICL) abilities.
Iterative Object Count Optimization for Text-to-image Diffusion Models
To solve this, we propose optimizing the generated image based on a counting loss derived from a counting model that aggregates an object\'s potential.
Still-Moving: Customized Video Generation without Customized Video Data
We assume access to a customized version of the T2I model, trained only on still image data (e. g., using DreamBooth or StyleDrop).
DeciMamba: Exploring the Length Extrapolation Potential of Mamba
Long-range sequence processing poses a significant challenge for Transformers due to their quadratic complexity in input length.
Knowledge Editing in Language Models via Adapted Direct Preference Optimization
We use the current knowledge as a negative sample and the new knowledge we want to introduce as a positive sample in a process called DPO.
LLM Questionnaire Completion for Automatic Psychiatric Assessment
We employ a Large Language Model (LLM) to convert unstructured psychological interviews into structured questionnaires spanning various psychiatric and personality domains.
Factor Graph Optimization of Error-Correcting Codes for Belief Propagation Decoding
This is performed via a novel complete graph tensor representation of the Belief Propagation algorithm, optimized over finite fields via backpropagation and coupled with an efficient line-search method.
Explaining Modern Gated-Linear RNNs via a Unified Implicit Attention Formulation
For the other architectures for which our method is the first to provide such a view, our method is effective and competitive in the relevant metrics compared to the results obtained by state-of-the-art Transformer explainability methods.
Learning Linear Block Error Correction Codes
In this work, we propose for the first time a unified encoder-decoder training of binary linear block codes.
Obtaining Favorable Layouts for Multiple Object Generation
To address this challenge, we propose a novel approach based on a guiding principle.
The Hidden Attention of Mamba Models
The Mamba layer offers an efficient selective state space model (SSM) that is highly effective in modeling multiple domains, including NLP, long-range sequence processing, and computer vision.
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space
Understanding how Transformer-based Language Models (LMs) learn and recall information is a key goal of the deep learning community.
Training-Free Consistent Text-to-Image Generation
Text-to-image models offer a new level of creative flexibility by allowing users to guide the image generation process through natural language.
Dynamic Layer Tying for Parameter-Efficient Transformers
In the pursuit of reducing the number of trainable parameters in deep transformer networks, we employ Reinforcement Learning to dynamically select layers during training and tie them together.
DiffMoog: a Differentiable Modular Synthesizer for Sound Matching
We introduce an open-source platform that comprises DiffMoog and an end-to-end sound matching framework.
PRILoRA: Pruned and Rank-Increasing Low-Rank Adaptation
With the proliferation of large pre-trained language models (PLMs), fine-tuning all model parameters becomes increasingly inefficient, particularly when dealing with numerous downstream tasks that entail substantial training and storage costs.
Efficient Verification-Based Face Identification
We study the problem of performing face verification with an efficient neural model $f$.
Degree-based stratification of nodes in Graph Neural Networks
Here, we propose a different approach that is based on a stratification of the graph nodes.
On the Long Range Abilities of Transformers
Despite their dominance in modern DL and, especially, NLP domains, transformer architectures exhibit sub-optimal performance on long-range tasks compared to recent layers that are specifically designed for this purpose.
Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption
This innovation enables us to perform secure inference on LMs with WikiText-103.
Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptation
The proposed method is based on a lightweight adaptor network, which learns to map the audio-based representation to the input representation expected by the text-to-video generation model.
Multi-Dimensional Hyena for Spatial Inductive Bias
Our empirical findings indicate that the proposed Hyena N-D layer boosts the performance of various Vision Transformer architectures, such as ViT, Swin, and DeiT across multiple datasets.
Box-based Refinement for Weakly Supervised and Unsupervised Localization Tasks
It has been established that training a box-based detector network can enhance the localization performance of weakly supervised and unsupervised methods.
Zero-Shot Audio Captioning via Audibility Guidance
Instead, captioning occurs as an inference process that involves three networks that correspond to the three desired qualities: (i) A Large Language Model, in our case, for reasons of convenience, GPT-2, (ii) A model that provides a matching score between an audio file and a text, for which we use a multimodal matching network called ImageBind, and (iii) A text classifier, trained using a dataset we collected automatically by instructing GPT-4 with prompts designed to direct the generation of both audible and inaudible sentences.
Ranked #3 on
Zero-shot Audio Captioning
on AudioCaps
(using extra training data)
Reconstructing the Hemodynamic Response Function via a Bimodal Transformer
The relationship between blood flow and neuronal activity is widely recognized, with blood flow frequently serving as a surrogate for neuronal activity in fMRI studies.
Annotator Consensus Prediction for Medical Image Segmentation with Diffusion Models
A major challenge in the segmentation of medical images is the large inter- and intra-observer variability in annotations provided by multiple experts.
2-D SSM: A General Spatial Layer for Visual Transformers
For example, vision transformers equipped with our layer exhibit effective performance even without positional encoding
AutoSAM: Adapting SAM to Medical Images by Overloading the Prompt Encoder
The recently introduced Segment Anything Model (SAM) combines a clever architecture and large quantities of training data to obtain remarkable image segmentation capabilities.
Ranked #4 on
Video Polyp Segmentation
on SUN-SEG-Hard (Unseen)
Decision S4: Efficient Sequence-Based RL via State Spaces Layers
Recently, sequence learning methods have been applied to the problem of off-policy Reinforcement Learning, including the seminal work on Decision Transformers, which employs transformers for this task.
Centered Self-Attention Layers
The self-attention mechanism in transformers and the message-passing mechanism in graph neural networks are repeatedly applied within deep learning architectures.
The Hidden Language of Diffusion Models
In this work, we present Conceptor, a novel method to interpret the internal representation of a textual concept by a diffusion model.
Anomaly Detection with Variance Stabilized Density Estimation
Then, we present a variance-stabilized density estimation problem for maximizing the likelihood of the observed samples while minimizing the variance of the density around normal samples.
Focus Your Attention (with Adaptive IIR Filters)
We present a new layer in which dynamic (i. e., input-dependent) Infinite Impulse Response (IIR) filters of order two are used to process the input sequence prior to applying conventional attention.
Ranked #1 on
Language Modelling
on Text8
AudioToken: Adaptation of Text-Conditioned Diffusion Models for Audio-to-Image Generation
In this paper, we propose a novel method utilizing latent diffusion models trained for text-to-image-generation to generate images conditioned on audio recordings.
Discriminative Class Tokens for Text-to-Image Diffusion Models
This approach has two disadvantages: (i) supervised datasets are generally small compared to large-scale scraped text-image datasets on which text-to-image models are trained, affecting the quality and diversity of the generated images, or (ii) the input is a hard-coded label, as opposed to free-form text, limiting the control over the generated images.
Improved Tree Search for Automatic Program Synthesis
In the task of automatic program synthesis, one obtains pairs of matching inputs and outputs and generates a computer program, in a particular domain-specific language (DSL), which given each sample input returns the matching output.
Energy Regularized RNNs for Solving Non-Stationary Bandit Problems
We consider a Multi-Armed Bandit problem in which the rewards are non-stationary and are dependent on past actions and potentially on past contexts.
Gradient Adjusting Networks for Domain Inversion
However, in order to manipulate a real-world image, one first needs to be able to retrieve its corresponding latent representation in StyleGAN's latent space that is decoded to an image as close as possible to the desired image.
Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models
Recent text-to-image generative models have demonstrated an unparalleled ability to generate diverse and creative imagery guided by a target text prompt.
Domain-Generalizable Multiple-Domain Clustering
This work generalizes the problem of unsupervised domain generalization to the case in which no labeled samples are available (completely unsupervised).
Deep Quantum Error Correction
Quantum error correction codes (QECC) are a key component for realizing the potential of quantum computing.
Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separation
Applying a diffusion model Vocoder that was pretrained to model single-speaker voices on the output of a deterministic separation model leads to state-of-the-art separation results.
Ranked #1 on
Speech Separation
on Libri20Mix
Similarity Maps for Self-Training Weakly-Supervised Phrase Grounding
A phrase grounding model receives an input image and a text phrase and outputs a suitable localization map.
Generating 2D and 3D Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution
The results we present demonstrate that it is possible to obtain a considerable coverage of the identities in the LFW or RFW datasets with less than 10 master faces, for six leading deep face recognition systems.
Describing Sets of Images with Textual-PCA
We seek to semantically describe a set of images, capturing both the attributes of single images and the variations within the set.
OCD: Learning to Overfit with Conditional Diffusion Models
We present a dynamic model in which the weights are conditioned on an input sample x and are learned to match those that would be obtained by finetuning a base model on x and its label y.
Ranked #1 on
Few-Shot Text Classification
on SST-5
Denoising Diffusion Error Correction Codes
Error correction code (ECC) is an integral part of the physical communication layer, ensuring reliable data transfer over noisy channels.
Semi-supervised Learning of Partial Differential Operators and Dynamical Flows
As a result, it successfully propagates initial conditions in continuous time steps by employing the general composition properties of the partial differential operators.
Zero-Shot Video Captioning with Evolving Pseudo-Tokens
We introduce a zero-shot video captioning method that employs two frozen networks: the GPT-2 language model and the CLIP image-text matching model.
FewGAN: Generating from the Joint Distribution of a Few Images
We then use a separate set of side images to model the structure of generated images using an autoregressive model trained on the learned patch embeddings of training images.
What is Where by Looking: Weakly-Supervised Open-World Phrase-Grounding without Text Inputs
Moreover, training takes place in a weakly supervised setting, where no bounding boxes are provided.
Zero-Shot Voice Conditioning for Denoising Diffusion TTS Models
At the heart of the method lies a sampling process that combines the estimation of the denoising model with a low-pass version of the new speaker's sample.
Optimizing Relevance Maps of Vision Transformers Improves Robustness
It has been observed that visual classification models often rely mostly on the image background, neglecting the foreground, which hurts their robustness to distribution changes.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
SepIt: Approaching a Single Channel Speech Separation Bound
We present an upper bound for the Single Channel Speech Separation task, which is based on an assumption regarding the nature of short segments of speech.
Ranked #2 on
Speech Separation
on Libri10Mix
Neural Inverse Kinematics
In this work, we propose a neural IK method that employs the hierarchical structure of the problem to sequentially sample valid joint angles conditioned on the desired position and on the preceding joints along the chain.
On Disentangled and Locally Fair Representations
This problem is tackled through the lens of disentangled and locally fair representations.
No Token Left Behind: Explainability-Aided Image Classification and Generation
To mitigate it, we present a novel explainability-based approach, which adds a loss term to ensure that CLIP focuses on all relevant semantic parts of the input, in addition to employing the CLIP similarity loss used in previous works.
End to End Lip Synchronization with a Temporal AutoEncoder
We study the problem of syncing the lip movement in a video with the audio stream.
Error Correction Code Transformer
Error correction code is a major part of the communication physical layer, ensuring the reliable transfer of data over noisy channels.
Dynamically-Scaled Deep Canonical Correlation Analysis
Canonical Correlation Analysis (CCA) is a method for feature extraction of two views by finding maximally correlated linear projections of them.
Dynamic Dual-Output Diffusion Models
In this paper, we reveal some of the causes that affect the generation quality of diffusion models, especially when sampling with few iterations, and come up with a simple, yet effective, solution to mitigate them.
XAI for Transformers: Better Explanations through Conservative Propagation
Transformers have become an important workhorse of machine learning, with numerous applications.
Ranked #6 on
Question Answering
on NewsQA
(using extra training data)
Explainable Artificial Intelligence (XAI)
Question Answering
Unsupervised Disentanglement with Tensor Product Representations on the Torus
This way, the obtained representations are naturally disentangled.
fMRI Neurofeedback Learning Patterns are Predictive of Personal and Clinical Traits
We obtain a personal signature of a person's learning progress in a self-neuromodulation task, guided by functional MRI (fMRI).
Self-Supervised Transformers for fMRI representation
We present TFF, which is a Transformer framework for the analysis of functional Magnetic Resonance Imaging (fMRI) data.
Locally Shifted Attention With Early Global Integration
The separation of the attention layer into local and global counterparts allows for a low computational cost in the number of patches, while still supporting data-dependent localization already at the first layer, as opposed to the static positioning in other visual transformers.
Learning Personal Representations from fMRIby Predicting Neurofeedback Performance
We present a deep neural network method for learning a personal representation for individuals that are performing a self neuromodulation task, guided by functional MRI (fMRI).
End-to-End Segmentation via Patch-wise Polygons Prediction
The leading segmentation methods represent the output map as a pixel grid.
Learning Query Expansion over the Nearest Neighbor Graph
In this work, a hierarchical model, Graph Query Expansion (GQE), is presented, which is learned in a supervised manner and performs aggregation over an extended neighborhood of the query, thus increasing the information used from the database when computing the query expansion, and using the structure of the nearest neighbors graph.
SegDiff: Image Segmentation with Diffusion Probabilistic Models
Since the diffusion model is probabilistic, it is applied multiple times, and the results are merged into a final segmentation map.
ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic
While such models can provide a powerful score for matching and subsequent zero-shot tasks, they are not capable of generating caption given an image.
Learning a Weight Map for Weakly-Supervised Localization
In the weakly supervised localization setting, supervision is given as an image-level label.
A-Muze-Net: Music Generation by Composing the Harmony based on the Generated Melody
The Midi is represented in a way that is invariant to the musical scale, and the melody is represented, for the purpose of conditioning the harmony, by the content of each bar, viewed as a chord.
Geometric Transformer for End-to-End Molecule Properties Prediction
Transformers have become methods of choice in many applications thanks to their ability to represent complex interactions between elements.
Image-Based CLIP-Guided Essence Transfer
We make the distinction between (i) style transfer, in which a source image is manipulated to match the textures and colors of a target image, and (ii) essence transfer, in which one edits the source image to include high-level semantic attributes from the target.
Video and Text Matching with Conditioned Embeddings
Traditionally video and text matching is done by learning a shared embedding space and the encoding of one modality is independent of the other.
Denoising Diffusion Gamma Models
Generative diffusion processes are an emerging and effective tool for image and speech generation.
Meta Internal Learning
Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image.
Revisiting Linear Decision Boundaries for Few-Shot Learning with Transformer Hypernetworks
Few-shot learning (FSL) methods aim to generalize a model to new unseen classes using only a small number of support examples.
Anomaly Detection for Tabular Data with Internal Contrastive Learning
We consider the task of finding out-of-class samples in tabular data, where little can be assumed on the structure of the data.
Local-Global Shifting Vision Transformers
Due to the expensive quadratic cost of the attention mechanism, either a large patch size is used, resulting in coarse-grained global interactions, or alternatively, attention is applied only on a local region of the image at the expense of long-range interactions.
Caption Enriched Samples for Improving Hateful Memes Detection
The recently introduced hateful meme challenge demonstrates the difficulty of determining whether a meme is hateful or not.
Mixing between the Cross Entropy and the Expectation Loss Terms
The cross entropy loss is widely used due to its effectiveness and solid theoretical grounding.
Generating Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution
A master face is a face image that passes face-based identity-authentication for a large portion of the population.
In Defense of the Learning Without Forgetting for Task Incremental Learning
Catastrophic forgetting is one of the major challenges on the road for continual learning systems, which are presented with an on-line stream of tasks.
Non Gaussian Denoising Diffusion Models
Moreover, we show that using a mixture of Gaussian noise variables in the diffusion process improves the performance over a diffusion process that is based on a single distribution.
Recovering AES Keys with a Deep Cold Boot Attack
In this work, we combine a novel cryptographic variant of a deep error correcting code technique with a modified SAT solver scheme to apply the attack on AES keys.
Natural Statistics of Network Activations and Implications for Knowledge Distillation
As we show, these statistics, similar to image statistics, follow a power law.
Identity and Attribute Preserving Thumbnail Upscaling
We consider the task of upscaling a low resolution thumbnail image of a person, to a higher resolution image, which preserves the person's identity and other attributes.
HyperHyperNetworks for the Design of Antenna Arrays
Our experiments demonstrate that our approach is able to design novel antennas and antenna arrays that are compliant with the design requirements, considerably better than the baseline methods.
A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection
We demonstrate the superiority of our method on both the one-shot and few-shot settings, on the datasets of Paris, CIFAR10, MNIST and FashionMNIST as well as in the setting of defect detection on MVTec.
Many-Speakers Single Channel Speech Separation with Optimal Permutation Training
Single channel speech separation has experienced great progress in the last few years.
Ranked #1 on
Speech Separation
on Libri15Mix
Noise Estimation for Generative Diffusion Models
Generative diffusion models have emerged as leading models in speech and image generation.
Adaptive Gradient Balancing for UndersampledMRI Reconstruction and Image-to-Image Translation
Recent accelerated MRI reconstruction models have used Deep Neural Networks (DNNs) to reconstruct relatively high-quality images from highly undersampled k-space data, enabling much faster MRI scanning.
Generative Adversarial Network
Image-to-Image Translation
+2
Maximal Multiverse Learning for Promoting Cross-Task Generalization of Fine-Tuned Language Models
Language modeling with BERT consists of two phases of (i) unsupervised pre-training on unlabeled text, and (ii) fine-tuning for a specific supervised task.
Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers
Transformers are increasingly dominating multi-modal reasoning tasks, such as visual question answering, achieving state-of-the-art results thanks to their ability to contextualize information using the self-attention and co-attention mechanisms.
Explainability Guided Multi-Site COVID-19 CT Classification
Radiologist examination of chest CT is an effective way for screening COVID-19 cases.
Weakly Supervised Recovery of Semantic Attributes
We consider the problem of the extraction of semantic attributes, supervised only with classification labels.
High Fidelity Speech Regeneration with Application to Speech Enhancement
Speech enhancement has seen great improvement in recent years mainly through contributions in denoising, speaker separation, and dereverberation methods that mostly deal with environmental effects on vocal audio.
Autoregressive Belief Propagation for Decoding Block Codes
We revisit recent methods that employ graph neural networks for decoding error correcting codes and employ messages that are computed in an autoregressive manner.
Fidelity-based Deep Adiabatic Scheduling
Adiabatic quantum computation is a form of computation that acts by slowly interpolating a quantum system between an easy to prepare initial state and a final state that represents a solution to a given computational problem.
Solving Non-Stationary Bandit Problems with an RNN and an Energy Minimization Loss
We consider a Multi-Armed Bandit problem in which the rewards are non-stationary and are dependent on past actions and potentially on past contexts.
Image Animation with Refined Masking
Conditioned on the source image, the transformed mask is then decoded by a multi-scale generator that renders a realistic image, in which the content of the source frame is animated by the pose in the driving video.
HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation
We present a novel, real-time, semantic segmentation network in which the encoder both encodes and generates the parameters (weights) of the decoder.
Ranked #6 on
Real-Time Semantic Segmentation
on CamVid
(using extra training data)
Transformer Interpretability Beyond Attention Visualization
Self-attention techniques, and specifically Transformers, are dominating the field of text processing and are becoming increasingly popular in computer vision classification tasks.
Visualization of Supervised and Self-Supervised Neural Networks via Attribution Guided Factorization
However, as we show, these methods are limited in their ability to identify the support for alternative classifications, an effect we name {\em the saliency bias} hypothesis.
Single-Shot Freestyle Dance Reenactment
The task of motion transfer between a source dancer and a target person is a special case of the pose transfer problem, in which the target person changes their pose in accordance with the motions of the dancer.
Image Animation with Perturbed Masks
We present a novel approach for image-animation of a source image by a driving video, both depicting the same type of object.
Single channel voice separation for unknown number of speakers under reverberant and noisy settings
The proposed approach is composed of several separation heads optimized together with a speaker classification branch.
Generating Correct Answers for Progressive Matrices Intelligence Tests
Raven's Progressive Matrices are multiple-choice intelligence tests, where one tries to complete the missing location in a $3\times 3$ grid of abstract images.
Permuted AdaIN: Reducing the Bias Towards Global Statistics in Image Classification
In the setting of robustness, our method improves on both ImageNet-C and Cifar-100-C for multiple architectures.
Scene Graph to Image Generation with Contextualized Object Layout Refinement
We empirically show on the COCO-STUFF dataset that our approach improves the quality of both the intermediate layout and the final image.
Scale-Localized Abstract Reasoning
First, it searches for relational patterns in multiple resolutions, which allows it to readily detect visual relations, such as location, in higher resolution, while allowing the lower resolution module to focus on semantic relations, such as shape type.
Hierarchical Timbre-Painting and Articulation Generation
We present a fast and high-fidelity method for music generation, based on specified f0 and loudness, such that the synthesized audio mimics the timbre and articulation of a target instrument.
DeepFake Detection Based on the Discrepancy Between the Face and its Context
Our approach involves two networks: (i) a face identification network that considers the face region bounded by a tight semantic segmentation, and (ii) a context recognition network that considers the face context (e. g., hair, ears, neck).
Unsupervised Cross-Domain Singing Voice Conversion
We present a wav-to-wav generative model for the task of singing voice conversion from any identity.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Shuffling Recurrent Neural Networks
We propose a novel recurrent neural network model, where the hidden state $h_t$ is obtained by permuting the vector elements of the previous hidden state $h_{t-1}$ and adding the output of a learned function $b(x_t)$ of the input $x_t$ at time $t$.
Data Augmenting Contrastive Learning of Speech Representations in the Time Domain
Contrastive Predictive Coding (CPC), based on predicting future segments of speech based on past segments is emerging as a powerful algorithm for representation learning of speech signal.
MTAdam: Automatic Balancing of Multiple Training Loss Terms
When training neural models, it is common to combine multiple loss terms.
A Novel Approach for Correcting Multiple Discrete Rigid In-Plane Motions Artefacts in MRI Scans
Motion artefacts created by patient motion during an MRI scan occur frequently in practice, often rendering the scans clinically unusable and requiring a re-scan.
Hierarchical Patch VAE-GAN: Generating Diverse Videos from a Single Sample
We consider the task of generating diverse and novel videos from a single video sample.
Wish You Were Here: Context-Aware Human Generation
We present a novel method for inserting objects, specifically humans, into existing images, such that they blend in a photorealistic manner, while respecting the semantic context of the scene.
Masked Based Unsupervised Content Transfer
We consider the problem of translating, in an unsupervised manner, between two domains where one contains some additional information compared to the other.
Evaluation Metrics for Conditional Image Generation
We present two new metrics for evaluating generative models in the class-conditional image generation setting.
Structural-analogy from a Single Image Pair
In this paper, we explore the capabilities of neural networks to understand image structure given only a single pair of images, A and B.
On Infinite-Width Hypernetworks
{\em Hypernetworks} are architectures that produce the weights of a task-specific {\em primary network}.
Voice Separation with an Unknown Number of Multiple Speakers
We present a new method for separating a mixed audio sequence, in which multiple voices speak simultaneously.
Ranked #2 on
Speech Separation
on WSJ0-4mix
ScopeFlow: Dynamic Scene Scoping for Optical Flow
We propose to modify the common training protocols of optical flow, leading to sizable accuracy improvements without adding to the computational complexity of the training process.
Ranked #12 on
Optical Flow Estimation
on Sintel-final
A Critical View of the Structural Causal Model
In the multivariate case, where one can ensure that the complexities of the cause and effect are balanced, we propose a new adversarial training method that mimics the disentangled structure of the causal model.
On the Modularity of Hypernetworks
Besides, we show that for a structured target function, the overall number of trainable parameters in a hypernetwork is smaller by orders of magnitude than the number of trainable parameters of a standard neural network and an embedding method.
Molecule Property Prediction and Classification with Graph Hypernetworks
In this work, we demonstrate that the replacement of the underlying networks with hypernetworks leads to a boost in performance, obtaining state of the art results in various benchmarks.
On Random Kernels of Residual Architectures
We derive finite width and depth corrections for the Neural Tangent Kernel (NTK) of ResNets and DenseNets.
Structured GANs
We present Generative Adversarial Networks (GANs), in which the symmetric property of the generated images is controlled.
A Formal Approach to Explainability
We regard explanations as a blending of the input sample and the model's output and offer a few definitions that capture various desired properties of the function that generates these explanations.
A Sample Selection Approach for Universal Domain Adaptation
We study the problem of unsupervised domain adaption in the universal scenario, in which only some of the classes are shared between the source and target domains.
Unsupervised Learning of the Set of Local Maxima
Two functions are learned: (i) a set indicator c, which is a binary classifier, and (ii) a comparator function h that given two nearby samples, predicts which sample has the higher value of the unknown function v. Loss terms are used to ensure that all training samples x are a local maxima of v, according to h and satisfy c(x)=1.
Microvascular Dynamics from 4D Microscopy Using Temporal Segmentation
Recently developed methods for rapid continuous volumetric two-photon microscopy facilitate the observation of neuronal activity in hundreds of individual neurons and changes in blood flow in adjacent blood vessels across a large volume of living brain at unprecedented spatio-temporal resolution.
Emerging Disentanglement in Auto-Encoder Based Unsupervised Image Content Transfer
Thus, in the above example, we can create, for every person without glasses a version with the glasses observed in any face image.
Single Image Depth Estimation Trained via Depth from Defocus Cues
We evaluate our method on data derived from five common datasets for depth estimation and lightfield images, and present results that are on par with supervised methods on KITTI and Make3D datasets and outperform unsupervised learning approaches.
Ranked #61 on
Monocular Depth Estimation
on KITTI Eigen split
On the Convex Behavior of Deep Neural Networks in Relation to the Layers' Width
The Hessian of neural networks can be decomposed into a sum of two matrices: (i) the positive semidefinite generalized Gauss-Newton matrix G, and (ii) the matrix H containing negative eigenvalues.
OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clustering
We present a method for simultaneously learning, in an unsupervised manner, (i) a conditional image generator, (ii) foreground extraction and segmentation, (iii) clustering into a two-level class hierarchy, and (iv) object removal and background completion, all done without any use of annotation.
Supervised and Unsupervised Learning of Parameterized Color Enhancement
We treat the problem of color enhancement as an image translation task, which we tackle using both supervised and unsupervised learning.
Meta Decision Trees for Explainable Recommendation Systems
We build the trees by applying learned regression functions to obtain the decision rules as well as the values at the leaf nodes.
End to End Trainable Active Contours via Differentiable Rendering
We present an image segmentation method that iteratively evolves a polygon.
Spectra2pix: Generating Nanostructure Images from Spectra
Our model architecture is not limited to a closed set of nanostructure shapes, and can be trained for the design of any geometry.
Computational Ceramicology
One method relies on the shape of the fracture outline of a sherd; the other is based on decorative features.
Live Face De-Identification in Video
We propose a method for face de-identification that enables fully automatic video modification at high frame rates.
A Gated Hypernet Decoder for Polar Codes
Hypernetworks were recently shown to improve the performance of message passing algorithms for decoding error correcting codes.
Electric Analog Circuit Design with Hypernetworks and a Differential Simulator
A two-stage network is used, which first generates a chain of circuit components and then predicts their parameters.
MML: Maximal Multiverse Learning for Robust Fine-Tuning of Language Models
In this work, we present a method that leverages BERT's fine-tuning phase to its fullest, by applying an extensive number of parallel classifier heads, which are enforced to be orthogonal, while adaptively eliminating the weaker heads during training.
Adaptive and Iteratively Improving Recurrent Lateral Connections
The current leading computer vision models are typically feed forward neural models, in which the output of one computational block is passed to the next one sequentially.
The Effect of Residual Architecture on the Per-Layer Gradient of Deep Networks
A critical part of the training process of neural networks takes place in the very first gradient steps post initialization.
Variable Complexity in the Univariate and Multivariate Structural Causal Model
Comparing the reconstruction errors of the two autoencoders, one for each variable, is shown to perform well on the accepted benchmarks of the field.
Fast Search with Poor OCR
The indexing and searching of historical documents have garnered attention in recent years due to massive digitization efforts of important collections worldwide.
Optical Character Recognition
Optical Character Recognition (OCR)
Specifying Object Attributes and Relations in Interactive Scene Generation
We introduce a method for the generation of images from an input scene graph.
Ranked #4 on
Layout-to-Image Generation
on COCO-Stuff 64x64
Hyper-Graph-Network Decoders for Block Codes
Neural decoders were shown to outperform classical message passing techniques for short BCH codes.
Bidirectional One-Shot Unsupervised Domain Mapping
We study the problem of mapping between a domain $A$, in which there is a single training sample and a domain $B$, for which we have a richer training set.
Domain Intersection and Domain Difference
We present a method for recovering the shared content between two visual domains as well as the content that is unique to each domain.
Deep Meta Functionals for Shape Representation
We present a new method for 3D shape reconstruction from a single image, in which a deep neural network directly maps an image to a vector of network weights.
Unsupervised Microvascular Image Segmentation Using an Active Contours Mimicking Neural Network
We present a novel deep learning method for unsupervised segmentation of blood vessels.
Mask Based Unsupervised Content Transfer
We consider the problem of translating, in an unsupervised manner, between two domains where one contains some additional information compared to the other.
Conditional WGANs with Adaptive Gradient Balancing for Sparse MRI Reconstruction
Recent sparse MRI reconstruction models have used Deep Neural Networks (DNNs) to reconstruct relatively high-quality images from highly undersampled k-space data, enabling much faster MRI scanning.
Autoencoder-based Music Translation
We present a method for translating music across musical instruments and styles.
TTS Skins: Speaker Conversion via ASR
We present a fully convolutional wav-to-wav network for converting between speakers' voices, without relying on text.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Vid2Game: Controllable Characters Extracted from Real-World Videos
The second network maps the current pose, the new pose, and a given background, to an output frame.
Speech Denoising by Accumulating Per-Frequency Modeling Fluctuations
The method is completely unsupervised and only trains on the specific audio clip that is being denoised.
Unsupervised Singing Voice Conversion
The proposed network is not conditioned on the text or on the notes, and it directly converts the audio of one singer to the voice of another.
Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks
As Deep Neural Networks (DNNs) have demonstrated superhuman performance in a variety of fields, there is an increasing interest in understanding the complex internal mechanisms of DNNs.
Unsupervised Polyglot Text To Speech
We present a TTS neural network that is able to produce speech in multiple languages.
Semi-Supervised Monaural Singing Voice Separation With a Masking Network Trained on Synthetic Mixtures
We study the problem of semi-supervised singing voice separation, in which the training data contains a set of samples of mixed music (singing and instrumental) and an unmatched set of instrumental music.
Regularizing by the Variance of the Activations' Sample-Variances
Normalization techniques play an important role in supporting efficient and often more effective training of deep neural networks.
Automatic Program Synthesis of Long Programs with a Learned Garbage Collector
We consider the problem of generating automatic code given sample input-output pairs.
Visual Analogies between Atari Games for Studying Transfer Learning in RL
For example, given a video frame in the target game, we map it to an analogous state in the source game and then attempt to play using a trained policy learned for the source game.
Risk Bounds for Unsupervised Cross-Domain Mapping with IPMs
The recent empirical success of unsupervised cross-domain mapping algorithms, between two domains that share common characteristics, is not well-supported by theoretical justifications.
One-Shot Unsupervised Cross Domain Translation
Given a single image x from domain A and a set of images from domain B, our task is to generate the analogous of x in B.
NAM: Non-Adversarial Unsupervised Domain Mapping
NAM relies on a pre-trained generative model of the target domain, and aligns each source image with an image synthesized from the target domain, while jointly optimizing the domain mapping function.
Confidence Prediction for Lexicon-Free OCR
Having a reliable accuracy score is crucial for real world applications of OCR, since such systems are judged by the number of false readings.
A Universal Music Translation Network
We present a method for translating music across musical instruments, genres, and styles.
Unsupervised Correlation Analysis
Linking between two data sources is a basic building block in numerous computer vision problems.
Learning the Multiple Traveling Salesmen Problem with Permutation Invariant Pooling Networks
While there are optimal TSP solvers, as well as recent learning-based approaches, the generalization of the TSP to the Multiple Traveling Salesmen Problem is much less studied.
Fitting New Speakers Based on a Short Untranscribed Sample
Learning-based Text To Speech systems have the potential to generalize from one speaker to the next and thus require a relatively short sample of any new voice.
Non-Adversarial Unsupervised Word Translation
We present a novel method that first aligns the second moment of the word distributions of the two languages and then iteratively refines the alignment.
Identifying Analogies Across Domains
We further show that the cross-domain mapping task can be broken into two parts: domain alignment and learning the mapping function.
Estimating the Success of Unsupervised Image to Image Translation
While in supervised learning, the validation error is an unbiased estimator of the generalization (test) error and complexity-based generalization bounds are abundant, no such bounds exist for learning a mapping in an unsupervised way.
DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess
We present an end-to-end learning method for chess, relying on deep neural networks.
End-to-End Supervised Product Quantization for Image Search and Retrieval
To our knowledge, this is the first work to introduce a dictionary-based representation that is inspired by Product Quantization and which is learned end-to-end, and thus benefits from the supervised signal.
A Two-Step Disentanglement Method
First, the part of the data that is correlated with the labels is extracted by training a classifier.
The Role of Minimal Complexity Functions in Unsupervised Learning of Semantic Mappings
We discuss the feasibility of the following learning problem: given unmatched samples from two domains and nothing else, learn a mapping between the two, which preserves semantics.
