Affine Coupling

Introduced by Dinh et al. in NICE: Non-linear Independent Components Estimation

Affine Coupling is a method for implementing a normalizing flow (where we stack a sequence of invertible bijective transformation functions). Affine coupling is one of these bijective transformation functions. Specifically, it is an example of a reversible transformation where the forward function, the reverse function and the log-determinant are computationally efficient. For the forward function, we split the input dimension into two parts:

$$ \mathbf{x}_{a}, \mathbf{x}_{b} = \text{split}\left(\mathbf{x}\right) $$

The second part stays the same $\mathbf{x}_{b} = \mathbf{y}_{b}$, while the first part $\mathbf{x}_{a}$ undergoes an affine transformation, where the parameters for this transformation are learnt using the second part $\mathbf{x}_{b}$ being put through a neural network. Together we have:

$$ \left(\log{\mathbf{s}, \mathbf{t}}\right) = \text{NN}\left(\mathbf{x}_{b}\right) $$

$$ \mathbf{s} = \exp\left(\log{\mathbf{s}}\right) $$

$$ \mathbf{y}_{a} = \mathbf{s} \odot \mathbf{x}_{a} + \mathbf{t} $$

$$ \mathbf{y}_{b} = \mathbf{x}_{b} $$

$$ \mathbf{y} = \text{concat}\left(\mathbf{y}_{a}, \mathbf{y}_{b}\right) $$

Image: GLOW

Source: NICE: Non-linear Independent Components Estimation

Latest Papers

PAPER DATE
Representational aspects of depth and conditioning in normalizing flows
Frederic KoehlerViraj MehtaAndrej Risteski
2020-10-02
An Algorithm to Attack Neural Network Encoder-based Out-Of-Distribution Sample Detector
Liang LiangLinhai MaLinchen QianJiasong Chen
2020-09-17
Same Same But DifferNet: Semi-Supervised Defect Detection with Normalizing Flows
| Marco RudolphBastian WandtBodo Rosenhahn
2020-08-28
Ultrasound-based Articulatory-to-Acoustic Mapping with WaveGlow Speech Synthesis
Tamás Gábor CsapóCsaba ZainkóLászló TóthGábor GosztolyaAlexandra Markó
2020-08-06
SeismoGlow -- Data augmentation for the class imbalance problem
Ruy Luiz MilidiúLuis Felipe Müller
2020-07-23
Discrete Point Flow Networks for Efficient Point Cloud Generation
| Roman KlokovEdmond BoyerJakob Verbeek
2020-07-20
Generative Flows with Matrix Exponential
| Changyi XiaoLigang Liu
2020-07-19
Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators
Takeshi TeshimaIsao IshikawaKoichi TojoKenta OonoMasahiro IkedaMasashi Sugiyama
2020-06-20
MoFlow: An Invertible Flow Model for Generating Molecular Graphs
Chengxi ZangFei Wang
2020-06-17
A comparison of Vietnamese Statistical Parametric Speech Synthesis Systems
Huy Kinh PhanViet Lam PhungTuan Anh DinhBao Quoc Nguyen
2020-05-26
Variational Autoencoders with Normalizing Flow Decoders
Rogan MorrowWei-Chen Chiu
2020-04-12
Modeling Contrary-to-Duty with CP-nets
Roberta CalegariAndrea LoreggiaEmiliano LoriniFrancesca RossiGiovanni Sartor
2020-03-23
Gaussianization Flows
Chenlin MengYang SongJiaming SongStefano Ermon
2020-03-04
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion
| Julian ChibaneThiemo AlldieckGerard Pons-Moll
2020-03-03
Emosaic: Visualizing Affective Content of Text at Varying Granularity
Philipp GeuderMarie Claire LeidingerMartin von LupinMarian DörkTobias Schröder
2020-02-24
Schoenberg-Rao distances: Entropy-based and geometry-aware statistical Hilbert distances
| Gaëtan HadjeresFrank Nielsen
2020-02-19
Multimodal Controller for Generative Models
Enmao DiaoJie DingVahid Tarokh
2020-02-07
Efficient Probabilistic Logic Reasoning with Graph Neural Networks
| Yuyu ZhangXinshi ChenYuan YangArun RamamurthyBo LiYuan QiLe Song
2020-01-29
Fair Transfer of Multiple Style Attributes in Text
Karan DabasNishtha MadanVijay AryaSameep MehtaGautam SinghTanmoy Chakraborty
2020-01-18
SqueezeWave: Extremely Lightweight Vocoders for On-device Speech Synthesis
Bohan ZhaiTianren GaoFlora XueDaniel RothchildBichen WuJoseph E. GonzalezKurt Keutzer
2020-01-16
Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)
Peter SorrensonCarsten RotherUllrich Köthe
2020-01-14
Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Ocurring in Data
Anonymous
2020-01-01
Disentanglement through Nonlinear ICA with General Incompressible-flow Networks (GIN)
Anonymous
2020-01-01
GENERALIZATION GUARANTEES FOR NEURAL NETS VIA HARNESSING THE LOW-RANKNESS OF JACOBIAN
Anonymous
2020-01-01
Training Deep Neural Networks with Partially Adaptive Momentum
Anonymous
2020-01-01
Word embedding re-examined: is the symmetrical factorization optimal?
Anonymous
2020-01-01
Shallow VAEs with RealNVP Prior Can Perform as Well as Deep Hierarchical VAEs
Anonymous
2020-01-01
Understanding l4-based Dictionary Learning: Interpretation, Stability, and Robustness
| Anonymous
2020-01-01
Avoiding Spurious Local Minima in Deep Quadratic Networks
Abbas KazemipourBrett W. LarsenShaul Druckmann
2019-12-31
Probing the phonetic and phonological knowledge of tones in Mandarin TTS models
| Jian Zhu
2019-12-23
WaveFlow: A Compact Flow-based Model for Raw Audio
| Wei PingKainan PengKexin ZhaoZhao Song
2019-12-03
How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?
Zixiang ChenYuan CaoDifan ZouQuanquan Gu
2019-11-27
Adversarial Robustness of Flow-Based Generative Models
Phillip PopeYogesh BalajiSoheil Feizi
2019-11-20
Any-Precision Deep Neural Networks
| Haichao YuHaoxiang LiHonghui ShiThomas S. HuangGang Hua
2019-11-17
A Configuration-Space Decomposition Scheme for Learning-based Collision Checking
Yiheng HanWang ZhaoJia PanZipeng YeRan YiYong-Jin Liu
2019-11-17
Speaker independence of neural vocoders and their effect on parametric resynthesis speech enhancement
Soumi MaitiMichael I Mandel
2019-11-14
Deep Generative Models Strike Back! Improving Understanding and Evaluation in Light of Unmet Expectations for OoD Data
John JustSambuddha Ghosal
2019-11-12
Incentive-aware Contextual Pricing with Non-parametric Market Noise
Negin GolrezaeiPatrick JailletJason Cheuk Nam Liang
2019-11-08
Small-GAN: Speeding Up GAN Training Using Core-sets
Samarth SinhaHan ZhangAnirudh GoyalYoshua BengioHugo LarochelleAugustus Odena
2019-10-29
Transferring neural speech waveform synthesizers to musical instrument sounds generation
Yi ZhaoXin WangLauri JuvelaJunichi Yamagishi
2019-10-27
Semi-Decentralized Coordinated Online Learning for Continuous Games with Coupled Constraints via Augmented Lagrangian
Ezra TampubolonHolger Boche
2019-10-21
An Unbiased Risk Estimator for Learning with Augmented Classes
Yu-Jie ZhangPeng ZhaoZhi-Hua Zhou
2019-10-21
Label-Conditioned Next-Frame Video Generation with Neural Flows
David Donahue
2019-10-16
Adaptive Exploration in Linear Contextual Bandit
Botao HaoTor LattimoreCsaba Szepesvari
2019-10-15
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks
Yu BaiJason D. Lee
2019-10-03
Tails of Lipschitz Triangular Flows
Priyank JainiIvan KobyzevYaoliang YuMarcus Brubaker
2019-07-10
Interpolated Spectral NGram Language Models
Ariadna QuattoniXavier Carreras
2019-07-01
Multi-Category Fairness in Sponsored Search Auctions
Shuchi ChawlaChristina IlventoMeena Jagadeesan
2019-06-20
Parametric Resynthesis with neural vocoders
| Soumi MaitiMichael I Mandel
2019-06-16
Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian
Samet OymakZalan FabianMingchen LiMahdi Soltanolkotabi
2019-06-12
Evaluating the Robustness of Nearest Neighbor Classifiers: A Primal-Dual Perspective
Lu WangXuanqing LiuJinfeng YiZhi-Hua ZhouCho-Jui Hsieh
2019-06-10
Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
Xiangyi ChenTiancong ChenHaoran SunZhiwei Steven WuMingyi Hong
2019-06-04
CIC at SemEval-2019 Task 5: Simple Yet Very Efficient Approach to Hate Speech Detection, Aggressive Behavior Detection, and Target Classification in Twitter
Iqra AmeerMuhammad Hammad Fahim SiddiquiGrigori SidorovAlex Gelbukher
2019-06-01
Structured Output Learning with Conditional Generative Flows
You LuBert Huang
2019-05-30
Discrete Flows: Invertible Generative Models of Discrete Data
| Dustin TranKeyon VafaKumar Krishna AgrawalLaurent DinhBen Poole
2019-05-24
Generative Flow via Invertible nxn Convolution
Thanh-Dat TruongKhoa LuuChi Nhan DuongNgan LeMinh-Triet Tran
2019-05-24
X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks
Xingde YingHeng GuoKai MaJian WuZhengxin WengYefeng Zheng
2019-05-16
DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
Hanlin TangXiangru LianChen YuTong ZhangJi Liu
2019-05-15
Deep Ordinal Reinforcement Learning
Alexander ZapTobias JoppenJohannes Fürnkranz
2019-05-06
$A^*$ sampling with probability matching
Yichi ZhouJun Zhu
2019-05-01
Padam: Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
Jinghui ChenQuanquan Gu
2019-05-01
Learning metrics for persistence-based summaries and applications for graph classification
Qi ZhaoYusu Wang
2019-04-27
Learning metrics for persistence-based summaries and applications for graph classification
Qi ZhaoYusu Wang
2019-04-27
A Skeleton-bridged Deep Learning Approach for Generating Meshesof Complex Topologies from Single RGB Image
| Jiapeng TangXiaoguang HanJunyi PanKui JiaXin Tong
2019-04-10
WaveCycleGAN2: Time-domain Neural Post-filter for Speech Waveform Generation
Kou TanakaHirokazu KameokaTakuhiro KanekoNobukatsu Hojo
2019-04-05
Conditional Adversarial Generative Flow for Controllable Image Synthesis
Rui LiuYu LiuXinyu GongXiaogang WangHongsheng Li
2019-04-03
A Skeleton-bridged Deep Learning Approach for Generating Meshes of Complex Topologies from Single RGB Images
| Jiapeng TangXiaoguang HanJunyi PanKui JiaXin Tong
2019-03-12
TzK: Flow-Based Conditional Generative Model
Micha LivneDavid Fleet
2019-02-05
Emerging Convolutions for Generative Normalizing Flows
| Emiel HoogeboomRianne van den BergMax Welling
2019-01-30
Throttling Malware Families in 2D
Mohamed NassarHaidar Safa
2019-01-29
How is Your Mood When Writing Sexist tweets? Detecting the Emotion Type and Intensity of Emotion Using Natural Language Processing Techniques
Sima SharifiradBorna JafarpourStan Matwin
2019-01-28
Semi-supervised learning in unbalanced and heterogeneous networks
Ting LiNingchen YingXianshi YuBin-Yi Jing
2019-01-07
Sparse Learning in reproducing kernel Hilbert space
Xin HeJunhui Wang
2019-01-03
On the Benefit of Width for Neural Networks: Disappearance of Bad Basins
Dawei LiTian DingRuoyu Sun
2018-12-28
Adversarial Defense of Image Classification Using a Variational Auto-Encoder
| Yi LuoHenry Pfister
2018-12-07
Feature selection with optimal coordinate ascent (OCA)
David SaltielEric Benhamou
2018-11-29
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
Difan ZouYuan CaoDongruo ZhouQuanquan Gu
2018-11-21
State Aggregation Learning from Markov Transition Data
Yaqi DuanZheng Tracy KeMengdi Wang
2018-11-06
Task Embedded Coordinate Update: A Realizable Framework for Multivariate Non-convex Optimization
Yiyang WangRisheng LiuLong MaXiaoliang Song
2018-11-05
WaveGlow: A Flow-based Generative Network for Speech Synthesis
| Ryan PrengerRafael ValleBryan Catanzaro
2018-10-31
Optimizing Waiting Thresholds Within A State Machine
Rohit PandeyYifan ChangCameron WhiteGaurav JagtianiAerin Young KimGil Lapid ShafririSathya Singh
2018-10-08
Online Dictionary Learning for Approximate Archetypal Analysis
Jieru MeiChunyu WangWenjun Zeng
2018-09-01
Making \emph{ordinary least squares} linear classfiers more robust
Babatunde M. Ayeni
2018-08-28
IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis
| Huaibo HuangZhihang LiRan HeZhenan SunTieniu Tan
2018-07-17
Payoff Control in the Iterated Prisoner's Dilemma
Dong HaoKai LiTao Zhou
2018-07-17
Glow: Generative Flow with Invertible 1x1 Convolutions
| Diederik P. KingmaPrafulla Dhariwal
2018-07-09
ReCoNet: Real-time Coherent Video Style Transfer Network
| Chang GaoDerun GuFangjun ZhangYizhou Yu
2018-07-03
Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval and Matrix Completion
Cong MaKaizheng WangYuejie ChiYuxin Chen
2018-07-01
Deep Neural Decision Trees
| Yongxin YangIrene Garcia MorilloTimothy M. Hospedales
2018-06-19
Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networks
| Jinghui ChenDongruo ZhouYiqi TangZiyan YangYuan CaoQuanquan Gu
2018-06-18
What Knowledge is Needed to Solve the RTE5 Textual Entailment Challenge?
Peter Clark
2018-06-10
Online Deep Metric Learning
Wenbin LiJing HuoYinghuan ShiYang GaoLei WangJiebo Luo
2018-05-15
Robust Blind Deconvolution via Mirror Descent
| Sathya N. RaviRonak MehtaVikas Singh
2018-03-21
Large Margin Deep Networks for Classification
| Gamaleldin F. ElsayedDilip KrishnanHossein MobahiKevin ReganSamy Bengio
2018-03-15
An Alternative View: When Does SGD Escape Local Minima?
Robert KleinbergYuanzhi LiYang Yuan
2018-02-17
Persistence Fisher Kernel: A Riemannian Manifold Kernel for Persistence Diagrams
| Tam LeMakoto Yamada
2018-02-10
Reparametrization of COM-Poisson Regression Models with Applications in the Analysis of Experimental Data
Eduardo E. Ribeiro JrWalmes M. ZevianiWagner H. BonatClarice G. B. DemétrioJohn Hinde
2018-01-29
Evaluation of generative networks through their data augmentation capacity
Timothée LesortFlorian BordesJean-Francois GoudouDavid Filliat
2018-01-01
A Spectral Approach to Generalization and Optimization in Neural Networks
Farzan FarniaJesse ZhangDavid Tse
2018-01-01
CNNs are Globally Optimal Given Multi-Layer Support
Chen HuangChen KongSimon Lucey
2017-12-07
On Frank-Wolfe and Equilibrium Computation
Jacob D. AbernethyJun-Kun Wang
2017-12-01
State Space LSTM Models with Particle MCMC Inference
Xun ZhengManzil ZaheerAmr AhmedYuan WangEric P XingAlexander J Smola
2017-11-30
Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval, Matrix Completion, and Blind Deconvolution
Cong MaKaizheng WangYuejie ChiYuxin Chen
2017-11-28
Proximal Alternating Direction Network: A Globally Converged Deep Unrolling Framework
Risheng LiuXin FanShichao ChengXiangyu WangZhongxuan Luo
2017-11-21
Wasserstein Auto-Encoders
| Ilya TolstikhinOlivier BousquetSylvain GellyBernhard Schoelkopf
2017-11-05
Analysis of planar ornament patterns via motif asymmetry assumption and local connections
Venera AdanovaSibel Tari
2017-10-12
Fast and Strong Convergence of Online Learning Algorithms
Zheng-Chu GuoLei Shi
2017-10-10
Generative Adversarial Mapping Networks
Jianbo GuoGuangxiang ZhuJian Li
2017-09-28
Order-Preserving Abstractive Summarization for Spoken Content Based on Connectionist Temporal Classification
Bo-Ru LuFrank ShyuYun-Nung ChenHung-Yi LeeLin-shan Lee
2017-09-16
Deep vs. Diverse Architectures for Classification Problems
Colleen M. Farrelly
2017-08-21
Geometric Enclosing Networks
Trung LeHung VuTu Dinh NguyenDinh Phung
2017-08-16
Follow the Moving Leader in Deep Learning
Shuai ZhengJames T. Kwok
2017-08-01
Restricted Eigenvalue from Stable Rank with Applications to Sparse Linear Regression
Shiva Prasad KasiviswanathanMark Rudelson
2017-07-25
Big Data Regression Using Tree Based Segmentation
Rajiv SambasivanSourish Das
2017-07-24
Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
John BradshawAlexander G. de G. MatthewsZoubin Ghahramani
2017-07-08
Maintaining cooperation in complex social dilemmas using deep reinforcement learning
Adam LererAlexander Peysakhovich
2017-07-04
Convergence Analysis of Two-layer Neural Networks with ReLU Activation
Yuanzhi LiYang Yuan
2017-05-28
MUTAN: Multimodal Tucker Fusion for Visual Question Answering
| Hedi Ben-younesRémi CadeneMatthieu CordNicolas Thome
2017-05-18
Generalized Ideals and Co-Granular Rough Sets
A Mani
2017-04-18
Identification of Multiword Expressions for Latvian and Lithuanian: Hybrid Approach
MJustina ravickait{\.e}Tomas Krilavi{\v{c}}ius
2017-04-01
On architectural choices in deep learning: From network structure to gradient convergence and parameter estimation
Vamsi K IthapuSathya N RaviVikas Singh
2017-02-28
Wages of wins: could an amateur make money from match outcome predictions?
Albrecht Zimmermann
2017-02-17
Bayesian System Identification based on Hierarchical Sparse Bayesian Learning and Gibbs Sampling with Application to Structural Damage Assessment
Yong HuangJames L. BeckHui Li
2017-01-13
Factor-Adjusted Regularized Model Selection
Jianqing FanYuan KeKaizheng Wang
2016-12-27
Saliency Driven Image Manipulation
| Roey MechrezEli ShechtmanLihi Zelnik-Manor
2016-12-07
Vectors or Graphs? On Differences of Representations for Distributional Semantic Models
Chris Biemann
2016-12-01
Optimizing Codes for Source Separation in Color Image Demosaicing and Compressive Video Recovery
Alankar KotwalAjit Rajwade
2016-09-07
Hybrid Jacobian and Gauss-Seidel proximal block coordinate update methods for linearly constrained convex programming
Yangyang Xu
2016-08-13
Density estimation using Real NVP
| Laurent DinhJascha Sohl-DicksteinSamy Bengio
2016-05-27
Interaction pursuit in high-dimensional multi-response regression via distance correlation
Yinfei KongDaoji LiYingying FanJinchi Lv
2016-05-11
The constrained Dantzig selector with enhanced consistency
Yinfei KongZemin ZhengJinchi Lv
2016-05-11
Makeup like a superstar: Deep Localized Makeup Transfer Network
Si LiuXinyu OuRuihe QianWei WangXiaochun Cao
2016-04-25
A strengthening of rational closure in DLs: reasoning about multiple aspects
Valentina Gliozzi
2016-04-01
Identity Mappings in Deep Residual Networks
| Kaiming HeXiangyu ZhangShaoqing RenJian Sun
2016-03-16
Local-likelihood transformation kernel density estimation for positive random variables
Gery GeenensCraig Wang
2016-02-15
Universal Dependency Analysis
Hoang-Vu NguyenJilles Vreeken
2015-10-28
Scalable Bayesian Non-Negative Tensor Factorization for Massive Count Data
Changwei HuPiyush RaiChangyou ChenMatthew HardingLawrence Carin
2015-08-18
A MAP approach for $\ell_q$-norm regularized sparse parameter estimation using the EM algorithm
Rodrigo CarvajalJuan C. AgüeroBoris I. GodoyDimitrios Katselis
2015-08-05
An SVM-like Approach for Expectile Regression
Muhammad FarooqIngo Steinwart
2015-07-14
Supervised Learning of Semantics-Preserving Hash via Deep Convolutional Neural Networks
| Huei-Fang YangKevin LinChu-Song Chen
2015-07-01
On Distributive Subalgebras of Qualitative Spatial and Temporal Calculi
Zhiguo LongSanjiang Li
2015-06-01
Provably Correct Algorithms for Matrix Column Subset Selection with Selectively Sampled Data
Yining WangAarti Singh
2015-05-17
Fast Guided Filter
| Kaiming HeJian Sun
2015-05-05
PEWA: Patch-based Exponentially Weighted Aggregation for image denoising
Charles Kervrann
2014-12-01
NICE: Non-linear Independent Components Estimation
| Laurent DinhDavid KruegerYoshua Bengio
2014-10-30

Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories