Reversible Residual Blocks are skipconnection blocks that learn reversible residual functions with reference to the layer inputs. It is proposed as part of the RevNet CNN architecture. Units in each layer are partitioned into two groups, denoted $x_{1}$ and $x_{2}$; the authors find what works best is partitioning the channels. Each reversible block takes inputs $\left(x_{1}, x_{2}\right)$ and produces outputs $\left(y_{1}, y_{2}\right)$ according to the following additive coupling rules – inspired by the transformation in NICE (nonlinear independent components estimation) – and residual functions $F$ and $G$ analogous to those in standard ResNets:
$$y_{1} = x_{1} + F\left(x_{2}\right)$$ $$y_{2} = x_{2} + G\left(y_{1}\right)$$
Each layer’s activations can be reconstructed from the next layer’s activations as follows:
$$ x_{2} = y_{2} − G\left(y_{1}\right)$$ $$ x_{1} = y_{1} − F\left(x_{2}\right)$$
Source: The Reversible Residual Network: Backpropagation Without Storing ActivationsPaper  Code  Results  Date  Stars 

Task  Papers  Share 

Language Modelling  2  5.56% 
Time Series  2  5.56% 
Time Series Forecasting  2  5.56% 
Machine Translation  2  5.56% 
Image Classification  2  5.56% 
Image Generation  2  5.56% 
Reading Comprehension  1  2.78% 
Image Restoration  1  2.78% 
BIGbench Machine Learning  1  2.78% 
Component  Type 


1x1 Convolution

Convolutions  
Batch Normalization

Normalization  (optional) 
Convolution

Convolutions  
ReLU

Activation Functions  (optional) 