Layer-wise Representation Fusion for Compositional Generalization

no code implementations • 20 Jul 2023 • Yafang Zheng, Lei Lin, Shuangtao Li, Yuxuan Yuan, Zhaohong Lai, Shan Liu, Biao Fu, Yidong Chen, Xiaodong Shi

Inspired by this, we propose LRF, a novel \textbf{L}ayer-wise \textbf{R}epresentation \textbf{F}usion framework for CG, which learns to fuse previous layers' information back into the encoding and decoding process effectively through introducing a \emph{fuse-attention module} at each encoder and decoder layer.

Decoder

Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalization

no code implementations • 20 May 2023 • Lei Lin, Shuangtao Li, Yafang Zheng, Biao Fu, Shan Liu, Yidong Chen, Xiaodong Shi

There is mounting evidence that one of the reasons hindering CG is the representation of the encoder uppermost layer is entangled, i. e., the syntactic and semantic representations of sequences are entangled.

Decoder