Transformers Get Stable: An End-to-End Signal Propagation Theory for Language Models

1 code implementation • 14 Mar 2024 • Akhil Kedia, Mohd Abbas Zaidi, Sushil Khyalia, Jungho Jung, Harshith Goka, Haejun Lee

In spite of their huge success, transformer models remain difficult to scale in depth.

Image Classification Language Modelling +1

Hole-robust Wireframe Detection

1 code implementation • 30 Nov 2021 • Naejin Kong, Kiwoong Park, Harshith Goka

In addition, we first time combine GAN in the model to let the model better predict underlying scene structure even beyond large holes.

Resolution-robust Large Mask Inpainting with Fourier Convolutions

7 code implementations • 15 Sep 2021 • Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, Victor Lempitsky

We find that one of the main reasons for that is the lack of an effective receptive field in both the inpainting network and the loss function.

Image Inpainting Seeing Beyond the Visible

An Analysis of Frame-skipping in Reinforcement Learning

no code implementations • 7 Feb 2021 • Shivaram Kalyanakrishnan, Siddharth Aravindan, Vishwajeet Bagdawat, Varun Bhatt, Harshith Goka, Archit Gupta, Kalpesh Krishna, Vihari Piratla

In this paper, we investigate the role of the parameter $d$ in RL; $d$ is called the "frame-skip" parameter, since states in the Atari domain are images.

Decision Making reinforcement-learning +1