Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models

no code implementations • 30 Aug 2023 • Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, Alham Fikri Aji, Zhiqiang Shen, Zhengzhong Liu, Natalia Vassilieva, Joel Hestness, Andy Hock, Andrew Feldman, Jonathan Lee, Andrew Jackson, Hector Xuguang Ren, Preslav Nakov, Timothy Baldwin, Eric Xing

We release two open versions of the model -- the foundation Jais model, and an instruction-tuned Jais-chat variant -- with the aim of promoting research on Arabic LLMs.

Fine-tuning Image Transformers using Learnable Memory

1 code implementation • CVPR 2022 • Mark Sandler, Andrey Zhmoginov, Max Vladymyrov, Andrew Jackson

In this paper we propose augmenting Vision Transformer models with learnable memory tokens.

Meta-Learning Bidirectional Update Rules

1 code implementation • 10 Apr 2021 • Mark Sandler, Max Vladymyrov, Andrey Zhmoginov, Nolan Miller, Andrew Jackson, Tom Madams, Blaise Aguera y Arcas

We show that classical gradient-based backpropagation in neural networks can be seen as a special case of a two-state network where one state is used for activations and another for gradients, with update rules derived from the chain rule.

Meta-Learning

Large-scale vortices and zonal flows in spherical rotating convection

no code implementations • 22 Dec 2020 • Yufeng Lin, Andrew Jackson

At Prandtl number of unity, as the thermal forcing (measured by the Rayleigh number) is increased above the value for the onset of convection, we find a relaxation oscillation regime, followed by a geostrophic turbulence regime.

Fluid Dynamics Solar and Stellar Astrophysics