INDUS: Effective and Efficient Language Models for Scientific Applications

no code implementations • 17 May 2024 • Bishwaranjan Bhattacharjee, Aashka Trivedi, Masayasu Muraoka, Muthukumaran Ramasubramanian, Takuma Udagawa, Iksha Gurung, Nishan Pantha, Rong Zhang, Bharath Dandala, Rahul Ramachandran, Manil Maskey, Kaylin Bugbee, Mike Little, Elizabeth Fancher, Irina Gerasimov, Armin Mehrabian, Lauren Sanders, Sylvain Costes, Sergi Blanco-Cuaresma, Kelly Lockhart, Thomas Allen, Felix Grezes, Megan Ansdell, Alberto Accomazzi, Yousef El-Kurdi, Davis Wertheimer, Birgit Pfitzmann, Cesar Berrospi Ramis, Michele Dolfi, Rafael Teixeira de Lima, Panagiotis Vagenas, S. Karthik Mukkavilli, Peter Staar, Sanaz Vahidinia, Ryan McGranaghan, Tsendgar Lee

The suite of models include: (1) an encoder model trained using domain-specific vocabulary and corpora to address NLP tasks, (2) a contrastive-learning based text embedding model trained using a diverse set of datasets to address information retrieval tasks and (3) smaller versions of these models created using knowledge distillation for applications which have latency or resource constraints.

Contrastive Learning Information Retrieval +4

Deep Learning for Space Weather Prediction: Bridging the Gap between Heliophysics Data and Theory

no code implementations • 27 Dec 2022 • John C. Dorelli, Chris Bard, Thomas Y. Chen, Daniel da Silva, Luiz Fernando Guides dos Santos, Jack Ireland, Michael Kirk, Ryan McGranaghan, Ayris Narock, Teresa Nieves-Chinchilla, Marilia Samara, Menelaos Sarantos, Pete Schuck, Barbara Thompson

Traditionally, data analysis and theory have been viewed as separate disciplines, each feeding into fundamentally different types of models.

Towards a Modular Ontology for Space Weather Research

no code implementations • 25 Sep 2020 • Cogan Shimizu, Ryan McGranaghan, Aaron Eberhart, Adam C. Kellerman

The interactions between the Sun, interplanetary space, near Earth space environment, the Earth's surface, and the power grid are, perhaps unsurprisingly, very complicated.

Data Integration

Machine Learning in Heliophysics and Space Weather Forecasting: A White Paper of Findings and Recommendations

no code implementations • 22 Jun 2020 • Gelu Nita, Manolis Georgoulis, Irina Kitiashvili, Viacheslav Sadykov, Enrico Camporeale, Alexander Kosovichev, Haimin Wang, Vincent Oria, Jason Wang, Rafal Angryk, Berkay Aydin, Azim Ahmadzadeh, Xiaoli Bai, Timothy Bastian, Soukaina Filali Boubrahimi, Bin Chen, Alisdair Davey, Sheldon Fereira, Gregory Fleishman, Dale Gary, Andrew Gerrard, Gregory Hellbourg, Katherine Herbert, Jack Ireland, Egor Illarionov, Natsuha Kuroda, Qin Li, Chang Liu, Yuexin Liu, Hyomin Kim, Dustin Kempton, Ruizhe Ma, Petrus Martens, Ryan McGranaghan, Edward Semones, John Stefan, Andrey Stejko, Yaireska Collado-Vega, Meiqi Wang, Yan Xu, Sijie Yu

The authors of this white paper met on 16-17 January 2020 at the New Jersey Institute of Technology, Newark, NJ, for a 2-day workshop that brought together a group of heliophysicists, data providers, expert modelers, and computer/data scientists.

BIG-bench Machine Learning Weather Forecasting