no code implementations • 4 Jun 2024 • Owen Dugan, Donato Manuel Jimenez Beneto, Charlotte Loh, Zhuo Chen, Rumen Dangovski, Marin Soljačić
Despite significant advancements in text generation and reasoning, Large Language Models (LLMs) still face challenges in accurately performing complex arithmetic operations.
1 code implementation • 31 May 2024 • Zhuo Chen, Rumen Dangovski, Charlotte Loh, Owen Dugan, Di Luo, Marin Soljačić
We propose Quantum-informed Tensor Adaptation (QuanTA), a novel, easy-to-implement, fine-tuning method with no inference overhead for large-scale pre-trained language models.
1 code implementation • 30 Nov 2023 • Viggo Moro, Charlotte Loh, Rumen Dangovski, Ali Ghorashi, Andrew Ma, Zhuo Chen, Samuel Kim, Peter Y. Lu, Thomas Christensen, Marin Soljačić
Artificial intelligence is transforming computational materials science, improving the prediction of material properties, and accelerating the discovery of novel materials.
no code implementations • 2 Apr 2023 • Ligong Han, Seungwook Han, Shivchander Sudalairaj, Charlotte Loh, Rumen Dangovski, Fei Deng, Pulkit Agrawal, Dimitris Metaxas, Leonid Karlinsky, Tsui-Wei Weng, Akash Srivastava
Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned.
no code implementations • 20 Mar 2023 • Adriano Hernandez, Rumen Dangovski, Peter Y. Lu, Marin Soljacic
Model stitching (Lenc & Vedaldi 2015) is a compelling methodology to compare different neural network representations, because it allows us to measure to what degree they may be interchanged.
1 code implementation • 4 Mar 2023 • Charlotte Loh, Seungwook Han, Shivchander Sudalairaj, Rumen Dangovski, Kai Xu, Florian Wenzel, Marin Soljacic, Akash Srivastava
In this work, we present Multi-Symmetry Ensembles (MSE), a framework for constructing diverse ensembles by capturing the multiplicity of hypotheses along symmetry axes, which explore the hypothesis space beyond stochastic perturbations of model weights and hyperparameters.
no code implementations • 23 Feb 2023 • Owen Dugan, Peter Y. Lu, Rumen Dangovski, Di Luo, Marin Soljačić
Studying the dynamics of open quantum systems can enable breakthroughs both in fundamental physics and applications to quantum engineering and quantum computation.
1 code implementation • NeurIPS 2023 • Di Luo, Jiayu Shen, Rumen Dangovski, Marin Soljačić
Quantum optimization, a key application of quantum computing, has traditionally been stymied by the linearly increasing complexity of gradient calculations with an increasing number of parameters.
no code implementations • 10 Oct 2022 • Charlotte Loh, Rumen Dangovski, Shivchander Sudalairaj, Seungwook Han, Ligong Han, Leonid Karlinsky, Marin Soljacic, Akash Srivastava
State-of-the-art (SOTA) semi-supervised learning (SSL) methods have been highly successful in leveraging a mix of labeled and unlabeled data by combining techniques of consistency regularization and pseudo-labeling.
1 code implementation • 2 Oct 2022 • Julia Balla, Sihao Huang, Owen Dugan, Rumen Dangovski, Marin Soljacic
In social science, formal and quantitative models, such as ones describing economic growth and collective action, are used to formulate mechanistic explanations, provide predictions, and uncover questions about observed phenomena.
1 code implementation • 31 Aug 2022 • Peter Y. Lu, Rumen Dangovski, Marin Soljačić
We test this new approach on a variety of physical systems and demonstrate that our method is able to both identify the number of conserved quantities and extract their values.
1 code implementation • NAACL 2022 • Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljačić, Shang-Wen Li, Wen-tau Yih, Yoon Kim, James Glass
We propose DiffCSE, an unsupervised contrastive learning framework for learning sentence embeddings.
Ranked #13 on
Semantic Textual Similarity
on STS16
no code implementations • 22 Dec 2021 • Ileana Rugina, Rumen Dangovski, Mark Veillette, Pooya Khorrami, Brian Cheung, Olga Simek, Marin Soljačić
In recent years, emerging fields such as meta-learning or self-supervised learning have been closing the gap between proof-of-concept results and real-life applications of machine learning by extending deep-learning to the semi-supervised and few-shot domains.
2 code implementations • 28 Oct 2021 • Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, Marin Soljačić
In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge.
1 code implementation • 15 Oct 2021 • Charlotte Loh, Thomas Christensen, Rumen Dangovski, Samuel Kim, Marin Soljacic
Deep learning techniques have been increasingly applied to the natural sciences, e. g., for property prediction and optimization or material discovery.
no code implementations • ICLR 2022 • Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, Marin Soljacic
In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge.
2 code implementations • 20 Nov 2020 • Ileana Rugina, Rumen Dangovski, Li Jing, Preslav Nakov, Marin Soljačić
Attention mechanisms play a crucial role in the neural revolution of Natural Language Processing (NLP).
no code implementations • EMNLP 2020 • Matthew Khoury, Rumen Dangovski, Longwu Ou, Preslav Nakov, Yichen Shen, Li Jing
To address this issue, we propose a novel vector-vector-matrix architecture (VVMA), which greatly reduces the latency at inference time for NMT.
1 code implementation • 18 Jul 2020 • Guillem Ramírez, Rumen Dangovski, Preslav Nakov, Marin Soljačić
We believe that our rethinking of the Wasserstein-Procrustes problem could enable further research, thus helping to develop better algorithms for aligning word embeddings across languages.
no code implementations • 17 Jul 2020 • Evan Vogelbaum, Rumen Dangovski, Li Jing, Marin Soljačić
We propose the implementation of contextualizers, which are generalizable prototypes that adapt to given examples and play a larger role in classification for gradient-based models.
4 code implementations • 16 Jul 2020 • Owen Dugan, Rumen Dangovski, Allan Costa, Samuel Kim, Pawan Goyal, Joseph Jacobson, Marin Soljačić
Neural networks' expressiveness comes at the cost of complex, black-box models that often extrapolate poorly beyond the domain of the training dataset, conflicting with the goal of finding compact analytic expressions to describe scientific data.
no code implementations • TACL 2019 • Rumen Dangovski, Li Jing, Preslav Nakov, Mi{\'c}o Tatalovi{\'c}, Marin Solja{\v{c}}i{\'c}
Stacking long short-term memory (LSTM) cells or gated recurrent units (GRUs) as part of a recurrent neural network (RNN) has become a standard approach to solving a number of tasks ranging from language modeling to text summarization.
no code implementations • 28 Nov 2018 • Li Jing, Rumen Dangovski, Marin Soljacic
We present a logarithmic-scale efficient convolutional neural network architecture for edge devices, named WaveletNet.
2 code implementations • ICLR 2018 • Rumen Dangovski, Li Jing, Marin Soljacic
We evaluate our model on synthetic memorization, question answering and language modeling tasks.
Ranked #5 on
Question Answering
on bAbi
(Accuracy (trained on 1k) metric)