MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining

1 code implementation • NeurIPS 2023 • Jacob Portes, Alex Trott, Sam Havens, Daniel King, Abhinav Venigalla, Moin Nadeem, Nikhil Sardana, Daya Khudia, Jonathan Frankle

Here, we introduce MosaicBERT, a BERT-style encoder architecture and training recipe that is empirically optimized for fast pretraining.

Language Modelling Masked Language Modeling

The Semantic Scholar Open Data Platform

1 code implementation • 24 Jan 2023 • Rodney Kinney, Chloe Anastasiades, Russell Authur, Iz Beltagy, Jonathan Bragg, Alexandra Buraczynski, Isabel Cachola, Stefan Candra, Yoganand Chandrasekhar, Arman Cohan, Miles Crawford, Doug Downey, Jason Dunkelberger, Oren Etzioni, Rob Evans, Sergey Feldman, Joseph Gorney, David Graham, Fangzhou Hu, Regan Huff, Daniel King, Sebastian Kohlmeier, Bailey Kuehl, Michael Langan, Daniel Lin, Haokun Liu, Kyle Lo, Jaron Lochner, Kelsey MacMillan, Tyler Murray, Chris Newell, Smita Rao, Shaurya Rohatgi, Paul Sayre, Zejiang Shen, Amanpreet Singh, Luca Soldaini, Shivashankar Subramanian, Amber Tanaka, Alex D. Wade, Linda Wagner, Lucy Lu Wang, Chris Wilhelm, Caroline Wu, Jiangjiang Yang, Angele Zamarron, Madeleine van Zuylen, Daniel S. Weld

The volume of scientific output is creating an urgent need for automated tools to help scientists keep up with developments in their field.

graph construction

ACCoRD: A Multi-Document Approach to Generating Diverse Descriptions of Scientific Concepts

1 code implementation • 14 May 2022 • Sonia K. Murthy, Kyle Lo, Daniel King, Chandra Bhagavatula, Bailey Kuehl, Sophie Johnson, Jonathan Borchardt, Daniel S. Weld, Tom Hope, Doug Downey

We present ACCoRD, an end-to-end system tackling the novel task of generating sets of descriptions of scientific concepts.

Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search

1 code implementation • 16 Mar 2022 • Daniel King, Zejiang Shen, Nishant Subramani, Daniel S. Weld, Iz Beltagy, Doug Downey

Based on our findings, we present PINOCCHIO, a new decoding method that improves the consistency of a transformer-based abstractive summarizer by constraining beam search to avoid hallucinations.

Abstractive Text Summarization

Reducing Annotating Load: Active Learning with Synthetic Images in Surgical Instrument Segmentation

1 code implementation • 7 Aug 2021 • Haonan Peng, Shan Lin, Daniel King, Yun-Hsuan Su, Randall A. Bly, Kris S. Moe, Blake Hannaford

Motivated by alleviating this workload, we propose a general embeddable method to decrease the usage of labeled real images, using active generated synthetic images.

Active Learning

High-Precision Extraction of Emerging Concepts from Scientific Literature

1 code implementation • 11 Jun 2020 • Daniel King, Doug Downey, Daniel S. Weld

From a corpus of computer science papers on arXiv, we find that our method achieves a Precision@1000 of 99%, compared to 86% for prior work, and a substantially better precision-yield trade-off across the top 15, 000 extractions.

Vocal Bursts Intensity Prediction

Pretrained Language Models for Sequential Sentence Classification

1 code implementation • IJCNLP 2019 • Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. Weld

As a step toward better document-level understanding, we explore classification of a sequence of sentences into their corresponding categories, a task that requires understanding sentences in context of the document.

Classification General Classification +2

Strong Baselines for Complex Word Identification across Multiple Languages

1 code implementation • NAACL 2019 • Pierre Finnimore, Elisabeth Fritzsch, Daniel King, Alison Sneyd, Aneeq Ur Rehman, Fernando Alva-Manchego, Andreas Vlachos

Complex Word Identification (CWI) is the task of identifying which words or phrases in a sentence are difficult to understand by a target audience.

Complex Word Identification Multi-Task Learning +1

ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing

1 code implementation • WS 2019 • Mark Neumann, Daniel King, Iz Beltagy, Waleed Ammar

Despite recent advances in natural language processing, many statistical models for processing text perform extremely poorly under domain shift.