Search Results for author:
Found 37 papers, 14 papers with code
Overview of the TREC 2023 NeuCLIR Track
The principal tasks are ranked retrieval of news in one of the three languages, using English topics.
FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions
We introduce our dataset FollowIR, which contains a rigorous instruction evaluation benchmark as well as a training set for helping IR models learn to better follow real-world instructions.
KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questions
To evaluate the capabilities of current LLMs on this task, we construct KIWI, a dataset of knowledge-intensive writing instructions in the scientific domain.
OLMo: Accelerating the Science of Language Models
Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs.
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Language models have become a critical technology to tackling a wide range of natural language processing tasks, yet many details about how the best-performing language models were developed are not reported.
AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters
Large language models' (LLMs) abilities are drawn from their pretraining data, and model development begins with data curation.
Paloma: A Benchmark for Evaluating Language Model Fit
We invite submissions to our benchmark and organize results by comparability based on compliance with guidelines such as removal of benchmark contamination from pretraining.
Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoders
Prevailing research practice today often relies on training dense retrievers on existing large datasets such as MSMARCO and then experimenting with ways to improve zero-shot generalization capabilities to unseen domains.
What's In My Big Data?
We open-source WIMBD's code and artifacts to provide a standard set of evaluations for new text-based corpora and to encourage more analyses and transparency around them.
The Surveillance AI Pipeline
Moreover, the majority of these technologies specifically enable extracting data about human bodies and body parts.
When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasets
Using large language models (LMs) for query or document expansion can improve generalization in information retrieval.
Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms
However, these auditing processes have been criticized for their failure to integrate the knowledge of marginalized communities and consider the power dynamics between auditors and the communities.
A Question Answering Framework for Decontextualizing User-facing Snippets from Scientific Documents
Many real-world applications (e. g., note taking, search) require extracting a sentence or paragraph from a document and showing that snippet to a human outside of the source document.
Overview of the TREC 2022 NeuCLIR Track
This is the first year of the TREC Neural CLIR (NeuCLIR) track, which aims to study the impact of neural approaches to cross-language information retrieval.
Queer In AI: A Case Study in Community-Led Participatory AI
We present Queer in AI as a case study for community-led participatory design in AI.
The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces
Scholarly publications are key to the transfer of knowledge from scholars to others.
One-Shot Labeling for Automatic Relevance Estimation
We then explore various approaches for predicting the relevance of unjudged documents with respect to a query and the known relevant document, including nearest neighbor, supervised, and prompting techniques.
The Semantic Scholar Open Data Platform
The volume of scientific output is creating an urgent need for automated tools to help scientists keep up with developments in their field.
Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval
Via extensive automatic and human evaluation, we determine: (1) state-of-the-art summarizers suffer large reductions in performance when applied to open-domain MDS, (2) additional training in the open-domain setting can reduce this sensitivity to imperfect retrieval, and (3) summarizers are insensitive to the retrieval of duplicate documents and the order of retrieved documents, but highly sensitive to other errors, like the retrieval of irrelevant documents.
Ranked #1 on
Multi-Document Summarization
on MS^2
Knowledge Transfer from Answer Ranking to Answer Generation
In this paper, we propose to train a GenQA model by transferring knowledge from a trained AS2 model, to overcome the aforementioned issue.
Embedding Recycling for Language Models
Real-world applications of neural language models often involve running many different models over the same corpus.
Pre-training Transformer Models with Sentence-Level Objectives for Answer Sentence Selection
An important task for designing QA systems is answer sentence selection (AS2): selecting the sentence containing (or constituting) the answer to a question from a set of retrieved relevant documents.
Ranked #1 on
Answer Selection
on ASNQ
Paragraph-based Transformer Pre-training for Multi-Sentence Inference
Our evaluation on three AS2 and one fact verification datasets demonstrates the superiority of our pre-training technique over the traditional ones for transformers used as joint models for multi-candidate inference tasks, as well as when used as cross-encoders for sentence-pair formulations of these tasks.
Ranked #3 on
Fact Verification
on FEVER
Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems
CERBERUS consists of two components: a stack of transformer layers that is used to encode inputs, and a set of ranking heads; unlike traditional distillation technique, each of them is trained by distilling a different large transformer architecture in a way that preserves the diversity of the ensemble members.
Cross-Lingual Open-Domain Question Answering with Answer Sentence Generation
Our cross-lingual generative system outperforms answer sentence selection baselines for all 5 languages and monolingual generative pipelines for three out of five languages studied.
Answer Generation for Retrieval-based Question Answering Systems
Recent advancements in transformer-based models have greatly improved the ability of Question Answering (QA) systems to provide correct answers; in particular, answer sentence selection (AS2) models, core components of retrieval-based systems, have achieved impressive results.
Modeling Context in Answer Sentence Selection Systems on a Latency Budget
In this work, we present an approach to efficiently incorporate contextual information in AS2 models.
Machine Reading Comprehension
Open-Domain Question Answering
+1
Multi-task Learning of Spoken Language Understanding by Integrating N-Best Hypotheses with Hierarchical Attention
The comparison shows that our model could recover the transcription by integrating the fragmented information among hypotheses and identifying the frequent error patterns of the ASR module, and even rewrite the query for a better understanding, which reveals the characteristic of multi-task learning of broadcasting knowledge.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
The Cascade Transformer: an Application for Efficient Answer Sentence Selection
Large transformer-based language models have been shown to be very effective in many classification tasks.
Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing
Virtual assistants such as Amazon Alexa, Apple Siri, and Google Assistant often rely on a semantic parsing component to understand which action(s) to execute for an utterance spoken by its users.
Improving Spoken Language Understanding By Exploiting ASR N-best Hypotheses
The NLU module usually uses the first best interpretation of a given speech in downstream tasks such as domain and intent classification.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-shot Learning
While billions of non-English speaking users rely on search engines every day, the problem of ad-hoc information retrieval is rarely studied for non-English languages.
RSDD-Time: Temporal Annotation of Self-Reported Mental Health Diagnoses
Self-reported diagnosis statements have been widely employed in studying language related to mental health in social media.
SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions
Mental health is a significant and growing public health concern.
Helping or Hurting? Predicting Changes in Users' Risk of Self-Harm Through Online Community Interactions
In recent years, online communities have formed around suicide and self-harm prevention.
GU IRLAB at SemEval-2018 Task 7: Tree-LSTMs for Scientific Relation Classification
SemEval 2018 Task 7 focuses on relation ex- traction and classification in scientific literature.
