Search Results for author:
Found 45 papers, 16 papers with code
Zero-shot Entity Linking with Less Data
Interestingly, we discovered that BLINK exhibits diminishing returns, i. e., it reaches 98% of its performance with just 1% of the training data and the remaining 99% of the data yields only a marginal increase of 2% in the performance.
Granite Embedding Models
We introduce the Granite Embedding models, a family of encoder-based embedding models designed for retrieval tasks, spanning dense-retrieval and sparse retrieval architectures, with both English and Multilingual capabilities.
Graph-based Uncertainty Metrics for Long-form Language Model Outputs
Recent advancements in Large Language Models (LLMs) have significantly improved text generation capabilities, but these systems are still known to hallucinate, and granular uncertainty estimation for long-form LLM generations remains challenging.
Retrieval Augmented Generation-Based Incident Resolution Recommendation System for IT Support
Clients wishing to implement generative AI in the domain of IT Support and AIOps face two critical issues: domain coverage and model size constraints due to model choice limitations.
Prompts as Auto-Optimized Training Hyperparameters: Training Best-in-Class IR Models from Scratch with 10 Gold Labels
We develop a method for training small-scale (under 100M parameter) neural information retrieval models with as few as 10 gold relevance labels.
From Multiple-Choice to Extractive QA: A Case Study for English and Arabic
The rapid evolution of Natural Language Processing (NLP) has favoured major languages such as English, leaving a significant gap for many others due to limited resources.
CLAPNQ: Cohesive Long-form Answers from Passages in Natural Questions for RAG systems
We present ClapNQ, a benchmark Long-form Question Answering dataset for the full RAG pipeline.
Self-Refinement of Language Models from External Proxy Metrics Feedback
It is often desirable for Large Language Models (LLMs) to capture multiple objectives when providing a response.
Ensemble-Instruct: Generating Instruction-Tuning Data with a Heterogeneous Mixture of LMs
Using in-context learning (ICL) for data generation, techniques such as Self-Instruct (Wang et al., 2023) or the follow-up Alpaca (Taori et al., 2023) can train strong conversational agents with only a small amount of human supervision.
Formally Specifying the High-Level Behavior of LLM-Based Agents
We demonstrate how the proposed framework can be used to implement recent LLM-based agents (e. g., ReACT), and show how the flexibility of our approach can be leveraged to define a new agent with more complex behavior, the Plan-Act-Summarize-Solve (PASS) agent.
MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Types
In this work, we propose a new evaluation scheme to model human judgments in 7 NLP tasks, based on the fine-grained mismatches between a pair of texts.
Scalable Learning of Latent Language Structure With Logical Offline Cycle Consistency
We introduce Logical Offline Cycle Consistency Optimization (LOCCO), a scalable, semi-supervised method for training a neural semantic parser.
Slide, Constrain, Parse, Repeat: Synchronous SlidingWindows for Document AMR Parsing
We evaluate our oracle and parser using the Abstract Meaning Representation (AMR) parsing 3. 0 corpus.
AMR Parsing with Instruction Fine-tuned Pre-trained Language Models
Instruction fine-tuned language models on a collection of instruction annotated datasets (FLAN) have shown highly effective to improve model performance and generalization to unseen tasks.
UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers
Many information retrieval tasks require large labeled datasets for fine-tuning.
PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development
The field of Question Answering (QA) has made remarkable progress in recent years, thanks to the advent of large pre-trained language models, newer realistic benchmark datasets with leaderboards, and novel algorithms for key components such as retrievers and readers.
Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking
Neural information retrieval (IR) systems have progressed rapidly in recent years, in large part due to the release of publicly available benchmarking tasks.
A Closer Look at the Calibration of Differentially Private Learners
Across 7 tasks, temperature scaling and Platt scaling with DP-SGD result in an average 3. 1-fold reduction in the in-domain expected calibration error and only incur at most a minor percent drop in accuracy.
Synthetic Target Domain Supervision for Open Retrieval QA
Neural passage retrieval is a new and promising approach in open retrieval question answering.
A Benchmark for Generalizable and Interpretable Temporal Question Answering over Knowledge Bases
Specifically, our benchmark is a temporal question answering dataset with the following advantages: (a) it is based on Wikidata, which is the most frequently curated, openly available knowledge base, (b) it includes intermediate sparql queries to facilitate the evaluation of semantic parsing based approaches for KBQA, and (c) it generalizes to multiple knowledge bases: Freebase and Wikidata.
DocAMR: Multi-Sentence AMR Representation and Evaluation
Despite extensive research on parsing of English sentences into Abstraction Meaning Representation (AMR) graphs, which are compared to gold graphs via the Smatch metric, full-document parsing into a unified graph representation lacks well-defined representation and evaluation.
Learning to Transpile AMR into SPARQL
We propose a transition-based system to transpile Abstract Meaning Representation (AMR) into SPARQL for Knowledge Base Question Answering (KBQA).
Abstract Meaning Representation
Knowledge Base Question Answering
+1
Maximum Bayes Smatch Ensemble Distillation for AMR Parsing
AMR parsing has experienced an unprecendented increase in performance in the last three years, due to a mixture of effects including architecture improvements and transfer learning.
Ranked #1 on
AMR Parsing
on LDC2020T02
(using extra training data)
Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing
We provide a detailed comparison with recent progress in AMR parsing and show that the proposed parser retains the desirable properties of previous transition-based approaches, while being simpler and reaching the new parsing state of the art for AMR 2. 0, without the need for graph re-categorization.
Ranked #9 on
AMR Parsing
on LDC2017T10
(using extra training data)
SYGMA: System for Generalizable Modular Question Answering OverKnowledge Bases
In addition, to demonstrate extensi-bility to additional reasoning types we evaluate on multi-hopreasoning datasets and a new Temporal KBQA benchmarkdataset on Wikidata, namedTempQA-WD1, introduced in thispaper.
Ranked #2 on
Question Answering
on TempQA-WD
Combining Rules and Embeddings via Neuro-Symbolic AI for Knowledge Base Completion
Recent interest in Knowledge Base Completion (KBC) has led to a plethora of approaches based on reinforcement learning, inductive logic programming and graph embeddings.
A Semantics-aware Transformer Model of Relation Linking for Knowledge Base Question Answering
Relation linking is a crucial component of Knowledge Base Question Answering systems.
Bootstrapping Multilingual AMR with Contextual Word Alignments
We develop high performance multilingualAbstract Meaning Representation (AMR) sys-tems by projecting English AMR annotationsto other languages with weak supervision.
Leveraging Abstract Meaning Representation for Knowledge Base Question Answering
Knowledge base question answering (KBQA)is an important task in Natural Language Processing.
End-to-End QA on COVID-19: Domain Adaptation with Synthetic Training
End-to-end question answering (QA) requires both information retrieval (IR) over a large document collection and machine reading comprehension (MRC) on the retrieved passages.
A Multilingual Reading Comprehension System for more than 100 Languages
We also show how M-GAAMA can be used in downstream tasks by incorporating it into an END-TO-END-QA system using CFO (Chakravarti et al., 2019).
Towards building a Robust Industry-scale Question Answering System
Building on top of the powerful BERTQA model, GAAMA provides a ∼2. 0{\%} absolute boost in F1 over the industry-scale state-of-the-art (SOTA) system on NQ.
Pushing the Limits of AMR Parsing with Self-Learning
Abstract Meaning Representation (AMR) parsing has experienced a notable growth in performance in the last two years, due both to the impact of transfer learning and the development of novel architectures specific to AMR.
Ranked #2 on
AMR Parsing
on LDC2014T12
Multi-Stage Pre-training for Low-Resource Domain Adaptation
Transfer learning techniques are particularly useful in NLP tasks where a sizable amount of high-quality annotated data is difficult to obtain.
ARES: A Reading Comprehension Ensembling Service
We introduce ARES (A Reading Comprehension Ensembling Service): a novel Machine Reading Comprehension (MRC) demonstration system which utilizes an ensemble of models to increase F1 by 2. 3 points.
Leveraging Semantic Parsing for Relation Linking over Knowledge Bases
Knowledgebase question answering systems are heavily dependent on relation extraction and linking modules.
Ranked #1 on
Relation Linking
on QALD-7
GPT-too: A language-model-first approach for AMR-to-text generation
Meaning Representations (AMRs) are broad-coverage sentence-level semantic graphs.
Ranked #10 on
AMR-to-Text Generation
on LDC2017T10
The TechQA Dataset
We introduce TechQA, a domain-adaptation question answering dataset for the technical support domain.
Path-Based Contextualization of Knowledge Graphs for Textual Entailment
In this paper, we introduce the problem of knowledge graph contextualization -- that is, given a specific NLP task, the problem of extracting meaningful and relevant sub-graphs from a given knowledge graph.
Ensembling Strategies for Answering Natural Questions
Many of the top question answering systems today utilize ensembling to improve their performance on tasks such as the Stanford Question Answering Dataset (SQuAD) and Natural Questions (NQ) challenges.
Frustratingly Easy Natural Question Answering
Existing literature on Question Answering (QA) mostly focuses on algorithmic novelty, data augmentation, or increasingly large pre-trained language models like XLNet and RoBERTa.
Ranked #5 on
Question Answering
on Natural Questions (long)
CFO: A Framework for Building Production NLP Systems
This paper introduces a novel orchestration framework, called CFO (COMPUTATION FLOW ORCHESTRATOR), for building, experimenting with, and deploying interactive NLP (Natural Language Processing) and IR (Information Retrieval) systems to production environments.
Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning
Our work involves enriching the Stack-LSTM transition-based AMR parser (Ballesteros and Al-Onaizan, 2017) by augmenting training with Policy Learning and rewarding the Smatch score of sampled graphs.
Ranked #24 on
AMR Parsing
on LDC2017T10
