Search Results for author:
Found 10 papers, 3 papers with code
On Synthetic Data Strategies for Domain-Specific Generative Retrieval
This paper investigates synthetic data generation strategies in developing generative retrieval models for domain-specific corpora, thereby addressing the scalability challenges inherent in manually annotating in-domain queries.
PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queries
Previous text-to-SQL datasets and systems have primarily focused on user questions with clear intentions that can be answered.
RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation
Despite Retrieval-Augmented Generation (RAG) showing promising capability in leveraging external knowledge, a comprehensive evaluation of RAG systems is still challenging due to the modular nature of RAG, evaluation of long-form responses and reliability of measurements.
Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall
By further decoupling model known and unknown knowledge, we find the degradation is attributed to exemplars that contradict a model's known knowledge, as well as the number of such exemplars.
UNITE: A Unified Benchmark for Text-to-SQL Evaluation
A practical text-to-SQL system should generalize well on a wide variety of natural language questions, unseen database schemas, and novel SQL query structures.
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness
Neural text-to-SQL models have achieved remarkable performance in translating natural language questions into SQL queries.
Importance of Synthesizing High-quality Data for Text-to-SQL Parsing
In this paper, we first examined the existing synthesized datasets and discovered that state-of-the-art text-to-SQL algorithms did not further improve on popular benchmarks when trained with augmented synthetic data.
Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding
When synthesizing a SQL query, there is no explicit semantic information of NLQ available to the parser which leads to undesirable generalization performance.
Learned Prioritization for Trading Off Accuracy and Speed
Users want natural language processing (NLP) systems to be both fast and accurate, but quality often comes at the cost of speed.
Message-Passing for Approximate MAP Inference with Latent Variables
We consider a general inference setting for discrete probabilistic graphical models where we seek maximum a posteriori (MAP) estimates for a subset of the random variables (max nodes), marginalizing over the rest (sum nodes).
