Search Results for author:
Found 77 papers, 53 papers with code
CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search
Neural rankers based on deep pretrained language models (LMs) have been shown to improve many information retrieval benchmarks.
OpenMatch: An Open Source Library for Neu-IR Research
OpenMatch is a Python-based library that serves for Neural Information Retrieval (Neu-IR) research.
Capturing Global Informativeness in Open Domain Keyphrase Extraction
Open-domain KeyPhrase Extraction (KPE) aims to extract keyphrases from documents without domain or quality restrictions, e. g., web pages with variant domains and qualities.
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
In this paper, we identify that the main bottleneck is in the training mechanisms, where the negative instances used in training are not representative of the irrelevant documents in testing.
Ranked #7 on
Passage Retrieval
on Natural Questions
Text Classification Using Label Names Only: A Language Model Self-Training Approach
In this paper, we explore the potential of only using the label name of each class to train classification models on unlabeled data, without using any labeled documents.
End-to-End Neural Ad-hoc Ranking with Kernel Pooling
Given a query and a set of documents, K-NRM uses a translation matrix that models word-level similarities via word embeddings, a new kernel-pooling technique that uses kernels to extract multi-level soft match features, and a learning-to-rank layer that combines those features into the final ranking score.
Cleaner Pretraining Corpus Curation with Neural Web Scraping
The web contains large-scale, diverse, and abundant information to satisfy the information-seeking needs of humans.
Fine-grained Fact Verification with Kernel Graph Attention Network
Fact Verification requires fine-grained natural language inference capability that finds subtle clues to identify the syntactical and semantically correct but not well-supported claims.
Ranked #5 on
Fact Verification
on FEVER
Adapting Open Domain Fact Extraction and Verification to COVID-FACT through In-Domain Language Modeling
With the epidemic of COVID-19, verifying the scientifically false online information, such as fake news and maliciously fabricated statements, has become crucial.
Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval
This paper presents the Entity-Duet Neural Ranking Model (EDRM), which introduces knowledge graphs to neural search systems.
Open Domain Web Keyphrase Extraction Beyond Language Modeling
This paper studies keyphrase extraction in real-world scenarios where documents are from diverse domains and have variant content quality.
An In-depth Look at Gemini's Language Abilities
The recently released Google Gemini class of models are the first to comprehensively report results that rival the OpenAI GPT series across a wide variety of tasks.
Target-Guided Open-Domain Conversation
We study the problem of imposing conversational goals on open-domain chat agents.
Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data
SANTA proposes two pretraining methods to make language models structure-aware and learn effective representations for structured data: 1) Structured Data Alignment, which utilizes the natural alignment relations between structured data and unstructured data for structure-aware pretraining.
Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs
Human conversations naturally evolve around related concepts and scatter to multi-hop concepts.
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining
The first token-level task, Corrective Language Modeling, is to detect and correct tokens replaced by the auxiliary model, in order to better capture token-level semantics.
TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network
Taxonomies consist of machine-interpretable semantics and provide valuable knowledge for many web applications.
TREC CAsT 2019: The Conversational Assistance Track Overview
A common theme through the runs is the use of BERT-based neural reranking methods.
ActiveRAG: Revealing the Treasures of Knowledge via Active Learning
Retrieval Augmented Generation (RAG) has introduced a new paradigm for Large Language Models (LLMs), aiding in the resolution of knowledge-intensive tasks.
Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention
Transformers have achieved new heights modeling natural language as a sequence of text tokens.
Ranked #42 on
Question Answering
on HotpotQA
Few-Shot Generative Conversational Query Rewriting
Conversational query rewriting aims to reformulate a concise conversational query to a fully specified, context-independent query that can be effectively handled by existing information retrieval systems.
COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learning
We present a new zero-shot dense retrieval (ZeroDR) method, COCO-DR, to improve the generalization ability of dense retrieval by combating the distribution shifts between source training tasks and target scenarios.
Ranked #1 on
Zero-shot Text Search
on CQADupStack
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In
Retrieval augmentation can aid language models (LMs) in knowledge-intensive tasks by supplying them with external information.
Less is More: Pre-train a Strong Text Encoder for Dense Retrieval Using a Weak Decoder
Dense retrieval requires high-quality text sequence embeddings to support effective search in the representation space.
Few-Shot Conversational Dense Retrieval
In this paper, we present a Conversational Dense Retrieval system, ConvDR, that learns contextualized embeddings for multi-turn conversational queries and retrieves documents solely using embedding dot products.
Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder
Dense retrieval requires high-quality text sequence embeddings to support effective search in the representation space.
Universal Vision-Language Dense Retrieval: Learning A Unified Representation Space for Multi-Modal Retrieval
To learn a unified embedding space for multi-modal retrieval, UniVL-DR proposes two techniques: 1) Universal embedding optimization strategy, which contrastively optimizes the embedding space using the modality-balanced hard negatives; 2) Image verbalization method, which bridges the modality gap between images and texts in the raw data space.
Selective Weak Supervision for Neural Information Retrieval
This paper democratizes neural information retrieval to scenarios where large scale relevance training signals are not available.
Text Matching Improves Sequential Recommendation by Reducing Popularity Biases
TASTE alleviates the cold start problem by representing long-tail items using full-text modeling and bringing the benefits of pretrained language models to recommendation systems.
Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators
We present a new framework AMOS that pretrains text encoders with an Adversarial learning curriculum via a Mixture Of Signals from multiple auxiliary generators.
Towards Interpretable Natural Language Understanding with Explanations as Latent Variables
In this paper, we develop a general framework for interpretable natural language understanding that requires only a small set of human annotated explanations for training.
Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers
This paper explores the effectiveness of model-generated signals in improving zero-shot generalization of text-to-text Transformers such as T5.
Reduce Catastrophic Forgetting of Dense Retrieval Training with Teleportation Negatives
In this paper, we investigate the instability in the standard dense retrieval training, which iterates between model training and hard negative selection using the being-trained model.
P^3 Ranker: Mitigating the Gaps between Pre-training and Ranking Fine-tuning with Prompt-based Learning and Pre-finetuning
In this paper, we identify and study the two mismatches between pre-training and ranking fine-tuning: the training schema gap regarding the differences in training objectives and model architectures, and the task knowledge gap considering the discrepancy between the knowledge needed in ranking and that learned during pre-training.
Long Document Ranking with Query-Directed Sparse Transformer
In this paper, we design Query-Directed Sparse attention that induces IR-axiomatic structures in transformer self-attention.
Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval
Complex question answering often requires finding a reasoning chain that consists of multiple evidence pieces.
More Robust Dense Retrieval with Contrastive Dual Learning
With contrastive learning, the dual training object of DANCE learns more tailored representations for queries and documents to keep the embedding space smooth and uniform, thriving on the ranking performance of DANCE on the MS MARCO document retrieval task.
TIAGE: A Benchmark for Topic-Shift Aware Dialog Modeling
In research on dialog systems, the ability to actively and smoothly transition to new topics is often ignored.
Data Augmentation for Abstractive Query-Focused Multi-Document Summarization
The progress in Query-focused Multi-Document Summarization (QMDS) has been limited by the lack of sufficient largescale high-quality training datasets.
Distantly-Supervised Evidence Retrieval Enables Question Answering without Evidence Annotation
Open-domain question answering answers a question based on evidence retrieved from a large corpus.
Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation
This paper investigates whether models can learn to find evidence from a large corpus, with only distant supervision from answer labels for model training, thereby generating no additional annotation cost.
Dimension Reduction for Efficient Dense Retrieval via Conditional Autoencoder
These embeddings need to be high-dimensional to fit training signals and guarantee the retrieval effectiveness of dense retrievers.
Ranked #1 on
Information Retrieval
on MS MARCO
MARVEL: Unlocking the Multi-Modal Capability of Dense Retrieval via Visual Module Plugin
To facilitate the multi-modal retrieval tasks, we build the ClueWeb22-MM dataset based on the ClueWeb22 dataset, which regards anchor texts as queries, and exacts the related text and image documents from anchor-linked web pages.
Contrastive Multi-document Question Generation
In the SL stage, a single-document question generator is trained.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback
This paper proposes ANCE-PRF, a new query encoder that uses pseudo relevance feedback (PRF) to improve query representations for dense retrieval.
Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision
To democratize the benefits of Neu-IR, this paper presents MetaAdaptRank, a domain adaptive learning method that generalizes Neu-IR models from label-rich source domains to few-shot target domains.
Say More with Less: Understanding Prompt Learning Behaviors through Gist Compression
It finetunes the compression plugin module and uses the representations of gist tokens to emulate the raw prompts in the vanilla language model.
Automatic Event Salience Identification
Our analyses demonstrate that our neural model captures interesting connections between salience and discourse unit relations (e. g., scripts and frame structures).
WebDRO: A Web-based Group-level Clustering and Reweighting Method for Unsupervised Dense Retrieval
In this paper, we introduce WebDRO, an efficient approach for clustering the web graph data and optimizing group weights to enhance the robustness of the pretraining process of dense retrieval models on web graphs.
Improving Multitask Retrieval by Promoting Task Specialization
In multitask retrieval, a single retriever is trained to retrieve relevant contexts for multiple tasks.
Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model
We first validate the efficacy of Toolink in harnessing the model's creativity and CoS ability on ChatGPT.
Unsupervised Dense Retrieval Training with Web Anchors
Based on their commonalities, we train an unsupervised dense retriever, Anchor-DR, with a contrastive learning task that matches the anchor text and the linked document.
Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling
The salience model also improves ad hoc search accuracy, providing effective ranking features by modeling the salience of query entities in candidate documents.
Word-Entity Duet Representations for Document Ranking
This paper presents a word-entity duet framework for utilizing knowledge bases in ad-hoc retrieval.
Consistency and Variation in Kernel Neural Ranking Model
This paper studies the consistency of the kernel-based neural ranking model K-NRM, a recent state-of-the-art neural IR model, which is important for reproducible research and deployment in the industry.
An Axiomatic Approach to Regularizing Neural Ranking Models
The training of these models involve a search for appropriate parameter values based on large quantities of labeled examples.
Understanding the Behaviors of BERT in Ranking
This paper studies the performances and behaviors of BERT in ranking tasks.
Convolutional Neural Networks for Soft Matching N-Grams in Ad-hoc Search
This paper presents Conv-KNRM, a Convolutional Kernel-based Neural Ranking Model that models n-gram soft matches for ad-hoc search.
Generic Intent Representation in Web Search
This paper presents GEneric iNtent Encoder (GEN Encoder) which learns a distributed representation space for user intent in search.
Neural Document Expansion with User Feedback
This paper presents a neural document expansion approach (NeuDEF) that enriches document representations for neural ranking models.
Latent Relation Language Models
In this paper, we propose Latent Relation Language Models (LRLMs), a class of language models that parameterizes the joint distribution over the words in a document and the entities that occur therein via knowledge graph relations.
Explore Entity Embedding Effectiveness in Entity Retrieval
Entity embedding learns lots of semantic information from the knowledge graph and represents entities with a low-dimensional representation, which provides an opportunity to establish interactions between query related entities and candidate entities for entity retrieval.
Proceedings of the KG-BIAS Workshop 2020 at AKBC 2020
The KG-BIAS 2020 workshop touches on biases and how they surface in knowledge graphs (KGs), biases in the source data that is used to create KGs, methods for measuring or remediating bias in KGs, but also identifying other biases such as how and which languages are represented in automatically constructed KGs or how personal KGs might incur inherent biases.
Pretrain Knowledge-Aware Language Models
Rather, we simply signal the existence of entities to the input of the transformer in pretraining, with an entity-extended tokenizer; and at the output, with an additional entity prediction task.
Complex Factoid Question Answering with a Free-Text Knowledge Graph
DELFT's advantage comes from both the high coverage of its free-text knowledge graph-more than double that of dbpedia relations-and the novel graph neural network which reasons on the rich but noisy free-text evidence.
Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature
A prominent case in point is the explosion of the biomedical literature on COVID-19, which swelled to hundreds of thousands of papers in a matter of months.
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representation
Dense retrieval (DR) methods conduct text retrieval by first encoding texts in the embedding space and then matching them by nearest neighbor search.
Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations
Dense retrieval (DR) methods conduct text retrieval by first encoding texts in the embedding space and then matching them by nearest neighbor search.
Neural Approaches to Conversational Information Retrieval
A conversational information retrieval (CIR) system is an information retrieval (IR) system with a conversational interface which allows users to interact with the system to seek information via multi-turn conversations of natural language, in spoken or written form.
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
We present an efficient method of pretraining large-scale autoencoding language models using training signals generated by an auxiliary model.
ClueWeb22: 10 Billion Web Documents with Visual and Semantic Information
ClueWeb22, the newest iteration of the ClueWeb line of datasets, provides 10 billion web pages affiliated with rich information.
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories
In this paper we improve the zero-shot generalization ability of language models via Mixture-Of-Memory Augmentation (MoMA), a mechanism that retrieves augmentation documents from multiple information corpora ("external memories"), with the option to "plug in" new memory at inference time.
Fusion-in-T5: Unifying Document Ranking Signals for Improved Information Retrieval
Common IR pipelines are typically cascade systems that may involve multiple rankers and/or fusion models to integrate different information step-by-step.
ED-Copilot: Reduce Emergency Department Wait Time with Language Model Diagnostic Assistance
Using publicly available patient data, we collaborate with ED clinicians to curate MIMIC-ED-Assist, a benchmark that measures the ability of AI systems in suggesting laboratory tests that minimize ED wait times, while correctly predicting critical outcomes such as death.
Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrieval
This study investigates the existence of positional biases in Transformer-based models for text representation learning, particularly in the context of web document retrieval.
