Search Results for author:
Found 19 papers, 11 papers with code
CooK: Empowering General-Purpose Language Models with Modular and Collaborative Knowledge
Large language models (LLMs) are increasingly adopted for knowledge-intensive tasks and contexts.
Can Language Models Solve Graph Problems in Natural Language?
We then propose Build-a-Graph Prompting and Algorithmic Prompting, two instruction-based approaches to enhance LLMs in solving natural language graph problems.
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation
In this work, we explore a useful but often neglected methodology for robustness analysis of text generation evaluation metrics: stress tests with synthetic data.
PCFG-based Natural Language Interface Improves Generalization for Controlled Text Generation
Existing work on controlled text generation (CTG) assumes a control interface of categorical attributes.
Controlling the Focus of Pretrained Language Generation Models
This work aims to develop a control mechanism by which a user can select spans of context as "highlights" for the model to focus on, and generate relevant output.
Revisiting Latent-Space Interpolation via a Quantitative Evaluation Framework
In this work, we show how data labeled with semantically continuous attributes can be utilized to conduct a quantitative evaluation of latent-space interpolation algorithms, for variational autoencoders.
An Empirical Study on Few-shot Knowledge Probing for Pretrained Language Models
Prompt-based knowledge probing for 1-hop relations has been used to measure how much world knowledge is stored in pretrained language models.
Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models
We find that mix-review effectively regularizes the finetuning process, and the forgetting problem is alleviated to some extent.
Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models
In this work, we explore joint energy-based model (EBM) training during the finetuning of pretrained text encoders (e. g., Roberta) for natural language understanding (NLU) tasks.
Quantifying Exposure Bias for Open-ended Language Generation
The exposure bias problem refers to the incrementally distorted generation induced by the training-generation discrepancy, in teacher-forcing training for auto-regressive neural network language models (LM).
A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation
On the other hand, we find that the set of sampling algorithms that satisfies these properties performs on par with the existing sampling algorithms.
AutoKG: Constructing Virtual Knowledge Graphs from Unstructured Documents for Question Answering
Knowledge graphs (KGs) have the advantage of providing fine-grained detail for question-answering systems.
Analyzing the Forgetting Problem in the Pretrain-Finetuning of Dialogue Response Models
We find that mix-review effectively regularizes the finetuning process, and the forgetting problem is alleviated to some extent.
From Data Quality to Model Quality: an Exploratory Study on Deep Learning
Nowadays, people strive to improve the accuracy of deep learning models.
Why gradient clipping accelerates training: A theoretical justification for adaptivity
We provide a theoretical explanation for the effectiveness of gradient clipping in training deep neural networks.
Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation?
Exposure bias has been regarded as a central problem for auto-regressive language models (LM).
Negative Training for Neural Dialogue Response Generation
Although deep learning models have brought tremendous advancements to the field of open-domain dialogue response generation, recent research results have revealed that the trained models have undesirable generation behaviors, such as malicious responses and generic (boring) responses.
Detecting egregious responses in neural sequence-to-sequence models
We adopt an empirical methodology, in which we first create lists of egregious output sequences, and then design a discrete optimization algorithm to find input sequences that will cause the model to generate them.
On Training Bi-directional Neural Network Language Model with Noise Contrastive Estimation
We propose to train bi-directional neural network language model(NNLM) with noise contrastive estimation(NCE).
