Search Results for author:
Found 6 papers, 4 papers with code
Reranking Laws for Language Generation: A Communication-Theoretic Perspective
In this paper, we draw a parallel between this strategy and the use of redundancy to decrease the error rate in noisy communication channels.
DOCE: Finding the Sweet Spot for Execution-Based Code Generation
Recently, a diverse set of decoding and reranking procedures have been shown effective for LLM-based code generation.
Uncertainty in Natural Language Generation: From Theory to Applications
Recent advances of powerful Language Models have allowed Natural Language Generation (NLG) to emerge as an important technology that can not only perform traditional tasks like summarisation or translation, but also serve as a natural language interface to a variety of applications.
Python Code Generation by Asking Clarification Questions
We hypothesize that the under-specification of a natural language description can be resolved by asking clarification questions.
When Do You Need Billions of Words of Pretraining Data?
We adopt four probing methods---classifier probing, information-theoretic probing, unsupervised relative acceptability judgment, and fine-tuning on NLU tasks---and draw learning curves that track the growth of these different measures of linguistic ability with respect to pretraining data volume using the MiniBERTas, a group of RoBERTa models pretrained on 1M, 10M, 100M and 1B words.
Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)
One reason pretraining on self-supervised linguistic tasks is effective is that it teaches models features that are helpful for language understanding.
