Search Results for author:
Found 28 papers, 7 papers with code
Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding
This collaborative prompting approach empowers a single LM to simultaneously act as a comprehensive orchestrator and a panel of diverse experts, significantly enhancing its performance across a wide array of tasks.
Calibrated Language Models Must Hallucinate
For "arbitrary" facts whose veracity cannot be determined from the training data, we show that hallucinations must occur at a certain rate for language models that satisfy a statistical calibration condition appropriate for generative language models.
Testing Language Model Agents Safely in the Wild
A prerequisite for safe autonomy-in-the-wild is safe testing-in-the-wild.
Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
In this work, we use a language-model-infused scaffolding program to improve itself.
Textbooks Are All You Need
Despite this small scale, phi-1 attains pass@1 accuracy 50. 6% on HumanEval and 55. 5% on MBPP.
Ranked #41 on
Code Generation
on HumanEval
Do Language Models Know When They're Hallucinating References?
In this work, we focus on hallucinated book and article references and present them as the "model organism" of language model hallucination research, due to their frequent and easy-to-discern nature.
Loss Minimization Yields Multicalibration for Large Neural Networks
We show that minimizing the squared loss over all neural nets of size $n$ implies multicalibration for all but a bounded number of unlucky values of $n$.
Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms
For example, on parity problems, the NN learns as well as Gaussian elimination, an efficient algorithm that can be succinctly described.
Partial Matrix Completion
This work establishes a new framework of partial matrix completion, where the goal is to identify a large subset of the entries that can be completed with high confidence.
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
We introduce a new type of test, called a Turing Experiment (TE), for evaluating to what extent a given language model, such as GPT models, can simulate different aspects of human behavior.
Language Models Can Teach Themselves to Program Better
We show that it is possible for an LM to synthesize programming problems and solutions, which are filtered for correctness by a Python interpreter.
Why GANs are overkill for NLP
We show that, while it may seem that maximizing likelihood is inherently different than minimizing distinguishability, this distinction is largely artificial and only holds for limited models.
Omnipredictors
We suggest a rigorous new paradigm for loss minimization in machine learning where the loss function can be ignored at the time of learning and only be taken into account when deciding an action.
Social Norm Bias: Residual Harms of Fairness-Aware Algorithms
Many modern machine learning algorithms mitigate bias by enforcing fairness constraints across coarsely-defined groups related to a sensitive attribute like gender or race.
Programming Puzzles
The dataset is comprehensive in that it spans problems of a range of difficulties and domains, ranging from trivial string manipulation problems, to classic programming puzzles (e. g., Tower of Hanoi), to interview/competitive-programming problems (e. g., dynamic programming), to longstanding open problems in algorithms and mathematics (e. g., factoring).
Towards optimally abstaining from prediction with OOD test examples
Our work builds on a recent abstention algorithm of Goldwasser, Kalais, and Montasser (2020) for transductive binary classification.
Beyond Perturbations: Learning Guarantees with Arbitrary Adversarial Test Examples
We present a transductive learning algorithm that takes as input training examples from a distribution $P$ and arbitrary (unlabeled) test examples, possibly chosen by an adversary.
ADAPTIVE GENERATION OF PROGRAMMING PUZZLES
Puzzles are objective in that one can easily test the correctness of a given solution x by seeing whether it satisfies f, unlike the most common representations for program synthesis: given input-output pairs or an English problem description, the correctness of a given solution is not determined and is debatable.
Learning to Prune: Speeding up Repeated Computations
We present an algorithm that learns to maximally prune the search space on repeated computations, thereby reducing runtime while provably outputting the correct solution each period with high probability.
What's in a Name? Reducing Bias in Bios without Access to Protected Attributes
In the context of mitigating bias in occupation classification, we propose a method for discouraging correlation between the predicted probability of an individual's true occupation and a word embedding of their name.
Humor in Word Embeddings: Cockamamie Gobbledegook for Nincompoops
While humor is often thought to be beyond the reach of Natural Language Processing, we show that several aspects of single-word humor correlate with simple linear directions in Word Embeddings.
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
We present a large-scale study of gender bias in occupation classification, a task where the use of machine learning may lead to negative outcomes on peoples' lives.
What are the biases in my word embedding?
The inputs to our algorithm are a list of target tokens, e. g. names, and a word embedding.
Unleashing Linear Optimizers for Group-Fair Learning and Optimization
Most systems and learning algorithms optimize average performance or average loss -- one reason being computational complexity.
Glass-Box Program Synthesis: A Machine Learning Approach
Our results show that (i) performing 4 rounds of our framework typically solves about 70% of the target problems, (ii) our framework can improve itself even in domain agnostic scenarios, and (iii) it can solve problems that would be otherwise too slow to solve with brute-force search.
Decoupled classifiers for fair and efficient machine learning
When it is ethical and legal to use a sensitive attribute (such as gender or race) in machine learning systems, the question remains how to do so.
Meta-Unsupervised-Learning: A supervised approach to unsupervised learning
We introduce a new paradigm to investigate unsupervised learning, reducing unsupervised learning to supervised learning.
Crowdsourcing Feature Discovery via Adaptively Chosen Comparisons
In addition we also ask the crowd to provide binary labels to the remaining examples based on the discovered features.
