Search Results for author:
Found 46 papers, 18 papers with code
ElementaryNet: A Non-Strategic Neural Network for Predicting Human Behavior in Normal-Form Games
Models of human behavior in game-theoretic settings often distinguish between strategic behavior, in which a player both reasons about how others will act and best responds to these beliefs, and "level-0" non-strategic behavior, in which they do not respond to explicit beliefs about others.
STEER-ME: Assessing the Microeconomic Reasoning of Large Language Models
How should one judge whether a given large language model (LLM) can reliably perform economic reasoning?
Understanding Understanding: A Pragmatic Framework Motivated by Large Language Models
Motivated by the rapid ascent of Large Language Models (LLMs) and debates about the extent to which they possess human-level qualities, we propose a framework for testing whether any agent (be it a machine or a human) understands a subject matter.
Utilitarian Algorithm Configuration for Infinite Parameter Spaces
Utilitarian algorithm configuration is a general-purpose technique for automatically searching the parameter space of a given algorithm to optimize its performance, as measured by a given utility function, on a given set of inputs.
What Can Natural Language Processing Do for Peer Review?
We hope that our work will help set the agenda for research in machine-assisted scientific quality control in the age of AI, within the NLP community and beyond.
Understanding Iterative Combinatorial Auction Designs via Multi-Agent Reinforcement Learning
Iterative combinatorial auctions are widely used in high stakes settings such as spectrum auctions.
STEER: Assessing the Economic Rationality of Large Language Models
We begin by surveying the economic literature on rational decision making, taxonomizing a large set of fine-grained "elements" that an agent should exhibit, along with dependencies between them.
Utilitarian Algorithm Configuration
We prove upper bounds on the runtime of these procedures that are similar to theoretical lower bounds, while also demonstrating their performance empirically.
Generating Benchmarks for Factuality Evaluation of Language Models
FACTOR automatically transforms a factual corpus of interest into a benchmark evaluating an LM's propensity to generate true facts from the corpus vs. similar but incorrect statements.
How to Evaluate Behavioral Models
Researchers building behavioral models, such as behavioral game theorists, use experimental data to evaluate predictive models of human behavior.
In-Context Retrieval-Augmented Language Models
Retrieval-Augmented Language Modeling (RALM) methods, which condition a language model (LM) on relevant documents from a grounding corpus during generation, were shown to significantly improve language modeling performance.
Parallel Context Windows for Large Language Models
We present Parallel Context Windows (PCW), a method that alleviates the context window restriction for any off-the-shelf LLM without further training.
UNSAT Solver Synthesis via Monte Carlo Forest Search
We introduce Monte Carlo Forest Search (MCFS), a class of reinforcement learning (RL) algorithms for learning policies in {tree MDPs}, for which policy execution involves traversing an exponential-sized tree.
Artificial Intelligence and Life in 2030: The One Hundred Year Study on Artificial Intelligence
In September 2016, Stanford's "One Hundred Year Study on Artificial Intelligence" project (AI100) issued the first report of its planned long-term periodic assessment of artificial intelligence (AI) and its impact on society.
Better Peer Grading through Bayesian Inference
Peer grading systems aggregate noisy reports from multiple students to approximate a true grade as closely as possible.
Formalizing Preferences Over Runtime Distributions
We propose a principled alternative, taking a utility-theoretic approach to characterize the scoring functions that describe preferences over algorithms.
MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning
Huge language models (LMs) have ushered in a new era for AI, serving as a gateway to natural-language-based knowledge tasks.
Standing on the Shoulders of Giant Frozen Language Models
To demonstrate this, we introduce three novel methods for leveraging frozen models: input-dependent prompt tuning, frozen readers, and recursive LMs, each of which vastly improves on current frozen-model approaches.
Matching Papers and Reviewers at Large Conferences
Peer-reviewed conferences, the main publication venues in CS, rely critically on matching highly qualified reviewers for each paper.
The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models
Supervised learning models often make systematic errors on rare subsets of the data.
The Perils of Learning Before Optimizing
Typically, learning the prediction model used to generate the optimization problem and solving that problem are performed in two separate stages.
ImpatientCapsAndRuns: Approximately Optimal Algorithm Configuration from an Infinite Pool
Algorithm configuration procedures optimize parameters of a given algorithm to perform well over a distribution of inputs.
Dynamic Weighted Matching with Heterogeneous Arrival and Departure Rates
The value of a match is determined by the types of the matched agents.
Computer Science and Game Theory Data Structures and Algorithms
Exemplar Guided Active Learning
The challenge is that we are not informed which labels are common and which are rare, and the true label distribution may exhibit extreme skew.
PMI-Masking: Principled masking of correlated spans
Specifically, we show experimentally that PMI-Masking reaches the performance of prior masking approaches in half the training time, and consistently improves performance at the end of training.
Valid Causal Inference with (Some) Invalid Instruments
The technique is simple to apply and is "black-box" in the sense that it may be used with any instrumental variable estimator as long as the treatment effect is identified for each valid instrument independently.
Learning under Invariable Bayesian Safety
A recent body of work addresses safety constraints in explore-and-exploit systems.
Smarter Parking: Using AI to Identify Parking Inefficiencies in Vancouver
To see where this might be true in downtown Vancouver, we used artificial intelligence techniques to estimate the amount of time it would take drivers to both park on and off street for destinations throughout the city.
Fiduciary Bandits
Recommendation systems often face exploration-exploitation tradeoffs: the system can only learn about the desirability of new options by recommending them to some user.
Procrastinating with Confidence: Near-Optimal, Anytime, Adaptive Algorithm Configuration
Unfortunately, Structured Procrastination is not $\textit{adaptive}$ to characteristics of the parameterized algorithm: it treats every input like the worst case.
Deep Models of Interactions Across Sets
In experiments, our models achieved surprisingly good generalization performance on this matrix extrapolation task, both within domains (e. g., new users and new movies drawn from the same distribution used for training) and even across domains (e. g., predicting music ratings after training on movies).
Ranked #3 on
Recommendation Systems
on YahooMusic Monti
Deep IV: A Flexible Approach for Counterfactual Prediction
Counterfactual prediction requires understanding causal relationships between so-called treatment and outcome variables.
Deep Optimization for Spectrum Repacking
A crucial element of the auction design was the construction of a solver, dubbed SATFC, that determined whether sets of stations could be "repacked" in this way; it needed to run every time a station was given a price quote.
Efficient Benchmarking of Algorithm Configuration Procedures via Model-Based Surrogates
In our experiments, we construct and evaluate surrogate benchmarks for hyperparameter optimization as well as for AC problems that involve performance optimization of solvers for hard combinatorial problems, drawing training data from the runs of existing AC procedures.
Counterfactual Prediction with Deep Instrumental Variables Networks
We are in the middle of a remarkable rise in the use and capability of artificial intelligence.
Deep Learning for Predicting Human Strategic Behavior
Predicting the behavior of human participants in strategic settings is an important problem in many domains.
ASlib: A Benchmark Library for Algorithm Selection
To address this problem, we introduce a standardized format for representing algorithm selection scenarios and a repository that contains a growing number of data sets from the literature.
The Configurable SAT Solver Challenge (CSSC)
It is well known that different solution strategies work well for different types of instances of hard combinatorial problems.
Computational Analysis of Perfect-Information Position Auctions
After experimentation with other designs, the major search engines converged on the weighted, generalized second-price auction (wGSP) for selling keyword advertisements.
Empirically Evaluating Multiagent Learning Algorithms
Most such claims in the literature are based on small experiments, which has hampered understanding as well as the development of new multiagent learning (MAL) algorithms.
ParamILS: An Automatic Algorithm Configuration Framework
The identification of performance-optimizing parameter settings is an important part of the development and application of algorithms.
Bayesian Optimization With Censored Response Data
Bayesian optimization (BO) aims to minimize a given blackbox function using a model that is updated whenever new evidence about the function becomes available.
Algorithm Runtime Prediction: Methods & Evaluation
We also comprehensively describe new and existing features for predicting algorithm runtime for propositional satisfiability (SAT), travelling salesperson (TSP) and mixed integer programming (MIP) problems.
Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms
Many different machine learning algorithms exist; taking into account each algorithm's hyperparameters, there is a staggeringly large number of possible alternatives overall.
Sequential Model-Based Optimization for General Algorithm Configuration
State-of-the-art algorithms for hard computational problems often expose many parameters that can be modified to improve empirical performance.
Bayesian Action-Graph Games
Games of incomplete information, or Bayesian games, are an important game-theoretic model and have many applications in economics.
