Search Results for author:
Found 31 papers, 21 papers with code
Humanity's Last Exam
However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities.
Ranked #4 on
Humanity's Last Exam
on Humanity's Last Exam
LiveBench: A Challenging, Contamination-Free LLM Benchmark
In this work, we introduce a new benchmark for LLMs designed to be immune to both test set contamination and the pitfalls of LLM judging and human crowdsourcing.
Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs
Existing neural operator architectures face challenges when solving multiphysics problems with coupled partial differential equations (PDEs) due to complex geometries, interactions between physical variables, and the limited amounts of high-resolution training data.
Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
In this work, first we show theoretically that the standard DPO loss can lead to a reduction of the model's likelihood of the preferred examples, as long as the relative probability between the preferred and dispreferred classes increases.
TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
Notably, TabPFN achieves very strong performance on small tabular datasets but is not designed to make predictions for datasets of size larger than 1000.
ForecastPFN: Synthetically-Trained Zero-Shot Forecasting
The vast majority of time-series forecasting approaches require a substantial training dataset.
Data Contamination Through the Lens of Time
Recent claims about the impressive abilities of large language models (LLMs) are often supported by evaluating publicly available benchmarks.
Guaranteed Approximation Bounds for Mixed-Precision Neural Operators
Neural operators, such as Fourier Neural Operators (FNO), form a principled approach for learning solution operators for PDEs and other mappings between function spaces.
When Do Neural Nets Outperform Boosted Trees on Tabular Data?
To this end, we conduct the largest tabular data analysis to date, comparing 19 algorithms across 176 datasets, and we find that the 'NN vs. GBDT' debate is overemphasized: for a surprisingly high number of datasets, either the performance difference between GBDTs and NNs is negligible, or light hyperparameter tuning on a GBDT is more important than choosing between NNs and GBDTs.
Neural Architecture Search: Insights from 1000 Papers
Specialized, high-performing neural architectures are crucial to the success of deep learning in these areas.
Natural Language Understanding
Neural Architecture Search
+2
Speeding up NAS with Adaptive Subset Selection
A majority of recent developments in neural architecture search (NAS) have been aimed at decreasing the computational cost of various techniques without affecting their final performance.
Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition
Our search outputs a suite of models which Pareto-dominate all other high-performance architectures and existing bias mitigation methods in terms of accuracy and fairness, often by large margins, on the two most widely used datasets for face identification, CelebA and VGGFace2.
AutoML for Climate Change: A Call to Action
The challenge that climate change poses to humanity has spurred a rapidly developing field of artificial intelligence research focused on climate change applications.
NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies
Zero-cost proxies (ZC proxies) are a recent architecture performance prediction technique aiming to significantly speed up algorithms for neural architecture search (NAS).
On the Generalizability and Predictability of Recommender Systems
By using far more meta-training data than prior work, RecZilla is able to substantially reduce the level of human involvement when faced with a new recommender system application.
NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy
The release of tabular benchmarks, such as NAS-Bench-101 and NAS-Bench-201, has significantly lowered the computational overhead for conducting scientific research in neural architecture search (NAS).
NAS-Bench-x11 and the Power of Learning Curves
While early research in neural architecture search (NAS) required extreme computational resources, the recent releases of tabular and surrogate benchmarks have greatly increased the speed and reproducibility of NAS research.
Synthetic Benchmarks for Scientific Research in Explainable Machine Learning
In this work, we address this issue by releasing XAI-Bench: a suite of synthetic datasets along with a library for benchmarking feature attribution algorithms.
How Powerful are Performance Predictors in Neural Architecture Search?
Early methods in the rapidly developing field of neural architecture search (NAS) required fully training thousands of neural networks.
A Study on Encodings for Neural Architecture Search
First we formally define architecture encodings and give a theoretical characterization on the scalability of the encodings we study Then we identify the main encoding-dependent subroutines which NAS algorithms employ, running experiments to show which encodings work best with each subroutine for many popular algorithms.
Intra-Processing Methods for Debiasing Neural Networks
Intra-processing methods are designed specifically to debias large models which have been trained on a generic dataset and fine-tuned on a more specific task.
Exploring the Loss Landscape in Neural Architecture Search
In this work, we show that (1) the simplest hill-climbing algorithm is a powerful baseline for NAS, and (2), when the noise in popular NAS benchmark datasets is reduced to a minimum, hill-climbing to outperforms many popular state-of-the-art algorithms.
BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search
Bayesian optimization (BO), which has long had success in hyperparameter optimization, has recently emerged as a very promising strategy for NAS when it is coupled with a neural predictor.
BANANAS: Bayesian Optimization with Neural Networks for Neural Architecture Search
We develop a path-based encoding scheme to featurize the neural architectures that are used to train the neural network model.
Data-Driven Clustering via Parameterized Lloyd's Families
Algorithms for clustering points in metric spaces is a long-studied area of research.
Clustering under Local Stability: Bridging the Gap between Worst-Case and Beyond Worst-Case Analysis
The typical idea is to design a clustering algorithm that outputs a near-optimal solution, provided the data satisfy a natural stability notion.
Robust Communication-Optimal Distributed Clustering Algorithms
In this work, we study the $k$-median and $k$-means clustering problems when the data is distributed across many servers and can contain outliers.
Learning-Theoretic Foundations of Algorithm Configuration for Combinatorial Partitioning Problems
We address this problem for clustering, max-cut, and other partitioning problems, such as integer quadratic programming, by designing computationally efficient and sample efficient learning algorithms which receive samples from an application-specific distribution over problem instances and learn a partitioning algorithm with high expected performance.
Learning Combinatorial Functions from Pairwise Comparisons
However, for real-valued functions, cardinal labels might not be accessible, or it may be difficult for an expert to consistently assign real-valued labels over the entire set of examples.
Data Driven Resource Allocation for Distributed Learning
In distributed machine learning, data is dispatched to multiple machines for processing.
$k$-center Clustering under Perturbation Resilience
In this work, we take this approach and provide strong positive results both for the asymmetric and symmetric $k$-center problems under a natural input stability (promise) condition called $\alpha$-perturbation resilience [Bilu and Linia 2012], which states that the optimal solution does not change under any alpha-factor perturbation to the input distances.
