Search Results for author:
Found 24 papers, 13 papers with code
Automated Benchmark Generation for Repository-Level Coding Tasks
The correctness of generated patches is then evaluated by executing a human-written test suite extracted from the repository after the issue's resolution.
Average Certified Radius is a Poor Metric for Randomized Smoothing
Randomized smoothing is a popular approach for providing certified robustness guarantees against adversarial attacks, and has become an active area of research.
Diagnosing Robotics Systems Issues with Large Language Models
Here, we extend this work to the challenging and largely unexplored domain of robotics systems.
Mitigating Catastrophic Forgetting in Language Transfer via Model Merging
As open-weight large language models (LLMs) achieve ever more impressive performances across a wide range of tasks in English, practitioners aim to adapt these models to different languages.
SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents
We find that LLMs generally perform surprisingly well at generating relevant test cases, with Code Agents designed for code repair exceeding the performance of systems designed specifically for test generation.
ConStat: Performance-Based Contamination Detection in Large Language Models
To overcome these limitations, we propose a novel definition of contamination as artificially inflated and non-generalizing benchmark performance instead of the inclusion of benchmark samples in the training data.
DAGER: Exact Gradient Inversion for Large Language Models
Federated learning works by aggregating locally computed gradients from multiple clients, thus enabling collaborative training without sharing private client data.
Gaussian Loss Smoothing Enables Certified Training with Tight Convex Relaxations
In this work, we theoretically show that Gaussian Loss Smoothing (GLS) can alleviate these issues.
SPEAR:Exact Gradient Inversion of Batches in Federated Learning
In this work, we propose SPEAR, the first algorithm reconstructing whole batches with $b >1$ exactly.
Evading Data Contamination Detection for Language Models is (too) Easy
Large language models are widespread, with their performance on benchmarks frequently guiding user preferences for one model over another.
Automated Classification of Model Errors on ImageNet
To address this, new label-sets and evaluation protocols have been proposed for ImageNet showing that state-of-the-art models already achieve over 95% accuracy and shifting the focus on investigating why the remaining errors persist.
Prompt Sketching for Large Language Models
This way, sketching grants users more control over the generation process, e. g., by providing a reasoning framework via intermediate instructions, leading to better overall results.
Expressivity of ReLU-Networks under Convex Relaxations
We show that: (i) more advanced relaxations allow a larger class of univariate functions to be expressed as precisely analyzable ReLU networks, (ii) more precise relaxations can allow exponentially larger solution spaces of ReLU networks encoding the same functions, and (iii) even using the most precise single-neuron relaxations, it is impossible to construct precisely analyzable ReLU networks that express multivariate, convex, monotone CPWL functions.
Understanding Certified Training with Interval Bound Propagation
We, then, derive sufficient and necessary conditions on weight matrices for IBP bounds to become exact and demonstrate that these impose strong regularization, explaining the empirically observed trade-off between robustness and accuracy in certified training.
TAPS: Connecting Certified and Adversarial Training
Training certifiably robust neural networks remains a notoriously hard problem.
Efficient Certified Training and Robustness Verification of Neural ODEs
Neural Ordinary Differential Equations (NODEs) are a novel neural architecture, built around initial value problems with learned dynamics which are solved during inference.
First Three Years of the International Verification of Neural Networks Competition (VNN-COMP)
This paper presents a summary and meta-analysis of the first three iterations of the annual International Verification of Neural Networks Competition (VNN-COMP) held in 2020, 2021, and 2022.
The Third International Verification of Neural Networks Competition (VNN-COMP 2022): Summary and Results
This report summarizes the 3rd International Verification of Neural Networks Competition (VNN-COMP 2022), held as a part of the 5th Workshop on Formal Methods for ML-Enabled Autonomous Systems (FoMLAS), which was collocated with the 34th International Conference on Computer-Aided Verification (CAV).
Certified Training: Small Boxes are All You Need
To obtain, deterministic guarantees of adversarial robustness, specialized training methods are used.
(De-)Randomized Smoothing for Decision Stump Ensembles
Whereas most prior work on randomized smoothing focuses on evaluating arbitrary base models approximately under input randomization, the key insight of our work is that decision stump ensembles enable exact yet efficient evaluation via dynamic programming.
Robust and Accurate -- Compositional Architectures for Randomized Smoothing
Randomized Smoothing (RS) is considered the state-of-the-art approach to obtain certifiably robust models for challenging tasks.
Abstract Interpretation of Fixpoint Iterators with Applications to Neural Networks
We present a new abstract interpretation framework for the precise over-approximation of numerical fixpoint iterators.
Boosting Randomized Smoothing with Variance Reduced Classifiers
Randomized Smoothing (RS) is a promising method for obtaining robustness certificates by evaluating a base model under noise.
PRIMA: General and Precise Neural Network Certification via Scalable Convex Hull Approximations
Formal verification of neural networks is critical for their safe adoption in real-world applications.
