Search Results for author:
Found 16 papers, 1 papers with code
Can You Trust Your Metric? Automatic Concatenation-Based Tests for Metric Validity
We applied these tests in a model safety scenario to assess the reliability of harmfulness detection metrics, uncovering a number of inconsistencies.
Generating Unseen Code Tests In Infinitum
Large Language Models (LLMs) are used for many tasks, including those related to coding.
Using Combinatorial Optimization to Design a High quality LLM Solution
Next, a set of interactions between the factors are defined and combinatorial optimization is used to create a small subset $P$ that ensures all desired interactions occur in $P$.
Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations
Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations.
Unveiling Safety Vulnerabilities of Large Language Models
As large language models become more prevalent, their possible harmful or inappropriate responses are a cause for concern.
Predicting Question-Answering Performance of Large Language Models through Semantic Consistency
Semantic consistency of a language model is broadly defined as the model's ability to produce semantically-equivalent outputs, given semantically-equivalent inputs.
Automatic Generation of Attention Rules For Containment of Machine Learning Model Errors
The approach we propose is to separate the observations that are the most likely to be predicted incorrectly into 'attention sets'.
Measuring the Measuring Tools: An Automatic Evaluation of Semantic Metrics for Text Corpora
The ability to compare the semantic similarity between text corpora is important in a variety of natural language processing applications.
Theory and Practice of Quality Assurance for Machine Learning Systems An Experiment Driven Approach
The crafting of machine learning (ML) based systems requires statistical control throughout its life cycle.
Classifier Data Quality: A Geometric Complexity Based Method for Automated Baseline And Insights Generation
Testing Machine Learning (ML) models and AI-Infused Applications (AIIAs), or systems that contain ML models, is highly challenging.
Automatically detecting data drift in machine learning classifiers
The assumption underlying statistical ML resulting in theoretical or empirical performance guarantees is that the distribution of the training data is representative of the production data distribution.
Detecting model drift using polynomial relations
We thus use a set of learned strong polynomial relations to identify drift.
Density-based interpretable hypercube region partitioning for mixed numeric and categorical data
A user may want to know where in the feature space observations are concentrated, and where it is sparse or empty.
FreaAI: Automated extraction of data slices to test machine learning models
In this paper we show the feasibility of automatically extracting feature models that result in explainable data slices over which the ML solution under-performs.
Machine Learning Model Drift Detection Via Weak Data Slices
Detecting drift in performance of Machine Learning (ML) models is an acknowledged challenge.
Detection of data drift and outliers affecting machine learning model performance over time
Drift is distribution change between the training and deployment data, which is concerning if model performance changes.
