Search Results for author:
Found 17 papers, 2 papers with code
Predicting Question-Answering Performance of Large Language Models through Semantic Consistency
Semantic consistency of a language model is broadly defined as the model's ability to produce semantically-equivalent outputs, given semantically-equivalent inputs.
Characterizing how 'distributional' NLP corpora distance metrics are
We quantify this quality by constructing a Known-Similarity Corpora set from two paraphrase corpora and calculating the distance between paired corpora from it.
Data Drift Monitoring for Log Anomaly Detection Pipelines
Logs enable the monitoring of infrastructure status and the performance of associated applications.
Reliable and Interpretable Drift Detection in Streams of Short Texts
Data drift is the change in model input data that is one of the key factors leading to machine learning models performance degradation over time.
Automatic Generation of Attention Rules For Containment of Machine Learning Model Errors
The approach we propose is to separate the observations that are the most likely to be predicted incorrectly into 'attention sets'.
Measuring the Measuring Tools: An Automatic Evaluation of Semantic Metrics for Text Corpora
The ability to compare the semantic similarity between text corpora is important in a variety of natural language processing applications.
Theory and Practice of Quality Assurance for Machine Learning Systems An Experiment Driven Approach
The crafting of machine learning (ML) based systems requires statistical control throughout its life cycle.
Classifier Data Quality: A Geometric Complexity Based Method for Automated Baseline And Insights Generation
Testing Machine Learning (ML) models and AI-Infused Applications (AIIAs), or systems that contain ML models, is highly challenging.
Automatically detecting data drift in machine learning classifiers
The assumption underlying statistical ML resulting in theoretical or empirical performance guarantees is that the distribution of the training data is representative of the production data distribution.
Using sequential drift detection to test the API economy
It is thus desirable to monitor the usage patterns and identify when the system is used in a way that was never used before.
Detecting model drift using polynomial relations
We thus use a set of learned strong polynomial relations to identify drift.
Density-based interpretable hypercube region partitioning for mixed numeric and categorical data
A user may want to know where in the feature space observations are concentrated, and where it is sparse or empty.
Towards API Testing Across Cloud and Edge
API economy is driving the digital transformation of business applications across the hybrid Cloud and edge environments.
FreaAI: Automated extraction of data slices to test machine learning models
In this paper we show the feasibility of automatically extracting feature models that result in explainable data slices over which the ML solution under-performs.
Machine Learning Model Drift Detection Via Weak Data Slices
Detecting drift in performance of Machine Learning (ML) models is an acknowledged challenge.
Detection of data drift and outliers affecting machine learning model performance over time
Drift is distribution change between the training and deployment data, which is concerning if model performance changes.
Sequential Drift Detection in Deep Learning Classifiers
We utilize neural network embeddings to detect data drift by formulating the drift detection within an appropriate sequential decision framework.
