Search Results for author:
Found 23 papers, 3 papers with code
Exploring Straightforward Conversational Red-Teaming
Large language models (LLMs) are increasingly used in business dialogue systems but they pose security and ethical risks.
Can You Trust Your Metric? Automatic Concatenation-Based Tests for Metric Validity
We applied these tests in a model safety scenario to assess the reliability of harmfulness detection metrics, uncovering a number of inconsistencies.
A Novel Metric for Measuring the Robustness of Large Language Models in Non-adversarial Scenarios
We evaluate the robustness of several large language models on multiple datasets.
Generating Unseen Code Tests In Infinitum
Large Language Models (LLMs) are used for many tasks, including those related to coding.
Using Combinatorial Optimization to Design a High quality LLM Solution
Next, a set of interactions between the factors are defined and combinatorial optimization is used to create a small subset $P$ that ensures all desired interactions occur in $P$.
Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations
Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations.
Unveiling Safety Vulnerabilities of Large Language Models
As large language models become more prevalent, their possible harmful or inappropriate responses are a cause for concern.
Predicting Question-Answering Performance of Large Language Models through Semantic Consistency
Semantic consistency of a language model is broadly defined as the model's ability to produce semantically-equivalent outputs, given semantically-equivalent inputs.
Characterizing how 'distributional' NLP corpora distance metrics are
We quantify this quality by constructing a Known-Similarity Corpora set from two paraphrase corpora and calculating the distance between paired corpora from it.
Data Drift Monitoring for Log Anomaly Detection Pipelines
Logs enable the monitoring of infrastructure status and the performance of associated applications.
Automatic Generation of Attention Rules For Containment of Machine Learning Model Errors
The approach we propose is to separate the observations that are the most likely to be predicted incorrectly into 'attention sets'.
Convex Bounds on the Softmax Function with Applications to Robustness Verification
The softmax function is a ubiquitous component at the output of neural networks and increasingly in intermediate layers as well.
Measuring the Measuring Tools: An Automatic Evaluation of Semantic Metrics for Text Corpora
The ability to compare the semantic similarity between text corpora is important in a variety of natural language processing applications.
Theory and Practice of Quality Assurance for Machine Learning Systems An Experiment Driven Approach
The crafting of machine learning (ML) based systems requires statistical control throughout its life cycle.
Using sequential drift detection to test the API economy
It is thus desirable to monitor the usage patterns and identify when the system is used in a way that was never used before.
Detecting model drift using polynomial relations
We thus use a set of learned strong polynomial relations to identify drift.
Density-based interpretable hypercube region partitioning for mixed numeric and categorical data
A user may want to know where in the feature space observations are concentrated, and where it is sparse or empty.
Towards API Testing Across Cloud and Edge
API economy is driving the digital transformation of business applications across the hybrid Cloud and edge environments.
Machine Learning Model Drift Detection Via Weak Data Slices
Detecting drift in performance of Machine Learning (ML) models is an acknowledged challenge.
Broadly Applicable Targeted Data Sample Omission Attacks
We show that, with a low attack budget, our attack's success rate is above 80%, and in some cases 100%, for white-box learning.
Detection of data drift and outliers affecting machine learning model performance over time
Drift is distribution change between the training and deployment data, which is concerning if model performance changes.
Sequential Drift Detection in Deep Learning Classifiers
We utilize neural network embeddings to detect data drift by formulating the drift detection within an appropriate sequential decision framework.
Defending via strategic ML selection
There are cases in which an adversary can strategically tamper with the input data to affect the outcome of the learning process.
