Search Results for author:
Found 4 papers, 0 papers with code
Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
In particular, we identify a phenomenon we dub \emph{criteria drift}: users need criteria to grade outputs, but grading outputs helps users define criteria.
Revisiting Prompt Engineering via Declarative Crowdsourcing
Large language models (LLMs) are incredibly powerful at comprehending and generating data in the form of text, but are brittle and error-prone.
Operationalizing Machine Learning: An Interview Study
Organizations rely on machine learning engineers (MLEs) to operationalize ML, i. e., deploy and maintain ML pipelines in production.
Rethinking Streaming Machine Learning Evaluation
While most work on evaluating machine learning (ML) models focuses on computing accuracy on batches of data, tracking accuracy alone in a streaming setting (i. e., unbounded, timestamp-ordered datasets) fails to appropriately identify when models are performing unexpectedly.
