Search Results for author:
Found 9 papers, 2 papers with code
ASAP: Interpretable Analysis and Summarization of AI-generated Image Patterns at Scale
Generative image models have emerged as a promising technology to produce realistic images.
InterVLS: Interactive Model Understanding and Improvement with Vision-Language Surrogates
Deep learning models are widely used in critical applications, highlighting the need for pre-deployment model understanding and improvement.
LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity
Cross-task generalization is a significant outcome that defines mastery in natural language understanding.
Real-Time Visual Feedback to Guide Benchmark Creation: A Human-and-Metric-in-the-Loop Workflow
In pursuit of creating better benchmarks, we propose VAIDA, a novel benchmark creation paradigm for NLP, that focuses on guiding crowdworkers, an under-explored facet of addressing benchmark idiosyncrasies.
Hardness of Samples Need to be Quantified for a Reliable Evaluation System: Exploring Potential Opportunities with a New Task
Evaluation of models on benchmarks is unreliable without knowing the degree of sample hardness; this subsequently overestimates the capability of AI systems and limits their adoption in real world applications.
A Survey of Parameters Associated with the Quality of Benchmarks in NLP
Inspired by successful quality indices in several domains such as power, food, and water, we take the first step towards a metric by identifying certain language properties that can represent various possible interactions leading to biases in a benchmark.
DQI: A Guide to Benchmark Evaluation
A `state of the art' model A surpasses humans in a benchmark B, but fails on similar benchmarks C, D, and E. What does B have that the other benchmarks do not?
Our Evaluation Metric Needs an Update to Encourage Generalization
In order to stop the inflation in model performance -- and thus overestimation in AI systems' capabilities -- we propose a simple and novel evaluation metric, WOOD Score, that encourages generalization during evaluation.
DQI: Measuring Data Quality in NLP
The data creation paradigm consists of several data visualizations to help data creators (i) understand the quality of data and (ii) visualize the impact of the created data instance on the overall quality.
