Search Results for author:
Found 8 papers, 7 papers with code
m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Tasks
With m&m's, we evaluate 6 popular LLMs with 2 planning strategies (multi-step vs. step-by-step planning), 2 plan formats (JSON vs. code), and 3 types of feedback (parsing/verification/execution).
SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality
In the last year alone, a surge of new benchmarks to measure compositional understanding of vision-language models have permeated the machine learning ecosystem.
Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design
Machine learning practitioners often end up tunneling on low-level technical details like model architectures and performance metrics.
CREPE: Can Vision-Language Foundation Models Reason Compositionally?
To measure systematicity, CREPE consists of a test dataset containing over $370K$ image-text pairs and three different seen-unseen splits.
ELIGN: Expectation Alignment as a Multi-Agent Intrinsic Reward
These results identify tasks where expectation alignment is a more useful strategy than curiosity-driven exploration for multi-agent coordination, enabling agents to do zero-shot coordination.
MobilePhys: Personalized Mobile Camera-Based Contactless Physiological Sensing
Camera-based contactless photoplethysmography refers to a set of popular techniques for contactless physiological measurement.
OpenAttack: An Open-source Textual Adversarial Attack Toolkit
Textual adversarial attacking has received wide and increasing attention in recent years.
