MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks

1 code implementation • 14 Oct 2024 • Jiacheng Chen, Tianhao Liang, Sherman Siu, Zhengqing Wang, Kai Wang, YuBo Wang, Yuansheng Ni, Wang Zhu, Ziyan Jiang, Bohan Lyu, Dongfu Jiang, Xuan He, YuAn Liu, Hexiang Hu, Xiang Yue, Wenhu Chen

We evaluate a wide variety of frontier vision-language models on MEGA-Bench to understand their capabilities across these dimensions.

VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation

no code implementations • 21 Jun 2024 • Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, Kai Wang, Quy Duc Do, Yuansheng Ni, Bohan Lyu, Yaswanth Narsupalli, Rongqi Fan, Zhiheng Lyu, YuChen Lin, Wenhu Chen

The main barrier is the lack of large-scale human-annotated dataset.

Video Generation Video Quality Assessment

Towards automating Codenames spymasters with deep reinforcement learning

no code implementations • 28 Dec 2022 • Sherman Siu

Although most reinforcement learning research has centered on competitive games, little work has been done on applying it to co-operative multiplayer games or text-based games.

Deep Reinforcement Learning reinforcement-learning +2