Search Results for author:
Found 14 papers, 9 papers with code
StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs
As Large Language Models (LLMs) become integral to software development workflows, their ability to generate structured outputs has become critically important.
QuickVideo: Real-Time Long Video Understanding with System Algorithm Co-Design
Long-video understanding has emerged as a crucial capability in real-world applications such as video surveillance, meeting summarization, educational lecture analysis, and sports broadcasting.
ACECODER: Acing Coder RL via Automated Test-Case Synthesis
Notably, we follow the R1-style training to start from Qwen2. 5-Coder-base directly and show that our RL training can improve model on HumanEval-plus by over 25\% and MBPP-plus by 6\% for merely 80 optimization steps.
MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks
We evaluate a wide variety of frontier vision-language models on MEGA-Bench to understand their capabilities across these dimensions.
VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation
The main barrier is the lack of large-scale human-annotated dataset.
WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences
Recent breakthroughs in vision-language models (VLMs) emphasize the necessity of benchmarking human preferences in real-world multimodal interactions.
GenAI Arena: An Open Evaluation Platform for Generative Models
Generative AI has made remarkable strides to revolutionize fields such as image and video generation.
MANTIS: Interleaved Multi-Image Instruction Tuning
We further evaluate Mantis on single-image benchmarks and demonstrate that Mantis also maintains a strong single-image performance on par with CogVLM and Emu2.
VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation
In the rapidly advancing field of conditional image generation research, challenges such as limited explainability lie in effectively evaluating the performance and capabilities of various models.
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning.
TIGERScore: Towards Building Explainable Metric for All Text Generation Tasks
To quantitatively assess our metric, we evaluate its correlation with human ratings on 5 held-in datasets, 2 held-out datasets and show that TIGERScore can achieve the open-source SoTA correlation with human ratings across these datasets and almost approaches GPT-4 evaluator.
LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion
We present LLM-Blender, an ensembling framework designed to attain consistently superior performance by leveraging the diverse strengths of multiple open-source large language models (LLMs).
PairReranker: Pairwise Reranking for Natural Language Generation
Pre-trained language models have been successful in natural language generation (NLG) tasks.
