Search Results for author:
Found 8 papers, 2 papers with code
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
Scaling test-time computation, or affording a generator large language model (LLM) extra compute during inference, typically employs the help of external non-generative evaluators (i. e., reward models).
Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings
While judge models -- LLMs finetuned to specialize in assessing and critiquing model outputs -- have been touted as general purpose evaluators, they are typically evaluated only on non-contextual scenarios, such as instruction following.
Direct Judgement Preference Optimization
Auto-evaluation is crucial for assessing response quality and offering feedback for model development.
SFR-RAG: Towards Contextually Faithful LLMs
Retrieval Augmented Generation (RAG), a paradigm that integrates external contextual information with large language models (LLMs) to enhance factual accuracy and relevance, has emerged as a pivotal area in generative AI.
Large Language Model Augmented Exercise Retrieval for Personalized Language Learning
We study the problem of zero-shot exercise retrieval in the context of online language learning, to give learners the ability to explicitly request personalized exercises via natural language.
HandsOff: Labeled Dataset Generation With No Additional Human Annotations
Recent work leverages the expressive power of generative adversarial networks (GANs) to generate labeled synthetic datasets.
Active metric learning and classification using similarity queries
Motivated by this, we propose a novel unified query framework that can be applied to any problem in which a key component is learning a representation of the data that reflects similarity.
Simultaneous Preference and Metric Learning from Paired Comparisons
The underlying assumption in this model is that a smaller distance between $\mathbf{u}$ and an item $\mathbf{x_j}$ indicates a stronger preference for $\mathbf{x_j}$.
