Search Results for author:
Found 23 papers, 7 papers with code
Focus-N-Fix: Region-Aware Fine-Tuning for Text-to-Image Generation
Text-to-image (T2I) generation has made significant advances in recent years, but challenges still remain in the generation of perceptual artifacts, misalignment with complex prompts, and safety.
Personalized and Sequential Text-to-Image Generation
We address the problem of personalized, interactive text-to-image (T2I) generation, designing a reinforcement learning (RL) agent which iteratively improves a set of generated images for a user through a sequence of prompt expansions.
Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation
Human feedback plays a critical role in learning and refining reward models for text-to-image generation, but the optimal form the feedback should take for learning an accurate reward function has not been conclusively established.
Online Bandit Learning with Offline Preference Data
We propose $\texttt{warmPref-PS}$, a posterior sampling algorithm for online learning that can be warm-started with an offline dataset with noisy preference feedback.
e-COP : Episodic Constrained Optimization of Policies
In this paper, we present the $\texttt{e-COP}$ algorithm, the first policy optimization algorithm for constrained Reinforcement Learning (RL) in episodic (finite horizon) settings.
Using Domain Knowledge to Guide Dialog Structure Induction via Neural Probabilistic Soft Logic
Dialog Structure Induction (DSI) is the task of inferring the latent dialog structure (i. e., a set of dialog states and their temporal transitions) of a given goal-oriented dialog.
Understanding Subjectivity through the Lens of Motivational Context in Model-Generated Image Satisfaction
Image generation models are poised to become ubiquitous in a range of applications.
Prompt Expansion for Adaptive Text-to-Image Generation
Text-to-image generation models are powerful but difficult to use.
Rich Human Feedback for Text-to-Image Generation
We show that the predicted rich human feedback can be leveraged to improve image generation, for example, by selecting high-quality training data to finetune and improve the generative models, or by creating masks with predicted heatmaps to inpaint the problematic regions.
Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking
However, even pretrain reward ensembles do not eliminate reward hacking: we show several qualitative reward hacking phenomena that are not mitigated by ensembling because all reward models in the ensemble exhibit similar error patterns.
Modeling subjectivity (by Mimicking Annotator Annotation) in toxic comment identification across diverse communities
The prevalence and impact of toxic discussions online have made content moderation crucial. Automated systems can play a vital role in identifying toxicity, and reducing the reliance on human moderation. Nevertheless, identifying toxic comments for diverse communities continues to present challenges that are addressed in this paper. The two-part goal of this study is to(1)identify intuitive variances from annotator disagreement using quantitative analysis and (2)model the subjectivity of these viewpoints. To achieve our goal, we published a new dataset\footnote{\url{https://github. com/XXX}} with expert annotators' annotations and used two other public datasets to identify the subjectivity of toxicity. Then leveraging the Large Language Model(LLM), we evaluate the model's ability to mimic diverse viewpoints on toxicity by varying size of the training data and utilizing same set of annotators as the test set used during model training and a separate set of annotators as the test set. We conclude that subjectivity is evident across all annotator groups, demonstrating the shortcomings of majority-rule voting.
Demystifying Embedding Spaces using Large Language Models
Embeddings have become a pivotal means to represent complex, multi-faceted information about entities, concepts, and relationships in a condensed and useful format.
TaskLAMA: Probing the Complex Task Understanding of Language Models
Our experiments reveal that LLMs are able to decompose complex tasks into individual steps effectively, with a relative improvement of 15% to 280% over the best baseline.
Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play
Standard empirical risk minimization (ERM) training can produce deep neural network (DNN) models that are accurate on average but under-perform in under-represented population subgroups, especially when there are imbalanced group distributions in the long-tailed training data.
Understanding Finetuning for Factual Knowledge Extraction from Language Models
Recently, it has been shown that finetuning LMs on a set of factual knowledge makes them produce better answers to queries from a different set, thus making finetuned LMs a good candidate for knowledge extraction and, consequently, knowledge graph construction.
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language
Remarkable progress has been made on automated reasoning with natural text, by using Language Models (LMs) and methods such as Chain-of-Thought and Selection-Inference.
Tackling Provably Hard Representative Selection via Graph Neural Networks
Representative Selection (RS) is the problem of finding a small subset of exemplars from a dataset that is representative of the dataset.
FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue
Task transfer, transferring knowledge contained in related tasks, holds the promise of reducing the quantity of labeled data required to fine-tune language models.
Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors
Interactive recommender systems have emerged as a promising paradigm to overcome the limitations of the primitive user feedback used by traditional recommender systems (e. g., clicks, item consumption, ratings).
Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering
Through a user preference study, we demonstrate that the oracle behavior of our proposed system that provides responses based on presupposition failure is preferred over the oracle behavior of existing QA systems.
Do Language Embeddings Capture Scales?
Pretrained Language Models (LMs) have been shown to possess significant linguistic, common sense, and factual knowledge.
How Large Are Lions? Inducing Distributions over Quantitative Attributes
Most current NLP systems have little knowledge about quantitative attributes of objects and events.
