Search Results for author:
Found 69 papers, 23 papers with code
ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback
The work presents our practices of aligning LLMs with human preferences, offering insights into the challenges and solutions in RLHF implementations.
Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation
We introduce Adversarial Policy Optimization (AdvPO), a novel solution to the pervasive issue of reward over-optimization in Reinforcement Learning from Human Feedback (RLHF) for Large Language Models (LLMs).
Federated Linear Contextual Bandits with Heterogeneous Clients
The demand for collaborative and private bandit learning across multiple agents is surging due to the growing quantity of data generated from distributed systems.
ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors
The safety of Large Language Models (LLMs) has gained increasing attention in recent years, but there still lacks a comprehensive approach for detecting safety issues within LLMs' responses in an aligned, customizable and explainable manner.
Stealthy Adversarial Attacks on Stochastic Multi-Armed Bandits
Our analysis shows that against two popularly employed MAB algorithms, UCB1 and $\epsilon$-greedy, the success of a stealthy attack depends on the environmental conditions and the realized reward of the arm pulled in the first round.
Incentivized Truthful Communication for Federated Bandits
To enhance the efficiency and practicality of federated bandit learning, recent advances have introduced incentives to motivate communication among clients, where a client participates only when the incentive offered by the server outweighs its participation cost.
AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback
We present AMOR, an agent framework based on open-source LLMs, which reasons with external knowledge bases and adapts to specific domains through human supervision to the reasoning process.
Towards Efficient and Exact Optimization of Language Model Alignment
We prove that EXO is guaranteed to optimize in the same direction as the RL algorithms asymptotically for arbitary parametrization of the policy, while enables efficient optimization by circumventing the complexities associated with RL algorithms.
The Impact of Snippet Reliability on Misinformation in Online Health Search
These results strongly suggest that an information need-focused approach can significantly improve the reliability of extracted snippets in online health search.
AlignBench: Benchmarking Chinese Alignment of Large Language Models
We will provide public APIs for evaluating AlignBench with CritiqueLLM to facilitate the evaluation of LLMs' Chinese alignment.
CritiqueLLM: Scaling LLM-as-Critic for Effective and Explainable Evaluation of Large Language Model Generation
Since the natural language processing (NLP) community started to make large language models (LLMs), such as GPT-4, act as a critic to evaluate the quality of generated texts, most of them only train a critique generation model of a specific scale on specific datasets.
Black-Box Prompt Optimization: Aligning Large Language Models without Model Training
However, these models are often not well aligned with human intents, which calls for additional treatments on them, that is, the alignment problem.
Language Model Decoding as Direct Metrics Optimization
And most importantly, we prove that this induced distribution is guaranteed to improve the perplexity on human texts, which suggests a better approximation to the underlying distribution of human texts.
Meta-Reinforcement Learning via Exploratory Task Clustering
In this paper, we explore the structured heterogeneity among tasks via clustering to improve meta-RL.
Debiasing Recommendation by Learning Identifiable Latent Confounders
Confounding bias arises due to the presence of unmeasured variables (e. g., the socio-economic status of a user) that can affect both a user's exposure and feedback.
How Bad is Top-$K$ Recommendation under Competing Content Creators?
Content creators compete for exposure on recommendation platforms, and such strategic behavior leads to a dynamic shift over the content distribution.
Disentangled Representation for Diversified Recommendations
This calls for a fine-grained understanding of a user's preferences over items, where one needs to recognize the user's choice is driven by the quality of the item itself, or the pre-selected attributes of the item.
MiddleGAN: Generate Domain Agnostic Samples for Unsupervised Domain Adaptation
We extend the theory of GAN to show that there exist optimal solutions for the parameters of the two discriminators and one generator in MiddleGAN, and empirically show that the samples generated by the MiddleGAN are similar to both samples from the source domain and samples from the target domain.
COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation
However, the bias inherent in user written text, often used for PTG model training, can inadvertently associate different levels of linguistic quality with users' protected attributes.
Spectral Augmentation for Self-Supervised Learning on Graphs
Graph contrastive learning (GCL), as an emerging self-supervised learning technique on graphs, aims to learn representations via instance discrimination.
Rethinking Conversational Recommendations: Is Decision Tree All You Need?
Fourthly, when the user rejects a recommendation, we adaptively choose the next decision tree to improve subsequent questions and recommendations.
Dynamic Global Sensitivity for Differentially Private Contextual Bandits
We provide a rigorous theoretical analysis over the amount of noise added via dynamic global sensitivity and the corresponding upper regret bound of our proposed algorithm.
Not Just Skipping. Understanding the Effect of Sponsored Content on Users' Decision-Making in Online Health Search
We further discover that the contrast between the indirect marketing ads and the viewpoint presented in the organic search results plays an important role in users' decision-making.
Scalable Exploration for Neural Online Learning to Rank with Perturbed Feedback
Deep neural networks (DNNs) demonstrate significant advantages in improving ranking performance in retrieval tasks.
Communication Efficient Distributed Learning for Kernelized Contextual Bandits
We tackle the communication efficiency challenge of learning kernelized contextual bandits in a distributed setting.
Meta Policy Learning for Cold-Start Conversational Recommendation
We propose to learn a meta policy and adapt it to new users with only a few trials of conversational recommendations.
Graph-based Extractive Explainer for Recommendations
Explanations in a recommender system assist users in making informed decisions among a set of recommended items.
Learning from a Learning User for Optimal Recommendations
In real-world recommendation problems, especially those with a formidably large item space, users have to gradually learn to estimate the utility of any fresh recommendations from their experience about previously consumed items.
Communication Efficient Federated Learning for Generalized Linear Bandits
Contextual bandit algorithms have been recently studied under the federated learning setting to satisfy the demand of keeping data decentralized and pushing the learning of bandit models to the client side.
E-ADDA: Unsupervised Adversarial Domain Adaptation Enhanced by a New Mahalanobis Distance Loss for Smart Computing
In the computer vision modality, the evaluation results suggest that we achieve new state-of-the-art performance on popular UDA benchmarks such as Office-31 and Office-Home, outperforming the second best-performing algorithms by up to 17. 9%.
Out-of-Distribution Detection
Unsupervised Domain Adaptation
IMO$^3$: Interactive Multi-Objective Off-Policy Optimization
This problem has been studied extensively in the setting of known objective functions.
Learning Neural Contextual Bandits Through Perturbed Rewards
Thanks to the power of representation learning, neural contextual bandit algorithms demonstrate remarkable performance improvement against their classical counterparts.
Learning Neural Ranking Models Online from Implicit User Feedback
Existing online learning to rank (OL2R) solutions are limited to linear models, which are incompetent to capture possible non-linear relations between queries and documents.
Calibrating Explore-Exploit Trade-off for Fair Online Learning to Rank
Online learning to rank (OL2R) has attracted great research interests in recent years, thanks to its advantages in avoiding expensive relevance labeling as required in offline supervised ranking model learning.
Graph Structural Attack by Perturbing Spectral Distance
Graph Convolutional Networks (GCNs) have fueled a surge of research interest due to their encouraging performance on graph learning tasks, but they are also shown vulnerability to adversarial attacks.
Comparative Explanations of Recommendations
As recommendation is essentially a comparative (or ranking) process, a good explanation should illustrate to users why an item is believed to be better than another, i. e., comparative explanations about the recommended items.
Graph Embedding with Hierarchical Attentive Membership
The exploitation of graph structures is the key to effectively learning representations of nodes that preserve useful information in graphs.
Unbiased Graph Embedding with Biased Graph Observations
In this paper, we propose a principled new way for unbiased graph embedding by learning node embeddings from an underlying bias-free graph, which is not influenced by sensitive node attributes.
When Are Linear Stochastic Bandits Attackable?
We study adversarial attacks on linear stochastic bandits: by manipulating the rewards, an adversary aims to control the behaviour of the bandit algorithm.
Learning the Optimal Recommendation from Explorative Users
We propose a new problem setting to study the sequential interactions between a recommender system and a user.
Asynchronous Upper Confidence Bound Algorithms for Federated Linear Bandits
In this paper, we study linear contextual bandit in a federated learning setting.
Improve Learning from Crowds via Generative Augmentation
This creates a sparsity issue and limits the quality of machine learning models trained on such data.
When and Whom to Collaborate with in a Changing Environment: A Collaborative Dynamic Bandit Solution
However, all existing collaborative bandit learning solutions impose a stationary assumption about the environment, i. e., both user preferences and the dependency among users are assumed static over time.
Incentivizing Exploration in Linear Bandits under Information Gap
We study the problem of incentivizing exploration for myopic users in linear bandits, where the users tend to exploit arm with the highest predicted reward instead of exploring.
PairRank: Online Pairwise Learning to Rank by Divide-and-Conquer
Online Learning to Rank (OL2R) eliminates the need of explicit relevance annotation by directly optimizing the rankers from their interactions with users.
Reversible Action Design for Combinatorial Optimization with Reinforcement Learning
Combinatorial optimization problem (COP) over graphs is a fundamental challenge in optimization.
Explanation as a Defense of Recommendation
At training time, the two learning tasks are joined by a latent sentiment vector, which is encoded by the recommendation module and used to make word choices for explanation generation.
Learning from Crowds by Modeling Common Confusions
Crowdsourcing provides a practical way to obtain large amounts of labeled data at a low cost.
Ranked #1 on
Image Classification
on LabelMe
Unifying Clustered and Non-stationary Bandits
Non-stationary bandits and online clustering of bandits lift the restrictive assumptions in contextual bandits and provide solutions to many important real-world scenarios.
Directional Multivariate Ranking
In this work, we propose a directional multi-aspect ranking criterion to enable a holistic ranking of items with respect to multiple aspects.
Non-Clicks Mean Irrelevant? Propensity Ratio Scoring As a Correction
Recent advances in unbiased learning to rank (LTR) count on Inverse Propensity Scoring (IPS) to eliminate bias in implicit feedback.
Déjà vu: A Contextualized Temporal Attention Mechanism for Sequential Recommendation
Predicting users' preferences based on their sequential behaviors in history is challenging and crucial for modern recommender systems.
A Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation
Reinforcement learning is effective in optimizing policies for recommender systems.
Generative Adversarial Network
Model-based Reinforcement Learning
+3
JNET: Learning User Representations via Joint Network Embedding and Topic Embedding
Inspired by the concept of user schema in social psychology, we take a new perspective to perform user representation learning by constructing a shared latent space to capture the dependency among different modalities of user-generated data.
Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation
Reinforcement learning is well suited for optimizing policies of recommender systems.
Generative Adversarial Network
Model-based Reinforcement Learning
+3
BPMR: Bayesian Probabilistic Multivariate Ranking
The framework naturally leads to a probabilistic multi-aspect ranking criterion, which generalizes the single-aspect ranking to a multivariate fashion.
Active Collaborative Sensing for Energy Breakdown
However, very few homes in the world have installed sub-meters (sensors measuring individual appliance energy); and the cost of retrofitting a home with extensive sub-metering eats into the funds available for energy saving retrofits.
Adversarial Domain Adaptation for Machine Reading Comprehension
In this paper, we focus on unsupervised domain adaptation for Machine Reading Comprehension (MRC), where the source domain has a large amount of labeled data, while only unlabeled passages are available in the target domain.
Variance Reduction in Gradient Exploration for Online Learning to Rank
We prove that the projected gradient is an unbiased estimation of the true gradient, and show that this lower-variance gradient estimation results in significant regret reduction.
Factorization Bandits for Online Influence Maximization
We capitalize on an important property of the influence maximization problem named network assortativity, which is ignored by most existing works in online influence maximization.
Context Attentive Document Ranking and Query Suggestion
We present a context-aware neural ranking model to exploit users' on-task search activities and enhance retrieval performance.
The FacT: Taming Latent Factor Models for Explainability with Factorization Trees
In this work, we integrate regression trees to guide the learning of latent factor models for recommendation, and use the learnt tree structure to explain the resulting latent factors.
Bandit Learning with Implicit Feedback
Implicit feedback, such as user clicks, although abundant in online information service systems, does not provide substantial evidence on users' evaluation of system's output.
Explainable Recommendation via Multi-Task Learning in Opinionated Text Data
Explaining automatically generated recommendations allows users to make more informed and accurate decisions about which results to utilize, and therefore improves their satisfaction.
Learning Contextual Bandits in a Non-stationary Environment
Multi-armed bandit algorithms have become a reference solution for handling the explore/exploit dilemma in recommender systems, and many other important real-world problems, such as display advertisement.
Efficient Exploration of Gradient Space for Online Learning to Rank
In this paper, we accelerate the online learning process by efficient exploration in the gradient space.
Multi-Task Learning for Document Ranking and Query Suggestion
We propose a multi-task learning framework to jointly learn document ranking and query suggestion for web search.
