Search Results for author: Lichan Hong

Found 31 papers, 8 papers with code

Aligning Large Language Models with Recommendation Knowledge

no code implementations30 Mar 2024 Yuwei Cao, Nikhil Mehta, Xinyang Yi, Raghunandan Keshavan, Lukasz Heldt, Lichan Hong, Ed H. Chi, Maheswaran Sathiamoorthy

Operations such as Masked Item Modeling (MIM) and Bayesian Personalized Ranking (BPR) have found success in conventional recommender systems.

Attribute Recommendation Systems +1

Wisdom of Committee: Distilling from Foundation Model to Specialized Application Model

no code implementations21 Feb 2024 Zichang Liu, Qingyun Liu, Yuening Li, Liang Liu, Anshumali Shrivastava, Shuchao Bi, Lichan Hong, Ed H. Chi, Zhe Zhao

Further, to accommodate the dissimilarity among the teachers in the committee, we introduce DiverseDistill, which allows the student to understand the expertise of each teacher and extract task knowledge.

Knowledge Distillation Transfer Learning

LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views

no code implementations7 Feb 2024 Yuji Roh, Qingyun Liu, Huan Gui, Zhe Yuan, Yujin Tang, Steven Euijong Whang, Liang Liu, Shuchao Bi, Lichan Hong, Ed H. Chi, Zhe Zhao

By combining two complementing models, LEVI effectively suppresses problematic features in both the fine-tuning data and pre-trained model and preserves useful features for new tasks.

Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systems

no code implementations10 Nov 2023 Huan Gui, Ruoxi Wang, Ke Yin, Long Jin, Maciej Kula, Taibai Xu, Lichan Hong, Ed H. Chi

We identify two key challenges for applying the vanilla Transformer architecture to web-scale recommender systems: (1) Transformer architecture fails to capture the heterogeneous feature interactions in the self-attention layer; (2) The serving latency of Transformer architecture might be too high to be deployed in web-scale recommender systems.

Recommendation Systems

Talking Models: Distill Pre-trained Knowledge to Downstream Models via Interactive Communication

no code implementations4 Oct 2023 Zhe Zhao, Qingyun Liu, Huan Gui, Bang An, Lichan Hong, Ed H. Chi

In this paper, we extend KD with an interactive communication process to help students of downstream tasks learn effectively from pre-trained foundation models.

Decoder Knowledge Distillation +1

Density Weighting for Multi-Interest Personalized Recommendation

no code implementations3 Aug 2023 Nikhil Mehta, Anima Singh, Xinyang Yi, Sagar Jain, Lichan Hong, Ed H. Chi

When the data distribution is highly skewed, the gains observed by learning multiple representations diminish since the model dominates on head items/interests, leading to poor performance on tail items.

Recommendation Systems

Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations

no code implementations13 Jun 2023 Anima Singh, Trung Vu, Nikhil Mehta, Raghunandan Keshavan, Maheswaran Sathiamoorthy, Yilin Zheng, Lichan Hong, Lukasz Heldt, Li Wei, Devansh Tandon, Ed H. Chi, Xinyang Yi

To strike a good balance of memorization and generalization, we propose to use Semantic IDs -- a compact discrete item representation learned from frozen content embeddings using RQ-VAE that captures the hierarchy of concepts in items -- as a replacement for random item ids.

Memorization Recommendation Systems

Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction

no code implementations10 May 2023 Wang-Cheng Kang, Jianmo Ni, Nikhil Mehta, Maheswaran Sathiamoorthy, Lichan Hong, Ed Chi, Derek Zhiyuan Cheng

In this paper, we conduct a thorough examination of both CF and LLMs within the classic task of user rating prediction, which involves predicting a user's rating for a candidate item based on their past ratings.

Collaborative Filtering World Knowledge

Empowering Long-tail Item Recommendation through Cross Decoupling Network (CDN)

no code implementations25 Oct 2022 Yin Zhang, Ruoxi Wang, Tiansheng Yao, Xinyang Yi, Lichan Hong, James Caverlee, Ed H. Chi, Derek Zhiyuan Cheng

In this work, we aim to improve tail item recommendations while maintaining the overall performance with less training and serving cost.

Memorization Recommendation Systems +1

Improving Multi-Task Generalization via Regularizing Spurious Correlation

no code implementations19 May 2022 Ziniu Hu, Zhe Zhao, Xinyang Yi, Tiansheng Yao, Lichan Hong, Yizhou Sun, Ed H. Chi

First, the risk of having non-causal knowledge is higher, as the shared MTL model needs to encode all knowledge from different tasks, and causal knowledge for one task could be potentially spurious to the other.

Multi-Task Learning Representation Learning

Learning to Embed Categorical Features without Embedding Tables for Recommendation

no code implementations21 Oct 2020 Wang-Cheng Kang, Derek Zhiyuan Cheng, Tiansheng Yao, Xinyang Yi, Ting Chen, Lichan Hong, Ed H. Chi

Embedding learning of categorical features (e. g. user/item IDs) is at the core of various recommendation models including matrix factorization and neural collaborative filtering.

Collaborative Filtering Natural Language Understanding +2

Beyond Point Estimate: Inferring Ensemble Prediction Variation from Neuron Activation Strength in Recommender Systems

no code implementations17 Aug 2020 Zhe Chen, Yuyan Wang, Dong Lin, Derek Zhiyuan Cheng, Lichan Hong, Ed H. Chi, Claire Cui

Despite deep neural network (DNN)'s impressive prediction performance in various domains, it is well known now that a set of DNN models trained with the same model specification and the same data can produce very different prediction results.

Model-based Reinforcement Learning Recommendation Systems

Small Towers Make Big Differences

no code implementations13 Aug 2020 Yuyan Wang, Zhe Zhao, Bo Dai, Christopher Fifty, Dong Lin, Lichan Hong, Ed H. Chi

A delicate balance between multi-task generalization and multi-objective optimization is therefore needed for finding a better trade-off between efficiency and generalization.

Multi-Task Learning

Self-supervised Learning for Large-scale Item Recommendations

1 code implementation25 Jul 2020 Tiansheng Yao, Xinyang Yi, Derek Zhiyuan Cheng, Felix Yu, Ting Chen, Aditya Menon, Lichan Hong, Ed H. Chi, Steve Tjoa, Jieqi Kang, Evan Ettinger

Our online results also verify our hypothesis that our framework indeed improves model performance even more on slices that lack supervision.

Data Augmentation Natural Language Understanding +3

Learning-to-Rank with Partitioned Preference: Fast Estimation for the Plackett-Luce Model

no code implementations9 Jun 2020 Jiaqi Ma, Xinyang Yi, Weijing Tang, Zhe Zhao, Lichan Hong, Ed H. Chi, Qiaozhu Mei

We investigate the Plackett-Luce (PL) model based listwise learning-to-rank (LTR) on data with partitioned preference, where a set of items are sliced into ordered and disjoint partitions, but the ranking of items within a partition is unknown.

Extreme Multi-Label Classification Learning-To-Rank +1

Recommending what video to watch next: a multitask ranking system

no code implementations RecSys 2019 Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, Ed Chi

In this paper, we introduce a large scale multi-objective ranking system for recommending what video to watch next on an industrial video sharing platform.

Fairness in Recommendation Ranking through Pairwise Comparisons

no code implementations2 Mar 2019 Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H. Chi, Cristos Goodrow

Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information.

Fairness Recommendation Systems

Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts

10 code implementations19 Jul 2018 Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, Ed Chi

In this work, we propose a novel multi-task learning approach, Multi-gate Mixture-of-Experts (MMoE), which explicitly learns to model task relationships from data.

Binary Classification Click-Through Rate Prediction +2

Wide & Deep Learning for Recommender Systems

36 code implementations24 Jun 2016 Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, Hemal Shah

Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more feature engineering effort.

Click-Through Rate Prediction Feature Engineering +3

Cannot find the paper you are looking for? You can Submit a new open access paper.