Search Results for author:
Found 50 papers, 8 papers with code
HyperFormer: Learning Expressive Sparse Feature Representations via Hypergraph Transformer
Learning expressive representations for high-dimensional yet sparse features has been a longstanding problem in information retrieval.
Large Language Models for User Interest Journeys
We argue, and demonstrate through extensive experiments, that LLMs as foundation models can reason through user activities, and describe their interests in nuanced and interesting ways, similar to how a human would.
Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals
We demonstrate that with a small amount of human-annotated counterfactual data (10%), we can generate a counterfactual augmentation dataset with learned labels, that provides an 18-20% improvement in robustness and a 14-21% reduction in errors on 6 out-of-domain datasets, comparable to that of a fully human-annotated counterfactual dataset for both sentiment classification and question paraphrase tasks.
Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems
Learning high-quality feature embeddings efficiently and effectively is critical for the performance of web-scale machine learning systems.
Recommender Systems with Generative Retrieval
Once we have the Semantic IDs for all the items, a Transformer based sequence-to-sequence model is trained to predict the Semantic ID of the next item.
What Are Effective Labels for Augmented Data? Improving Calibration and Robustness with AutoLabel
A wide breadth of research has devised data augmentation approaches that can improve both accuracy and generalization performance for neural networks.
Improving Training Stability for Multitask Ranking Models in Recommender Systems
Recommender systems play an important role in many content platforms.
Latent User Intent Modeling for Sequential Recommenders
We demonstrate the effectiveness of the latent user intent modeling via offline analyses as well as live experiments on a large-scale industrial recommendation platform.
Empowering Long-tail Item Recommendation through Cross Decoupling Network (CDN)
Most existing methods mainly attempt to reduce the bias from the prior perspective, which ignores the discrepancy in the conditional probability.
Scaling Instruction-Finetuned Language Models
We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation).
Ranked #1 on
Coreference Resolution
on WSC
Cross-Lingual Question Answering
Multi-task Language Understanding
+1
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
BIG-Bench (Srivastava et al., 2022) is a diverse evaluation suite that focuses on tasks believed to be beyond the capabilities of current language models.
Simpson's Paradox in Recommender Fairness: Reconciling differences between per-user and aggregated evaluations
We reconcile these notions and show that the tension is due to differences in distributions of users where items are relevant, and break down the important factors of the user's recommendations.
Reward Shaping for User Satisfaction in a REINFORCE Recommender
How might we design Reinforcement Learning (RL)-based recommenders that encourage aligning user trajectories with the underlying user satisfaction?
Emergent Abilities of Large Language Models
Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks.
Improving Multi-Task Generalization via Regularizing Spurious Correlation
First, the risk of having non-causal knowledge is higher, as the shared MTL model needs to encode all knowledge from different tasks, and causal knowledge for one task could be potentially spurious to the other.
Learning to Augment for Casual User Recommendation
Users who come to recommendation platforms are heterogeneous in activity levels.
HyperPrompt: Prompt-based Task-Conditioning of Transformers
Prompt-Tuning is a new paradigm for finetuning pre-trained language models in a parameter-efficient way.
Nonlinear Initialization Methods for Low-Rank Neural Networks
We propose a novel low-rank initialization framework for training low-rank deep neural networks -- networks where the weight parameters are re-parameterized by products of two low-rank matrices.
DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning
State-of-the-art MoE models use a trainable sparse gate to select a subset of the experts for each input example.
Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning
This presents a multi-dimensional Pareto frontier on (1) the trade-off between group fairness and accuracy with respect to each task, as well as (2) the trade-offs across multiple tasks.
Measuring Model Fairness under Noisy Covariates: A Theoretical Perspective
In this work we study the problem of measuring the fairness of a machine learning model under noisy information.
Towards Content Provider Aware Recommender Systems: A Simulation Study on the Interplay between User and Provider Utilities
We then build a REINFORCE recommender agent, coined EcoAgent, to optimize a joint objective of user utility and the counterfactual utility lift of the provider associated with the recommended content, which we show to be equivalent to maximizing overall user utility and the utilities of all providers on the platform under some mild assumptions.
Measuring Recommender System Effects with Simulated Users
Using this simulation framework, we can (a) isolate the effect of the recommender system from the user preferences, and (b) examine how the system performs not just on average for an "average user" but also the extreme experiences under atypical user behavior.
Learned Indexes for a Google-scale Disk-based Database
There is great excitement about learned index structures, but understandable skepticism about the practicality of a new method uprooting decades of research on B-Trees.
A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendation
It is also very encouraging that our framework further improves head items and overall performance on top of the gains on tail items.
Learning to Embed Categorical Features without Embedding Tables for Recommendation
Embedding learning of categorical features (e. g. user/item IDs) is at the core of various recommendation models including matrix factorization and neural collaborative filtering.
DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
Learning effective feature crosses is the key behind building recommender systems.
Ranked #4 on
Click-Through Rate Prediction
on Criteo
Beyond Point Estimate: Inferring Ensemble Prediction Variation from Neuron Activation Strength in Recommender Systems
Despite deep neural network (DNN)'s impressive prediction performance in various domains, it is well known now that a set of DNN models trained with the same model specification and the same data can produce very different prediction results.
Small Towers Make Big Differences
A delicate balance between multi-task generalization and multi-objective optimization is therefore needed for finding a better trade-off between efficiency and generalization.
Zero-Shot Heterogeneous Transfer Learning from Recommender Systems to Cold-Start Search Retrieval
In light of these problems, we observed that most online content platforms have both a search and a recommender system that, while having heterogeneous input spaces, can be connected through their common output item space and a shared semantic representation.
Self-supervised Learning for Large-scale Item Recommendations
Our online results also verify our hypothesis that our framework indeed improves model performance even more on slices that lack supervision.
Improving Calibration through the Relationship with Adversarial Robustness
To this end, we propose Adversarial Robustness based Adaptive Label Smoothing (AR-AdaLS) that integrates the correlations of adversarial robustness and calibration into training by adaptively softening labels for an example based on how easily it can be attacked by an adversary.
Fairness without Demographics through Adversarially Reweighted Learning
Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns.
Learning-to-Rank with Partitioned Preference: Fast Estimation for the Plackett-Luce Model
We investigate the Plackett-Luce (PL) model based listwise learning-to-rank (LTR) on data with partitioned preference, where a set of items are sliced into ordered and disjoint partitions, but the ranking of items within a partition is unknown.
Developing a Recommendation Benchmark for MLPerf Training and Inference
Deep learning-based recommendation models are used pervasively and broadly, for example, to recommend movies, products, or other information most relevant to users, in order to enhance the user experience.
Learning Multi-granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems
In this paper, we seek to learn highly compact embeddings for large-vocab sparse features in recommender systems (recsys).
Understanding and Improving Knowledge Distillation
Knowledge Distillation (KD) is a model-agnostic technique to improve model quality while having a fixed capacity budget.
Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems
In addition to the theoretical results, we find on multiple datasets -- including a large-scale real-world recommender system -- that the overall system's end-to-end fairness is largely achievable by improving fairness in individual components.
Toward a better trade-off between performance and fairness with kernel-based distribution matching
As recent literature has demonstrated how classifiers often carry unintended biases toward some subgroups, deploying machine learned models to users demands careful consideration of the social consequences.
The Dynamics of Signal Propagation in Gated Recurrent Neural Networks
We demonstrate the efficacy of our initialization scheme on multiple sequence tasks, on which it enables successful training while a standard initialization either fails completely or is orders of magnitude slower.
Transfer of Machine Learning Fairness across Domains
A model trained for one setting may be picked up and used in many others, particularly as is common with pre-training and cloud APIs.
Quantifying Long Range Dependence in Language and User Behavior to improve RNNs
Characterizing temporal dependence patterns is a critical step in understanding the statistical properties of sequential data.
Fairness in Recommendation Ranking through Pairwise Comparisons
Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information.
AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks
In this paper, we draw connections between recurrent networks and ordinary differential equations.
Towards Neural Mixture Recommender for Long Range Dependent User Sequences
Our approach employs a mixture of models, each with a different temporal range.
Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs
We demonstrate the efficacy of our initialization scheme on multiple sequence tasks, on which it enables successful training while a standard initialization either fails completely or is orders of magnitude slower.
Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements
In this paper we provide a case-study on the application of fairness in machine learning research to a production classification system, and offer new insights in how to measure and address algorithmic fairness issues.
Counterfactual Fairness in Text Classification through Robustness
In this paper, we study counterfactual fairness in text classification, which asks the question: How would the prediction change if the sensitive attribute referenced in the example were different?
The Case for Learned Index Structures
Indexes are models: a B-Tree-Index can be seen as a model to map a key to the position of a record within a sorted array, a Hash-Index as a model to map a key to a position of a record within an unsorted array, and a BitMap-Index as a model to indicate if a data record exists or not.
Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
How can we learn a classifier that is "fair" for a protected or sensitive group, when we do not know if the input to the classifier belongs to the protected group?
