Search Results for author: Julian McAuley

Found 177 papers, 92 papers with code

Interview: Large-scale Modeling of Media Dialog with Discourse Patterns and Knowledge Grounding

no code implementations EMNLP 2020 Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, Julian McAuley

In this work, we perform the first large-scale analysis of discourse in media dialog and its impact on generative modeling of dialog turns, with a focus on interrogative patterns and use of external knowledge.

SkipBERT: Efficient Inference with Shallow Layer Skipping

1 code implementation ACL 2022 Jue Wang, Ke Chen, Gang Chen, Lidan Shou, Julian McAuley

In this paper, we propose SkipBERT to accelerate BERT inference by skipping the computation of shallow layers.

Auto-Encoding or Auto-Regression? A Reality Check on Causality of Self-Attention-Based Sequential Recommenders

1 code implementation4 Jun 2024 Yueqi Wang, Zhankui He, Zhenrui Yue, Julian McAuley, Dong Wang

We start by tracing the AE/AR debate back to its origin through a systematic re-evaluation of SASRec and BERT4Rec, discovering that AR models generally surpass AE models in sequential recommendation.

feature selection Inductive Bias +1

DITTO-2: Distilled Diffusion Inference-Time T-Optimization for Music Generation

no code implementations30 May 2024 Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick, Nicholas Bryan

We propose Distilled Diffusion Inference-Time T -Optimization (or DITTO-2), a new method to speed up inference-time optimization-based control and unlock faster-than-real-time generation for a wide-variety of applications such as music inpainting, outpainting, intensity, melody, and musical structure control.

Music Generation

Multi-Behavior Generative Recommendation

1 code implementation27 May 2024 Zihan Liu, Yupeng Hou, Julian McAuley

We formulate the MBSR task into a consecutive two-step process: (1) given item sequences, MBGen first predicts the next behavior type to frame the user intention, (2) given item sequences and a target behavior type, MBGen then predicts the next items.

Sequential Recommendation

Large Scale Knowledge Washing

1 code implementation26 May 2024 Yu Wang, Ruihan Wu, Zexue He, Xiusi Chen, Julian McAuley

To this end, we propose LAW (Large Scale Washing) to update the MLP layers in decoder-only large language models to perform knowledge washing, as inspired by model editing methods and based on the hypothesis that knowledge and reasoning are disentanglable.

Decoder Memorization +2

Reindex-Then-Adapt: Improving Large Language Models for Conversational Recommendation

no code implementations20 May 2024 Zhankui He, Zhouhang Xie, Harald Steck, Dawen Liang, Rahul Jha, Nathan Kallus, Julian McAuley

The RTA framework marries the benefits of both LLMs and traditional recommender systems (RecSys): understanding complex queries as LLMs do; while efficiently controlling the recommended item distributions in conversational recommendations as traditional RecSys do.

Recommendation Systems

A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law

no code implementations2 May 2024 Zhiyu Zoey Chen, Jing Ma, Xinlu Zhang, Nan Hao, An Yan, Armineh Nourbakhsh, Xianjun Yang, Julian McAuley, Linda Petzold, William Yang Wang

In the fast-evolving domain of artificial intelligence, large language models (LLMs) such as GPT-3 and GPT-4 are revolutionizing the landscapes of finance, healthcare, and law: domains characterized by their reliance on professional expertise, challenging data acquisition, high-stakes, and stringent regulatory compliance.


List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs

1 code implementation25 Apr 2024 An Yan, Zhengyuan Yang, Junda Wu, Wanrong Zhu, Jianwei Yang, Linjie Li, Kevin Lin, JianFeng Wang, Julian McAuley, Jianfeng Gao, Lijuan Wang

Set-of-Mark (SoM) Prompting unleashes the visual grounding capability of GPT-4V, by enabling the model to associate visual objects with tags inserted on the image.

Visual Grounding Visual Question Answering +1

Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs

no code implementations24 Apr 2024 Yu Xia, Rui Wang, Xu Liu, Mingyan Li, Tong Yu, Xiang Chen, Julian McAuley, Shuai Li

Chain-of-Thought (CoT) has been a widely adopted prompting method, eliciting impressive reasoning abilities of Large Language Models (LLMs).

A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)

1 code implementation31 Mar 2024 Yashar Deldjoo, Zhankui He, Julian McAuley, Anton Korikov, Scott Sanner, Arnau Ramisa, René Vidal, Maheswaran Sathiamoorthy, Atoosa Kasirzadeh, Silvia Milano

Traditional recommender systems (RS) have used user-item rating histories as their primary data source, with collaborative filtering being one of the principal methods.

Collaborative Filtering Recommendation Systems +1

Few-shot Dialogue Strategy Learning for Motivational Interviewing via Inductive Reasoning

no code implementations23 Mar 2024 Zhouhang Xie, Bodhisattwa Prasad Majumder, Mengjie Zhao, Yoshinori Maeda, Keiichi Yamada, Hiromi Wakaki, Julian McAuley

We consider the task of building a dialogue system that can motivate users to adopt positive lifestyle changes: Motivational Interviewing.

Instruction Following

Large Language Models and Causal Inference in Collaboration: A Comprehensive Survey

no code implementations14 Mar 2024 Xiaoyu Liu, Paiheng Xu, Junda Wu, Jiaxin Yuan, Yifan Yang, YuHang Zhou, Fuxiao Liu, Tianrui Guan, Haoliang Wang, Tong Yu, Julian McAuley, Wei Ai, Furong Huang

Causal inference has shown potential in enhancing the predictive accuracy, fairness, robustness, and explainability of Natural Language Processing (NLP) models by capturing causal relationships among variables.

Causal Inference Fairness

CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendation

no code implementations11 Mar 2024 Junda Wu, Cheng-Chun Chang, Tong Yu, Zhankui He, Jianing Wang, Yupeng Hou, Julian McAuley

Based on the retrieved user-item interactions, the LLM can analyze shared and distinct preferences among users, and summarize the patterns indicating which types of users would be attracted by certain items.

Recommendation Systems Reinforcement Learning (RL) +1

Bridging Language and Items for Retrieval and Recommendation

1 code implementation6 Mar 2024 Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, Julian McAuley

This paper introduces BLaIR, a series of pretrained sentence embedding models specialized for recommendation scenarios.

Retrieval Sentence +2

Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding

no code implementations29 Feb 2024 Guangyi Liu, Yu Wang, Zeyu Feng, Qiyu Wu, Liping Tang, Yuan Gao, Zhen Li, Shuguang Cui, Julian McAuley, Zichao Yang, Eric P. Xing, Zhiting Hu

The vast applications of deep generative models are anchored in three core capabilities -- generating new instances, reconstructing inputs, and learning compact representations -- across various data types, such as discrete text/protein sequences and continuous images.

Decoder Denoising

LVCHAT: Facilitating Long Video Comprehension

1 code implementation19 Feb 2024 Yu Wang, Zeyuan Zhang, Julian McAuley, Zexue He

To address this issue, we propose Long Video Chat (LVChat), where Frame-Scalable Encoding (FSE) is introduced to dynamically adjust the number of embeddings in alignment with the duration of the video to ensure long videos are not overly compressed into a few embeddings.

Video Captioning

Foundation Models for Recommender Systems: A Survey and New Perspectives

no code implementations17 Feb 2024 Chengkai Huang, Tong Yu, Kaige Xie, Shuai Zhang, Lina Yao, Julian McAuley

Recently, Foundation Models (FMs), with their extensive knowledge bases and complex architectures, have offered unique opportunities within the realm of recommender systems (RSs).

Recommendation Systems Representation Learning

InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment

1 code implementation13 Feb 2024 Jianing Wang, Junda Wu, Yupeng Hou, Yao Liu, Ming Gao, Julian McAuley

In this paper, we propose InstructGraph, a framework that empowers LLMs with the abilities of graph reasoning and generation by instruction tuning and preference alignment.


MEMORYLLM: Towards Self-Updatable Large Language Models

1 code implementation7 Feb 2024 Yu Wang, Yifan Gao, Xiusi Chen, Haoming Jiang, Shiyang Li, Jingfeng Yang, Qingyu Yin, Zheng Li, Xian Li, Bing Yin, Jingbo Shang, Julian McAuley

We aim to build models containing a considerable portion of self-updatable parameters, enabling the model to integrate new knowledge effectively and efficiently.

Model Editing

FINEST: Stabilizing Recommendations by Rank-Preserving Fine-Tuning

no code implementations5 Feb 2024 Sejoon Oh, Berk Ustun, Julian McAuley, Srijan Kumar

Modern recommender systems may output considerably different recommendations due to small perturbations in the training data.

Recommendation Systems

InfoRank: Unbiased Learning-to-Rank via Conditional Mutual Information Minimization

no code implementations23 Jan 2024 Jiarui Jin, Zexue He, Mengyue Yang, Weinan Zhang, Yong Yu, Jun Wang, Julian McAuley

Subsequently, we minimize the mutual information between the observation estimation and the relevance estimation conditioned on the input features.

Learning-To-Rank Recommendation Systems

DITTO: Diffusion Inference-Time T-Optimization for Music Generation

no code implementations22 Jan 2024 Zachary Novack, Julian McAuley, Taylor Berg-Kirkpatrick, Nicholas J. Bryan

We propose Diffusion Inference-Time T-Optimization (DITTO), a general-purpose frame-work for controlling pre-trained text-to-music diffusion models at inference-time via optimizing initial noise latents.

Computational Efficiency Music Generation

Deciphering Compatibility Relationships with Textual Descriptions via Extraction and Explanation

1 code implementation17 Dec 2023 Yu Wang, Zexue He, Zhankui He, Hao Xu, Julian McAuley

This fine-tuning allows the model to generate explanations that convey the compatibility relationships between items.

Equipping Pretrained Unconditional Music Transformers with Instrument and Genre Controls

no code implementations21 Nov 2023 Weihan Xu, Julian McAuley, Shlomo Dubnov, Hao-Wen Dong

We then propose a simple technique to equip this pretrained unconditional music transformer model with instrument and genre controls by finetuning the model with additional control tokens.

Music Generation

Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving

1 code implementation25 Oct 2023 Jessica Echterhoff, An Yan, Kyungtae Han, Amr Abdelraouf, Rohit Gupta, Julian McAuley

In the context of human-assisted or autonomous driving, explainability models can help user acceptance and understanding of decisions made by the autonomous vehicle, which can be used to rationalize and explain driver or vehicle behavior.

Autonomous Driving

Extending Input Contexts of Language Models through Training on Segmented Sequences

no code implementations23 Oct 2023 Petros Karypis, Julian McAuley, George Karypis

Our method benefits both models trained with absolute positional embeddings, by extending their input contexts, as well as popular relative positional embedding methods showing a reduced perplexity on sequences longer than they were trained on.

Unsupervised Lead Sheet Generation via Semantic Compression

1 code implementation16 Oct 2023 Zachary Novack, Nikita Srivatsan, Taylor Berg-Kirkpatrick, Julian McAuley

Lead sheets have become commonplace in generative music research, being used as an initial compressed representation for downstream tasks like multitrack music generation and automatic arrangement.

Music Compression Music Generation +1

Farzi Data: Autoregressive Data Distillation

no code implementations15 Oct 2023 Noveen Sachdeva, Zexue He, Wang-Cheng Kang, Jianmo Ni, Derek Zhiyuan Cheng, Julian McAuley

We study data distillation for auto-regressive machine learning tasks, where the input and output have a strict left-to-right causal structure.

Language Modelling Sequential Recommendation

SelfVC: Voice Conversion With Iterative Refinement using Self Transformations

no code implementations14 Oct 2023 Paarth Neekhara, Shehzeen Hussain, Rafael Valle, Boris Ginsburg, Rishabh Ranjan, Shlomo Dubnov, Farinaz Koushanfar, Julian McAuley

In this work, instead of explicitly disentangling attributes with loss terms, we present a framework to train a controllable voice conversion model on entangled speech representations derived from self-supervised learning (SSL) and speaker verification models.

Self-Supervised Learning Speaker Verification +2

AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems

no code implementations13 Oct 2023 Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian McAuley, Wayne Xin Zhao, Leyu Lin, Ji-Rong Wen

The optimized agents can also propagate their preferences to other agents in subsequent interactions, implicitly capturing the collaborative filtering idea.

Collaborative Filtering Decision Making +2

Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models

no code implementations4 Oct 2023 An Yan, Yu Wang, Yiwu Zhong, Zexue He, Petros Karypis, Zihan Wang, chengyu dong, Amilcare Gentili, Chun-Nan Hsu, Jingbo Shang, Julian McAuley

Medical image classification is a critical problem for healthcare, with the potential to alleviate the workload of doctors and facilitate diagnoses of patients.

Image Classification Language Modelling +1

Automatic Pair Construction for Contrastive Post-training

1 code implementation3 Oct 2023 Canwen Xu, Corby Rosset, Ethan C. Chau, Luciano del Corro, Shweti Mahajan, Julian McAuley, Jennifer Neville, Ahmed Hassan Awadallah, Nikhil Rao

Remarkably, our automatic contrastive post-training further improves the performance of Orca, already a state-of-the-art instruction learning model tuned with GPT-4 outputs, to outperform ChatGPT.

Linear Recurrent Units for Sequential Recommendation

1 code implementation3 Oct 2023 Zhenrui Yue, Yueqi Wang, Zhankui He, Huimin Zeng, Julian McAuley, Dong Wang

State-of-the-art sequential recommendation relies heavily on self-attention-based recommender models.

Language Modelling Sequential Recommendation

Reinforcement Learning for Generative AI: A Survey

no code implementations28 Aug 2023 Yuanjiang Cao, Quan Z. Sheng, Julian McAuley, Lina Yao

Deep Generative AI has been a long-standing essential topic in the machine learning community, which can impact a number of application areas like text generation and computer vision.

Inductive Bias Language Modelling +3

On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems

no code implementations22 Aug 2023 Xiaocong Chen, Siyu Wang, Julian McAuley, Dietmar Jannach, Lina Yao

Offline reinforcement learning empowers agents to glean insights from offline datasets and deploy learned policies in online settings.

Recommendation Systems reinforcement-learning

Large Language Models as Zero-Shot Conversational Recommenders

1 code implementation19 Aug 2023 Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, Julian McAuley

In this paper, we present empirical studies on conversational recommendation tasks using representative large language models in a zero-shot setting with three primary contributions.

Learning Concise and Descriptive Attributes for Visual Recognition

2 code implementations ICCV 2023 An Yan, Yu Wang, Yiwu Zhong, chengyu dong, Zexue He, Yujie Lu, William Wang, Jingbo Shang, Julian McAuley

Recent advances in foundation models present new opportunities for interpretable visual recognition -- one can first query Large Language Models (LLMs) to obtain a set of attributes that describe each class, then apply vision-language models to classify images via these attributes.


LongCoder: A Long-Range Pre-trained Language Model for Code Completion

1 code implementation26 Jun 2023 Daya Guo, Canwen Xu, Nan Duan, Jian Yin, Julian McAuley

In this paper, we introduce a new task for code completion that focuses on handling long code input and propose a sparse Transformer model, called LongCoder, to address this task.

Code Completion Language Modelling

RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems

1 code implementation5 Jun 2023 Tianyang Liu, Canwen Xu, Julian McAuley

Large Language Models (LLMs) have greatly advanced code auto-completion systems, with a potential for substantial productivity enhancements for developers.

Benchmarking C++ code +2

KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations

no code implementations5 Jun 2023 Myeongjun Jang, Bodhisattwa Prasad Majumder, Julian McAuley, Thomas Lukasiewicz, Oana-Maria Camburu

While recent works have been considerably improving the quality of the natural language explanations (NLEs) generated by a model to justify its predictions, there is very limited research in detecting and alleviating inconsistencies among generated NLEs.

Adversarial Attack

Generative Flow Network for Listwise Recommendation

1 code implementation4 Jun 2023 Shuchang Liu, Qingpeng Cai, Zhankui He, Bowen Sun, Julian McAuley, Dong Zheng, Peng Jiang, Kun Gai

In this work, we aim to learn a policy that can generate sufficiently diverse item lists for users while maintaining high recommendation quality.

Recommendation Systems

Text Is All You Need: Learning Language Representations for Sequential Recommendation

1 code implementation23 May 2023 Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, Julian McAuley

In this paper, we propose to model user preferences and item features as language representations that can be generalized to new items and datasets.

Representation Learning Sentence +1

Large Language Models are Zero-Shot Rankers for Recommender Systems

1 code implementation15 May 2023 Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, Wayne Xin Zhao

Recently, large language models (LLMs) (e. g., GPT-4) have demonstrated impressive general-purpose task-solving abilities, including the potential to approach recommendation tasks.

Recommendation Systems

"Nothing Abnormal": Disambiguating Medical Reports via Contrastive Knowledge Infusion

no code implementations15 May 2023 Zexue He, An Yan, Amilcare Gentili, Julian McAuley, Chun-Nan Hsu

Based on our analysis, we define a disambiguation rewriting task to regenerate an input to be unambiguous while preserving information about the original content.

Small Models are Valuable Plug-ins for Large Language Models

1 code implementation15 May 2023 Canwen Xu, Yichong Xu, Shuohang Wang, Yang Liu, Chenguang Zhu, Julian McAuley

Large language models (LLMs) such as GPT-3 and GPT-4 are powerful but their weights are often publicly unavailable and their immense sizes make the models difficult to be tuned with common hardware.

In-Context Learning

Unsupervised Improvement of Factual Knowledge in Language Models

1 code implementation4 Apr 2023 Nafis Sadeq, Byungkyu Kang, Prarit Lamba, Julian McAuley

In this work, we propose an approach for influencing MLM pretraining in a way that can improve language model performance on a variety of knowledge-intensive tasks.

Language Modelling Masked Language Modeling +3

Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data

5 code implementations3 Apr 2023 Canwen Xu, Daya Guo, Nan Duan, Julian McAuley

Furthermore, we propose a new technique called Self-Distill with Feedback, to further improve the performance of the Baize models with feedback from ChatGPT.

Chatbot Language Modelling +1

Mirror: A Natural Language Interface for Data Querying, Summarization, and Visualization

1 code implementation15 Mar 2023 Canwen Xu, Julian McAuley, Penghan Wang

We present Mirror, an open-source platform for data exploration and analysis powered by large language models.

Data Distillation: A Survey

1 code implementation11 Jan 2023 Noveen Sachdeva, Julian McAuley

The popularity of deep learning has led to the curation of a vast number of massive and multifarious datasets.

Recommendation Systems

Synthetic Pre-Training Tasks for Neural Machine Translation

no code implementations19 Dec 2022 Zexue He, Graeme Blackwood, Rameswar Panda, Julian McAuley, Rogerio Feris

Pre-training models with large crawled corpora can lead to issues such as toxicity and bias, as well as copyright and privacy concerns.

Machine Translation NMT +1

CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos

1 code implementation14 Dec 2022 Hao-Wen Dong, Naoya Takahashi, Yuki Mitsufuji, Julian McAuley, Taylor Berg-Kirkpatrick

Further, videos in the wild often contain off-screen sounds and background noise that may hinder the model from learning the desired audio-textual correspondence.

Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders

1 code implementation22 Oct 2022 Yupeng Hou, Zhankui He, Julian McAuley, Wayne Xin Zhao

Based on this representation scheme, we further propose an enhanced contrastive pre-training approach, using semi-synthetic and mixed-domain code representations as hard negatives.

Language Modelling Recommendation Systems +1

Efficiently Tuned Parameters are Task Embeddings

1 code implementation21 Oct 2022 Wangchunshu Zhou, Canwen Xu, Julian McAuley

Thus, we propose to exploit these efficiently tuned parameters as off-the-shelf task embeddings for the efficient selection of source datasets for intermediate-task transfer.

Question Answering Text Classification

InforMask: Unsupervised Informative Masking for Language Model Pretraining

1 code implementation21 Oct 2022 Nafis Sadeq, Canwen Xu, Julian McAuley

In this paper, we propose InforMask, a new unsupervised masking strategy for training masked language models.

Language Modelling Masked Language Modeling +2

InterFair: Debiasing with Natural Language Feedback for Fair Interpretable Predictions

no code implementations14 Oct 2022 Bodhisattwa Prasad Majumder, Zexue He, Julian McAuley

In the other setup, human feedback was able to disentangle associated bias and predictive information from the input leading to superior bias mitigation and improved task performance (4-5%) simultaneously.


Controlling Bias Exposure for Fair Interpretable Predictions

1 code implementation14 Oct 2022 Zexue He, Yu Wang, Julian McAuley, Bodhisattwa Prasad Majumder

However, when sensitive information is semantically entangled with the task information of the input, e. g., gender information is predictive for a profession, a fair trade-off between task performance and bias mitigation is difficult to achieve.

Attribute Task 2 +2

UCEpic: Unifying Aspect Planning and Lexical Constraints for Generating Explanations in Recommendation

1 code implementation28 Sep 2022 Jiacheng Li, Zhankui He, Jingbo Shang, Julian McAuley

Then, to obtain personalized explanations under this framework of insertion-based generation, we design a method of incorporating aspect planning and personalized references into the insertion process.

Explainable Recommendation Explanation Generation +2

On Faithfulness and Coherence of Language Explanations for Recommendation Systems

no code implementations12 Sep 2022 Zhouhang Xie, Julian McAuley, Bodhisattwa Prasad Majumder

Reviews contain rich information about product characteristics and user interests and thus are commonly used to boost recommender system performance.

Recommendation Systems Review Generation

SPOT: Knowledge-Enhanced Language Representations for Information Extraction

no code implementations20 Aug 2022 Jiacheng Li, Yannis Katsis, Tyler Baldwin, Ho-Cheol Kim, Andrew Bartko, Julian McAuley, Chun-Nan Hsu

To address these problems, we propose a new pre-trained model that learns representations of both entities and relationships from token spans and span pairs in the text respectively.

Relation Extraction

Plug-and-Play Model-Agnostic Counterfactual Policy Synthesis for Deep Reinforcement Learning based Recommendation

no code implementations10 Aug 2022 Siyu Wang, Xiaocong Chen, Lina Yao, Sally Cripps, Julian McAuley

Recent advances in recommender systems have proved the potential of Reinforcement Learning (RL) to handle the dynamic evolution processes between users and recommender systems.

counterfactual Data Augmentation +3

Generating Negative Samples for Sequential Recommendation

no code implementations7 Aug 2022 Yongjun Chen, Jia Li, Zhiwei Liu, Nitish Shirish Keskar, Huan Wang, Julian McAuley, Caiming Xiong

Due to the dynamics of users' interests and model updates during training, considering randomly sampled items from a user's non-interacted item set as negatives can be uninformative.

Sequential Recommendation

Bundle MCR: Towards Conversational Bundle Recommendation

1 code implementation26 Jul 2022 Zhankui He, Handong Zhao, Tong Yu, Sungchul Kim, Fan Du, Julian McAuley

MCR, which uses a conversational paradigm to elicit user interests by asking user preferences on tags (e. g., categories or attributes) and handling user feedback across multiple rounds, is an emerging recommendation setting to acquire user feedback and narrow down the output space, but has not been explored in the context of bundle recommendation.

Recommendation Systems

Contrastive Learning for Interactive Recommendation in Fashion

no code implementations25 Jul 2022 Karin Sevegnani, Arjun Seshadri, Tian Wang, Anurag Beniwal, Julian McAuley, Alan Lu, Gerard Medioni

Recommender systems and search are both indispensable in facilitating personalization and ease of browsing in online fashion platforms.

Contrastive Learning Recommendation Systems +1

Multitrack Music Transformer

2 code implementations14 Jul 2022 Hao-Wen Dong, Ke Chen, Shlomo Dubnov, Julian McAuley, Taylor Berg-Kirkpatrick

Existing approaches for generating multitrack music with transformer models have been limited in terms of the number of instruments, the length of the music segments and slow inference.

Music Generation

Personalized Showcases: Generating Multi-Modal Explanations for Recommendations

no code implementations30 Jun 2022 An Yan, Zhankui He, Jiacheng Li, Tianyang Zhang, Julian McAuley

In this paper, to further enrich explanations, we propose a new task named personalized showcases, in which we provide both textual and visual information to explain our recommendations.

Contrastive Learning

Infinite Recommendation Networks: A Data-Centric Approach

5 code implementations3 Jun 2022 Noveen Sachdeva, Mehak Preet Dhaliwal, Carole-Jean Wu, Julian McAuley

We leverage the Neural Tangent Kernel and its equivalence to training infinitely-wide neural networks to devise $\infty$-AE: an autoencoder with infinitely-wide bottleneck layers.

 Ranked #1 on Recommendation Systems on Douban (AUC metric)

Information Retrieval Recommendation Systems

Assistive Recipe Editing through Critiquing

no code implementations5 May 2022 Diego Antognini, Shuyang Li, Boi Faltings, Julian McAuley

Prior studies have used pre-trained language models, or relied on small paired recipe data (e. g., a recipe paired with a similar one that satisfies a dietary constraint).

Denoising Language Modelling

Automatic Multi-Label Prompting: Simple and Interpretable Few-Shot Classification

1 code implementation NAACL 2022 Han Wang, Canwen Xu, Julian McAuley

Prompt-based learning (i. e., prompting) is an emerging paradigm for exploiting knowledge learned by a pretrained language model.

Few-Shot Text Classification Language Modelling +1

FaceSigns: Semi-Fragile Neural Watermarks for Media Authentication and Countering Deepfakes

1 code implementation5 Apr 2022 Paarth Neekhara, Shehzeen Hussain, Xinqiao Zhang, Ke Huang, Julian McAuley, Farinaz Koushanfar

We demonstrate that FaceSigns can embed a 128 bit secret as an imperceptible image watermark that can be recovered with a high bit recovery accuracy at several compression levels, while being non-recoverable when unseen Deepfake manipulations are applied.

Face Swapping Image Compression +1

Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection

1 code implementation ACL 2022 Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, Julian McAuley

In this paper, we propose a post-hoc knowledge-injection technique where we first retrieve a diverse set of relevant knowledge snippets conditioned on both the dialog history and an initial response from an existing dialog model.

Informativeness Specificity

LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrieval

1 code implementation Findings (ACL) 2022 Canwen Xu, Daya Guo, Nan Duan, Julian McAuley

Experimental results show that LaPraDoR achieves state-of-the-art performance compared with supervised dense retrieval models, and further analysis reveals the effectiveness of our training strategy and objectives.

Contrastive Learning Re-Ranking +3

Leashing the Inner Demons: Self-Detoxification for Language Models

no code implementations6 Mar 2022 Canwen Xu, Zexue He, Zhankui He, Julian McAuley

Language models (LMs) can reproduce (or amplify) toxic language seen during training, which poses a risk to their practical application.

A Survey on Model Compression and Acceleration for Pretrained Language Models

no code implementations15 Feb 2022 Canwen Xu, Julian McAuley

Despite achieving state-of-the-art performance on many NLP tasks, the high energy cost and long inference delay prevent Transformer-based pretrained language models (PLMs) from seeing broader adoption including for edge and mobile computing.

Model Compression

A Survey on Dynamic Neural Networks for Natural Language Processing

no code implementations15 Feb 2022 Canwen Xu, Julian McAuley

Effectively scaling large Transformer models is a main driver of recent advances in natural language processing.

Intent Contrastive Learning for Sequential Recommendation

1 code implementation5 Feb 2022 Yongjun Chen, Zhiwei Liu, Jia Li, Julian McAuley, Caiming Xiong

Specifically, we introduce a latent variable to represent users' intents and learn the distribution function of the latent variable via clustering.

Contrastive Learning Model Optimization +3

Rank List Sensitivity of Recommender Systems to Interaction Perturbations

no code implementations29 Jan 2022 Sejoon Oh, Berk Ustun, Julian McAuley, Srijan Kumar

We introduce a measure of stability for recommender systems, called Rank List Sensitivity (RLS), which measures how rank lists generated by a given recommender system at test time change as a result of a perturbation in the training data.

Recommendation Systems

On Sampling Collaborative Filtering Datasets

1 code implementation13 Jan 2022 Noveen Sachdeva, Carole-Jean Wu, Julian McAuley

We study the practical consequences of dataset sampling strategies on the ranking performance of recommendation algorithms.

Collaborative Filtering Recommendation Systems

Diversity-boosted Generalization-Specialization Balancing for Zero-shot Learning

no code implementations6 Jan 2022 Yun Li, Zhe Liu, Xiaojun Chang, Julian McAuley, Lina Yao

We further propose a differentiable dataset-level balance and update the weights in a linear annealing schedule to simulate network pruning and thus obtain the optimal structure for BSNet with dataset-level balance achieved.

Meta-Learning Network Pruning +1

Self-Supervised Bot Play for Conversational Recommendation with Justifications

no code implementations9 Dec 2021 Shuyang Li, Bodhisattwa Prasad Majumder, Julian McAuley

Conversational recommender systems offer the promise of interactive, engaging ways for users to find items they enjoy.

Recommendation Systems

Rethink, Revisit, Revise: A Spiral Reinforced Self-Revised Network for Zero-Shot Learning

no code implementations1 Dec 2021 Zhe Liu, Yun Li, Lina Yao, Julian McAuley, Sam Dixon

Our framework outperforms state-of-the-art algorithms on four benchmark datasets in both zero-shot and generalized zero-shot settings, which demonstrates the effectiveness of spiral learning in learning generalizable and complex correlations.

Attribute Zero-Shot Learning

An Entropy-guided Reinforced Partial Convolutional Network for Zero-Shot Learning

no code implementations3 Nov 2021 Yun Li, Zhe Liu, Lina Yao, Xianzhi Wang, Julian McAuley, Xiaojun Chang

Zero-Shot Learning (ZSL) aims to transfer learned knowledge from observed classes to unseen classes via semantic correlations.

Generalized Zero-Shot Learning

Locality-Sensitive Experience Replay for Online Recommendation

no code implementations21 Oct 2021 Xiaocong Chen, Lina Yao, Xianzhi Wang, Julian McAuley

Existing studies encourage the agent to learn from past experience via experience replay (ER).

Recommendation Systems

Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation

1 code implementation Findings (EMNLP) 2021 An Yan, Zexue He, Xing Lu, Jiang Du, Eric Chang, Amilcare Gentili, Julian McAuley, Chun-Nan Hsu

Radiology report generation aims at generating descriptive text from radiology images automatically, which may present an opportunity to improve radiology reporting and interpretation.

Contrastive Learning Descriptive +2

Modeling Dynamic Attributes for Next Basket Recommendation

no code implementations23 Sep 2021 Yongjun Chen, Jia Li, Chenghao Liu, Chenxi Li, Markus Anderle, Julian McAuley, Caiming Xiong

However, properly integrating them into user interest models is challenging since attribute dynamics can be diverse such as time-interval aware, periodic patterns (etc.

Attribute Next-basket recommendation

A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions

no code implementations8 Sep 2021 Xiaocong Chen, Lina Yao, Julian McAuley, Guanglin Zhou, Xianzhi Wang

In light of the emergence of deep reinforcement learning (DRL) in recommender systems research and several fruitful results in recent years, this survey aims to provide a timely and comprehensive overview of the recent trends of deep reinforcement learning in recommender systems.

Recommendation Systems reinforcement-learning +1

Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression

1 code implementation EMNLP 2021 Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

Recent studies on compression of pretrained language models (e. g., BERT) usually use preserved accuracy as the metric for evaluation.

Knowledge Distillation Quantization

Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction

1 code implementation1 Sep 2021 Zhenrui Yue, Zhankui He, Huimin Zeng, Julian McAuley

Under this setting, we propose an API-based model extraction method via limited-budget synthetic data generation and knowledge distillation.

Data Poisoning Knowledge Distillation +5

Contrastive Self-supervised Sequential Recommendation with Robust Augmentation

1 code implementation14 Aug 2021 Zhiwei Liu, Yongjun Chen, Jia Li, Philip S. Yu, Julian McAuley, Caiming Xiong

In this paper, we investigate the application of contrastive Self-Supervised Learning (SSL) to the sequential recommendation, as a way to alleviate some of these issues.

Contrastive Learning Self-Supervised Learning +1

Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music

1 code implementation13 Jul 2021 Hao-Wen Dong, Chris Donahue, Taylor Berg-Kirkpatrick, Julian McAuley

In this paper, we aim to further extend this idea and examine the feasibility of automatic instrumentation -- dynamically assigning instruments to notes in solo music during performance.

Multi-class Classification

SVP-CF: Selection via Proxy for Collaborative Filtering Data

no code implementations11 Jul 2021 Noveen Sachdeva, Carole-Jean Wu, Julian McAuley

As we demonstrate, commonly-used data sampling schemes can have significant consequences on algorithm performance -- masking performance deficiencies in algorithms or altering the relative performance of algorithms, as compared to models trained on the complete dataset.

Collaborative Filtering Recommendation Systems

Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations

no code implementations25 Jun 2021 Bodhisattwa Prasad Majumder, Oana-Maria Camburu, Thomas Lukasiewicz, Julian McAuley

Our framework improves over previous methods by: (i) reaching SOTA task performance while also providing explanations, (ii) providing two types of explanations, while existing models usually provide only one type, and (iii) beating by a large margin the previous SOTA in terms of quality of both types of explanations.

Decision Making

Unsupervised Enrichment of Persona-grounded Dialog with Background Stories

1 code implementation ACL 2021 Bodhisattwa Prasad Majumder, Taylor Berg-Kirkpatrick, Julian McAuley, Harsh Jhamtani

Humans often refer to personal narratives, life experiences, and events to make a conversation more engaging and rich.

BERT Learns to Teach: Knowledge Distillation with Meta Learning

1 code implementation ACL 2022 Wangchunshu Zhou, Canwen Xu, Julian McAuley

We present Knowledge Distillation with Meta Learning (MetaDistil), a simple yet effective alternative to traditional knowledge distillation (KD) methods where the teacher model is fixed during training.

Knowledge Distillation Meta-Learning

SHARE: a System for Hierarchical Assistive Recipe Editing

1 code implementation17 May 2021 Shuyang Li, Yufei Li, Jianmo Ni, Julian McAuley

The large population of home cooks with dietary restrictions is under-served by existing cooking resources and recipe generation models.

Recipe Generation

Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledge

1 code implementation NAACL 2021 Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley, Julian McAuley

The ability to generate clarification questions i. e., questions that identify useful missing information in a given context, is important in reducing ambiguity.

Question Generation Question-Generation

Cross-modal Adversarial Reprogramming

1 code implementation15 Feb 2021 Paarth Neekhara, Shehzeen Hussain, Jinglong Du, Shlomo Dubnov, Farinaz Koushanfar, Julian McAuley

Recent works on adversarial reprogramming have shown that it is possible to repurpose neural networks for alternate tasks without modifying the network architecture or parameters.

Classification General Classification +1

Expressive Neural Voice Cloning

no code implementations30 Jan 2021 Paarth Neekhara, Shehzeen Hussain, Shlomo Dubnov, Farinaz Koushanfar, Julian McAuley

In this work, we propose a controllable voice cloning method that allows fine-grained control over various style aspects of the synthesized speech for an unseen speaker.

Speech Synthesis Style Transfer +1

Like hiking? You probably enjoy nature: Persona-grounded Dialog with Commonsense Expansions

1 code implementation EMNLP 2020 Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, Julian McAuley

Existing persona-grounded dialog models often fail to capture simple implications of given persona descriptions, something which humans are able to do seamlessly.

Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rays

no code implementations Findings of the Association for Computational Linguistics 2020 Jianmo Ni, Chun-Nan Hsu, Amilcare Gentili, Julian McAuley

In this work, we focus on reporting abnormal findings on radiology images; instead of training on complete radiology reports, we propose a method to identify abnormal findings from the reports in addition to grouping them with unsupervised clustering and minimal rules.

Clustering Cross-Modal Retrieval +3

MusPy: A Toolkit for Symbolic Music Generation

2 code implementations5 Aug 2020 Hao-Wen Dong, Ke Chen, Julian McAuley, Taylor Berg-Kirkpatrick

MusPy provides easy-to-use tools for essential components in a music generation system, including dataset management, data I/O, data preprocessing and model evaluation.

Management Music Generation

BERT Loses Patience: Fast and Robust Inference with Early Exit

1 code implementation NeurIPS 2020 Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, Furu Wei

In this paper, we propose Patience-based Early Exit, a straightforward yet effective inference method that can be used as a plug-and-play technique to simultaneously improve the efficiency and robustness of a pretrained language model (PLM).

Language Modelling

How Useful are Reviews for Recommendation? A Critical Review and Potential Improvements

1 code implementation25 May 2020 Noveen Sachdeva, Julian McAuley

We investigate a growing body of work that seeks to improve recommender systems through the use of review text.

Recommendation Systems

Speech Recognition and Multi-Speaker Diarization of Long Conversations

3 code implementations16 May 2020 Huanru Henry Mao, Shuyang Li, Julian McAuley, Garrison Cottrell

Speech recognition (ASR) and speaker diarization (SD) models have traditionally been trained separately to produce rich conversation transcripts with speaker labels.

Data Augmentation speaker-diarization +3

Interview: A Large-Scale Open-Source Corpus of Media Dialog

no code implementations7 Apr 2020 Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, Julian McAuley

Compared to existing large-scale proxies for conversational data, language models trained on our dataset exhibit better zero-shot out-of-domain performance on existing spoken dialog datasets, demonstrating its usefulness in modeling real-world conversations.

Developing a Recommendation Benchmark for MLPerf Training and Inference

no code implementations16 Mar 2020 Carole-Jean Wu, Robin Burke, Ed H. Chi, Joseph Konstan, Julian McAuley, Yves Raimond, Hao Zhang

Deep learning-based recommendation models are used pervasively and broadly, for example, to recommend movies, products, or other information most relevant to users, in order to enhance the user experience.

Image Classification object-detection +3

ReZero is All You Need: Fast Convergence at Large Depth

13 code implementations10 Mar 2020 Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, Garrison W. Cottrell, Julian McAuley

Deep networks often suffer from vanishing or exploding gradients due to inefficient signal propagation, leading to long training times or convergence difficulties.

Language Modelling

TiSASRec: Time Interval Aware Self-Attention for Sequential Recommendation

5 code implementations1 Jan 2020 Jiacheng Li, Yujie Wang, Julian McAuley

Sequential recommender systems seek to exploit the order of users' interactions, in order to predict their next action based on the context of what they have done recently.

Sequential Recommendation

Addressing Marketing Bias in Product Recommendations

1 code implementation4 Dec 2019 Mengting Wan, Jianmo Ni, Rishabh Misra, Julian McAuley

However, these interactions can be biased by how the product is marketed, for example due to the selection of a particular human model in a product image.

Collaborative Filtering Fairness +2

Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects

no code implementations IJCNLP 2019 Jianmo Ni, Jiacheng Li, Julian McAuley

Several recent works have considered the problem of generating reviews (or {`}tips{'}) as a form of explanation as to why a recommendation might match a customer{'}s interests.

Decision Making Language Modelling

Candidate Generation with Binary Codes for Large-Scale Top-N Recommendation

no code implementations12 Sep 2019 Wang-Cheng Kang, Julian McAuley

Generating the Top-N recommendations from a large corpus is computationally expensive to perform at scale.

Recommendation Systems Re-Ranking

Scalable and Accurate Dialogue State Tracking via Hierarchical Sequence Generation

1 code implementation IJCNLP 2019 Liliang Ren, Jianmo Ni, Julian McAuley

Experiments on both the multi-domain and the single domain dialogue state tracking dataset show that our model not only scales easily with the increasing number of pre-defined domains and slots but also reaches the state-of-the-art performance.

Decoder Dialogue State Tracking +1

Generating Personalized Recipes from Historical User Preferences

1 code implementation IJCNLP 2019 Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, Julian McAuley

Existing approaches to recipe generation are unable to create recipes for users with culinary preferences but incomplete knowledge of ingredients in specific dishes.

Recipe Generation Text Generation

Fine-Grained Spoiler Detection from Large-Scale Review Corpora

no code implementations ACL 2019 Mengting Wan, Rishabh Misra, Ndapa Nakashole, Julian McAuley

This paper presents computational approaches for automatically detecting critical plot twists in reviews of media products.


Universal Adversarial Perturbations for Speech Recognition Systems

no code implementations9 May 2019 Paarth Neekhara, Shehzeen Hussain, Prakhar Pandey, Shlomo Dubnov, Julian McAuley, Farinaz Koushanfar

In this work, we demonstrate the existence of universal adversarial audio perturbations that cause mis-transcription of audio signals by automatic speech recognition (ASR) systems.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Expediting TTS Synthesis with Adversarial Vocoding

1 code implementation16 Apr 2019 Paarth Neekhara, Chris Donahue, Miller Puckette, Shlomo Dubnov, Julian McAuley

Recent approaches in text-to-speech (TTS) synthesis employ neural network strategies to vocode perceptually-informed spectrogram representations directly into listenable waveforms.

Embryo staging with weakly-supervised region selection and dynamically-decoded predictions

no code implementations9 Apr 2019 Tingfung Lau, Nathan Ng, Julian Gingold, Nina Desai, Julian McAuley, Zachary C. Lipton

First, noting that in each image the embryo occupies a small subregion, we jointly train a region proposal network with the downstream classifier to isolate the embryo.

Decoder Region Proposal

Complete the Look: Scene-based Complementary Product Recommendation

1 code implementation CVPR 2019 Wang-Cheng Kang, Eric Kim, Jure Leskovec, Charles Rosenberg, Julian McAuley

We design an approach to extract training data for this task, and propose a novel way to learn the scene-product compatibility from fashion or interior design images.

Product Recommendation

Recommendation Through Mixtures of Heterogeneous Item Relationships

2 code implementations29 Aug 2018 Wang-Cheng Kang, Mengting Wan, Julian McAuley

Recommender Systems have proliferated as general-purpose approaches to model a wide variety of consumer interaction data.

Knowledge Graph Embeddings Recommendation Systems

Self-Attentive Sequential Recommendation

8 code implementations20 Aug 2018 Wang-Cheng Kang, Julian McAuley

Sequential dynamics are a key feature of many modern recommender systems, which seek to capture the `context' of users' activities on the basis of actions they have performed recently.

Sequential Recommendation

Visually-Aware Personalized Recommendation using Interpretable Image Representations

no code implementations26 Jun 2018 Charles Packer, Julian McAuley, Arnau Ramisa

Visually-aware recommender systems use visual signals present in the underlying data to model the visual characteristics of items and users' preferences towards them.

Recommendation Systems

The NES Music Database: A multi-instrumental dataset with expressive performance attributes

2 code implementations12 Jun 2018 Chris Donahue, Huanru Henry Mao, Julian McAuley

Existing research on music generation focuses on composition, but often ignores the expressive performance characteristics required for plausible renditions of resultant pieces.

Music Generation

Adversarial Audio Synthesis

21 code implementations ICLR 2019 Chris Donahue, Julian McAuley, Miller Puckette

Audio signals are sampled at high temporal resolutions, and learning to synthesize audio requires capturing structure across a range of timescales.

Audio Generation Audio Synthesis +1

Does mitigating ML's impact disparity require treatment disparity?

1 code implementation NeurIPS 2018 Zachary C. Lipton, Alexandra Chouldechova, Julian McAuley

Following related work in law and policy, two notions of disparity have come to shape the study of fairness in algorithmic decision-making.

Decision Making Fairness

Visually-Aware Fashion Recommendation and Design with Generative Image Models

no code implementations7 Nov 2017 Wang-Cheng Kang, Chen Fang, Zhaowen Wang, Julian McAuley

Here, we seek to extend this contribution by showing that recommendation performance can be significantly improved by learning `fashion aware' image representations directly, i. e., by training the image representation (from the pixel level) and the recommender system jointly; this contribution is related to recent work using Siamese CNNs, though we are able to show improvements over state-of-the-art recommendation techniques such as BPR and variants that make use of pre-trained visual features.

Recommendation Systems

Estimating Reactions and Recommending Products with Generative Models of Reviews

no code implementations IJCNLP 2017 Jianmo Ni, Zachary C. Lipton, Sharad Vikram, Julian McAuley

Natural language approaches that model information like product reviews have proved to be incredibly useful in improving the performance of such methods, as reviews provide valuable auxiliary information that can be used to better estimate latent user preferences and item properties.

Collaborative Filtering Language Modelling +2

Translation-based Recommendation

1 code implementation8 Jul 2017 Ruining He, Wang-Cheng Kang, Julian McAuley

Modeling the complex interactions between users and items as well as amongst items themselves is at the core of designing successful recommender systems.

Recommendation Systems Translation

Dance Dance Convolution

1 code implementation ICML 2017 Chris Donahue, Zachary C. Lipton, Julian McAuley

For the step placement task, we combine recurrent and convolutional neural networks to ingest spectrograms of low-level audio features to predict steps, conditioned on chart difficulty.

Predicting Surgery Duration with Neural Heteroscedastic Regression

no code implementations17 Feb 2017 Nathan Ng, Rodney A Gabriel, Julian McAuley, Charles Elkan, Zachary C. Lipton

Scheduling surgeries is a challenging task due to the fundamental uncertainty of the clinical environment, as well as the risks and costs associated with under- and over-booking.

regression Scheduling