Search Results for author: Linqi Song

Found 48 papers, 18 papers with code

Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation

1 code implementation5 Apr 2024 Tianqi Zhong, Zhaoyi Li, Quan Wang, Linqi Song, Ying WEI, Defu Lian, Zhendong Mao

Compositional generalization, representing the model's ability to generate text with new attribute combinations obtained by recombining single attributes from the training data, is a crucial property for multi-aspect controllable text generation (MCTG) methods.

Attribute Benchmarking +2

Learning From Correctness Without Prompting Makes LLM Efficient Reasoner

no code implementations28 Mar 2024 Yuxuan Yao, Han Wu, Zhijiang Guo, Biyan Zhou, Jiahui Gao, Sichun Luo, Hanxu Hou, Xiaojin Fu, Linqi Song

Large language models (LLMs) have demonstrated outstanding performance across various tasks, yet they still exhibit limitations such as hallucination, unfaithful reasoning, and toxic content.


MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric

no code implementations12 Mar 2024 Haokun Lin, Haoli Bai, Zhili Liu, Lu Hou, Muyi Sun, Linqi Song, Ying WEI, Zhenan Sun

We find that directly using smaller pre-trained models and applying magnitude-based pruning on CLIP models leads to inflexibility and inferior performance.

Can LLM Substitute Human Labeling? A Case Study of Fine-grained Chinese Address Entity Recognition Dataset for UAV Delivery

1 code implementation10 Mar 2024 Yuxuan Yao, Sichun Luo, Haohan Zhao, Guanzhi Deng, Linqi Song

We present CNER-UAV, a fine-grained \textbf{C}hinese \textbf{N}ame \textbf{E}ntity \textbf{R}ecognition dataset specifically designed for the task of address resolution in \textbf{U}nmanned \textbf{A}erial \textbf{V}ehicle delivery systems.


Understanding and Patching Compositional Reasoning in LLMs

no code implementations22 Feb 2024 Zhaoyi Li, Gangwei Jiang, Hong Xie, Linqi Song, Defu Lian, Ying WEI

LLMs have marked a revolutonary shift, yet they falter when faced with compositional reasoning tasks.

MUSTARD: Mastering Uniform Synthesis of Theorem and Proof Data

1 code implementation14 Feb 2024 Yinya Huang, Xiaohan Lin, Zhengying Liu, Qingxing Cao, Huajian Xin, Haiming Wang, Zhenguo Li, Linqi Song, Xiaodan Liang

Recent large language models (LLMs) have witnessed significant advancement in various tasks, including mathematical reasoning and theorem proving.

Automated Theorem Proving Language Modelling +3

Truncated Non-Uniform Quantization for Distributed SGD

no code implementations2 Feb 2024 Guangfeng Yan, Tan Li, Yuanzhang Xiao, Congduan Li, Linqi Song

To address the communication bottleneck challenge in distributed learning, our work introduces a novel two-stage quantization strategy designed to enhance the communication efficiency of distributed Stochastic Gradient Descent (SGD).


Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning

no code implementations2 Feb 2024 Guangfeng Yan, Tan Li, Yuanzhang Xiao, Hanxu Hou, Linqi Song

We consider a general family of heavy-tail gradients that follow a power-law distribution, we aim to minimize the error resulting from quantization, thereby determining optimal values for two critical parameters: the truncation threshold and the quantization density.


Integrating Large Language Models into Recommendation via Mutual Augmentation and Adaptive Aggregation

no code implementations25 Jan 2024 Sichun Luo, Yuxuan Yao, Bowei He, Yinya Huang, Aojun Zhou, Xinyi Zhang, Yuanzhang Xiao, Mingjie Zhan, Linqi Song

Conventional recommendation methods have achieved notable advancements by harnessing collaborative or sequential information from user behavior.

Data Augmentation

3D Shape Completion on Unseen Categories:A Weakly-supervised Approach

no code implementations19 Jan 2024 Lintai Wu, Junhui Hou, Linqi Song, Yong Xu

Specifically, we construct a prior bank consisting of representative shapes from the seen categories.

RecRanker: Instruction Tuning Large Language Model as Ranker for Top-k Recommendation

1 code implementation26 Dec 2023 Sichun Luo, Bowei He, Haohan Zhao, Wei Shao, Yanlin Qi, Yinya Huang, Aojun Zhou, Yuxuan Yao, Zongpeng Li, Yuanzhang Xiao, Mingjie Zhan, Linqi Song

Large Language Models (LLMs) have demonstrated remarkable capabilities and have been extensively deployed across various domains, including recommender systems.

In-Context Learning Language Modelling +3

Fine-grained Conversational Decoding via Isotropic and Proximal Search

1 code implementation12 Oct 2023 Yuxuan Yao, Han Wu, Qiling Xu, Linqi Song

General-purpose text decoding approaches are usually adopted for dialogue response generation.

Informativeness Response Generation

MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning

1 code implementation5 Oct 2023 Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, Hongsheng Li

In this paper, we present a method to fine-tune open-source language models, enabling them to use code for modeling and deriving math equations and, consequently, enhancing their mathematical reasoning abilities.

Ranked #4 on Math Word Problem Solving on SVAMP (using extra training data)

Arithmetic Reasoning GSM8K +2

Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification

1 code implementation15 Aug 2023 Aojun Zhou, Ke Wang, Zimu Lu, Weikang Shi, Sichun Luo, Zipeng Qin, Shaoqing Lu, Anya Jia, Linqi Song, Mingjie Zhan, Hongsheng Li

We found that its success can be largely attributed to its powerful skills in generating and executing code, evaluating the output of code execution, and rectifying its solution when receiving unreasonable outputs.

Arithmetic Reasoning Math +1

PerFedRec++: Enhancing Personalized Federated Recommendation with Self-Supervised Pre-Training

no code implementations11 May 2023 Sichun Luo, Yuanzhang Xiao, Xinyi Zhang, Yang Liu, Wenbo Ding, Linqi Song

Each user learns a personalized model by combining the global federated model, the cluster-level federated model, and its own fine-tuned local model.

Federated Learning Graph Learning +3

VCSUM: A Versatile Chinese Meeting Summarization Dataset

1 code implementation9 May 2023 Han Wu, Mingjie Zhan, Haochen Tan, Zhaohui Hou, Ding Liang, Linqi Song

Compared to news and chat summarization, the development of meeting summarization is hugely decelerated by the limited data.

Meeting Summarization Retrieval +1

Killing Two Birds with One Stone: Quantization Achieves Privacy in Distributed Learning

no code implementations26 Apr 2023 Guangfeng Yan, Tan Li, Kui Wu, Linqi Song

Communication efficiency and privacy protection are two critical issues in distributed machine learning.


A Scalable Test Problem Generator for Sequential Transfer Optimization

2 code implementations17 Apr 2023 Xiaoming Xue, Cuie Yang, Liang Feng, Kai Zhang, Linqi Song, Kay Chen Tan

Lastly, a benchmark suite with 12 STO problems featured by a variety of customized similarity relationships is developed using the proposed generator.

Gap-closing Matters: Perceptual Quality Evaluation and Optimization of Low-Light Image Enhancement

no code implementations22 Feb 2023 Baoliang Chen, Lingyu Zhu, Hanwei Zhu, Wenhan Yang, Linqi Song, Shiqi Wang

Subsequently, we propose an objective quality assessment measure that plays a critical role in bridging the gap between visual quality and enhancement.

Image Quality Assessment Low-Light Image Enhancement

Adaptive Top-K in SGD for Communication-Efficient Distributed Learning

no code implementations24 Oct 2022 Mengzhe Ruan, Guangfeng Yan, Yuanzhang Xiao, Linqi Song, Weitao Xu

This paper proposes a novel adaptive Top-K in SGD framework that enables an adaptive degree of sparsification for each gradient descent step to optimize the convergence performance by balancing the trade-off between communication cost and convergence error.

Hiding Images in Deep Probabilistic Models

no code implementations5 Oct 2022 Haoyu Chen, Linqi Song, Zhenxing Qian, Xinpeng Zhang, Kede Ma

As an instantiation, we adopt a SinGAN, a pyramid of generative adversarial networks (GANs), to learn the patch distribution of one cover image.

Towards Communication Efficient and Fair Federated Personalized Sequential Recommendation

no code implementations23 Aug 2022 Sichun Luo, Yuanzhang Xiao, Yang Liu, Congduan Li, Linqi Song

Federated recommendations leverage the federated learning (FL) techniques to make privacy-preserving recommendations.

Fairness Federated Learning +2

HySAGE: A Hybrid Static and Adaptive Graph Embedding Network for Context-Drifting Recommendations

no code implementations20 Aug 2022 Sichun Luo, Xinyi Zhang, Yuanzhang Xiao, Linqi Song

For example, in a mobile game recommendation, contextual features like locations, battery, and storage levels of mobile devices are frequently drifting over time.

Collaborative Filtering Graph Embedding

Personalized Federated Recommendation via Joint Representation Learning, User Clustering, and Model Adaptation

no code implementations19 Aug 2022 Sichun Luo, Yuanzhang Xiao, Linqi Song

In this paper, we propose a Graph Neural Network based Personalized Federated Recommendation (PerFedRec) framework via joint representation learning, user clustering, and model adaptation.

Attribute Clustering +3

Towards Better Understanding with Uniformity and Explicit Regularization of Embeddings in Embedding-based Neural Topic Models

no code implementations16 Jun 2022 Wei Shao, Lei Huang, Shuqi Liu, Shihua Ma, Linqi Song

In this paper, we propose an embedding regularized neural topic model, which applies the specially designed training constraints on word embedding and topic embedding to reduce the optimization space of parameters.

Topic Models

Learning Locality and Isotropy in Dialogue Modeling

1 code implementation29 May 2022 Han Wu, Haochen Tan, Mingjie Zhan, Gangming Zhao, Shaoqing Lu, Ding Liang, Linqi Song

Existing dialogue modeling methods have achieved promising performance on various dialogue tasks with the aid of Transformer and the large-scale pre-trained language models.

Zero-shot Cross-lingual Conversational Semantic Role Labeling

1 code implementation Findings (NAACL) 2022 Han Wu, Haochen Tan, Kun Xu, Shuqi Liu, Lianwei Wu, Linqi Song

While conversational semantic role labeling (CSRL) has shown its usefulness on Chinese conversational tasks, it is still under-explored in non-Chinese languages due to the lack of multilingual CSRL annotations for the parser training.

Response Generation Semantic Role Labeling

Privacy-Preserving Communication-Efficient Federated Multi-Armed Bandits

no code implementations2 Nov 2021 Tan Li, Linqi Song

Our bandit learning algorithms are based on epoch-wise sub-optimal arm eliminations at each agent and agents exchange learning knowledge with the server/each other at the end of each epoch.

Decision Making Multi-Armed Bandits +1

Byzantine-robust Federated Learning through Collaborative Malicious Gradient Filtering

3 code implementations13 Sep 2021 Jian Xu, Shao-Lun Huang, Linqi Song, Tian Lan

To this end, previous work either makes use of auxiliary data at parameter server to verify the received gradients (e. g., by computing validation error rate) or leverages statistic-based methods (e. g. median and Krum) to identify and remove malicious gradients from Byzantine clients.

Federated Learning Model Poisoning +2

DQ-SGD: Dynamic Quantization in SGD for Communication-Efficient Distributed Learning

no code implementations30 Jul 2021 Guangfeng Yan, Shao-Lun Huang, Tian Lan, Linqi Song

Gradient quantization is an emerging technique in reducing communication costs in distributed learning.


Domain-Adaptive Pretraining Methods for Dialogue Understanding

no code implementations ACL 2021 Han Wu, Kun Xu, Linfeng Song, Lifeng Jin, Haisong Zhang, Linqi Song

Language models like BERT and SpanBERT pretrained on open-domain data have obtained impressive gains on various NLP tasks.

Dialogue Understanding

Conversational Semantic Role Labeling

no code implementations11 Apr 2021 Kun Xu, Han Wu, Linfeng Song, Haisong Zhang, Linqi Song, Dong Yu

Semantic role labeling (SRL) aims to extract the arguments for each predicate in an input sentence.

coreference-resolution Dialogue Understanding +3

EGFI: Drug-Drug Interaction Extraction and Generation with Fusion of Enriched Entity and Sentence Information

1 code implementation25 Jan 2021 Lei Huang, Jiecong Lin, Xiangtao Li, Linqi Song, Ka-Chun Wong

To address such a problem, we propose EGFI for extracting and consolidating drug interactions from large-scale medical literature text data.

Classification Drug–drug Interaction Extraction +3


no code implementations1 Jan 2021 Guangfeng Yan, Shao-Lun Huang, Tian Lan, Linqi Song

This paper addresses this issue by proposing a novel dynamic quantized SGD (DQSGD) framework, which enables us to optimize the quantization strategy for each gradient descent step by exploring the trade-off between communication cost and modeling error.


Distributed Thompson Sampling

no code implementations3 Dec 2020 Jing Dong, Tan Li, Shaolei Ren, Linqi Song

To further improve the performance of distributed Thompson Sampling, we propose a distributed Elimination based Thompson Sampling algorithm that allow the agents to learn collaboratively.

Multi-Armed Bandits Thompson Sampling

Semantic Role Labeling Guided Multi-turn Dialogue ReWriter

no code implementations EMNLP 2020 Kun Xu, Haochen Tan, Linfeng Song, Han Wu, Haisong Zhang, Linqi Song, Dong Yu

For multi-turn dialogue rewriting, the capacity of effectively modeling the linguistic knowledge in dialog context and getting rid of the noises is essential to improve its performance.

Dialogue Rewriting Semantic Role Labeling

Federated Recommendation System via Differential Privacy

no code implementations14 May 2020 Tan Li, Linqi Song, Christina Fragouli

In this paper, we are interested in what we term the federated private bandits framework, that combines differential privacy with multi-agent bandit learning.

Federated Learning

The Tradeoff Between Privacy and Accuracy in Anomaly Detection Using Federated XGBoost

1 code implementation16 Jul 2019 Mengwei Yang, Linqi Song, Jie Xu, Congduan Li, Guozhen Tan

Our proposed federated XGBoost algorithm incorporates data aggregation and sparse federated update processes to balance the tradeoff between privacy and learning performance.

Anomaly Detection Federated Learning +1

Data Encoding for Byzantine-Resilient Distributed Optimization

no code implementations5 Jul 2019 Deepesh Data, Linqi Song, Suhas Diggavi

In this paper, we propose a method based on data encoding and error correction over real numbers to combat adversarial attacks.

Distributed Optimization

Equivariant neural networks and equivarification

1 code implementation16 Jun 2019 Erkao Bao, Linqi Song

We provide a process to modify a neural network to an equivariant one, which we call equivarification.

General Classification Image Classification

Regret vs. Bandwidth Trade-off for Recommendation Systems

no code implementations15 Oct 2018 Linqi Song, Christina Fragouli, Devavrat Shah

We consider recommendation systems that need to operate under wireless bandwidth constraints, measured as number of broadcast transmissions, and demonstrate a (tight for some instances) tradeoff between regret and bandwidth for two scenarios: the case of multi-armed bandit with context, and the case where there is a latent structure in the message space that we can exploit to reduce the learning phase.

Recommendation Systems

A Contextual Bandit Approach for Stream-Based Active Learning

no code implementations24 Jan 2017 Linqi Song, Jie Xu

The key feature of our algorithm is that in addition to sending a query to an annotator for the ground truth, prior information about the ground truth learned by the learner is sent together, thereby reducing the query cost.

Active Learning Decision Making

Stream-based Online Active Learning in a Contextual Multi-Armed Bandit Framework

no code implementations11 Jul 2016 Linqi Song

In a stream-based active learning setting, obtaining the ground truth of the reward is costly, and the conventional contextual multi-armed bandit algorithm fails to achieve a sublinear regret due to this cost.

Active Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.