Search Results for author:
Found 62 papers, 33 papers with code
Data-Juicer: A One-Stop Data Processing System for Large Language Models
A data recipe is a mixture of data from different sources for training LLMs, which plays a vital role in LLMs' performance.
FederatedScope: A Flexible Federated Learning Platform for Heterogeneity
Although remarkable progress has been made by existing federated learning (FL) platforms to provide infrastructures for development, these platforms may not well tackle the challenges brought by various types of heterogeneity, including the heterogeneity in participants' local data, resources, behaviors and learning goals.
FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning
The incredible development of federated learning (FL) has benefited various tasks in the domains of computer vision and natural language processing, and the existing frameworks such as TFF and FATE has made the deployment easy in real-world applications.
A Benchmark for Federated Hetero-Task Learning
To investigate the heterogeneity in federated learning in real-world scenarios, we generalize the classic federated learning to federated hetero-task learning, which emphasizes the inconsistency across the participants in federated learning in terms of both data distribution and learning tasks.
FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimization
Due to this uniqueness, existing HPO benchmarks no longer satisfy the need to compare HPO methods in the FL setting.
pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning
Personalized Federated Learning (pFL), which utilizes and deploys distinct local models, has gained increasing attention in recent years due to its success in handling the statistical heterogeneity of FL clients.
Collaborating Heterogeneous Natural Language Processing Tasks via Federated Learning
In this study, we further broaden the application scope of FL in NLP by proposing an Assign-Then-Contrast (denoted as ATC) framework, which enables clients with heterogeneous NLP tasks to construct an FL course and learn useful knowledge from each other.
Revisiting Personalized Federated Learning: Robustness Against Backdoor Attacks
We conduct the first study of backdoor attacks in the pFL framework, testing 4 widely used backdoor attacks against 6 pFL methods on benchmark datasets FEMNIST and CIFAR-10, a total of 600 experiments.
Efficient Personalized Federated Learning via Sparse Model-Adaptation
To overcome these challenges, we propose a novel approach named pFedGate for efficient personalized FL by adaptively and efficiently learning sparse local models.
FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning
When several entities have similar interested tasks, but their data cannot be shared because of privacy concerns regulations, federated learning (FL) is a mainstream solution to leverage the data of different entities.
Tunable Soft Prompts are Messengers in Federated Learning
As the global model itself is not required to be shared and the local training is conducted based on an auxiliary model with fewer parameters than the global model, the proposed approach provides protection for the global model while reducing communication and computation costs in FL.
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes
Pre-trained large language models (LLMs) need fine-tuning to improve their responsiveness to natural language instructions.
AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search
Motivated by the necessity and benefits of task-oriented BERT compression, we propose a novel compression method, AdaBERT, that leverages differentiable Neural Architecture Search to automatically compress BERT into task-adaptive small models for specific tasks.
AgentScope: A Flexible yet Robust Multi-Agent Platform
With the rapid advancement of Large Language Models (LLMs), significant progress has been made in multi-agent applications.
Simple and Deep Graph Convolutional Networks
We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques: {\em Initial residual} and {\em Identity mapping}.
Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation
Our explorations highlight open-source LLMs' potential in Text-to-SQL, as well as the advantages and disadvantages of the supervised fine-tuning.
Ranked #2 on
Text-To-SQL
on spider
Towards Universal Sequence Representation Learning for Recommender Systems
In order to develop effective sequential recommenders, a series of sequence representation learning (SRL) methods are proposed to model historical user behaviors.
Improving Utility and Security of the Shuffler-based Differential Privacy
When collecting information, local differential privacy (LDP) alleviates privacy concerns of users because their private information is randomized before being sent it to the central aggregator.
Contrastive Learning for Sequential Recommendation
Sequential recommendation methods play a crucial role in modern recommender systems because of their ability to capture a user's dynamic interest from her/his historical interactions.
VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition
End-to-end AutoML has attracted intensive interests from both academia and industry, which automatically searches for ML pipelines in a space induced by feature engineering, algorithm/model selection, and hyper-parameter tuning.
Automated Relational Meta-learning
In order to efficiently learn with small amount of data on new tasks, meta-learning transfers knowledge learned from previous tasks to the new ones.
Recommendation Unlearning
From the perspective of utility, if a system's utility is damaged by some bad data, the system needs to forget these data to regain utility.
EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism
We present EE-LLM, a framework for large-scale training and inference of early-exit large language models (LLMs).
EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models
This work introduces EE-Tuning, a lightweight and economical solution to training/tuning early-exit large language models (LLMs).
CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting
In this work, we design a special Transformer, i. e., Channel Aligned Robust Blend Transformer (CARD for short), that addresses key shortcomings of CI type Transformer in time series forecasting.
Scalable Graph Neural Networks via Bidirectional Propagation
Most notably, GBP can deliver superior performance on a graph with over 60 million nodes and 1. 8 billion edges in less than half an hour on a single machine.
Lero: A Learning-to-Rank Query Optimizer
In this paper, we introduce a learning-to-rank query optimizer, called Lero, which builds on top of a native query optimizer and continuously learns to improve the optimization performance.
Sequential Recommendation with Self-Attentive Multi-Adversarial Network
Recently, deep learning has made significant progress in the task of sequential recommendation.
EvenNet: Ignoring Odd-Hop Neighbors Improves Robustness of Graph Neural Networks
Despite their extraordinary predictive accuracy, existing approaches, such as GCN and GPRGNN, are not robust in the face of homophily changes on test graphs, rendering these models vulnerable to graph structural attacks and with limited capacity in generalizing to graphs of varied homophily levels.
Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation
However, existing metrics either neglect the intrinsic cause of the factual inconsistency or rely on auxiliary tasks, leading to an unsatisfied correlation with human judgments or increasing the inconvenience of usage in practice.
Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study
Different from previous studies focused on overall performance, this work aims to investigate the impact of quantization on \emph{emergent abilities}, which are important characteristics that distinguish LLMs from small language models.
Learning to be a Statistician: Learned Estimator for Number of Distinct Values
We derive conditions of the learning framework under which the learned model is workload agnostic, in the sense that the model/estimator can be trained with synthetically generated training data, and then deployed into any data warehouse simply as, e. g., user-defined functions (UDFs), to offer efficient (within microseconds on CPU) and accurate NDV estimations for unseen tables and workloads.
Towards Personalized Answer Generation in E-Commerce via Multi-Perspective Preference Modeling
Recently, Product Question Answering (PQA) on E-Commerce platforms has attracted increasing attention as it can act as an intelligent online shopping assistant and improve the customer shopping experience.
Learn-Memorize-Recall-Reduce A Robotic Cloud Computing Paradigm
The rise of robotic applications has led to the generation of a huge volume of unstructured data, whereas the current cloud infrastructure was designed to process limited amounts of structured data.
Towards Differentially Private Truth Discovery for Crowd Sensing Systems
To better utilize sensory data, the problem of truth discovery, whose goal is to estimate user quality and infer reliable aggregated results through quality-aware data aggregation, has emerged as a hot topic.
ABC: Efficient Selection of Machine Learning Configuration on Large Dataset
A machine learning configuration refers to a combination of preprocessor, learner, and hyperparameters.
Collecting Telemetry Data Privately
In particular, existing LDP algorithms are not suitable for repeated collection of counter data such as daily app usage statistics.
Continuous Integration of Machine Learning Models with ease.ml/ci: Towards a Rigorous Yet Practical Treatment
Continuous integration is an indispensable step of modern software engineering practices to systematically manage the life cycles of system development.
Practical Data Poisoning Attack against Next-Item Recommendation
In real-world recommendation systems, the cost of retraining recommendation models is high, and the interaction frequency between users and a recommendation system is restricted. Given these real-world restrictions, we propose to let the agent interact with a recommender simulator instead of the target recommendation system and leverage the transferability of the generated adversarial samples to poison the target system.
FIVES: Feature Interaction Via Edge Search for Large-Scale Tabular Data
Then we instantiate this search strategy by optimizing both a dedicated graph neural network (GNN) and the adjacency tensor associated with the defined feature graph.
PURE: An Uncertainty-aware Recommendation Framework for Maximizing Expected Posterior Utility of Platform
In this paper, we propose a novel recommendation framework which effectively utilizes the information of user uncertainty over different item dimensions and explicitly takes into consideration the impact of display policy on user in order to achieve maximal expected posterior utility for the platform.
Learning to Mutate with Hypergradient Guided Population
Computing the gradient of model hyperparameters, i. e., hypergradient, enables a promising and natural way to solve the hyperparameter optimization task.
A Pluggable Learned Index Method via Sampling and Gap Insertion
In this paper, we propose a formal machine learning based framework to quantify the index learning objective, and study two general and pluggable techniques to enhance the learning efficiency and learning effectiveness for learned indexes.
FlashP: An Analytical Pipeline for Real-time Forecasting of Time-Series Relational Data
Our scalable real-time forecasting system FlashP (Flash Prediction) is built based on this idea, with two major challenges to be resolved in this paper: first, we need to figure out how approximate aggregations affect the fitting of forecasting models, and forecasting results; and second, accordingly, what sampling algorithms we should use to obtain these approximate aggregations and how large the samples are.
Competitive Information Design for Pandora's Box
We study a natural competitive-information-design variant for the Pandora's Box problem (Weitzman 1979), where each box is associated with a strategic information sender who can design what information about the box's prize value to be revealed to the agent when the agent inspects the box.
Computer Science and Game Theory
Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning
However, existing methods mainly target at solving one or two of these three decision-making problems in CRS with separated conversation and recommendation components, which restrict the scalability and generality of CRS and fall short of preserving a stable training procedure.
CausCF: Causal Collaborative Filtering for RecommendationEffect Estimation
It is based on the idea that similar users not only have a similar taste on items, but also have similar treatment effect under recommendations.
Coarformer: Transformer for large graph via graph coarsening
We get rid of these obstacles by exploiting the complementary natures of GNN and Transformer, and trade the fine-grained long-range information for the efficiency of Transformer.
Learned Index with Dynamic $\epsilon$
We theoretically analyze prediction error bounds that link $\epsilon$ with data characteristics for an illustrative learned index method.
iFlood: A Stable and Effective Regularizer
However, our further studies uncover that the design of the loss function of Flooding can lead to a discrepancy between its objective and implementation, and cause the instability issue.
Path-specific Causal Fair Prediction via Auxiliary Graph Structure Learning
To tackle these challenges, we propose a novel casual graph based fair prediction framework, which integrates graph structure learning into fair prediction to ensure that unfair pathways are excluded in the causal graph.
Glue: Adaptively Merging Single Table Cardinality to Estimate Join Query Size
Cardinality estimation (CardEst), a central component of the query optimizer, plays a significant role in generating high-quality query plans in DBMS.
Baihe: SysML Framework for AI-driven Databases
Based on the high level architecture, we then describe a concrete implementation of Baihe for PostgreSQL and present example use cases for learned query optimizers.
Studying the Impact of Data Disclosure Mechanism in Recommender Systems via Simulation
Then we study users' privacy decision making under different data disclosure mechanisms and recommendation models, and how their data disclosure decisions affect the recommender system's performance.
ID-Agnostic User Behavior Pre-training for Sequential Recommendation
Instead of explicitly learning representations for item IDs, IDA-SR directly learns item representations from rich text information.
FS-Real: Towards Real-World Cross-Device Federated Learning
Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data, which attracts increasing attention in both academia and industry.
SoK: Privacy-Preserving Data Synthesis
As the prevalence of data analysis grows, safeguarding data privacy has become a paramount concern.
Deep Efficient Private Neighbor Generation for Subgraph Federated Learning
Behemoth graphs are often fragmented and separately stored by multiple data owners as distributed subgraphs in many realistic applications.
An Auction-based Marketplace for Model Trading in Federated Learning
This FL market allows clients to gain monetary reward by selling their own models and improve local model performance through the purchase of others' models.
DiffsFormer: A Diffusion Transformer on Stock Factor Augmentation
Our research demonstrates the efficacy of leveraging AIGS and the DiffsFormer architecture to mitigate data scarcity in stock forecasting tasks.
A Bargaining-based Approach for Feature Trading in Vertical Federated Learning
We analyze the proposed bargaining model under perfect and imperfect performance information settings, proving the existence of an equilibrium that optimizes the parties' objectives.
Improving LoRA in Privacy-preserving Federated Learning
Low-rank adaptation (LoRA) is one of the most popular task-specific parameter-efficient fine-tuning (PEFT) methods on pre-trained language models for its good performance and computational efficiency.
