Search Results for author:
Found 41 papers, 24 papers with code
Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals
We first empirically show that the features of either clean signals or adversarial perturbations are redundant and span in low-dimensional linear subspaces respectively with minimal overlap, and the classical low-dimensional subspace projection can suppress perturbation features out of the subspace of clean signals.
A Survey on Temporal Knowledge Graph: Representation Learning and Applications
Knowledge graph representation learning aims to learn low-dimensional vector embeddings for entities and relations in a knowledge graph.
Temporal Knowledge Graph Completion with Time-sensitive Relations in Hypercomplex Space
Temporal knowledge graph completion (TKGC) aims to fill in missing facts within a given temporal knowledge graph at a specific time.
Knowledge Graph Completion
Temporal Knowledge Graph Completion
Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
In this paper, we propose R$^3$: Learning Reasoning through Reverse Curriculum Reinforcement Learning (RL), a novel method that employs only outcome supervision to achieve the benefits of process supervision for large language models.
StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
The advancement of large language models (LLMs) has significantly propelled the field of code generation.
MouSi: Poly-Visual-Expert Vision-Language Models
This technique introduces a fusion network to unify the processing of outputs from different visual experts, while bridging the gap between image encoders and pre-trained LLMs.
Ranked #39 on
Visual Question Answering
on MM-Vet
Secrets of RLHF in Large Language Models Part II: Reward Modeling
We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset and fully leverage high-quality preference data.
Compositional Inductive Invariant Based Verification of Neural Network Controlled Systems
Verifying the inductiveness of a candidate inductive invariant in the context of NNCS is hard because of the scale and nonlinearity of neural networks.
LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks.
The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing
We present a unified probabilistic formulation for diffusion-based image editing, where a latent variable is edited in a task-specific manner and generally deviates from the corresponding marginal distribution induced by the original stochastic or ordinary differential equation (SDE or ODE).
Improving Generalization of Alignment with Human Preferences through Group Invariant Learning
In this work, we propose a novel approach that can learn a consistent policy via RL across various data groups or domains.
PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning
Personalized FL (PFL) addresses this by synthesizing personalized models from a global model via training on local data.
The Rise and Potential of Large Language Model Based Agents: A Survey
Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks.
Federated cINN Clustering for Accurate Clustered Federated Learning
However, a significant challenge arises when coordinating FL with crowd intelligence which diverse client groups possess disparate objectives due to data heterogeneity or distinct tasks.
Secrets of RLHF in Large Language Models Part I: PPO
Therefore, we explore the PPO-max, an advanced version of PPO algorithm, to efficiently improve the training stability of the policy model.
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement
For example, with Text-davinci-003, our method boosts the performance of standard few-shot prompting by $8. 0\%$ on GSM8K and $17. 8\%$ on MultiArith; it also improves the performance of CoT by $6. 0\%$ on GSM8K and $6. 0\%$ on MathQA, respectively.
Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergence
Reducing communication overhead in federated learning (FL) is challenging but crucial for large-scale distributed privacy-preserving machine learning.
Personalized Federated Learning with Hidden Information on Personalized Prior
Federated learning (FL for simplification) is a distributed machine learning technique that utilizes global servers and collaborative clients to achieve privacy-preserving global model training without direct data sharing.
Ranked #1 on
Image Classification
on Fashion-MNIST
(Accuracy metric)
Robust Lottery Tickets for Pre-trained Language Models
Recent works on Lottery Ticket Hypothesis have shown that pre-trained language models (PLMs) contain smaller matching subnetworks(winning tickets) which are capable of reaching accuracy comparable to the original models.
DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
The commonly-used fast sampler for guided sampling is DDIM, a first-order diffusion ODE solver that generally needs 100 to 250 steps for high-quality samples.
DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
In this work, we propose an exact formulation of the solution of diffusion ODEs.
Fast Instrument Learning with Faster Rates
We investigate nonlinear instrumental variable (IV) regression given high-dimensional instruments.
DeFTA: A Plug-and-Play Decentralized Replacement for FedAvg
Federated learning (FL) is identified as a crucial enabler for large-scale distributed machine learning (ML) without the need for local raw dataset sharing, substantially reducing privacy concerns and alleviating the isolated data problem.
Fuse Local and Global Semantics in Representation Learning
We propose Fuse Local and Global Semantics in Representation Learning (FLAGS) to generate richer representations.
Gradient Estimation with Discrete Stein Operators
Gradient estimation -- approximating the gradient of an expectation with respect to the parameters of a distribution -- is central to the solution of many machine learning problems.
Retrieval and Localization with Observation Constraints
The localization pipeline is designed as a coarse-to-fine paradigm.
Quasi-Bayesian Dual Instrumental Variable Regression
Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking.
Scalable Quasi-Bayesian Inference for Instrumental Variable Regression
Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking.
LANA: Towards Personalized Deep Knowledge Tracing Through Distinguishable Interactive Sequences
With pivot module reconstructed the decoder for individual students and leveled learning specialized encoders for groups, personalized DKT was achieved.
Communication-Efficient Federated Learning with Compensated Overlap-FedAvg
Petabytes of data are generated each day by emerging Internet of Things (IoT), but only few of them can be finally collected and used for Machine Learning (ML) purposes due to the apprehension of data & privacy leakage, which seriously retarding ML's growth.
Limits of PageRank-based ranking methods in sports data
We propose a new PageRank variant which outperforms PageRank in all evaluated settings, yet shares its sensitivity to increased randomness in the data.
HPSGD: Hierarchical Parallel SGD With Stale Gradients Featuring
While distributed training significantly speeds up the training process of the deep neural network (DNN), the utilization of the cluster is relatively low due to the time-consuming data synchronizing between workers.
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training
Specifically, the performance of each worker is evaluatedfirst based on the fact in the previous epoch, and then the batch size and datasetpartition are dynamically adjusted in consideration of the current performanceof the worker, thereby improving the utilization of the cluster.
Visual Localization Using Semantic Segmentation and Depth Prediction
In this paper, we propose a monocular visual localization pipeline leveraging semantic and depth cues.
Learning to Simulate Dynamic Environments with GameGAN
Simulation is a crucial component of any robotic system.
Nonparametric Score Estimators
Estimating the score, i. e., the gradient of log density function, from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models that involve flexible yet intractable densities.
Neural Graph Evolution: Towards Efficient Automatic Robot Design
To address the two challenges, we formulate automatic robot design as a graph search problem and perform evolution search in graph space.
Neural Graph Evolution: Automatic Robot Design
To address the two challenges, we formulate automatic robot design as a graph search problem and perform evolution search in graph space.
ReXCam: Resource-Efficient, Cross-Camera Video Analytics at Scale
Enterprises are increasingly deploying large camera networks for video analytics.
Now You Shake Me: Towards Automatic 4D Cinema
We are interested in enabling automatic 4D cinema by parsing physical and special effects from untrimmed movies.
ZhuSuan: A Library for Bayesian Deep Learning
In this paper we introduce ZhuSuan, a python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and deep learning.
