Search Results for author:
Found 28 papers, 7 papers with code
Effective and Efficient Masked Image Generation Models
Additionally, on ImageNet 512x512, with only about 60% of the NFE, eMIGM outperforms the state-of-the-art continuous diffusion models.
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs
Ultimately, our experimental findings demonstrate that a 300B MoE LLM can be effectively trained on lower-performance devices while achieving comparable performance to models of a similar scale, including dense and MoE models.
Toward Stable and Consistent Evaluation Results: A New Methodology for Base Model Evaluation
To address these issues, we propose Base model Oriented Systematic Evaluation (BOSE), a method specifically designed to optimize the evaluation of base models.
Large Language Diffusion Models
Autoregressive models (ARMs) are widely regarded as the cornerstone of large language models (LLMs).
Keypoint-based Progressive Chain-of-Thought Distillation for LLMs
Chain-of-thought distillation is a powerful technique for transferring reasoning abilities from large language models (LLMs) to smaller student models.
Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equations
Bayesian flow networks (BFNs) iteratively refine the parameters, instead of the samples in diffusion models (DMs), of distributions at various noise levels through Bayesian inference.
AntDT: A Self-Adaptive Distributed Training Framework for Leader and Straggler Nodes
Previous works only address part of the stragglers and could not adaptively solve various stragglers in practice.
Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors
However, we argue that a critical obstacle remains in deploying LLMs for practical use: the efficiency of LLMs when processing long textual user behaviors.
G-Meta: Distributed Meta Learning in GPU Clusters for Large-Scale Recommender Systems
However, the existing systems are not tailored for meta learning based DLRM models and have critical problems regarding efficiency in distributed training in the GPU cluster.
Efficient Model Stealing Defense with Noise Transition Matrix
With the escalating complexity and investment cost of training deep neural networks safeguarding them from unauthorized usage and intellectual property theft has become imperative.
An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training
To address these issues, we propose a flexible model placement framework that offers two general and agile model placement strategies.
One Model for All: Large Language Models are Domain-Agnostic Recommendation Systems
We expect that by mixing the user's behaviors across different domains, we can exploit the common knowledge encoded in the pre-trained language model to alleviate the problems of data sparsity and cold start problems.
Rethinking Memory and Communication Cost for Efficient Large Language Model Training
In this paper, we rethink the impact of memory consumption and communication costs on the training speed of large language models, and propose a memory-communication balanced strategy set Partial Redundancy Optimizer (PaRO).
AntM$^{2}$C: A Large Scale Dataset For Multi-Scenario Multi-Modal CTR Prediction
3) AntM$^{2}$C provides 1 billion CTR data with 200 features, including 200 million users and 6 million items.
Towards Open Temporal Graph Neural Networks
This will bring two big challenges to the existing dynamic GNN methods: (i) How to dynamically propagate appropriate information in an open temporal graph, where new class nodes are often linked to old class nodes.
BadDet: Backdoor Attacks on Object Detection
We propose four kinds of backdoor attacks for object detection task: 1) Object Generation Attack: a trigger can falsely generate an object of the target class; 2) Regional Misclassification Attack: a trigger can change the prediction of a surrounding object to the target class; 3) Global Misclassification Attack: a single trigger can change the predictions of all objects in an image to the target class; and 4) Object Disappearance Attack: a trigger can make the detector fail to detect the object of the target class.
Improving Generative Adversarial Networks via Adversarial Learning in Latent Space
Although many efforts have been made in terms of backbone architecture design, loss function, and training techniques, few results have been obtained on how the sampling in latent space can affect the final performance, and existing works on latent space mainly focus on controllability.
Improving Transferability of Adversarial Patches on Face Recognition with Generative Models
However, deep CNNs are vulnerable to adversarial patches, which are physically realizable and stealthy, raising new security concerns on the real-world applications of these models.
Progressive-Scale Boundary Blackbox Attack via Projective Gradient Estimation
In this paper, we show that such efficiency highly depends on the scale at which the attack is applied, and attacking at the optimal scale significantly improves the efficiency.
Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks
We aim to bridge the gap between the two by investigating how to efficiently estimate gradient based on a projected low-dimensional space.
QEBA: Query-Efficient Boundary-Based Blackbox Attack
Such adversarial attacks can be achieved by adding a small magnitude of perturbation to the input to mislead model prediction.
Data-Free Adversarial Perturbations for Practical Black-Box Attack
Adversarial examples often exhibit black-box attacking transferability, which allows that adversarial examples crafted for one model can fool another model.
Characterizing Membership Privacy in Stochastic Gradient Langevin Dynamics
Based on this framework, we demonstrate that SGLD can prevent the information leakage of the training dataset to a certain extent.
Pruning from Scratch
Network pruning is an important research field aiming at reducing computational costs of neural networks.
Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protection
In this paper, we aim to understand the generalization properties of generative adversarial networks (GANs) from a new perspective of privacy protection.
P3SGD: Patient Privacy Preserving SGD for Regularizing Deep CNNs in Pathological Image Classification
(2) privacy leakage: the model trained using a conventional method may involuntarily reveal the private information of the patients in the training dataset.
Infinite Curriculum Learning for Efficiently Detecting Gastric Ulcers in WCE Images
This paper considers WCE-based gastric ulcer detection, in which the major challenge is to detect the lesions in a local region.
G2C: A Generator-to-Classifier Framework Integrating Multi-Stained Visual Cues for Pathological Glomerulus Classification
Given an input image from a specified stain, several generators are first applied to estimate its appearances in other staining methods, and a classifier follows to combine visual cues from different stains for prediction (whether it is pathological, or which type of pathology it has).
