Search Results for author:
Found 47 papers, 18 papers with code
Wukong: Towards a Scaling Law for Large-Scale Recommendation
Scaling laws play an instrumental role in the sustainable improvement in model quality.
Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendation
We study a mismatch between the deep learning recommendation models' flat architecture, common distributed training paradigm and hierarchical data center topology.
Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning
To support various applications, business owners often seek the customized models that are obtained by fine-tuning a pre-trained LLM through the API provided by LLM owners or cloud servers.
xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein
We propose a unified protein language model, xTrimoPGLM, to address these two types of tasks simultaneously through an innovative pre-training framework.
Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection
Despite considerable advances in automated fake news detection, due to the timely nature of news, it remains a critical open question how to effectively predict the veracity of news articles based on limited fact-checks.
On the Performance of Multidimensional Constellation Shaping for Linear and Nonlinear Optical Fiber Channel
Multidimensional constellation shaping of up to 32 dimensions with different spectral efficiencies are compared through AWGN and fiber-optic simulations.
A Robust Integrated Multi-Strategy Bus Control System via Deep Reinforcement Learning
An efficient urban bus control system has the potential to significantly reduce travel delays and streamline the allocation of transportation resources, thereby offering enhanced and user-friendly transit services to passengers.
Proximity-Informed Calibration for Deep Neural Networks
We examine the problem over 504 pretrained ImageNet models and observe that: 1) Proximity bias exists across a wide variety of model architectures and sizes; 2) Transformer-based models are relatively more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms like temperature scaling; 4) Models tend to overfit more heavily on low proximity samples than on high proximity samples.
How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence
In general, a large scale of testing in simulation environment is conducted and then the learned driving knowledge is transferred to the real world, so how to adapt driving knowledge learned in simulation to reality becomes a critical issue.
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
It is widely acknowledged that large models have the potential to deliver superior performance across a broad range of domains.
Trust, but Verify: Using Self-Supervised Probing to Improve Trustworthiness
Trustworthy machine learning is of primary importance to the practical deployment of deep learning models.
Probabilistic Knowledge Distillation of Face Ensembles
Mean ensemble (i. e. averaging predictions from multiple models) is a commonly-used technique in machine learning that improves the performance of each individual model.
Birds of a Feather Trust Together: Knowing When to Trust a Classifier via Adaptive Neighborhood Aggregation
How do we know when the predictions made by a classifier can be trusted?
Coordinating CAV Swarms at Intersections with a Deep Learning Model
Connected and automated vehicles (CAVs) are viewed as a special kind of robots that have the potential to significantly improve the safety and efficiency of traffic.
lo-fi: distributed fine-tuning without communication
When fine-tuning DeiT-base and DeiT-large on ImageNet, this procedure matches accuracy in-distribution and improves accuracy under distribution shift compared to the baseline, which observes the same amount of data but communicates gradients at each step.
Neural PCA for Flow-Based Representation Learning
Without exploiting any label information, the principal components recovered store the most informative elements in their \emph{leading} dimensions and leave the negligible in the \emph{trailing} ones, allowing for clear performance improvements of $5\%$-$10\%$ in downstream tasks.
Temporal Logic Imitation: Learning Plan-Satisficing Motion Policies from Demonstrations
In this work, we identify the roots of this challenge as the failure of a learned continuous policy to satisfy the discrete plan implicit in the demonstration.
Parameter-Efficient Sparsity for Large Language Models Fine-Tuning
With the dramatically increased number of parameters in language models, sparsity methods have received ever-increasing research focus to compress and accelerate the models.
Neighborhood Attention Transformer
We present Neighborhood Attention (NA), the first efficient and scalable sliding-window attention mechanism for vision.
Ranked #119 on
Semantic Segmentation
on ADE20K
A General Framework for Debiasing in CTR Prediction
Most of the existing methods for debaising in click-through rate (CTR) prediction depend on an oversimplified assumption, i. e., the click probability is the product of observation probability and relevance probability.
Probabilistic Contrastive Loss for Self-Supervised Learning
This paper proposes a probabilistic contrastive loss function for self-supervised learning.
Enhancing Top-N Item Recommendations by Peer Collaboration
In this paper, we plan to exploit such redundancy phenomena to improve the performance of RS.
R4: A Framework for Route Representation and Route Recommendation
For user representation, we utilize a series of historical navigation to extract user preference.
Set-based State Estimation with Probabilistic Consistency Guarantee under Epistemic Uncertainty
Consistent state estimation is challenging, especially under the epistemic uncertainties arising from learned (nonlinear) dynamic and observation models.
Supervised Bayesian Specification Inference from Demonstrations
When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task.
Spherical Confidence Learning for Face Recognition
Probabilistic Face Embeddings (PFE) is the first attempt to address this dilemma.
You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient
Motivated by the necessity of efficient inference across various constraints on BERT, we propose a novel approach, YOCO-BERT, to achieve compress once and deploy everywhere.
1xN Pattern for Pruning Convolutional Neural Networks
We also provide a workflow of filter rearrangement that first rearranges the weight matrix in the output channel dimension to derive more influential blocks for accuracy improvements and then applies similar rearrangement to the next-layer weights in the input channel dimension to ensure correct convolutional operations.
Tackling Variabilities in Autonomous Driving
In this work, we aim to extensively explore the above system design challenges and these challenges motivate us to propose a comprehensive framework that synergistically handles the heterogeneous hardware accelerator design principles, system design criteria, and task scheduling mechanism.
Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking
This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN), for visual object tracking.
PipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers
PipeTransformer automatically adjusts the pipelining and data parallelism by identifying and freezing some layers during the training, and instead allocates resources for training of the remaining active layers.
Hypersphere Face Uncertainty Learning
To address these issues, in this paper, we propose a novel framework for face uncertainty learning in hyperspherical space.
PyTorch Distributed: Experiences on Accelerating Data Parallel Training
This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module.
Quantum Inspired Word Representation and Computation
Word meaning has different aspects, while the existing word representation "compresses" these aspects into a single vector, and it needs further analysis to recover the information in different dimensions.
Identifying through Flows for Recovering Latent Representations
Yet, most deep generative models do not address the question of identifiability, and thus fail to deliver on the promise of the recovery of the true latent sources that generate the observations.
Self-Balanced Dropout
Dropout is known as an effective way to reduce overfitting via preventing co-adaptations of units.
Diachronic Sense Modeling with Deep Contextualized Word Embeddings: An Ecological View
Diachronic word embeddings have been widely used in detecting temporal changes.
A Prism Module for Semantic Disentanglement in Name Entity Recognition
To solve this problem, we propose a prism module to disentangle the semantic aspects of words and reduce noise at the input layer of a model.
Ranked #52 on
Named Entity Recognition (NER)
on CoNLL 2003 (English)
Bayesian Inference of Temporal Task Specifications from Demonstrations
When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task.
Generalize Symbolic Knowledge With Neural Rule Engine
As neural networks have dominated the state-of-the-art results in a wide range of NLP tasks, it attracts considerable attention to improve the performance of neural models by integrating symbolic knowledge.
From Random to Supervised: A Novel Dropout Mechanism Integrated with Global Information
Dropout is used to avoid overfitting by randomly dropping units from the neural networks during training.
Analogical Reasoning on Chinese Morphological and Semantic Relations
Analogical reasoning is effective in capturing linguistic regularities.
Achievable Information Rates for Nonlinear Fiber Communication via End-to-end Autoencoder Learning
Machine learning is used to compute achievable information rates (AIRs) for a simplified fiber channel.
RDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations
Recent advances in deep learning have led various applications to unprecedented achievements, which could potentially bring higher intelligence to a broad spectrum of mobile and ubiquitous applications.
Ngram2vec: Learning Improved Word Representations from Ngram Co-occurrence Statistics
The existing word representation methods mostly limit their information source to word co-occurrence statistics.
Initializing Convolutional Filters with Semantic Features for Text Classification
Convolutional Neural Networks (CNNs) are widely used in NLP tasks.
