Search Results for author:
Found 52 papers, 25 papers with code
Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System
Catastrophic forgetting is a challenge for model deployment in industrial real-time systems, which requires the model to quickly master a new task without forgetting the old one.
QFA2SR: Query-Free Adversarial Transfer Attacks to Speaker Recognition Systems
Current adversarial attacks against speaker recognition systems (SRSs) require either white-box access or heavy black-box queries to the target SRS, thus still falling behind practical attacks against proprietary commercial APIs and voice-controlled devices.
Recouple Event Field via Probabilistic Bias for Event Extraction
Event Extraction (EE), aiming to identify and classify event triggers and arguments from event mentions, has benefited from pre-trained language models (PLMs).
Weight-Inherited Distillation for Task-Agnostic BERT Compression
Knowledge Distillation (KD) is a predominant approach for BERT compression.
Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
Distilling high-accuracy Graph Neural Networks~(GNNs) to low-latency multilayer perceptrons~(MLPs) on graph tasks has become a hot research topic.
Fast as CHITA: Neural Network Pruning with Combinatorial Optimization
Our approach, CHITA, extends the classical Optimal Brain Surgeon framework and results in significant improvements in speed, memory, and performance over existing optimization-based approaches for network pruning.
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities
The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework.
QVIP: An ILP-based Formal Verification Approach for Quantized Neural Networks
Experimental results on QNNs with different quantization bits confirm the effectiveness and efficiency of our approach, e. g., two orders of magnitude faster and able to solve more verification tasks in the same time limit than the state-of-the-art methods.
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning
Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries, which are expensive and impractical for low-resource languages.
SAMP: A Toolkit for Model Inference with Self-Adaptive Mixed-Precision
The latest industrial inference engines, such as FasterTransformer1 and TurboTransformers, have verified that half-precision floating point (FP16) and 8-bit integer (INT8) quantization can greatly improve model inference speed.
Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching
Previous studies have proved that cross-lingual knowledge distillation can significantly improve the performance of pre-trained models for cross-lingual similarity matching tasks.
Scalable Bayesian Inference for Detection and Deblending in Astronomical Images
We present a new probabilistic method for detecting, deblending, and cataloging astronomical sources called the Bayesian Light Source Separator (BLISS).
AS2T: Arbitrary Source-To-Target Adversarial Attack on Speaker Recognition Systems
Recent work has illuminated the vulnerability of speaker recognition systems (SRSs) against adversarial attacks, raising significant security concerns in deploying SRSs.
Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognition
According to the characteristic of SRSs, we present 22 diverse transformations and thoroughly evaluate them using 7 recent promising adversarial attacks (4 white-box and 3 black-box) on speaker recognition.
Improving Multi-Task Generalization via Regularizing Spurious Correlation
First, the risk of having non-causal knowledge is higher, as the shared MTL model needs to encode all knowledge from different tasks, and causal knowledge for one task could be potentially spurious to the other.
HyperPrompt: Prompt-based Task-Conditioning of Transformers
Prompt-Tuning is a new paradigm for finetuning pre-trained language models in a parameter-efficient way.
Transformer Memory as a Differentiable Search Index
In this paper, we demonstrate that information retrieval can be accomplished with a single Transformer, in which all information about the corpus is encoded in the parameters of the model.
Semantic Matching from Different Perspectives
In this paper, we pay attention to the issue which is usually overlooked, i. e., \textit{similarity should be determined from different perspectives}.
Adversarial Attacks on ML Defense Models Competition
Due to the vulnerability of deep neural networks (DNNs) to adversarial examples, a large number of defense techniques have been proposed to alleviate this problem in recent years.
Efficiently Identifying Task Groupings for Multi-Task Learning
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
SEC4SR: A Security Analysis Platform for Speaker Recognition
To bridge this gap, we present SEC4SR, the first platform enabling researchers to systematically and comprehensively evaluate adversarial attacks and defenses in SR. SEC4SR incorporates 4 white-box and 2 black-box attacks, 24 defenses including our novel feature-level transformations.
The Benchmark Lottery
The world of empirical machine learning (ML) strongly relies on benchmarks in order to determine the relative effectiveness of different algorithms and methods.
DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning
State-of-the-art MoE models use a trainable sparse gate to select a subset of the experts for each input example.
Attack as Defense: Characterizing Adversarial Examples using Robustness
Though various defense mechanisms have been proposed to improve robustness of deep learning software, many of them are ineffective against adaptive attacks.
BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networks
Verifying and explaining the behavior of neural networks is becoming increasingly important, especially when they are deployed in safety-critical applications.
Synthesizer: Rethinking Self-Attention for Transformer Models
The dot product self-attention is known to be central and indispensable to state-of-the-art Transformer models.
Information Transfer in Multi-Task Learning
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
HyperGrid Transformers: Towards A Single Model for Multiple Tasks
Specifically, we propose a decomposable hypernetwork that learns grid-wise projections that help to specialize regions in weight matrices for different tasks.
Measuring and Harnessing Transference in Multi-Task Learning
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
Small Towers Make Big Differences
A delicate balance between multi-task generalization and multi-objective optimization is therefore needed for finding a better trade-off between efficiency and generalization.
HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections
The proposed approach is based on a decomposable hypernetwork that learns grid-wise projections that help to specialize regions in weight matrices for different tasks.
Learning-to-Rank with Partitioned Preference: Fast Estimation for the Plackett-Luce Model
We investigate the Plackett-Luce (PL) model based listwise learning-to-rank (LTR) on data with partitioned preference, where a set of items are sliced into ordered and disjoint partitions, but the ranking of items within a partition is unknown.
Synthesizer: Rethinking Self-Attention in Transformer Models
The dot product self-attention is known to be central and indispensable to state-of-the-art Transformer models.
Ranked #1 on
Dialogue Generation
on Persona-Chat
(BLEU-1 metric)
CLUE: A Chinese Language Understanding Evaluation Benchmark
The advent of natural language understanding (NLU) benchmarks for English, such as GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of tasks.
FastBERT: a Self-distilling BERT with Adaptive Inference Time
Pre-trained language models like BERT have proven to be highly performant.
Understanding and Improving Knowledge Distillation
Knowledge Distillation (KD) is a model-agnostic technique to improve model quality while having a fixed capacity budget.
Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems
In this paper, we conduct the first comprehensive and systematic study of the adversarial attacks on SR systems (SRSs) to understand their security weakness in the practical blackbox setting.
K-BERT: Enabling Language Representation with Knowledge Graph
For machines to achieve this capability, we propose a knowledge-enabled language representation model (K-BERT) with knowledge graphs (KGs), in which triples are injected into the sentences as domain knowledge.
Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations
However, batch loss is subject to sampling bias which could severely restrict model performance, particularly in the case of power-law distribution.
UER: An Open-Source Toolkit for Pre-training Models
Existing works, including ELMO and BERT, have revealed the importance of pre-training for NLP tasks.
Recommending what video to watch next: a multitask ranking system
In this paper, we introduce a large scale multi-objective ranking system for recommending what video to watch next on an industrial video sharing platform.
Taking Care of The Discretization Problem: A Comprehensive Study of the Discretization Problem and A Black-Box Adversarial Attack in Discrete Integer Domain
In this work, we first conduct a comprehensive study of existing methods and tools for crafting.
Fairness in Recommendation Ranking through Pairwise Comparisons
Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information.
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
In this work, we propose a novel multi-task learning approach, Multi-gate Mixture-of-Experts (MMoE), which explicitly learns to model task relationships from data.
Analogical Reasoning on Chinese Morphological and Semantic Relations
Analogical reasoning is effective in capturing linguistic regularities.
Initializing Convolutional Filters with Semantic Features for Text Classification
Convolutional Neural Networks (CNNs) are widely used in NLP tasks.
Investigating Different Syntactic Context Types and Context Representations for Learning Word Embeddings
The number of word embedding models is growing every year.
Ngram2vec: Learning Improved Word Representations from Ngram Co-occurrence Statistics
The existing word representation methods mostly limit their information source to word co-occurrence statistics.
Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
How can we learn a classifier that is "fair" for a protected or sensitive group, when we do not know if the input to the classifier belongs to the protected group?
Weighted Neural Bag-of-n-grams Model: New Baselines for Text Classification
We train n-gram embeddings and use NB weighting to guide the neural models to focus on important words.
Learning Document Embeddings by Predicting N-grams for Sentiment Classification of Long Movie Reviews
Many document embeddings methods have been proposed to capture semantics, but they still can't outperform bag-of-ngram based methods on this task.
