Search Results for author:
Found 62 papers, 30 papers with code
Learning Document Embeddings by Predicting N-grams for Sentiment Classification of Long Movie Reviews
Many document embeddings methods have been proposed to capture semantics, but they still can't outperform bag-of-ngram based methods on this task.
Weighted Neural Bag-of-n-grams Model: New Baselines for Text Classification
We train n-gram embeddings and use NB weighting to guide the neural models to focus on important words.
Investigating Different Syntactic Context Types and Context Representations for Learning Word Embeddings
The number of word embedding models is growing every year.
Initializing Convolutional Filters with Semantic Features for Text Classification
Convolutional Neural Networks (CNNs) are widely used in NLP tasks.
Ngram2vec: Learning Improved Word Representations from Ngram Co-occurrence Statistics
The existing word representation methods mostly limit their information source to word co-occurrence statistics.
Analogical Reasoning on Chinese Morphological and Semantic Relations
Analogical reasoning is effective in capturing linguistic regularities.
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
In this work, we propose a novel multi-task learning approach, Multi-gate Mixture-of-Experts (MMoE), which explicitly learns to model task relationships from data.
Fairness in Recommendation Ranking through Pairwise Comparisons
Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information.
Taking Care of The Discretization Problem: A Comprehensive Study of the Discretization Problem and A Black-Box Adversarial Attack in Discrete Integer Domain
In this work, we first conduct a comprehensive study of existing methods and tools for crafting.
Recommending what video to watch next: a multitask ranking system
In this paper, we introduce a large scale multi-objective ranking system for recommending what video to watch next on an industrial video sharing platform.
UER: An Open-Source Toolkit for Pre-training Models
Existing works, including ELMO and BERT, have revealed the importance of pre-training for NLP tasks.
Sampling-Bias-Corrected Neural Modeling for Large Corpus Item Recommendations
However, batch loss is subject to sampling bias which could severely restrict model performance, particularly in the case of power-law distribution.
K-BERT: Enabling Language Representation with Knowledge Graph
For machines to achieve this capability, we propose a knowledge-enabled language representation model (K-BERT) with knowledge graphs (KGs), in which triples are injected into the sentences as domain knowledge.
Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems
In this paper, we conduct the first comprehensive and systematic study of the adversarial attacks on SR systems (SRSs) to understand their security weakness in the practical blackbox setting.
Understanding and Improving Knowledge Distillation
Knowledge Distillation (KD) is a model-agnostic technique to improve model quality while having a fixed capacity budget.
FastBERT: a Self-distilling BERT with Adaptive Inference Time
Pre-trained language models like BERT have proven to be highly performant.
CLUE: A Chinese Language Understanding Evaluation Benchmark
The advent of natural language understanding (NLU) benchmarks for English, such as GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of tasks.
Synthesizer: Rethinking Self-Attention in Transformer Models
The dot product self-attention is known to be central and indispensable to state-of-the-art Transformer models.
Ranked #1 on
Dialogue Generation
on Persona-Chat
(BLEU-1 metric, using extra
training data)
Learning-to-Rank with Partitioned Preference: Fast Estimation for the Plackett-Luce Model
We investigate the Plackett-Luce (PL) model based listwise learning-to-rank (LTR) on data with partitioned preference, where a set of items are sliced into ordered and disjoint partitions, but the ranking of items within a partition is unknown.
HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections
The proposed approach is based on a decomposable hypernetwork that learns grid-wise projections that help to specialize regions in weight matrices for different tasks.
Small Towers Make Big Differences
A delicate balance between multi-task generalization and multi-objective optimization is therefore needed for finding a better trade-off between efficiency and generalization.
Measuring and Harnessing Transference in Multi-Task Learning
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
HyperGrid Transformers: Towards A Single Model for Multiple Tasks
Specifically, we propose a decomposable hypernetwork that learns grid-wise projections that help to specialize regions in weight matrices for different tasks.
Information Transfer in Multi-Task Learning
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
Synthesizer: Rethinking Self-Attention for Transformer Models
The dot product self-attention is known to be central and indispensable to state-of-the-art Transformer models.
BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networks
Verifying and explaining the behavior of neural networks is becoming increasingly important, especially when they are deployed in safety-critical applications.
Attack as Defense: Characterizing Adversarial Examples using Robustness
Though various defense mechanisms have been proposed to improve robustness of deep learning software, many of them are ineffective against adaptive attacks.
DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning
State-of-the-art MoE models use a trainable sparse gate to select a subset of the experts for each input example.
The Benchmark Lottery
The world of empirical machine learning (ML) strongly relies on benchmarks in order to determine the relative effectiveness of different algorithms and methods.
SEC4SR: A Security Analysis Platform for Speaker Recognition
To bridge this gap, we present SEC4SR, the first platform enabling researchers to systematically and comprehensively evaluate adversarial attacks and defenses in SR. SEC4SR incorporates 4 white-box and 2 black-box attacks, 24 defenses including our novel feature-level transformations.
Efficiently Identifying Task Groupings for Multi-Task Learning
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
Adversarial Attacks on ML Defense Models Competition
Due to the vulnerability of deep neural networks (DNNs) to adversarial examples, a large number of defense techniques have been proposed to alleviate this problem in recent years.
Transformer Memory as a Differentiable Search Index
In this paper, we demonstrate that information retrieval can be accomplished with a single Transformer, in which all information about the corpus is encoded in the parameters of the model.
Semantic Matching from Different Perspectives
In this paper, we pay attention to the issue which is usually overlooked, i. e., \textit{similarity should be determined from different perspectives}.
HyperPrompt: Prompt-based Task-Conditioning of Transformers
Prompt-Tuning is a new paradigm for finetuning pre-trained language models in a parameter-efficient way.
Improving Multi-Task Generalization via Regularizing Spurious Correlation
First, the risk of having non-causal knowledge is higher, as the shared MTL model needs to encode all knowledge from different tasks, and causal knowledge for one task could be potentially spurious to the other.
Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognition
According to the characteristic of SRSs, we present 22 diverse transformations and thoroughly evaluate them using 7 recent promising adversarial attacks (4 white-box and 3 black-box) on speaker recognition.
AS2T: Arbitrary Source-To-Target Adversarial Attack on Speaker Recognition Systems
Recent work has illuminated the vulnerability of speaker recognition systems (SRSs) against adversarial attacks, raising significant security concerns in deploying SRSs.
Scalable Bayesian Inference for Detection and Deblending in Astronomical Images
We present a new probabilistic method for detecting, deblending, and cataloging astronomical sources called the Bayesian Light Source Separator (BLISS).
Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching
Previous studies have proved that cross-lingual knowledge distillation can significantly improve the performance of pre-trained models for cross-lingual similarity matching tasks.
SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision
The latest industrial inference engines, such as FasterTransformer and TurboTransformers, have verified that half-precision floating point (FP16) and 8-bit integer (INT8) quantization can greatly improve model inference speed.
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning
Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries, which are expensive and impractical for low-resource languages.
QVIP: An ILP-based Formal Verification Approach for Quantized Neural Networks
Experimental results on QNNs with different quantization bits confirm the effectiveness and efficiency of our approach, e. g., two orders of magnitude faster and able to solve more verification tasks in the same time limit than the state-of-the-art methods.
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities
The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework.
Fast as CHITA: Neural Network Pruning with Combinatorial Optimization
Our approach, CHITA, extends the classical Optimal Brain Surgeon framework and results in significant improvements in speed, memory, and performance over existing optimization-based approaches for network pruning.
Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
Distilling high-accuracy Graph Neural Networks~(GNNs) to low-latency multilayer perceptrons~(MLPs) on graph tasks has become a hot research topic.
Weight-Inherited Distillation for Task-Agnostic BERT Compression
Knowledge Distillation (KD) is a predominant approach for BERT compression.
Recouple Event Field via Probabilistic Bias for Event Extraction
Event Extraction (EE), aiming to identify and classify event triggers and arguments from event mentions, has benefited from pre-trained language models (PLMs).
QFA2SR: Query-Free Adversarial Transfer Attacks to Speaker Recognition Systems
Current adversarial attacks against speaker recognition systems (SRSs) require either white-box access or heavy black-box queries to the target SRS, thus still falling behind practical attacks against proprietary commercial APIs and voice-controlled devices.
COMET: Learning Cardinality Constrained Mixture of Experts with Trees and Local Search
To deal with this challenge, we propose a novel, permutation-based local search method that can complement first-order methods in training any sparse gate, e. g., Hash routing, Top-k, DSelect-k, and COMET.
EVD Surgical Guidance with Retro-Reflective Tool Tracking and Spatial Reconstruction using Head-Mounted Augmented Reality Device
The head surface is reconstructed using depth data for spatial registration, avoiding fixing tracking targets rigidly on the patient's skull.
$\rm SP^3$: Enhancing Structured Pruning via PCA Projection
Structured pruning is a widely used technique for reducing the size of pre-trained language models (PLMs), but current methods often overlook the potential of compressing the hidden dimension (d) in PLMs, a dimension critical to model size and efficiency.
Create and Find Flatness: Building Flat Training Spaces in Advance for Continual Learning
Therefore, we shift the attention to the current task learning stage, presenting a novel framework, C&F (Create and Find Flatness), which builds a flat training space for each task in advance.
Talking Models: Distill Pre-trained Knowledge to Downstream Models via Interactive Communication
In this paper, we extend KD with an interactive communication process to help students of downstream tasks learn effectively from pre-trained foundation models.
LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views
By combining two complementing models, LEVI effectively suppresses problematic features in both the fine-tuning data and pre-trained model and preserves useful features for new tasks.
Wisdom of Committee: Distilling from Foundation Model to Specialized Application Model
Further, to accommodate the dissimilarity among the teachers in the committee, we introduce DiverseDistill, which allows the student to understand the expertise of each teacher and extract task knowledge.
Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction
We evaluate DRA across various open-source and close-source models, showcasing state-of-the-art jailbreak success rates and attack efficiency.
KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation
Parameter-efficient finetuning (PEFT) is a key technique for adapting large language models (LLMs) to downstream tasks.
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
Kullback-Leiber divergence has been widely used in Knowledge Distillation (KD) to compress Large Language Models (LLMs).
Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System
Catastrophic forgetting is a challenge for model deployment in industrial real-time systems, which requires the model to quickly master a new task without forgetting the old one.
