Search Results for author:
Found 40 papers, 18 papers with code
Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns
We present a comprehensive study of sparse attention patterns in Transformer models.
“Average” Approximates “First Principal Component”? An Empirical Analysis on Representations from Neural Language Models
In this paper, we present an empirical property of these representations—”average” approximates “first principal component”.
Esports Data-to-commentary Generation on Large-scale Data-to-text Dataset
Therefore, in this study, we introduce a task of generating game commentaries from structured data records to address the problem.
Reconstructing Training Data from Model Gradient, Provably
Understanding when and how much a model gradient leaks information about the training sample is an important question in privacy.
Improving ECG-based COVID-19 diagnosis and mortality predictions using pre-pandemic medical records at population-scale
Pandemic outbreaks such as COVID-19 occur unexpectedly, and need immediate action due to their potential devastating consequences on global health.
Multilingual Speech Emotion Recognition With Multi-Gating Mechanism and Neural Architecture Search
While Speech Emotion Recognition (SER) is a common application for popular languages, it continues to be a problem for low-resourced languages, i. e., languages with no pretrained speech-to-text recognition models.
GLM-130B: An Open Bilingual Pre-trained Model
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters.
Ranked #1 on
Language Modelling
on CLUE (CMRC2018)
Waveformer: Linear-Time Attention with Forward and Backward Wavelet Transform
We propose Waveformer that learns attention mechanism in the wavelet coefficient space, requires only linear time complexity, and enjoys universal approximating power.
Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations
Specifically, we design a masked policy network with a binary mask to block certain modalities.
M^4I: Multi-modal Models Membership Inference
To achieve this, we propose Multi-modal Models Membership Inference (M^4I) with two attack methods to infer the membership status, named metric-based (MB) M^4I and feature-based (FB) M^4I, respectively.
CV 3315 Is All You Need : Semantic Segmentation Competition
This competition focus on Urban-Sense Segmentation based on the vehicle camera view.
Debiasing Learning for Membership Inference Attacks Against Recommender Systems
To address the above limitations, we propose a Debiasing Learning for Membership Inference Attacks against recommender systems (DL-MIA) framework that has four main components: (1) a difference vector generator, (2) a disentangled encoder, (3) a weight estimator, and (4) an attack model.
SPGNet: Spatial Projection Guided 3D Human Pose Estimation in Low Dimensional Space
We propose a method SPGNet for 3D human pose estimation that mixes multi-dimensional re-projection into supervised learning.
Ranked #37 on
3D Human Pose Estimation
on Human3.6M
Rethinking the Setting of Semi-supervised Learning on Graphs
graph benchmark (IGB) consisting of 4 datasets.
Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition
Fine-tuning pre-trained language models has recently become a common practice in building NLP models for various tasks, especially few-shot tasks.
WeDef: Weakly Supervised Backdoor Defense for Text Classification
Inspired by this observation, in WeDef, we define the reliability of samples based on whether the predictions of the weak classifier agree with their labels in the poisoned training set.
Beyond the Granularity: Multi-Perspective Dialogue Collaborative Selection for Dialogue State Tracking
However, no matter how the dialogue history is used, each existing model uses its own consistent dialogue history during the entire state tracking process, regardless of which slot is updated.
Effectively Using Long and Short Sessions for Multi-Session-based Recommendations
The RNN often used in previous work is not suitable to process short sessions, because RNN only focuses on the sequential relationship, which we find is not the only relationship between items in short sessions.
HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification
However, in this paradigm, there exists a huge gap between the classification tasks with sophisticated label hierarchy and the masked language model (MLM) pretraining tasks of PLMs and thus the potentials of PLMs can not be fully tapped.
Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification
Hierarchical text classification is a challenging subtask of multi-label classification due to its complex label hierarchy.
Weakly Supervised Correspondence Learning
Correspondence learning is a fundamental problem in robotics, which aims to learn a mapping between state, action pairs of agents of different dynamics or embodiments.
Learning from Imperfect Demonstrations via Adversarial Confidence Transfer
Existing learning from demonstration algorithms usually assume access to expert demonstrations.
An Interactive Visualization Tool for Understanding Active Learning
Despite recent progress in artificial intelligence and machine learning, many state-of-the-art methods suffer from a lack of explainability and transparency.
Membership Inference Attacks Against Recommender Systems
In this paper, we make the first attempt on quantifying the privacy leakage of recommender systems through the lens of membership inference.
Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking
However, the overwhelming majority of the slots in each turn should simply inherit the slot values from the previous turn.
UCPhrase: Unsupervised Context-aware Quality Phrase Tagging
Training a conventional neural tagger based on silver labels usually faces the risk of overfitting phrase surface names.
Ranked #1 on
Phrase Tagging
on KPTimes
Cross-Domain Contract Element Extraction with a Bi-directional Feedback Clause-Element Relation Network
First, contract elements are far more fine-grained than named entities, which hinders the transfer of extractors.
Cross-Domain Named Entity Recognition
named-entity-recognition
+1
XCrossNet: Feature Structure-Oriented Learning for Click-Through Rate Prediction
Motivated by this, we propose a novel Extreme Cross Network, abbreviated XCrossNet, which aims at learning dense and sparse feature interactions in an explicit manner.
"Average" Approximates "First Principal Component"? An Empirical Analysis on Representations from Neural Language Models
In this paper, we present an empirical property of these representations -- "average" approximates "first principal component".
Low-Power Wireless Wearable ECG Monitoring Chestbelt Based on Ferroelectric Microprocessor
The ECG monitoring device, abbreviated as ECGM, is designed based on ferroelectric microprocessor which provides ultra-low power consumption and contains four parts-MCU, BLE, Sensors and Power.
X-Class: Text Classification with Extremely Weak Supervision
Finally, we pick the most confident documents from each cluster to train a text classifier.
Emora: An Inquisitive Social Chatbot Who Cares For You
Inspired by studies on the overwhelming presence of experience-sharing in human-human conversations, Emora, the social chatbot developed by Emory University, aims to bring such experience-focused interaction to the current field of conversational AI.
Extending Multilingual BERT to Low-Resource Languages
Multilingual BERT (M-BERT) has been a huge success in both supervised and zero-shot cross-lingual transfer learning.
Cross-Lingual Ability of Multilingual BERT: An Empirical Study
Recent work has exhibited the surprising cross-lingual abilities of multilingual BERT (M-BERT) -- surprising since it is trained without any cross-lingual objective and with no aligned data.
Learning to Order Sub-questions for Complex Question Answering
Answering complex questions involving multiple entities and relations is a challenging task.
CrossWeigh: Training Named Entity Tagger from Imperfect Annotations
Therefore, we manually correct these label mistakes and form a cleaner test set.
Ranked #3 on
Named Entity Recognition
on CoNLL++
(using extra training data)
Discriminative Topic Mining via Category-Name Guided Text Embedding
We propose a new task, discriminative topic mining, which leverages a set of user-provided category names to mine discriminative topics from text corpora.
Raw-to-End Name Entity Recognition in Social Media
Our model neither requires the conversion from character sequences to word sequences, nor assumes tokenizer can correctly detect all word boundaries.
ECG Identification under Exercise and Rest Situations via Various Learning Methods
By applying various existing learning methods to our ECG dataset, we find that current methods which can well support the identification of individuals under rests, do not suffice to present satisfying ECGID performance under exercise situations, therefore exposing the deficiency of existing ECG identification methods.
A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies
Learning effective visuomotor policies for robots purely from data is challenging, but also appealing since a learning-based system should not require manual tuning or calibration.
