Search Results for author:
Found 114 papers, 41 papers with code
RGL: A Simple yet Effective Relation Graph Augmented Prompt-based Tuning Approach for Few-Shot Learning
In this work, we show that Relation Graph augmented Learning (RGL) can improve the performance of few-shot natural language understanding tasks.
Scalable Differential Privacy with Certified Robustness in Adversarial Learning
In this paper, we aim to develop a scalable algorithm to preserve differential privacy (DP) in adversarial learning for deep neural networks (DNNs), with certified robustness to adversarial examples.
Parameter-Efficient Domain Knowledge Integration from Multiple Sources for Biomedical Pre-trained Language Models
These knowledge adapters are pre-trained for individual domain knowledge sources and integrated via an attention-based knowledge controller to enrich PLMs.
Adversarial Attack against Cross-lingual Knowledge Graph Alignment
Recent literatures have shown that knowledge graph (KG) learning models are highly vulnerable to adversarial attacks.
Video4MRI: An Empirical Study on Brain Magnetic Resonance Image Analytics with CNN-based Video Classification Frameworks
Due to the high similarity between MRI data and videos, we conduct extensive empirical studies on video recognition techniques for MRI classification to answer the questions: (1) can we directly use video recognition models for MRI classification, (2) which model is more appropriate for MRI, (3) are the common tricks like data augmentation in video recognition still useful for MRI classification?
Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement
In this work, we propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder (BVAE) equipped with diffusion, denoise, and disentanglement, namely D3VAE.
Towards Table-to-Text Generation with Pretrained Language Model: A Table Structure Understanding and Text Deliberating Approach
Although remarkable progress on the neural table-to-text methods has been made, the generalization issues hinder the applicability of these models due to the limited source tables.
Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution
Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning.
Temporal Output Discrepancy for Loss Estimation-based Active Learning
Inspired by the fact that the samples with higher loss are usually more informative to the model than the samples with lower loss, in this paper we present a novel deep active learning approach that queries the oracle for data annotation when the unlabeled sample is believed to incorporate high loss.
Learning from Training Dynamics: Identifying Mislabeled Data Beyond Manually Designed Features
While mislabeled or ambiguously-labeled samples in the training set could negatively affect the performance of deep models, diagnosing the dataset and identifying mislabeled samples helps to improve the generalization power.
A Contextual Master-Slave Framework on Urban Region Graph for Urban Village Detection
The core idea of such a framework is to firstly pre-train a basis (or master) model over the URG, and then to adaptively derive specific (or slave) models from the basis model for different regions.
Multi-Job Intelligent Scheduling with Cross-Device Federated Learning
The system model enables a parallel training process of multiple jobs, with a cost model based on the data fairness and the training time of diverse devices during the parallel training process.
Textual Data Augmentation for Patient Outcomes Prediction
Deep learning models have demonstrated superior performance in various healthcare applications.
SDWPF: A Dataset for Spatial Dynamic Wind Power Forecasting Challenge at KDD Cup 2022
Thus, Wind Power Forecasting (WPF) has been widely recognized as one of the most critical issues in wind power integration and operation.
$\textbf{P$^2$A}$: A Dataset and Benchmark for Dense Action Detection from Table Tennis Match Broadcasting Videos
While deep learning has been widely used for video analytics, such as video classification and action detection, dense action detection with fast-moving subjects from sports videos is still challenging.
Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resources
Although more layers and more parameters generally improve the accuracy of the models, such big models generally have high computational complexity and require big memory, which exceed the capacity of small devices for inference and incurs long training time.
Accelerated Federated Learning with Decoupled Adaptive Optimization
The federated learning (FL) framework enables edge clients to collaboratively learn a shared inference model while keeping privacy of training data on clients.
Pareto Optimization for Active Learning under Out-of-Distribution Data Scenarios
In this paper, we propose a sampling scheme, Monte-Carlo Pareto Optimization for Active Learning (POAL), which selects optimal subsets of unlabeled samples with fixed batch size from the unlabeled data pool.
Distilling Ensemble of Explanations for Weakly-Supervised Pre-Training of Image Segmentation Models
Though image classification datasets could provide the backbone networks with rich visual features and discriminative ability, they are incapable of fully pre-training the target model (i. e., backbone+segmentation modules) in an end-to-end manner.
FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity
As the efficiency of training in the ring topology prefers devices with homogeneous resources, the classification based on the computing capacity mitigates the impact of straggler effects.
Fine-tuning Pre-trained Language Models with Noise Stability Regularization
The advent of large-scale pre-trained language models has contributed greatly to the recent progress in natural language processing.
A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications
Finally, we discuss the challenges and unsolved problems in this area and to facilitate sports analytics, we develop a toolbox using PaddlePaddle, which supports football, basketball, table tennis and figure skating action recognition.
Deep Active Learning with Noise Stability
Uncertainty estimation for unlabeled data is crucial to active learning.
Feature Forgetting in Continual Representation Learning
To study the feature forgetting problem, we create a synthetic dataset to identify and visualize the prevalence of feature forgetting in neural networks.
A Simple yet Effective Framework for Active Learning to Rank
To handle the diverse query requests from users at web-scale, Baidu has done tremendous efforts in understanding users' queries, retrieve relevant contents from a pool of trillions of webpages, and rank the most relevant webpages on the top of results.
FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server
Despite achieving remarkable performance, Federated Learning (FL) suffers from two critical challenges, i. e., limited computational resources and low training efficiency.
Structure-aware Protein Self-supervised Learning
Furthermore, we propose to leverage the available protein language model pretrained on protein sequences to enhance the self-supervised learning.
A Comparative Survey of Deep Active Learning
In this work, We construct a DAL toolkit, DeepAL+, by re-implementing 19 highly-cited DAL methods.
Inadequately Pre-trained Models are Better Feature Extractors
Based on the contradictory phenomenon between FE and FT that better feature extractor fails to be fine-tuned better accordingly, we conduct comprehensive analyses on features before softmax layer to provide insightful explanations.
Learning Moving-Object Tracking with FMCW LiDAR
In this paper, we propose a learning-based moving-object tracking method utilizing our newly developed LiDAR sensor, Frequency Modulated Continuous Wave (FMCW) LiDAR.
Predicting Patient Readmission Risk from Medical Text via Knowledge Graph Enhanced Multiview Graph Convolution
Unplanned intensive care unit (ICU) readmission rate is an important metric for evaluating the quality of hospital care.
Efficient Device Scheduling with Multi-Job Federated Learning
However, the scheduling of devices for multiple jobs with FL remains a critical and open problem.
Generalized DataWeighting via Class-Level Gradient Manipulation
To be specific, GDW unrolls the loss gradient to class-level gradients by the chain rule and reweights the flow of each gradient separately.
Validating the Lottery Ticket Hypothesis with Inertial Manifold Theory
Despite achieving remarkable efficiency, traditional network pruning techniques often follow manually-crafted heuristics to generate pruned sparse networks.
HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments
The training process generally exploits distributed computing resources to reduce training time.
Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification
Short text classification is a fundamental task in natural language processing.
Generalized Data Weighting via Class-level Gradient Manipulation
To be specific, GDW unrolls the loss gradient to class-level gradients by the chain rule and reweights the flow of each gradient separately.
SenseMag: Enabling Low-Cost Traffic Monitoring using Non-invasive Magnetic Sensing
Extensive experiment based on real-word field deployment (on the highways in Shenzhen, China) shows that SenseMag significantly outperforms the existing methods in both classification accuracy and the granularity of vehicle types (i. e., 7 types by SenseMag versus 4 types by the existing work in comparisons).
AgFlow: Fast Model Selection of Penalized PCA via Implicit Regularization Effects of Gradient Flow
Principal component analysis (PCA) has been widely used as an effective technique for feature extraction and dimension reduction.
Exploring the Common Principal Subspace of Deep Features in Neural Networks
Specifically, we design a new metric $\mathcal{P}$-vector to represent the principal subspace of deep features learned in a DNN, and propose to measure angles between the principal subspaces using $\mathcal{P}$-vectors.
GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction
Though graph contrastive learning (GCL) methods have achieved extraordinary performance with insufficient labeled data, most focused on designing data augmentation schemes for general graphs.
Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study
Existing interpretation algorithms have found that, even deep models make the same and right predictions on the same image, they might rely on different sets of input features for classification.
Semi-Supervised Active Learning with Temporal Output Discrepancy
To lower the cost of data annotation, active learning has been proposed to interactively query an oracle to annotate a small proportion of informative samples in an unlabeled dataset.
Structure-aware Interactive Graph Neural Networks for the Prediction of Protein-Ligand Binding Affinity
To this end, we propose a structure-aware interactive graph neural network (SIGN) which consists of two components: polar-inspired graph attention layers (PGAL) and pairwise interactive pooling (PiPool).
Property-Aware Relation Networks for Few-Shot Molecular Property Prediction
the target property, such that the limited labels can be effectively propagated among similar molecules.
MugRep: A Multi-Task Hierarchical Graph Representation Learning Framework for Real Estate Appraisal
Real estate appraisal refers to the process of developing an unbiased opinion for real property's market value, which plays a vital role in decision-making for various players in the marketplace (e. g., real estate agents, appraisers, lenders, and buyers).
Noise Stability Regularization for Improving BERT Fine-tuning
The brittleness of this process is often reflected by the sensitivity to random seeds.
From Personalized Medicine to Population Health: A Survey of mHealth Sensing Techniques
Mobile Sensing Apps have been widely used as a practical approach to collect behavioral and health-related information from individuals and provide timely intervention to promote health and well-beings, such as mental health and chronic cares.
Robust Matrix Factorization with Grouping Effect
Compared with existing algorithms, the proposed GRMF can automatically learn the grouping structure and sparsity in MF without prior knowledge, by introducing a naturally adjustable non-convex regularization to achieve simultaneous sparsity and grouping effect.
Practical Assessment of Generalization Performance Robustness for Deep Networks via Contrastive Examples
Incorporating with a set of randomized strategies for well-designed data transformations over the training set, ContRE adopts classification errors and Fisher ratios on the generated contrastive examples to assess and analyze the generalization performance of deep models in complement with a testing set.
JIZHI: A Fast and Cost-Effective Model-As-A-Service System for Web-Scale Online Inference at Baidu
In this work, we present JIZHI - a Model-as-a-Service system - that per second handles hundreds of millions of online inference requests to huge deep models with more than trillions of sparse parameters, for over twenty real-time recommendation services at Baidu, Inc.
From Distributed Machine Learning to Federated Learning: A Survey
Because of laws or regulations, the distributed data and computing resources cannot be directly shared among different regions or organizations for machine learning tasks.
Cross-lingual Entity Alignment with Adversarial Kernel Embedding and Adversarial Knowledge Translation
With the guidance of known aligned entities in the context of multiple random walks, an adversarial knowledge translation model is developed to fill and translate masked entities in pairwise random walks from two KGs.
Semantic Oppositeness Assisted Deep Contextual Modeling for Automatic Rumor Detection in Social Networks
Social networks face a major challenge in the form of rumors and fake news, due to their intrinsic nature of connecting users to millions of others, and of giving any individual the power to post anything.
ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
The forward inference projects input images into deep features, while the backward inference remaps deep features back to input images in a lossless and unbiased way.
SMILE: Self-Distilled MIxup for Efficient Transfer LEarning
Performing mixup for transfer learning with pre-trained models however is not that simple, a high capacity pre-trained model with a large fully-connected (FC) layer could easily overfit to the target dataset even with samples-to-labels mixed up.
Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond
Then, to understand the interpretation results, we also survey the performance metrics for evaluating interpretation algorithms.
Adaptive Consistency Regularization for Semi-Supervised Transfer Learning
To better exploit the value of both pre-trained weights and unlabeled target examples, we introduce adaptive consistency regularization that consists of two complementary components: Adaptive Knowledge Consistency (AKC) on the examples between the source and target model, and Adaptive Representation Consistency (ARC) on the target model between labeled and unlabeled examples.
Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning
Electric Vehicle (EV) has become a preferable choice in the modern transportation system due to its environmental and energy sustainability.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Out-of-Town Recommendation with Travel Intention Modeling
Second, a user-specific travel intention is formulated as an aggregation combining home-town preference and generic travel intention together, where the generic travel intention is regarded as a mixture of inherent intentions that can be learned by Neural Topic Model (NTM).
Can We Use Gradient Norm as a Measure of Generalization Error for Model Selection in Practice?
The recent theoretical investigation (Li et al., 2020) on the upper bound of generalization error of deep neural networks (DNNs) demonstrates the potential of using the gradient norm as a measure that complements validation accuracy for model selection in practice.
Democratizing Evaluation of Deep Model Interpretability through Consensus
Given any task/dataset, Consensus first obtains the interpretation results using existing tools, e. g., LIME (Ribeiro et al., 2016), for every model in the committee, then aggregates the results from the entire committee and approximates the “ground truth” of interpretations through voting.
Model information as an analysis tool in deep learning
We demonstrate using model information as a general analysis tool to gain insight into problems that arise in deep learning.
Information distance for neural network functions
We provide a practical distance measure in the space of functions parameterized by neural networks.
Implicit Regularization Effects of Unbiased Random Label Noises with SGD
Random label noises (or observational noises) widely exist in practical machinelearning settings.
Empirical Studies on the Convergence of Feature Spaces in Deep Learning
While deep learning is effective to learn features/representations from data, the distributions of samples in feature spaces learned by various architectures for different training tasks (e. g., latent layers of AEs and feature vectors in CNN classifiers) have not been well-studied or compared.
Joint Air Quality and Weather Prediction Based on Multi-Adversarial Spatiotemporal Networks
Specifically, we first propose a heterogeneous recurrent graph neural network to model the spatiotemporal autocorrelation among air quality and weather monitoring stations.
C-Watcher: A Framework for Early Detection of High-Risk Neighborhoods Ahead of COVID-19 Outbreak
Furthermore, to transfer the firsthand knowledge (witted in epicenters) to the target city before local outbreaks, we adopt a novel adversarial encoder framework to learn "city-invariant" representations from the mobility-related features for precise early detection of high-risk neighborhoods, even before any confirmed cases known, in the target city.
Distance-aware Molecule Graph Attention Network for Drug-Target Binding Affinity Prediction
The hierarchical attentive aggregation can capture spatial dependencies among atoms, as well as fuse the position-enhanced information with the capability of discriminating multiple spatial relations among atoms.
Temporal Relational Modeling with Self-Supervision for Action Segmentation
The main reason is that large number of nodes (i. e., video frames) makes GCNs hard to capture and model temporal relations in videos.
Ranked #17 on
Action Segmentation
on Breakfast
Adversarial Attacks on Deep Graph Matching
Despite achieving remarkable performance, deep graph learning models, such as node classification and network embedding, suffer from harassment caused by small adversarial perturbations.
Exploiting Node Content for Multiview Graph Convolutional Network and Adversarial Regularization
Network representation learning (NRL) is crucial in the area of graph learning.
Introducing Syntactic Structures into Target Opinion Word Extraction with Deep Learning
In this work, we propose to incorporate the syntactic structures of the sentences into the deep learning models for TOWE, leveraging the syntax-based opinion possibility scores and the syntactic connections between the words.
Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional Networks and Syntax-based Regulation
In addition, we propose a mechanism to obtain the importance scores for each word in the sentences based on the dependency trees that are then injected into the model to improve the representation vectors for ABSA.
Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement
Fine-tuning deep neural networks pre-trained on large scale datasets is one of the most practical transfer learning paradigm given limited quantity of training samples.
Discriminative Sounding Objects Localization via Self-supervised Audiovisual Matching
First, we propose to learn robust object representations by aggregating the candidate sound localization results in the single source scenes.
Measuring Information Transfer in Neural Networks
We propose a practical measure of the generalizable information in a neural network model based on prequential coding, which we term Information Transfer ($L_{IT}$).
XMixup: Efficient Transfer Learning with Auxiliary Samples by Cross-domain Mixup
While the existing multitask learning algorithms need to run backpropagation over both the source and target datasets and usually consume a higher gradient complexity, XMixup transfers the knowledge from source to target tasks more efficiently: for every class of the target task, XMixup selects the auxiliary samples from the source dataset and augments training samples via the simple mixup strategy.
Generating Person Images with Appearance-aware Pose Stylizer
Generation of high-quality person images is challenging, due to the sophisticated entanglements among image factors, e. g., appearance, pose, foreground, background, local details, global structures, etc.
Representation Transfer by Optimal Transport
We use optimal transport to quantify the match between two representations, yielding a distance that embeds some invariances inherent to the representation of deep networks.
RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayEr
RIFLE brings meaningful updates to the weights of deep CNN layers and improves low-level feature learning, while the effects of randomization can be easily converged throughout the overall learning procedure.
Exploiting the Syntax-Model Consistency for Neural Relation Extraction
In order to overcome these issues, we propose a novel deep learning model for RE that uses the dependency trees to extract the syntax-based importance scores for the words, serving as a tree representation to introduce syntactic information into the models with greater generalization.
Cross-Task Transfer for Geotagged Audiovisual Aerial Scene Recognition
With the help of this dataset, we evaluate three proposed approaches for transferring the sound event knowledge to the aerial scene recognition task in a multimodal learning framework, and show the benefit of exploiting the audio information for the aerial scene recognition.
Ambient Sound Helps: Audiovisual Crowd Counting in Extreme Conditions
Visual crowd counting has been recently studied as a way to enable people counting in crowd scenes from images.
Quantifying the Economic Impact of COVID-19 in Mainland China Using Human Mobility Data
While these strategies have effectively dealt with the critical situations of outbreaks, the combination of the pandemic and mobility controls has slowed China's economic growth, resulting in the first quarterly decline of Gross Domestic Product (GDP) since GDP began to be calculated, in 1992.
Pay Attention to Features, Transfer Learn Faster CNNs
Deep convolutional neural networks are now widely deployed in vision applications, but a limited size of training data can restrict their task performance.
COLAM: Co-Learning of Deep Neural Networks and Soft Labels via Alternating Minimization
Softening labels of training datasets with respect to data representations has been frequently used to improve the training of deep neural networks (DNNs).
Ontology-based Interpretable Machine Learning for Textual Data
In this paper, we introduce a novel interpreting framework that learns an interpretable model based on an ontology-based sampling technique to explain agnostic prediction models.
Parameter-Free Style Projection for Arbitrary Style Transfer
This paper further presents a real-time feed-forward model to leverage Style Projection for arbitrary image style transfer, which includes a regularization term for matching the semantics between input contents and stylized outputs.
Label-guided Learning for Text Classification
In this study, we propose a label-guided learning framework LguidedLearn for text representation and classification.
Curriculum Audiovisual Learning
Associating sound and its producer in complex audiovisual scene is a challenging task, especially when we are lack of annotated training data.
Searching for Stage-wise Neural Graphs In the Limit
Search space is a key consideration for neural architecture search.
An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
We demonstrate that training the networks to have interpretable gradients improves their robustness to adversarial perturbations.
A Joint Model for Definition Extraction with Syntactic Connection and Semantic Consistency
In this work, we propose a novel model for DE that simultaneously performs the two tasks in a single framework to benefit from their inter-dependencies.
Differential Privacy in Adversarial Learning with Provable Robustness
In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in adversarial learning for deep neural networks, with provable robustness to adversarial examples.
Language-independent Cross-lingual Contextual Representations
Contextual representation models like BERT have achieved state-of-the-art performance on a diverse range of NLP tasks.
NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks
The problem of explaining deep learning models, and model predictions generally, has attracted intensive interest recently.
Improving Adversarial Robustness via Attention and Adversarial Logit Pairing
Finally, we conduct extensive experiments using a wide range of datasets and the experiment results show that our AT+ALP achieves the state of the art defense performance.
Learning Conceptual-Contextual Embeddings for Medical Text
External knowledge is often useful for natural language understanding tasks.
Improving Cross-Domain Performance for Relation Extraction via Dependency Prediction and Information Flow Control
The current deep learning models for relation extraction has mainly exploited this dependency information by guiding their computation along the structures of the dependency trees.
Graph based Neural Networks for Event Factuality Prediction using Syntactic and Semantic Structures
In this work, we introduce a novel graph-based neural network for EFP that can integrate the semantic and syntactic information more effectively.
Heterogeneous Gaussian Mechanism: Preserving Differential Privacy in Deep Learning with Provable Robustness
In this paper, we propose a novel Heterogeneous Gaussian Mechanism (HGM) to preserve differential privacy in deep neural networks, with provable robustness against adversarial examples.
Preserving Differential Privacy in Adversarial Learning with Provable Robustness
In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in adversarial learning for deep neural networks, with provable robustness to adversarial examples.
Cryptography and Security
Logic Rules Powered Knowledge Graph Embedding
And then, to put both triples and mined logic rules within the same semantic space, all triples in the knowledge graph are represented as first-order logic.
Delta Embedding Learning
Evaluation also confirms the tuned word embeddings have better semantic properties.
On Adversarial Examples for Character-Level Neural Machine Translation
Evaluating on adversarial examples has become a standard procedure to measure robustness of deep learning models.
HotFlip: White-Box Adversarial Examples for Text Classification
We propose an efficient method to generate white-box adversarial examples to trick a character-level neural classifier.
Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning
In this paper, we focus on developing a novel mechanism to preserve differential privacy in deep neural networks, such that: (1) The privacy budget consumption is totally independent of the number of training steps; (2) It has the ability to adaptively inject noise into features based on the contribution of each to the output; and (3) It could be applied in a variety of different deep neural networks.
Preserving Differential Privacy in Convolutional Deep Belief Networks
However, only a few scientific studies on preserving privacy in deep learning have been conducted.
A Joint Sentiment-Target-Stance Model for Stance Classification in Tweets
Classifying the stance expressed in online microblogging social media is an emerging problem in opinion mining.
A Probabilistic Approach to Knowledge Translation
In this paper, we focus on a novel knowledge reuse scenario where the knowledge in the source schema needs to be translated to a semantically heterogeneous target schema.
Ontology Matching with Knowledge Rules
We use a probabilistic framework to integrate this new knowledge-based strategy with standard terminology-based and structure-based strategies.
