Search Results for author:
Found 58 papers, 21 papers with code
Time-Consistent Self-Supervision for Semi-Supervised Learning
In this paper, we study the dynamics of neural net outputs in SSL and show that selecting and using first the unlabeled samples with more consistent outputs over the course of training (i. e., "time-consistency") can improve the final test accuracy and save computation.
Aerial Diffusion: Text Guided Ground-to-Aerial View Translation from a Single Image using Diffusion Models
Aerial Diffusion leverages a pretrained text-image diffusion model for prior knowledge.
It Takes One to Tango but More Make Trouble? In-Context Training with Different Number of Demonstrations
Moreover, ICL (with and w/o CoT) using only one correct demo significantly outperforms all-demo ICL adopted by most previous works, indicating the weakness of LLMs in finding correct demo(s) for input queries, which is difficult to evaluate on the biased datasets.
Voting from Nearest Tasks: Meta-Vote Pruning of Pre-trained Models for Downstream Tasks
To address these challenges, we create a small model for a new task from the pruned models of similar tasks.
Federated Recommendation with Additive Personalization
Moreover, a curriculum learning mechanism has been applied for additive personalization on item embeddings by gradually increasing regularization weights to mitigate the performance degradation caused by large variances among client-specific item embeddings.
Dual Personalization on Federated Recommendation
Moreover, we provide visualizations and in-depth analysis of the personalization techniques in item embedding, which shed novel insights on the design of RecSys in federated settings.
Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach
In experiments, we show that our method consistently brings non-trivial improvements to the three aforementioned learning tasks from both efficiency and final performance, either or not combined with strong pre-defined augmentations, e. g., on medical images when domain knowledge is unavailable and the existing augmentation techniques perform poorly.
TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack
We present Twin Answer Sentences Attack (TASA), an adversarial attack method for question answering (QA) models that produces fluent and grammatical adversarial contexts while maintaining gold answers.
Federated Learning from Pre-Trained Models: A Contrastive Learning Approach
To prevent these issues from hindering the deployment of FL systems, we propose a lightweight framework where clients jointly learn to fuse the representations generated by multiple fixed pre-trained models rather than training a large-scale model from scratch.
Phrase-level Textual Adversarial Attack with Label Preservation
Generating high-quality textual adversarial examples is critical for investigating the pitfalls of natural language processing (NLP) models and further promoting their robustness.
FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels
Federated learning (FL) aims at training a global model on the server side while the training data are collected and located at the local devices.
Token Dropping for Efficient BERT Pretraining
Transformer-based models generally allocate the same amount of computation for each token in a given sequence.
Personalized Federated Learning With Graph
We propose a novel structured federated learning (SFL) framework to learn both the global and personalized models simultaneously using client-wise relation graphs and clients' private data.
On the Convergence of Clustered Federated Learning
Knowledge sharing and model personalization are essential components to tackle the non-IID challenge in federated learning (FL).
Learning To Collaborate in Decentralized Learning of Personalized Models
Decentralized learning (DL) can exploit the images distributed over devices on a network topology to train a global model but is not designed to train personalized models for different tasks or optimize the topology.
Constrained Robust Submodular Partitioning
Robust submodular partitioning promotes the diversity of every block in the partition.
CO-PILOT: COllaborative Planning and reInforcement Learning On sub-Task curriculum
Next, a bottom-up traversal of the tree trains the RL agent from easier sub-tasks with denser rewards on bottom layers to harder ones on top layers and collects its cost on each sub-task train the planner in the next episode.
Class-Disentanglement and Applications in Adversarial Detection and Defense
In this paper, we propose ''class-disentanglement'' that trains a variational autoencoder $G(\cdot)$ to extract this class-dependent information as $x - G(x)$ via a trade-off between reconstructing $x$ by $G(x)$ and classifying $x$ by $D(x-G(x))$, where the former competes with the latter in decomposing $x$ so the latter retains only necessary information for classification in $x-G(x)$.
Vote for Nearest Neighbors Meta-Pruning of Self-Supervised Networks
Can we find a better initialization for a new task, e. g., a much smaller network closer to the final pruned model, by exploiting its similar tasks?
Diverse Client Selection for Federated Learning via Submodular Maximization
In every communication round of federated learning, a random subset of clients communicate their model updates back to the server which then aggregates them all.
Identity-Disentangled Adversarial Augmentation for Self-supervised Learning
We then adversarially perturb $G(x)$ in the VAE's bottleneck space and adds it back to the original $R(x)$ as an augmentation, which is therefore sufficiently challenging for contrastive learning and meanwhile preserves the sample identity intact.
EAT-C: Environment-Adversarial sub-Task Curriculum for Efficient Reinforcement Learning
They are complementary in acquiring more informative feedback for RL: the planning policy provides dense reward of finishing easier sub-tasks while the environment policy modifies these sub-tasks to be adequately challenging and diverse so the RL agent can quickly adapt to different tasks/environments.
Pareto Policy Pool for Model-based Offline Reinforcement Learning
Model-based offline RL instead trains an environment model using a dataset of pre-collected experiences so online RL methods can learn in an offline manner by solely interacting with the model.
Eliminating Sentiment Bias for Aspect-Level Sentiment Classification with Unsupervised Opinion Extraction
Aspect-level sentiment classification (ALSC) aims at identifying the sentiment polarity of a specified aspect in a sentence.
Aspect-Based Sentiment Analysis (ABSA)
Representation Learning
+1
Multi-Center Federated Learning: Clients Clustering for Better Personalization
By comparison, a mixture of multiple global models could capture the heterogeneity across various clients if assigning the client to different global models (i. e., centers) in FL.
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly
This mutual-training process between BO and the loss-prediction model allows us to limit the training steps invested in the BO search.
FedProto: Federated Prototype Learning across Heterogeneous Clients
Heterogeneity across clients in federated learning (FL) usually hinders the optimization convergence and generalization performance when the aggregation of clients' knowledge occurs in the gradient space.
Isometric Propagation Network for Generalized Zero-shot Learning
To resolve this problem, we propose Isometric Propagation Network (IPN), which learns to strengthen the relation between classes within each space and align the class dependency in the two spaces.
Robust Curriculum Learning: from clean label detection to noisy label self-correction
Neural nets training can easily overfit to noisy labels and end with poor generalization performance.
Extract Local Inference Chains of Deep Neural Nets
In this paper, we introduce an efficient method, \name, to extract the local inference chains by optimizing a differentiable sparse scoring for the filters and layers to preserve the outputs on given data from a local region.
MASP: Model-Agnostic Sample Propagation for Few-shot learning
Few-shot learning aims to train a classifier given only a few samples per class that are highly insufficient to describe the whole data distribution.
Curriculum Learning by Dynamic Instance Hardness
Compared to existing CL methods: (1) DIH is more stable over time than using only instantaneous hardness, which is noisy due to stochastic training and DNN's non-smoothness; (2) DIHCL is computationally inexpensive since it uses only a byproduct of back-propagation and thus does not require extra inference.
Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention
Then, facilitated by the proposed base model, we introduce collaborating relation features shared among relations in the hierarchies to promote the relation-augmenting process and balance the training data for long-tail relations.
Attribute Propagation Network for Graph Zero-shot Learning
To address this challenging task, most ZSL methods relate unseen test classes to seen(training) classes via a pre-defined set of attributes that can describe all classes in the same semantic space, so the knowledge learned on the training classes can be adapted to unseen classes.
Many-Class Few-Shot Learning on Multi-Granularity Class Hierarchy
We study many-class few-shot (MCFS) problem in both supervised learning and meta-learning settings.
Multi-Center Federated Learning: Clients Clustering for Better Personalization
However, due to the diverse nature of user behaviors, assigning users' gradients to different global models (i. e., centers) can better capture the heterogeneity of data distributions across users.
Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion
In experiments, we achieve state-of-the-art performance on three benchmarks and a zero-shot dataset for link prediction, with highlights of inference costs reduced by 1-2 orders of magnitude compared to a textual encoding method.
Ranked #4 on
Link Prediction
on UMLS
Omni-Scale CNNs: a simple and effective kernel size configuration for time series classification
Particularly, it is a set of kernel sizes that can efficiently cover the best RF size across different datasets via consisting of multiple prime numbers according to the length of the time series.
Conditional Self-Attention for Query-based Summarization
Thereby, the contextual dependencies modeled by CSA will be highly relevant to the query.
Collaborative Inference for Efficient Remote Monitoring
While current machine learning models have impressive performance over a wide range of applications, their large size and complexity render them unsuitable for tasks such as remote monitoring on edge devices with limited storage and computational power.
Curriculum-guided Hindsight Experience Replay
This ``Goal-and-Curiosity-driven Curriculum Learning'' leads to ``Curriculum-guided HER (CHER)'', which adaptively and dynamically controls the exploration-exploitation trade-off during the learning process via hindsight experience selection.
Self-Attention Enhanced Selective Gate with Entity-Aware Embedding for Distantly Supervised Relation Extraction
Distantly supervised relation extraction intrinsically suffers from noisy labels due to the strong assumption of distant supervision.
Dynamic Instance Hardness
The advantages of DIHCL, compared to other curriculum learning approaches, are: (1) DIHCL does not require additional inference steps over the data not selected by DIHCL in each epoch, (2) the dynamic instance hardness, compared to static instance hardness (e. g., instantaneous loss), is more stable as it integrates information over the entire training history up to the present time.
Learning to Propagate for Graph Meta-Learning
It can significantly improve tasks that suffer from insufficient training data, e. g., few shot learning.
Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot Learning on Category Graph
The resulting graph of prototypes can be continually re-used and updated for new tasks and classes.
Bias Also Matters: Bias Attribution for Deep Neural Network Explanation
In particular, we study how to attribute a DNN's bias to its input features.
Jumpout: Improved Dropout for Deep Neural Networks with Rectified Linear Units
In this paper, we discuss three novel observations about dropout to better understand the generalization of DNNs with rectified linear unit (ReLU) activations: 1) dropout is a smoothing technique that encourages each local linear model of a DNN to be trained on data points from nearby regions; 2) a constant dropout rate can result in effective neural-deactivation rates that are significantly different for layers with different fractions of activated neurons; and 3) the rescaling factor of dropout causes an inconsistency to occur between the normalization during training and testing conditions when batch normalization is also used.
MahiNet: A Neural Network for Many-Class Few-Shot Learning with Class Hierarchy
It addresses the ``many-class'' problem by exploring the class hierarchy, e. g., the coarse-class label that covers a subset of fine classes, which helps to narrow down the candidates for the fine class and is cheaper to obtain.
Diverse Ensemble Evolution: Curriculum Data-Model Marriage
We study a new method (``Diverse Ensemble Evolution (DivE$^2$)'') to train an ensemble of machine learning models that assigns data to models at each training epoch based on each model's current expertise and an intra- and inter-model diversity reward.
Tensorized Self-Attention: Efficiently Modeling Pairwise and Global Dependencies Together
Neural networks equipped with self-attention have parallelizable computation, light-weight structure, and the ability to capture both long-range and local dependencies.
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
In this paper, we propose a model, called "bi-directional block self-attention network (Bi-BloSAN)", for RNN/CNN-free sequence encoding.
Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
In this paper, we integrate both soft and hard attention into one context fusion model, "reinforced self-attention (ReSA)", for the mutual benefit of each other.
Ranked #56 on
Natural Language Inference
on SNLI
Minimax Curriculum Learning: Machine Teaching with Desirable Difficulties and Scheduled Diversity
We introduce and study minimax curriculum learning (MCL), a new method for adaptively selecting a sequence of training subsets for a succession of stages in machine learning.
DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
Recurrent neural nets (RNN) and convolutional neural nets (CNN) are widely used on NLP tasks to capture the long-term and local dependencies, respectively.
Ranked #69 on
Natural Language Inference
on SNLI
Scaling Submodular Maximization via Pruned Submodularity Graphs
We propose a new random pruning method (called "submodular sparsification (SS)") to reduce the cost of submodular maximization.
Stream Clipper: Scalable Submodular Maximization on Stream
We propose a streaming submodular maximization algorithm "stream clipper" that performs as well as the offline greedy algorithm on document/video summarization in practice.
Divide-and-Conquer Learning by Anchoring a Conical Hull
We reduce a broad class of machine learning problems, usually addressed by EM or sampling, to the problem of finding the $k$ extremal rays spanning the conical hull of a data point set.
Unmixing Incoherent Structures of Big Data by Randomized or Greedy Decomposition
Learning big data by matrix decomposition always suffers from expensive computation, mixing of complicated structures and noise.
