Search Results for author:
Found 98 papers, 55 papers with code
Rethinking Knowledge Transfer in Learning Using Privileged Information
Research on learning using privileged information (LUPI) aims to transfer the knowledge captured in PI onto a model that can perform inference without PI.
A Probabilistic Framework for Adapting to Changing and Recurring Concepts in Data Streams
Existing streaming approaches either do not consider experience to change in relevance over time and thus cannot handle concept drift, or only consider the recency of experience and thus cannot handle recurring concepts, or only sparsely evaluate relevance and thus fail when concept drift is missed.
Robust Active Learning (RoAL): Countering Dynamic Adversaries in Active Learning with Elastic Weight Consolidation
Despite significant advancements in active learning and adversarial attacks, the intersection of these two fields remains underexplored, particularly in developing robust active learning frameworks against dynamic adversarial threats.
Unveiling the Power of Sparse Neural Networks for Feature Selection
Sparse Neural Networks (SNNs) have emerged as powerful tools for efficient feature selection.
(PASS) Visual Prompt Locates Good Structure Sparsity through a Recurrent HyperNetwork
In this paper, we investigate a charming possibility - \textit{leveraging visual prompts to capture the channel importance and derive high-quality structural sparsity}.
Nerva: a Truly Sparse Implementation of Neural Networks
We introduce Nerva, a fast neural network library under development in C++.
Dynamic Data Pruning for Automatic Speech Recognition
The recent success of Automatic Speech Recognition (ASR) is largely attributed to the ever-growing amount of training data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Boosting Robustness in Preference-Based Reinforcement Learning with Dynamic Sparsity
It becomes essential that agents learn to focus on the subset of task-relevant environment features.
One-Shot Federated Learning with Bayesian Pseudocoresets
Optimization-based techniques for federated learning (FL) often come with prohibitive communication cost, as high dimensional model parameters need to be communicated repeatedly between server and clients.
Efficient Exploration in Average-Reward Constrained Reinforcement Learning: Achieving Near-Optimal Regret With Posterior Sampling
We present a new algorithm based on posterior sampling for learning in Constrained Markov Decision Processes (CMDP) in the infinite-horizon undiscounted setting.
The Neutrality Fallacy: When Algorithmic Fairness Interventions are (Not) Positive Action
While individuals and organizations have an obligation to avoid discrimination, the use of fairness-aware machine learning interventions has also been described as amounting to 'algorithmic positive action' under European Union (EU) non-discrimination law.
Learning Efficient and Fair Policies for Uncertainty-Aware Collaborative Human-Robot Order Picking
We develop a discrete-event simulation model, which we use to train and evaluate the proposed DRL approach.
Investigating Gender Fairness in Machine Learning-driven Personalized Care for Chronic Pain
In this article, we study gender fairness in personalized pain care recommendations using a real-world application of reinforcement learning (Piette et al., 2022a).
Large Language Models Are Neurosymbolic Reasoners
A wide range of real-world applications is characterized by their symbolic nature, necessitating a strong capability for symbolic reasoning.
MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learning
Our algorithm improves the agent's focus with useful masks, while its efficient Masker network only adds 0. 2% more parameters to the original structure, in contrast to previous work.
GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models
In this work, we propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs (e. g., GPT-4 \cite{openai2023gpt4}) to assess bias in models.
A Structural-Clustering Based Active Learning for Graph Neural Networks
To address this problem, we propose the Structural-Clustering PageRank method for improved Active learning (SPA) specifically designed for graph-structured data.
E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation
E2ENet achieves comparable accuracy on the large-scale challenge AMOS-CT, while saving over 68\% parameter count and 29\% FLOPs in the inference phase, compared with the previous best-performing method.
REST: Enhancing Group Robustness in DNNs through Reweighted Sparse Training
These bias attributes are strongly spuriously correlated with the target variable, causing the models to be biased towards spurious correlations (i. e., \textit{bias-conflicting}).
Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective
The rapid development of large-scale deep learning models questions the affordability of hardware platforms, which necessitates the pruning to reduce their computational and memory footprints.
KeyGen2Vec: Learning Document Embedding via Multi-label Keyword Generation in Question-Answering
Interestingly, although in general the absolute advantage of learning embeddings through label supervision is highly positive across evaluation datasets, KeyGen2Vec is shown to be competitive with classifier that exploits topic label supervision in Yahoo!
Heterophily-Based Graph Neural Network for Imbalanced Classification
This leads to suboptimal performance of standard GNNs on imbalanced graphs.
Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
Large Language Models (LLMs), renowned for their remarkable performance across diverse domains, present a challenge when it comes to practical deployment due to their colossal model size.
Provably Efficient Exploration in Constrained Reinforcement Learning:Posterior Sampling Is All You Need
We present a new algorithm based on posterior sampling for learning in constrained Markov decision processes (CMDP) in the infinite-horizon undiscounted setting.
Enhancing Adversarial Training via Reweighting Optimization Trajectory
Despite the fact that adversarial training has become the de facto method for improving the robustness of deep neural networks, it is well-known that vanilla adversarial training suffers from daunting robust overfitting, resulting in unsatisfactory robust generalization.
Are Large Kernels Better Teachers than Transformers for ConvNets?
We hereby carry out a first-of-its-kind study unveiling that modern large-kernel ConvNets, a compelling competitor to Vision Transformers, are remarkably more effective teachers for small-kernel ConvNets, due to more similar architectures.
Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers
Efficient time series forecasting has become critical for real-world applications, particularly with deep neural networks (DNNs).
Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approach
While the majority of current approaches construct the reward redistribution in an uninterpretable manner, we propose to explicitly model the contributions of state and action from a causal perspective, resulting in an interpretable reward redistribution and preserving policy invariance.
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models
We evaluate two popular pretrained Chinese conversational models, CDial-GPT and EVA2. 0, using CHBias.
NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist
Our proposed framework provides access: (i) for verifying whether automatic metrics are faithful to human preference, regardless of their correlation level to human; and (ii) for inspecting the strengths and limitations of NLG systems via pairwise evaluation.
Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree
In this paper, we aim to illustrate to what extent European Union (EU) non-discrimination law coincides with notions of algorithmic fairness proposed in computer science literature and where they differ.
Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
The field of automated machine learning (AutoML) introduces techniques that automate parts of the development of machine learning (ML) systems, accelerating the process and reducing barriers for novices.
Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks
Feature selection that selects an informative subset of variables from data not only enhances the model interpretability and performance but also alleviates the resource demands.
Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning
Tomorrow's robots will need to distinguish useful information from noise when performing different tasks.
Dynamic Sparse Network for Time Series Classification: Learning What to "see''
The receptive field (RF), which determines the region of time series to be ``seen'' and used, is critical to improve the performance for time series classification (TSC).
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
Recent works have impressively demonstrated that there exists a subnetwork in randomly initialized convolutional neural networks (CNNs) that can match the performance of the fully trained dense networks at initialization, without any optimization of the weights of the network (i. e., untrained networks).
Where to Pay Attention in Sparse Training for Feature Selection?
Our proposed approach outperforms the state-of-the-art methods in terms of selecting informative features while reducing training iterations and computational costs substantially.
FAL-CUR: Fair Active Learning using Uncertainty and Representativeness on Fair Clustering
Active Learning (AL) techniques have proven to be highly effective in reducing data labeling costs across a range of machine learning tasks.
An Empirical Evaluation of Posterior Sampling for Constrained Reinforcement Learning
We study a posterior sampling approach to efficient exploration in constrained reinforcement learning.
Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost
We call our method Lottery Pools.
Memory-free Online Change-point Detection: A Novel Neural Network Approach
We show that ALACPD, on average, ranks first among state-of-the-art CPD algorithms in terms of quality of the time series segmentation, and it is on par with the best performer in terms of the accuracy of the estimated change-points.
More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity
Transformers have quickly shined in the computer vision world since the emergence of Vision Transformers (ViTs).
Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training
Recent works on sparse neural network training (sparse training) have shown that a compelling trade-off between performance and efficiency can be achieved by training intrinsically sparse neural networks from scratch.
Phrase-level Textual Adversarial Attack with Label Preservation
Generating high-quality textual adversarial examples is critical for investigating the pitfalls of natural language processing (NLP) models and further promoting their robustness.
Survey on Fair Reinforcement Learning: Theory and Practice
In this article, we provide an extensive overview of fairness approaches that have been implemented via a reinforcement learning (RL) framework.
Are There Exceptions to Goodhart's Law? On the Moral Justification of Fairness-Aware Machine Learning
We use the framework to distil moral and empirical assumptions under which particular fairness metrics correspond to a fair distribution of outcomes.
The Impact of Batch Learning in Stochastic Linear Bandits
Our main theoretical results show that the impact of batch learning is a multiplicative factor of batch size relative to the regret of online behavior.
The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training
In this paper, we focus on sparse training and highlight a perhaps counter-intuitive finding, that random pruning at initialization can be quite powerful for the sparse training of modern neural networks.
Semantic-Based Few-Shot Learning by Interactive Psychometric Testing
In this work, we advance the few-shot learning towards this more challenging scenario, the semantic-based few-shot learning, and propose a method to address the paradigm by capturing the inner semantic relationships using interactive psychometric learning.
The Impact of Batch Learning in Stochastic Bandits
We consider a special case of bandit problems, namely batched bandits.
Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks
To address this challenge, we propose a new CL method, named AFAF, that aims to Avoid Forgetting and Allow Forward transfer in class-IL using fix-capacity models.
Calibrated Adversarial Training
In this paper, we present the Calibrated Adversarial Training, a method that reduces the adverse effects of semantic perturbations in adversarial training.
Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles
Specifically, we propose to leverage causal knowledge by regarding the distributional shifts in subpopulations and deployment environments as the results of interventions on the underlying system.
ProtoInfoMax: Prototypical Networks with Mutual Information Maximization for Out-of-Domain Detection
The ability to detect Out-of-Domain (OOD) inputs has been a critical requirement in many real-world NLP applications.
Unbiased Cascade Bandits: Mitigating Exposure Bias in Online Learning to Rank Recommendation
This is especially problematic when bias is amplified over time as a few popular items are repeatedly over-represented in recommendation lists.
A Graph-based Approach for Mitigating Multi-sided Exposure Bias in Recommender Systems
Fairness is a critical system-level objective in recommender systems that has been the subject of extensive recent research.
Hierarchical Semantic Segmentation using Psychometric Learning
One of the major challenges in the supervised learning approaches is expressing and collecting the rich knowledge that experts have with respect to the meaning present in the image data.
On Generalization of Graph Autoencoders with Adversarial Training
Adversarial training is an approach for increasing model's resilience against adversarial perturbations.
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
Our framework, FreeTickets, is defined as the ensemble of these relatively cheap sparse subnetworks.
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised a lot of attention currently on post-training pruning (iterative magnitude pruning), and before-training pruning (pruning at initialization).
Ranked #3 on
Sparse Learning
on ImageNet
Dynamic Sparse Training for Deep Reinforcement Learning
In this paper, we introduce for the first time a dynamic sparse training approach for deep reinforcement learning to accelerate the training process.
Direction-Aggregated Attack for Transferable Adversarial Examples
Deep neural networks are vulnerable to adversarial examples that are crafted by imposing imperceptible changes to the inputs.
Hop-Count Based Self-Supervised Anomaly Detection on Attributed Networks
Although various approaches have been proposed to solve this problem, two major limitations exist: (1) unsupervised approaches usually work much less efficiently due to the lack of supervisory signal, and (2) existing anomaly detection methods only use local contextual information to detect anomalous nodes, e. g., one- or two-hop information, but ignore the global contextual information.
Self-Supervised Anomaly Detection
Supervised Anomaly Detection
Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training
By starting from a random sparse network and continuously exploring sparse connectivities during training, we can perform an Over-Parameterization in the space-time manifold, closing the gap in the expressibility between sparse training and dense training.
Ranked #4 on
Sparse Learning
on ImageNet
Self-Attention Meta-Learner for Continual Learning
In this paper, we propose a new method, named Self-Attention Meta-Learner (SAM), which learns a prior knowledge for continual learning that permits learning a sequence of tasks, while avoiding catastrophic forgetting.
Selfish Sparse RNN Training
Sparse neural networks have been widely applied to reduce the computational demands of training and deploying over-parameterized deep neural networks.
Learning Invariant Representation for Continual Learning
Finally, we analyze the role of the shared invariant representation in mitigating the forgetting problem especially when the number of replayed samples for each previous task is small.
Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders
This method, named QuickSelection, introduces the strength of the neuron in sparse neural networks as a criterion to measure the feature importance.
Ranked #2 on
Dimensionality Reduction
on EMNIST
ViDi: Descriptive Visual Data Clustering as Radiologist Assistant in COVID-19 Streamline Diagnostic
In the light of the COVID-19 pandemic, deep learning methods have been widely investigated in detecting COVID-19 from chest X-rays.
Bridging the Performance Gap between FGSM and PGD Adversarial Training
In addition, it achieves comparable performance of adversarial robustness on MNIST dataset under white-box attack, and it achieves better performance than adv. PGD under white-box attack and effectively defends the transferable adversarial attack on CIFAR-10 dataset.
ResGCN: Attention-based Deep Residual Modeling for Anomaly Detection on Attributed Networks
Effectively detecting anomalous nodes in attributed networks is crucial for the success of many real-world applications such as fraud and intrusion detection.
Feedback Loop and Bias Amplification in Recommender Systems
Recommendation algorithms are known to suffer from popularity bias; a few popular items are recommended frequently while the majority of other items are ignored.
SpaceNet: Make Free Space For Continual Learning
Regularization-based methods maintain a fixed model capacity; however, previous studies showed the huge performance degradation of these methods when the task identity is not available during inference (e. g. class incremental learning scenario).
Topological Insights into Sparse Neural Networks
However, comparing different sparse topologies and determining how sparse topologies evolve during training, especially for the situation in which the sparse structure optimization is involved, remain as challenging open questions.
Knowledge Elicitation using Deep Metric Learning and Psychometric Testing
The main reason for such a reductionist approach is the difficulty in eliciting the domain knowledge from the experts.
Investigating Potential Factors Associated with Gender Discrimination in Collaborative Recommender Systems
The proliferation of personalized recommendation technologies has raised concerns about discrepancies in their recommendation performance across different genders, age groups, and racial or ethnic populations.
Novelty Producing Synaptic Plasticity
A learning process with the plasticity property often requires reinforcement signals to guide the process.
Causal Discovery from Incomplete Data: A Deep Learning Approach
As systems are getting more autonomous with the development of artificial intelligence, it is important to discover the causal knowledge from observational sensory inputs.
The Relationship between the Consistency of Users' Ratings and Recommendation Calibration
In this paper, we aim to explore the relationship between the consistency of users' ratings behavior and the degree of calibrated recommendations they receive.
BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation
This study mainly investigates two decoding problems in neural keyphrase generation: sequence length bias and beam diversity.
Bias Disparity in Collaborative Recommendation: Algorithmic Evaluation and Comparison
Research on fairness in machine learning has been recently extended to recommender systems.
Case-Based Reasoning for Assisting Domain Experts in Processing Fraud Alerts of Black-Box Machine Learning Models
In many contexts, it can be useful for domain experts to understand to what extent predictions made by a machine learning model can be trusted.
A Human-Grounded Evaluation of SHAP for Alert Processing
In this paper we present the results of a human-grounded evaluation of SHAP, an explanation method that has been well-received in the XAI and related communities.
On improving deep learning generalization with adaptive sparse connectivity
Large neural networks are very successful in various tasks.
Adversarial Balancing-based Representation Learning for Causal Effect Inference with Observational Data
The challenges for this problem are two-fold: on the one hand, we have to derive a causal estimator to estimate the causal quantity from observational data, where there exists confounding bias; on the other hand, we have to deal with the identification of CATE when the distribution of covariates in treatment and control groups are imbalanced.
Evolving Plasticity for Autonomous Learning under Changing Environmental Conditions
Hebbian learning is a biologically plausible mechanism for modeling the plasticity property in artificial neural networks (ANNs), based on the local interactions of neurons.
Learning with Delayed Synaptic Plasticity
Inspired by biology, plasticity can be modeled in artificial neural networks by using Hebbian learning rules, i. e. rules that update synapses based on the neuron activations and reinforcement signals.
A Brain-inspired Algorithm for Training Highly Sparse Neural Networks
Concretely, by exploiting the cosine similarity metric to measure the importance of the connections, our proposed method, Cosine similarity-based and Random Topology Exploration (CTRE), evolves the topology of sparse neural networks by adding the most important connections to the network without calculating dense gradient in the backward.
Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware
Despite the success of ANNs, it is challenging to train and deploy modern ANNs on commodity hardware due to the ever-increasing model size and the unprecedented growth in the data volumes.
Intrinsically Sparse Long Short-Term Memory Networks
However, LSTMs are prone to be memory-bandwidth limited in realistic applications and need an unbearable period of training and inference time as the model size is ever-increasing.
Looking Deeper into Deep Learning Model: Attribution-based Explanations of TextCNN
Layer-wise Relevance Propagation (LRP) and saliency maps have been recently used to explain the predictions of Deep Learning models, specifically in the domain of text classification.
Controversy Rules - Discovering Regions Where Classifiers (Dis-)Agree Exceptionally
Finding regions for which there is higher controversy among different classifiers is insightful with regards to the domain and our models.
struc2gauss: Structural Role Preserving Network Embedding via Gaussian Embedding
Almost all previous methods represent a node into a point in space and focus on local structural information, i. e., neighborhood information.
Limited Evaluation Cooperative Co-evolutionary Differential Evolution for Large-scale Neuroevolution
Many real-world control and classification tasks involve a large number of features.
On Formalizing Fairness in Prediction with Machine Learning
Machine learning algorithms for prediction are increasingly being used in critical decisions affecting human lives.
Rule-based Emotion Detection on Social Media: Putting Tweets on Plutchik's Wheel
We study sentiment analysis beyond the typical granularity of polarity and instead use Plutchik's wheel of emotions model.
Predictive User Modeling with Actionable Attributes
We study how to learn to choose the value of an actionable attribute in order to maximize the probability of a desired outcome in predictive modeling.
