Search Results for author: Mykola Pechenizkiy

Found 85 papers, 48 papers with code

NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist

7 code implementations15 May 2023 Iftitahu Ni'mah, Meng Fang, Vlado Menkovski, Mykola Pechenizkiy

Our proposed framework provides access: (i) for verifying whether automatic metrics are faithful to human preference, regardless of their correlation level to human; and (ii) for inspecting the strengths and limitations of NLG systems via pairwise evaluation.

Controllable Language Modelling Dialogue Generation +3

Are Large Kernels Better Teachers than Transformers for ConvNets?

1 code implementation30 May 2023 Tianjin Huang, Lu Yin, Zhenyu Zhang, Li Shen, Meng Fang, Mykola Pechenizkiy, Zhangyang Wang, Shiwei Liu

We hereby carry out a first-of-its-kind study unveiling that modern large-kernel ConvNets, a compelling competitor to Vision Transformers, are remarkably more effective teachers for small-kernel ConvNets, due to more similar architectures.

Knowledge Distillation

Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware

4 code implementations26 Jan 2019 Shiwei Liu, Decebal Constantin Mocanu, Amarsagar Reddy Ramapuram Matavalam, Yulong Pei, Mykola Pechenizkiy

Despite the success of ANNs, it is challenging to train and deploy modern ANNs on commodity hardware due to the ever-increasing model size and the unprecedented growth in the data volumes.

Topological Insights into Sparse Neural Networks

3 code implementations24 Jun 2020 Shiwei Liu, Tim Van der Lee, Anil Yaman, Zahra Atashgahi, Davide Ferraro, Ghada Sokar, Mykola Pechenizkiy, Decebal Constantin Mocanu

However, comparing different sparse topologies and determining how sparse topologies evolve during training, especially for the situation in which the sparse structure optimization is involved, remain as challenging open questions.

The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training

1 code implementation ICLR 2022 Shiwei Liu, Tianlong Chen, Xiaohan Chen, Li Shen, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy

In this paper, we focus on sparse training and highlight a perhaps counter-intuitive finding, that random pruning at initialization can be quite powerful for the sparse training of modern neural networks.

Adversarial Robustness Out-of-Distribution Detection

Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training

4 code implementations4 Feb 2021 Shiwei Liu, Lu Yin, Decebal Constantin Mocanu, Mykola Pechenizkiy

By starting from a random sparse network and continuously exploring sparse connectivities during training, we can perform an Over-Parameterization in the space-time manifold, closing the gap in the expressibility between sparse training and dense training.

Image Classification Sparse Learning

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity

1 code implementation8 Oct 2023 Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Mykola Pechenizkiy, Yi Liang, Zhangyang Wang, Shiwei Liu

Large Language Models (LLMs), renowned for their remarkable performance across diverse domains, present a challenge when it comes to practical deployment due to their colossal model size.

Network Pruning

Sparse Training via Boosting Pruning Plasticity with Neuroregeneration

2 code implementations NeurIPS 2021 Shiwei Liu, Tianlong Chen, Xiaohan Chen, Zahra Atashgahi, Lu Yin, Huanyu Kou, Li Shen, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin Mocanu

Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised a lot of attention currently on post-training pruning (iterative magnitude pruning), and before-training pruning (pruning at initialization).

Network Pruning Sparse Learning

Dynamic Sparse Network for Time Series Classification: Learning What to "see''

1 code implementation19 Dec 2022 Qiao Xiao, Boqian Wu, Yu Zhang, Shiwei Liu, Mykola Pechenizkiy, Elena Mocanu, Decebal Constantin Mocanu

The receptive field (RF), which determines the region of time series to be ``seen'' and used, is critical to improve the performance for time series classification (TSC).

Time Series Time Series Analysis +1

MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learning

1 code implementation23 Dec 2023 Bram Grooten, Tristan Tomilin, Gautham Vasan, Matthew E. Taylor, A. Rupam Mahmood, Meng Fang, Mykola Pechenizkiy, Decebal Constantin Mocanu

Our algorithm improves the agent's focus with useful masks, while its efficient Masker network only adds 0. 2% more parameters to the original structure, in contrast to previous work.

Data Augmentation

Memory-free Online Change-point Detection: A Novel Neural Network Approach

1 code implementation8 Jul 2022 Zahra Atashgahi, Decebal Constantin Mocanu, Raymond Veldhuis, Mykola Pechenizkiy

We show that ALACPD, on average, ranks first among state-of-the-art CPD algorithms in terms of quality of the time series segmentation, and it is on par with the best performer in terms of the accuracy of the estimated change-points.

Change Point Detection Time Series +1

Learning Invariant Representation for Continual Learning

1 code implementation15 Jan 2021 Ghada Sokar, Decebal Constantin Mocanu, Mykola Pechenizkiy

Finally, we analyze the role of the shared invariant representation in mitigating the forgetting problem especially when the number of replayed samples for each previous task is small.

Class Incremental Learning Incremental Learning +2

Selfish Sparse RNN Training

1 code implementation22 Jan 2021 Shiwei Liu, Decebal Constantin Mocanu, Yulong Pei, Mykola Pechenizkiy

Sparse neural networks have been widely applied to reduce the computational demands of training and deploying over-parameterized deep neural networks.

Dynamic Sparse Training for Deep Reinforcement Learning

1 code implementation8 Jun 2021 Ghada Sokar, Elena Mocanu, Decebal Constantin Mocanu, Mykola Pechenizkiy, Peter Stone

In this paper, we introduce for the first time a dynamic sparse training approach for deep reinforcement learning to accelerate the training process.

Continuous Control Decision Making +3

Where to Pay Attention in Sparse Training for Feature Selection?

1 code implementation26 Nov 2022 Ghada Sokar, Zahra Atashgahi, Mykola Pechenizkiy, Decebal Constantin Mocanu

Our proposed approach outperforms the state-of-the-art methods in terms of selecting informative features while reducing training iterations and computational costs substantially.

feature selection

You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets

1 code implementation28 Nov 2022 Tianjin Huang, Tianlong Chen, Meng Fang, Vlado Menkovski, Jiaxu Zhao, Lu Yin, Yulong Pei, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy, Shiwei Liu

Recent works have impressively demonstrated that there exists a subnetwork in randomly initialized convolutional neural networks (CNNs) that can match the performance of the fully trained dense networks at initialization, without any optimization of the weights of the network (i. e., untrained networks).

Out-of-Distribution Detection

Adversarial Balancing-based Representation Learning for Causal Effect Inference with Observational Data

2 code implementations30 Apr 2019 Xin Du, Lei Sun, Wouter Duivesteijn, Alexander Nikolaev, Mykola Pechenizkiy

The challenges for this problem are two-fold: on the one hand, we have to derive a causal estimator to estimate the causal quantity from observational data, where there exists confounding bias; on the other hand, we have to deal with the identification of CATE when the distribution of covariates in treatment and control groups are imbalanced.

Causal Inference Representation Learning +2

Self-Attention Meta-Learner for Continual Learning

1 code implementation28 Jan 2021 Ghada Sokar, Decebal Constantin Mocanu, Mykola Pechenizkiy

In this paper, we propose a new method, named Self-Attention Meta-Learner (SAM), which learns a prior knowledge for continual learning that permits learning a sequence of tasks, while avoiding catastrophic forgetting.

Continual Learning Split-CIFAR-10 +1

Phrase-level Textual Adversarial Attack with Label Preservation

1 code implementation Findings (NAACL) 2022 Yibin Lei, Yu Cao, Dianqi Li, Tianyi Zhou, Meng Fang, Mykola Pechenizkiy

Generating high-quality textual adversarial examples is critical for investigating the pitfalls of natural language processing (NLP) models and further promoting their robustness.

Adversarial Attack Sentence

E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation

1 code implementation7 Dec 2023 Boqian Wu, Qiao Xiao, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Decebal Constantin Mocanu, Maurice van Keulen, Elena Mocanu

E2ENet achieves comparable accuracy on the large-scale challenge AMOS-CT, while saving over 68\% parameter count and 29\% FLOPs in the inference phase, compared with the previous best-performing method.

Brain Tumor Segmentation Image Segmentation +2

SpaceNet: Make Free Space For Continual Learning

1 code implementation15 Jul 2020 Ghada Sokar, Decebal Constantin Mocanu, Mykola Pechenizkiy

Regularization-based methods maintain a fixed model capacity; however, previous studies showed the huge performance degradation of these methods when the task identity is not available during inference (e. g. class incremental learning scenario).

Class Incremental Learning Incremental Learning +1

Hop-Count Based Self-Supervised Anomaly Detection on Attributed Networks

1 code implementation16 Apr 2021 Tianjin Huang, Yulong Pei, Vlado Menkovski, Mykola Pechenizkiy

Although various approaches have been proposed to solve this problem, two major limitations exist: (1) unsupervised approaches usually work much less efficiently due to the lack of supervisory signal, and (2) existing anomaly detection methods only use local contextual information to detect anomalous nodes, e. g., one- or two-hop information, but ignore the global contextual information.

Self-Supervised Anomaly Detection Supervised Anomaly Detection

Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective

1 code implementation3 Dec 2023 Can Jin, Tianjin Huang, Yihua Zhang, Mykola Pechenizkiy, Sijia Liu, Shiwei Liu, Tianlong Chen

The rapid development of large-scale deep learning models questions the affordability of hardware platforms, which necessitates the pruning to reduce their computational and memory footprints.

Image Classification Visual Prompting

A Brain-inspired Algorithm for Training Highly Sparse Neural Networks

2 code implementations17 Mar 2019 Zahra Atashgahi, Joost Pieterse, Shiwei Liu, Decebal Constantin Mocanu, Raymond Veldhuis, Mykola Pechenizkiy

Concretely, by exploiting the cosine similarity metric to measure the importance of the connections, our proposed method, Cosine similarity-based and Random Topology Exploration (CTRE), evolves the topology of sparse neural networks by adding the most important connections to the network without calculating dense gradient in the backward.

Learning Theory

Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks

1 code implementation10 Mar 2023 Zahra Atashgahi, Xuhao Zhang, Neil Kichler, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Raymond Veldhuis, Decebal Constantin Mocanu

Feature selection that selects an informative subset of variables from data not only enhances the model interpretability and performance but also alleviates the resource demands.

feature selection

Large Language Models Are Neurosymbolic Reasoners

1 code implementation17 Jan 2024 Meng Fang, Shilong Deng, Yudi Zhang, Zijing Shi, Ling Chen, Mykola Pechenizkiy, Jun Wang

A wide range of real-world applications is characterized by their symbolic nature, necessitating a strong capability for symbolic reasoning.

Common Sense Reasoning Math +2

The Impact of Batch Learning in Stochastic Linear Bandits

1 code implementation14 Feb 2022 Danil Provodin, Pratik Gajane, Mykola Pechenizkiy, Maurits Kaptein

Our main theoretical results show that the impact of batch learning is a multiplicative factor of batch size relative to the regret of online behavior.

CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models

1 code implementation18 May 2023 Jiaxu Zhao, Meng Fang, Zijing Shi, Yitong Li, Ling Chen, Mykola Pechenizkiy

We evaluate two popular pretrained Chinese conversational models, CDial-GPT and EVA2. 0, using CHBias.

Response Generation

Direction-Aggregated Attack for Transferable Adversarial Examples

1 code implementation19 Apr 2021 Tianjin Huang, Vlado Menkovski, Yulong Pei, Yuhao Wang, Mykola Pechenizkiy

Deep neural networks are vulnerable to adversarial examples that are crafted by imposing imperceptible changes to the inputs.

Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks

1 code implementation11 Oct 2021 Ghada Sokar, Decebal Constantin Mocanu, Mykola Pechenizkiy

To address this challenge, we propose a new CL method, named AFAF, that aims to Avoid Forgetting and Allow Forward transfer in class-IL using fix-capacity models.

Class Incremental Learning Incremental Learning +2

Enhancing Adversarial Training via Reweighting Optimization Trajectory

1 code implementation25 Jun 2023 Tianjin Huang, Shiwei Liu, Tianlong Chen, Meng Fang, Li Shen, Vlaod Menkovski, Lu Yin, Yulong Pei, Mykola Pechenizkiy

Despite the fact that adversarial training has become the de facto method for improving the robustness of deep neural networks, it is well-known that vanilla adversarial training suffers from daunting robust overfitting, resulting in unsatisfactory robust generalization.

Adversarial Robustness

REST: Enhancing Group Robustness in DNNs through Reweighted Sparse Training

1 code implementation5 Dec 2023 Jiaxu Zhao, Lu Yin, Shiwei Liu, Meng Fang, Mykola Pechenizkiy

These bias attributes are strongly spuriously correlated with the target variable, causing the models to be biased towards spurious correlations (i. e., \textit{bias-conflicting}).

A Structural-Clustering Based Active Learning for Graph Neural Networks

1 code implementation7 Dec 2023 Ricky Maulana Fajri, Yulong Pei, Lu Yin, Mykola Pechenizkiy

To address this problem, we propose the Structural-Clustering PageRank method for improved Active learning (SPA) specifically designed for graph-structured data.

Active Learning Clustering +2

On Formalizing Fairness in Prediction with Machine Learning

no code implementations9 Oct 2017 Pratik Gajane, Mykola Pechenizkiy

Machine learning algorithms for prediction are increasingly being used in critical decisions affecting human lives.

BIG-bench Machine Learning Fairness

struc2gauss: Structural Role Preserving Network Embedding via Gaussian Embedding

no code implementations25 May 2018 Yulong Pei, Xin Du, Jianpeng Zhang, George Fletcher, Mykola Pechenizkiy

Almost all previous methods represent a node into a point in space and focus on local structural information, i. e., neighborhood information.

Clustering Network Embedding

Rule-based Emotion Detection on Social Media: Putting Tweets on Plutchik's Wheel

no code implementations15 Dec 2014 Erik Tromp, Mykola Pechenizkiy

We study sentiment analysis beyond the typical granularity of polarity and instead use Plutchik's wheel of emotions model.

Sentiment Analysis

Predictive User Modeling with Actionable Attributes

no code implementations23 Dec 2013 Indre Zliobaite, Mykola Pechenizkiy

We study how to learn to choose the value of an actionable attribute in order to maximize the probability of a desired outcome in predictive modeling.

Attribute Marketing

Controversy Rules - Discovering Regions Where Classifiers (Dis-)Agree Exceptionally

no code implementations22 Aug 2018 Oren Zeev-Ben-Mordehai, Wouter Duivesteijn, Mykola Pechenizkiy

Finding regions for which there is higher controversy among different classifiers is insightful with regards to the domain and our models.

General Classification General Knowledge

Looking Deeper into Deep Learning Model: Attribution-based Explanations of TextCNN

no code implementations8 Nov 2018 Wenting Xiong, Iftitahu Ni'mah, Juan M. G. Huesca, Werner van Ipenburg, Jan Veldsink, Mykola Pechenizkiy

Layer-wise Relevance Propagation (LRP) and saliency maps have been recently used to explain the predictions of Deep Learning models, specifically in the domain of text classification.

Sentence text-classification +1

Intrinsically Sparse Long Short-Term Memory Networks

no code implementations26 Jan 2019 Shiwei Liu, Decebal Constantin Mocanu, Mykola Pechenizkiy

However, LSTMs are prone to be memory-bandwidth limited in realistic applications and need an unbearable period of training and inference time as the model size is ever-increasing.

Model Compression Sentiment Analysis

Learning with Delayed Synaptic Plasticity

no code implementations22 Mar 2019 Anil Yaman, Giovanni Iacca, Decebal Constantin Mocanu, George Fletcher, Mykola Pechenizkiy

Inspired by biology, plasticity can be modeled in artificial neural networks by using Hebbian learning rules, i. e. rules that update synapses based on the neuron activations and reinforcement signals.

Evolving Plasticity for Autonomous Learning under Changing Environmental Conditions

no code implementations2 Apr 2019 Anil Yaman, Giovanni Iacca, Decebal Constantin Mocanu, Matt Coler, George Fletcher, Mykola Pechenizkiy

Hebbian learning is a biologically plausible mechanism for modeling the plasticity property in artificial neural networks (ANNs), based on the local interactions of neurons.

A Human-Grounded Evaluation of SHAP for Alert Processing

no code implementations7 Jul 2019 Hilde J. P. Weerts, Werner van Ipenburg, Mykola Pechenizkiy

In this paper we present the results of a human-grounded evaluation of SHAP, an explanation method that has been well-received in the XAI and related communities.

BIG-bench Machine Learning Decision Making +1

Case-Based Reasoning for Assisting Domain Experts in Processing Fraud Alerts of Black-Box Machine Learning Models

no code implementations7 Jul 2019 Hilde J. P. Weerts, Werner van Ipenburg, Mykola Pechenizkiy

In many contexts, it can be useful for domain experts to understand to what extent predictions made by a machine learning model can be trusted.

BIG-bench Machine Learning

BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation

no code implementations17 Sep 2019 Iftitahu Ni'mah, Vlado Menkovski, Mykola Pechenizkiy

This study mainly investigates two decoding problems in neural keyphrase generation: sequence length bias and beam diversity.

Keyphrase Generation

The Relationship between the Consistency of Users' Ratings and Recommendation Calibration

no code implementations3 Nov 2019 Masoud Mansoury, Himan Abdollahpouri, Joris Rombouts, Mykola Pechenizkiy

In this paper, we aim to explore the relationship between the consistency of users' ratings behavior and the degree of calibrated recommendations they receive.

Fairness Recommendation Systems

Causal Discovery from Incomplete Data: A Deep Learning Approach

no code implementations15 Jan 2020 Yuhao Wang, Vlado Menkovski, Hao Wang, Xin Du, Mykola Pechenizkiy

As systems are getting more autonomous with the development of artificial intelligence, it is important to discover the causal knowledge from observational sensory inputs.

Causal Discovery Imputation

Novelty Producing Synaptic Plasticity

no code implementations10 Feb 2020 Anil Yaman, Giovanni Iacca, Decebal Constantin Mocanu, George Fletcher, Mykola Pechenizkiy

A learning process with the plasticity property often requires reinforcement signals to guide the process.

Investigating Potential Factors Associated with Gender Discrimination in Collaborative Recommender Systems

no code implementations18 Feb 2020 Masoud Mansoury, Himan Abdollahpouri, Jessie Smith, Arman Dehpanah, Mykola Pechenizkiy, Bamshad Mobasher

The proliferation of personalized recommendation technologies has raised concerns about discrepancies in their recommendation performance across different genders, age groups, and racial or ethnic populations.

Fairness Recommendation Systems

Knowledge Elicitation using Deep Metric Learning and Psychometric Testing

no code implementations14 Apr 2020 Lu Yin, Vlado Menkovski, Mykola Pechenizkiy

The main reason for such a reductionist approach is the difficulty in eliciting the domain knowledge from the experts.

Metric Learning

FairMatch: A Graph-based Approach for Improving Aggregate Diversity in Recommender Systems

no code implementations3 May 2020 Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, Robin Burke

That leads to low coverage of items in recommendation lists across users (i. e. low aggregate diversity) and unfair distribution of recommended items.

Fairness Recommendation Systems

Feedback Loop and Bias Amplification in Recommender Systems

no code implementations25 Jul 2020 Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, Robin Burke

Recommendation algorithms are known to suffer from popularity bias; a few popular items are recommended frequently while the majority of other items are ignored.

Recommendation Systems

ResGCN: Attention-based Deep Residual Modeling for Anomaly Detection on Attributed Networks

1 code implementation30 Sep 2020 Yulong Pei, Tianjin Huang, Werner van Ipenburg, Mykola Pechenizkiy

Effectively detecting anomalous nodes in attributed networks is crucial for the success of many real-world applications such as fraud and intrusion detection.

Anomaly Detection Intrusion Detection

Bridging the Performance Gap between FGSM and PGD Adversarial Training

1 code implementation7 Nov 2020 Tianjin Huang, Vlado Menkovski, Yulong Pei, Mykola Pechenizkiy

In addition, it achieves comparable performance of adversarial robustness on MNIST dataset under white-box attack, and it achieves better performance than adv. PGD under white-box attack and effectively defends the transferable adversarial attack on CIFAR-10 dataset.

Adversarial Attack Adversarial Robustness

ViDi: Descriptive Visual Data Clustering as Radiologist Assistant in COVID-19 Streamline Diagnostic

no code implementations30 Nov 2020 Sahithya Ravi, Samaneh Khoshrou, Mykola Pechenizkiy

In the light of the COVID-19 pandemic, deep learning methods have been widely investigated in detecting COVID-19 from chest X-rays.

Clustering Decision Making +2

Hierarchical Semantic Segmentation using Psychometric Learning

no code implementations7 Jul 2021 Lu Yin, Vlado Menkovski, Shiwei Liu, Mykola Pechenizkiy

One of the major challenges in the supervised learning approaches is expressing and collecting the rich knowledge that experts have with respect to the meaning present in the image data.

Image Segmentation Metric Learning +2

Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles

no code implementations21 Sep 2021 Xin Du, Subramanian Ramamoorthy, Wouter Duivesteijn, Jin Tian, Mykola Pechenizkiy

Specifically, we propose to leverage causal knowledge by regarding the distributional shifts in subpopulations and deployment environments as the results of interventions on the underlying system.

Calibrated Adversarial Training

1 code implementation1 Oct 2021 Tianjin Huang, Vlado Menkovski, Yulong Pei, Mykola Pechenizkiy

In this paper, we present the Calibrated Adversarial Training, a method that reduces the adverse effects of semantic perturbations in adversarial training.

Does the End Justify the Means? On the Moral Justification of Fairness-Aware Machine Learning

no code implementations17 Feb 2022 Hilde Weerts, Lambèr Royakkers, Mykola Pechenizkiy

In this paper, we present a framework for moral reasoning about the justification of fairness metrics and explore the moral implications of the use of fair-ml algorithms that optimize for them.

Ethics Fairness

Semantic-Based Few-Shot Learning by Interactive Psychometric Testing

no code implementations16 Dec 2021 Lu Yin, Vlado Menkovski, Yulong Pei, Mykola Pechenizkiy

In this work, we advance the few-shot learning towards this more challenging scenario, the semantic-based few-shot learning, and propose a method to address the paradigm by capturing the inner semantic relationships using interactive psychometric learning.

Few-Shot Learning

Survey on Fair Reinforcement Learning: Theory and Practice

no code implementations20 May 2022 Pratik Gajane, Akrati Saxena, Maryam Tavakol, George Fletcher, Mykola Pechenizkiy

In this article, we provide an extensive overview of fairness approaches that have been implemented via a reinforcement learning (RL) framework.

Decision Making Fairness +3

Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training

no code implementations30 May 2022 Lu Yin, Vlado Menkovski, Meng Fang, Tianjin Huang, Yulong Pei, Mykola Pechenizkiy, Decebal Constantin Mocanu, Shiwei Liu

Recent works on sparse neural network training (sparse training) have shown that a compelling trade-off between performance and efficiency can be achieved by training intrinsically sparse neural networks from scratch.

FAL-CUR: Fair Active Learning using Uncertainty and Representativeness on Fair Clustering

1 code implementation21 Sep 2022 Ricky Fajri, Akrati Saxena, Yulong Pei, Mykola Pechenizkiy

Active Learning (AL) techniques have proven to be highly effective in reducing data labeling costs across a range of machine learning tasks.

Active Learning Clustering +1

Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML

no code implementations15 Mar 2023 Hilde Weerts, Florian Pfisterer, Matthias Feurer, Katharina Eggensperger, Edward Bergman, Noor Awad, Joaquin Vanschoren, Mykola Pechenizkiy, Bernd Bischl, Frank Hutter

The field of automated machine learning (AutoML) introduces techniques that automate parts of the development of machine learning (ML) systems, accelerating the process and reducing barriers for novices.

AutoML Fairness

Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree

no code implementations5 May 2023 Hilde Weerts, Raphaële Xenidis, Fabien Tarissan, Henrik Palmer Olsen, Mykola Pechenizkiy

In this paper, we aim to illustrate to what extent European Union (EU) non-discrimination law coincides with notions of algorithmic fairness proposed in computer science literature and where they differ.

Fairness Legal Reasoning

Provably Efficient Exploration in Constrained Reinforcement Learning:Posterior Sampling Is All You Need

no code implementations27 Sep 2023 Danil Provodin, Pratik Gajane, Mykola Pechenizkiy, Maurits Kaptein

We present a new algorithm based on posterior sampling for learning in constrained Markov decision processes (CMDP) in the infinite-horizon undiscounted setting.

Efficient Exploration

KeyGen2Vec: Learning Document Embedding via Multi-label Keyword Generation in Question-Answering

no code implementations30 Oct 2023 Iftitahu Ni'mah, Samaneh Khoshrou, Vlado Menkovski, Mykola Pechenizkiy

Interestingly, although in general the absolute advantage of learning embeddings through label supervision is highly positive across evaluation datasets, KeyGen2Vec is shown to be competitive with classifier that exploits topic label supervision in Yahoo!

Document Embedding Keyphrase Generation +1

Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approach

no code implementations NeurIPS 2023 Yudi Zhang, Yali Du, Biwei Huang, Ziyan Wang, Jun Wang, Meng Fang, Mykola Pechenizkiy

While the majority of current approaches construct the reward redistribution in an uninterpretable manner, we propose to explicitly model the contributions of state and action from a causal perspective, resulting in an interpretable reward redistribution and preserving policy invariance.

reinforcement-learning

GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models

no code implementations11 Dec 2023 Jiaxu Zhao, Meng Fang, Shirui Pan, Wenpeng Yin, Mykola Pechenizkiy

In this work, we propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs (e. g., GPT-4 \cite{openai2023gpt4}) to assess bias in models.

Cannot find the paper you are looking for? You can Submit a new open access paper.