Search Results for author: Bahram Zonooz

Found 47 papers, 31 papers with code

The Effectiveness of Random Forgetting for Robust Generalization

no code implementations18 Feb 2024 Vijaya Raghavan T Ramkumar, Bahram Zonooz, Elahe Arani

However, a key challenge of AT is robust overfitting, where the network's robust performance on test data deteriorates with further training, thus hindering generalization.

Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Training

1 code implementation26 Jan 2024 Shruthi Gowda, Bahram Zonooz, Elahe Arani

Adversarial training improves the robustness of neural networks against adversarial attacks, albeit at the expense of the trade-off between standard and robust generalization.

Memorization

Transformers in Unsupervised Structure-from-Motion

1 code implementation16 Dec 2023 Hemang Chawla, Arnav Varma, Elahe Arani, Bahram Zonooz

Transformers have revolutionized deep learning based computer vision with improved performance as well as robustness to natural corruptions and adversarial attacks.

Decision Making Image Classification +4

Continual Learning of Unsupervised Monocular Depth from Videos

1 code implementation4 Nov 2023 Hemang Chawla, Arnav Varma, Elahe Arani, Bahram Zonooz

Spatial scene understanding, including monocular depth estimation, is an important problem in various applications, such as robotics and autonomous driving.

Autonomous Driving Continual Learning +4

Dual Cognitive Architecture: Incorporating Biases and Multi-Memory Systems for Lifelong Learning

2 code implementations17 Oct 2023 Shruthi Gowda, Bahram Zonooz, Elahe Arani

Artificial neural networks (ANNs) exhibit a narrow scope of expertise on stationary independent data.

Inductive Bias

TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion

1 code implementation NeurIPS 2023 Preetha Vijayan, Prashant Bhat, Elahe Arani, Bahram Zonooz

Continual learning (CL) has remained a persistent challenge for deep neural networks due to catastrophic forgetting (CF) of previously learned tasks.

Continual Learning

Towards Brain Inspired Design for Addressing the Shortcomings of ANNs

no code implementations30 Jun 2023 Fahad Sarfraz, Elahe Arani, Bahram Zonooz

As our understanding of the mechanisms of brain function is enhanced, the value of insights gained from neuroscience to the development of AI algorithms deserves further consideration.

Enhancing Performance of Vision Transformers on Small Datasets through Local Inductive Bias Incorporation

no code implementations15 May 2023 Ibrahim Batuhan Akkaya, Senthilkumar S. Kathiresan, Elahe Arani, Bahram Zonooz

Vision transformers (ViTs) achieve remarkable performance on large datasets, but tend to perform worse than convolutional neural networks (CNNs) when trained from scratch on smaller datasets, possibly due to a lack of local inductive bias in the architecture.

Image Classification Inductive Bias +3

BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning

1 code implementation8 May 2023 Kishaan Jeeveswaran, Prashant Bhat, Bahram Zonooz, Elahe Arani

The ability of deep neural networks to continually learn and adapt to a sequence of tasks has remained challenging due to catastrophic forgetting of previously learned tasks.

Continual Learning

LSFSL: Leveraging Shape Information in Few-shot Learning

no code implementations13 Apr 2023 Deepan Chakravarthi Padmanabhan, Shruthi Gowda, Elahe Arani, Bahram Zonooz

Few-shot learning (FSL) techniques seek to learn the underlying patterns in data using fewer samples, analogous to how humans learn from limited experience.

Few-Shot Learning

A Study of Biologically Plausible Neural Network: The Role and Interactions of Brain-Inspired Mechanisms in Continual Learning

1 code implementation13 Apr 2023 Fahad Sarfraz, Elahe Arani, Bahram Zonooz

Humans excel at continually acquiring, consolidating, and retaining information from an ever-changing environment, whereas artificial neural networks (ANNs) exhibit catastrophic forgetting.

Continual Learning

Learn, Unlearn and Relearn: An Online Learning Paradigm for Deep Neural Networks

1 code implementation18 Mar 2023 Vijaya Raghavan T. Ramkumar, Elahe Arani, Bahram Zonooz

Deep neural networks (DNNs) are often trained on the premise that the complete training data set is provided ahead of time.

Task-Aware Information Routing from Common Representation Space in Lifelong Learning

1 code implementation14 Feb 2023 Prashant Bhat, Bahram Zonooz, Elahe Arani

Thus, inspired by the Global Workspace Theory of conscious information access in the brain, we propose TAMiL, a continual learning method that entails task-attention modules to capture task-specific information from the common representation space.

Continual Learning

Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learning

1 code implementation14 Feb 2023 Fahad Sarfraz, Elahe Arani, Bahram Zonooz

To this end, we propose \textit{ESMER} which employs a principled mechanism to modulate error sensitivity in a dual-memory rehearsal-based system.

Continual Learning

Dynamically Modular and Sparse General Continual Learning

1 code implementation2 Jan 2023 Arnav Varma, Elahe Arani, Bahram Zonooz

Real-world applications often require learning continuously from a stream of data under ever-changing conditions.

Continual Learning

Sparse Coding in a Dual Memory System for Lifelong Learning

1 code implementation28 Dec 2022 Fahad Sarfraz, Elahe Arani, Bahram Zonooz

Efficient continual learning in humans is enabled by a rich set of neurophysiological mechanisms and interactions between multiple memory systems.

Continual Learning

Curbing Task Interference using Representation Similarity-Guided Multi-Task Feature Sharing

1 code implementation19 Aug 2022 Naresh Kumar Gurulingan, Elahe Arani, Bahram Zonooz

However, increased sharing exposes more parameters to task interference which likely hinders both generalization and robustness.

Computational Efficiency Inductive Bias +1

Differencing based Self-supervised pretraining for Scene Change Detection

1 code implementation11 Aug 2022 Vijaya Raghavan T. Ramkumar, Elahe Arani, Bahram Zonooz

SCD is challenging due to noisy changes in illumination, seasonal variations, and perspective differences across a pair of views.

Change Detection Scene Change Detection +1

Adversarial Attacks on Monocular Pose Estimation

1 code implementation14 Jul 2022 Hemang Chawla, Arnav Varma, Elahe Arani, Bahram Zonooz

While studies evaluating the impact of adversarial attacks on monocular depth estimation exist, a systematic demonstration and analysis of adversarial perturbations against pose estimation are lacking.

Monocular Depth Estimation Object Detection +3

Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach

1 code implementation13 Jul 2022 Prashant Bhat, Bahram Zonooz, Elahe Arani

Furthermore, the domain shift between pre-training data distribution and the task distribution reduces the generalizability of the learned representations.

Continual Learning

Consistency is the key to further mitigating catastrophic forgetting in continual learning

1 code implementation11 Jul 2022 Prashant Bhat, Bahram Zonooz, Elahe Arani

Therefore, we examine the role of consistency regularization in ER framework under various continual learning scenarios.

Continual Learning Self-Supervised Learning

InBiaseD: Inductive Bias Distillation to Improve Generalization and Robustness through Shape-awareness

1 code implementation12 Jun 2022 Shruthi Gowda, Bahram Zonooz, Elahe Arani

Humans rely less on spurious correlations and trivial cues, such as texture, compared to deep neural networks which lead to better generalization and robustness.

Inductive Bias

Transformers in Self-Supervised Monocular Depth Estimation with Unknown Camera Intrinsics

1 code implementation7 Feb 2022 Arnav Varma, Hemang Chawla, Bahram Zonooz, Elahe Arani

While recent works have compared transformers against their CNN counterparts for tasks such as image classification, no study exists that investigates the impact of using transformers for self-supervised monocular depth estimation.

Autonomous Driving Depth Prediction +3

Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System

1 code implementation ICLR 2022 Elahe Arani, Fahad Sarfraz, Bahram Zonooz

Humans excel at continually learning from an ever-changing environment whereas it remains a challenge for deep neural networks which exhibit catastrophic forgetting.

Continual Learning

A Comprehensive Study of Vision Transformers on Dense Prediction Tasks

no code implementations21 Jan 2022 Kishaan Jeeveswaran, Senthilkumar Kathiresan, Arnav Varma, Omar Magdy, Bahram Zonooz, Elahe Arani

Convolutional Neural Networks (CNNs), architectures consisting of convolutional layers, have been the standard choice in vision tasks.

Object object-detection +3

Does Thermal data make the detection systems more reliable?

1 code implementation9 Nov 2021 Shruthi Gowda, Bahram Zonooz, Elahe Arani

To overcome these challenges, we explore the idea of leveraging a different data modality that is disparate yet complementary to the visual data.

Autonomous Driving

Improving the Efficiency of Transformers for Resource-Constrained Devices

no code implementations30 Jun 2021 Hamid Tabani, Ajay Balasubramaniam, Shabbir Marzban, Elahe Arani, Bahram Zonooz

Transformers provide promising accuracy and have become popular and used in various domains such as natural language processing and computer vision.

Highlighting the Importance of Reducing Research Bias and Carbon Emissions in CNNs

no code implementations6 Jun 2021 Ahmed Badar, Arnav Varma, Adrian Staniec, Mahmoud Gamal, Omar Magdy, Haris Iqbal, Elahe Arani, Bahram Zonooz

We highlight that there is a need to rethink the design and evaluation of CNNs to alleviate the issue of research bias and carbon emissions.

Fairness

AI Driven Road Maintenance Inspection

no code implementations4 Jun 2021 Ratnajit Mukherjee, Haris Iqbal, Shabbir Marzban, Ahmed Badar, Terence Brouns, Shruthi Gowda, Elahe Arani, Bahram Zonooz

Road infrastructure maintenance inspection is typically a labour-intensive and critical task to ensure the safety of all the road users.

object-detection Object Detection +1

Challenges and Obstacles Towards Deploying Deep Learning Models on Mobile Devices

no code implementations6 May 2021 Hamid Tabani, Ajay Balasubramaniam, Elahe Arani, Bahram Zonooz

From computer vision and speech recognition to forecasting trajectories in autonomous vehicles, deep learning approaches are at the forefront of so many domains.

Autonomous Vehicles speech-recognition +1

Distill on the Go: Online knowledge distillation in self-supervised learning

1 code implementation20 Apr 2021 Prashant Bhat, Elahe Arani, Bahram Zonooz

To address the issue of self-supervised pre-training of smaller models, we propose Distill-on-the-Go (DoGo), a self-supervised learning paradigm using single-stage online knowledge distillation to improve the representation quality of the smaller models.

Knowledge Distillation Self-Supervised Learning

Perceptual Loss for Robust Unsupervised Homography Estimation

1 code implementation20 Apr 2021 Daniel Koguciuk, Elahe Arani, Bahram Zonooz

We use an additional photometric distortion step in the synthetic COCO dataset generation to better represent the illumination variation of the real-world scenarios.

Homography Estimation Representation Learning

Noisy Concurrent Training for Efficient Learning under Label Noise

2 code implementations17 Sep 2020 Fahad Sarfraz, Elahe Arani, Bahram Zonooz

Thus, we propose Noisy Concurrent Training (NCT) which leverages collaborative learning to use the consensus between two models as an additional source of supervision.

Image Classification Memorization

Adversarial Concurrent Training: Optimizing Robustness and Accuracy Trade-off of Deep Neural Networks

1 code implementation16 Aug 2020 Elahe Arani, Fahad Sarfraz, Bahram Zonooz

Adversarial training has been proven to be an effective technique for improving the adversarial robustness of models.

Adversarial Robustness

Crowdsourced 3D Mapping: A Combined Multi-View Geometry and Self-Supervised Learning Approach

1 code implementation25 Jul 2020 Hemang Chawla, Matti Jukola, Terence Brouns, Elahe Arani, Bahram Zonooz

The ability to efficiently utilize crowdsourced visual data carries immense potential for the domains of large scale dynamic mapping and autonomous driving.

Autonomous Driving Motion Estimation +1

Knowledge Distillation Beyond Model Compression

no code implementations3 Jul 2020 Fahad Sarfraz, Elahe Arani, Bahram Zonooz

Knowledge distillation (KD) is commonly deemed as an effective model compression technique in which a compact model (student) is trained under the supervision of a larger pretrained model or an ensemble of models (teacher).

Knowledge Distillation Model Compression +1

RGPNet: A Real-Time General Purpose Semantic Segmentation

no code implementations3 Dec 2019 Elahe Arani, Shabbir Marzban, Andrei Pata, Bahram Zonooz

We propose a real-time general purpose semantic segmentation architecture, RGPNet, which achieves significant performance gain in complex environments.

Segmentation Semantic Segmentation

Noise as a Resource for Learning in Knowledge Distillation

no code implementations11 Oct 2019 Elahe Arani, Fahad Sarfraz, Bahram Zonooz

In doing so, we propose three different methods that target the common challenges in deep neural networks: minimizing the performance gap between a compact model and large model (Fickle Teacher), training high performance compact adversarially robust models (Soft Randomization), and training models efficiently under label noise (Messy Collaboration).

Knowledge Distillation

Cannot find the paper you are looking for? You can Submit a new open access paper.