Search Results for author:
Found 24 papers, 17 papers with code
Hyperparameter Selection in Continual Learning
In continual learning (CL) -- where a learner trains on a stream of data -- standard hyperparameter optimisation (HPO) cannot be applied, as a learner does not have access to all of the data at the same time.
PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition
In this paper, we further adapt the selective scanning process of Mamba to the visual domain, enhancing its ability to learn features from two-dimensional images by (i) a continuous 2D scanning process that improves spatial continuity by ensuring adjacency of tokens in the scanning sequence, and (ii) direction-aware updating which enables the model to discern the spatial relations of tokens by encoding directional information.
EgoPoseFormer: A Simple Baseline for Egocentric 3D Human Pose Estimation
We also show that our method can be seamlessly extended to monocular settings, which achieves state-of-the-art performance on the SceneEgo dataset, improving MPJPE by 25. 5mm (21% improvement) compared to the best existing method with only 60. 7% model parameters and 36. 4% FLOPs.
WidthFormer: Toward Efficient Transformer-based BEV View Transformation
In this work, we present WidthFormer, a novel transformer-based Bird's-Eye-View (BEV) 3D detection method tailored for real-time autonomous-driving applications.
DLAS: An Exploration and Assessment of the Deep Learning Acceleration Stack
Deep Neural Networks (DNNs) are extremely computationally demanding, which presents a large barrier to their deployment on resource-constrained devices.
Generate Your Own Scotland: Satellite Image Generation Conditioned on Maps
Despite recent advancements in image generation, diffusion models still remain largely underexplored in Earth Observation.
GPViT: A High Resolution Non-Hierarchical Vision Transformer with Group Propagation
In each GP Block, features are first grouped together by a fixed number of learnable group tokens; we then perform Group Propagation where global information is exchanged between the grouped features; finally, global information in the updated grouped features is returned back to the image features through a transformer decoder.
Plug and Play Active Learning for Object Detection
To overcome this challenge, we introduce Plug and Play Active Learning (PPAL), a simple and effective AL strategy for object detection.
Prediction-Guided Distillation for Dense Object Detection
Based on this, we propose Prediction-Guided Distillation (PGD), which focuses distillation on these key predictive regions of the teacher and yields considerable gains in performance over many existing KD baselines.
Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning
The goal of contrastive learning based pre-training is to leverage large quantities of unlabeled data to produce a model that can be readily adapted downstream.
Neural Architecture Search as Program Transformation Exploration
This unification allows us to express existing NAS operations as combinations of simpler transformations.
Optimizing Grouped Convolutions on Edge Devices
We observe that our new implementation scales well with the number of groups and provides the best inference times in all settings, improving the existing implementations of grouped convolutions in TVM, PyTorch and TensorFlow Lite by 3. 4x, 8x and 4x on average respectively.
Neural Architecture Search without Training
In this work, we examine the overlap of activations between datapoints in untrained networks and motivate how this can give a measure which is usefully indicative of a network's trained performance.
Performance Aware Convolutional Neural Network Channel Pruning for Embedded GPUs
We evaluate higher level libraries, which analyze the input characteristics of a convolutional layer, based on which they produce optimized OpenCL (Arm Compute Library and TVM) and CUDA (cuDNN) code.
Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels
Recently, different machine learning methods have been introduced to tackle the challenging few-shot learning scenario that is, learning from a small labeled dataset related to a specific task.
BlockSwap: Fisher-guided Block Substitution for Network Compression on a Budget
The desire to map neural networks to varying-capacity devices has led to the development of a wealth of compression techniques, many of which involve replacing standard convolutional blocks in a large network with cheap alternative blocks.
Separable Layers Enable Structured Efficient Linear Substitutions
In response to the development of recent efficient dense layers, this paper shows that something as simple as replacing linear components in pointwise convolutions with structured linear decompositions also produces substantial gains in the efficiency/accuracy tradeoff.
Dilated DenseNets for Relational Reasoning
Despite their impressive performance in many tasks, deep neural networks often struggle at relational reasoning.
Distilling with Performance Enhanced Students
The task of accelerating large neural networks on general purpose hardware has, in recent years, prompted the use of channel pruning to reduce network size.
Pruning neural networks: is it time to nip it in the bud?
First, when time-constrained, it is better to train a simple, smaller network from scratch than prune a large network.
A Closer Look at Structured Pruning for Neural Network Compression
Structured pruning is a popular method for compressing a neural network: given a large trained network, one alternates between removing channel connections and fine-tuning; reducing the overall width of the network.
CINIC-10 is not ImageNet or CIFAR-10
In this brief technical report we introduce the CINIC-10 dataset as a plug-in extended alternative for CIFAR-10.
Ranked #6 on
Image Classification
on CINIC-10
Characterising Across-Stack Optimisations for Deep Convolutional Neural Networks
Convolutional Neural Networks (CNNs) are extremely computationally demanding, presenting a large barrier to their deployment on resource-constrained devices.
Moonshine: Distilling with Cheap Convolutions
Many engineers wish to deploy modern neural networks in memory-limited settings; but the development of flexible methods for reducing memory use is in its infancy, and there is little knowledge of the resulting cost-benefit.
