Search Results for author:
Found 52 papers, 19 papers with code
Hardware and Software Platform Inference
We evaluate HSPI against models served on different real hardware and find that in a white-box setting we can distinguish between different \GPU{}s with between $83. 9\%$ and $100\%$ accuracy.
Absorb & Escape: Overcoming Single Model Limitations in Generating Genomic Sequences
To overcome the limitations of both approaches, we propose a post-training sampling method, termed Absorb & Escape (A&E) to perform compositional generation from AR models and DMs.
Scaling Laws for Mixed quantization in Large Language Models
Post-training quantization of Large Language Models (LLMs) has proven effective in reducing the computational requirements for running inference on these models.
QERA: an Analytical Framework for Quantization Error Reconstruction
However, these heuristic methods lack an analytical solution to guide the design of quantization error reconstruction terms.
GV-Rep: A Large-Scale Dataset for Genetic Variant Representation Learning
Genetic variants (GVs) are defined as differences in the DNA sequences among individuals and play a crucial role in diagnosing and treating genetic diseases.
Unlocking the Global Synergies in Low-Rank Adapters
We present HeteroLoRA, a light-weight search algorithm that leverages zero-cost proxies to allocate the limited LoRA trainable parameters across the model for better fine-tuned performance.
Optimised Grouped-Query Attention Mechanism for Transformers
In this work, we propose AsymGQA, an activation-informed approach to asymmetrically grouping an MHA to a GQA for better model performance.
HASS: Hardware-Aware Sparsity Search for Dataflow DNN Accelerator
We propose a Hardware-Aware Sparsity Search (HASS) to systematically determine an efficient sparsity solution for dataflow accelerators.
$Δ$-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers
Based on this insight, we propose an overall training-free inference acceleration framework $\Delta$-DiT: using a designed cache mechanism to accelerate the rear DiT blocks in the early sampling stages and the front DiT blocks in the later stages.
Locking Machine Learning Models into Hardware
We demonstrate that locking mechanisms are feasible by either targeting efficiency of model representations, such making models incompatible with quantisation, or tie the model's operation on specific characteristics of hardware, such as number of cycles for arithmetic operations.
Enhancing Node Representations for Real-World Complex Networks with Topological Augmentation
In this paper, we present Topological Augmentation (TopoAug), a novel graph augmentation method that builds a combinatorial complex from the original graph by constructing virtual hyperedges directly from the raw data.
Architectural Neural Backdoors from First Principles
In this work we construct an arbitrary trigger detector which can be used to backdoor an architecture with no human supervision.
DiscDiff: Latent Diffusion Model for DNA Sequence Generation
This paper introduces a novel framework for DNA sequence generation, comprising two key components: DiscDiff, a Latent Diffusion Model (LDM) tailored for generating discrete DNA sequences, and Absorb-Escape, a post-training algorithm designed to refine these sequences.
LQER: Low-Rank Quantization Error Reconstruction for LLMs
Post-training quantization of Large Language Models (LLMs) is challenging.
Latent Diffusion Model for DNA Sequence Generation
On the other hand, Diffusion Models are a promising new class of generative models that are not burdened with these problems, enabling them to reach the state-of-the-art in domains such as image generation.
Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?
In this work, we explore the statistical and learning properties of the LLM layer and attribute the bottleneck of LLM quantisation to numerical scaling offsets.
LLM4DV: Using Large Language Models for Hardware Test Stimuli Generation
Test stimuli generation has been a crucial but labor-intensive task in hardware design verification.
Will More Expressive Graph Neural Networks do Better on Generative Tasks?
Moreover, we show that GCPN and GraphAF with advanced GNNs can achieve state-of-the-art results across 17 other non-GNN-based graph generative approaches, such as variational autoencoders and Bayesian optimisation models, on the proposed molecular generative objectives (DRD2, Median1, Median2), which are impor- tant metrics for de-novo molecular design.
Hybrid Graph: A Unified Graph Representation with Datasets and Benchmarks for Complex Graphs
Additionally, though many Graph Neural Networks (GNNs) have been proposed for representation learning on higher-order graphs, they are usually only evaluated on simple graph datasets.
Genomic Interpreter: A Hierarchical Genomic Deep Neural Network with 1D Shifted Window Transformer
This work presents Genomic Interpreter: a novel architecture for genomic assay prediction.
The Curse of Recursion: Training on Generated Data Makes Models Forget
It is now clear that large language models (LLMs) are here to stay, and will bring about drastic change in the whole ecosystem of online text and images.
Revisiting Automated Prompting: Are We Actually Doing Better?
Current literature demonstrates that Large Language Models (LLMs) are great few-shot learners, and prompting significantly increases their performance on a range of downstream tasks in a few-shot learning setting.
Dynamic Stashing Quantization for Efficient Transformer Training
Large Language Models (LLMs) have demonstrated impressive performance on a range of Natural Language Processing (NLP) tasks.
Task-Agnostic Graph Neural Network Evaluation via Adversarial Collaboration
Existing GNN benchmarking methods for molecular representation learning focus on comparing the GNNs' performances on some node/graph classification/regression tasks on certain datasets.
Adaptive Channel Sparsity for Federated Learning Under System Heterogeneity
Owing to the non-i. i. d.
Flareon: Stealthy any2any Backdoor Injection via Poisoned Augmentation
Open software supply chain attacks, once successful, can exact heavy costs in mission-critical applications.
Revisiting Structured Dropout
Large neural networks are often overparameterised and prone to overfitting, Dropout is a widely used regularization technique to combat overfitting and improve model generalization.
DARTFormer: Finding The Best Type Of Attention
Given the wide and ever growing range of different efficient Transformer attention mechanisms, it is important to identify which attention is most effective when given a task.
Wide Attention Is The Way Forward For Transformers?
We demonstrate that wide single layer Transformer models can compete with or outperform deeper ones in a variety of Natural Language Processing (NLP) tasks when both are trained from scratch.
ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks
These backdoors are impossible to detect during the training or data preparation processes, because they are not yet present.
Augmentation Backdoors
It is well known that backdoors can be inserted into machine learning models through serving a modified dataset to train on.
Efficient Adversarial Training With Data Pruning
In this paper we demonstrate data pruning-a method for increasing adversarial training efficiency through data sub-sampling. We empirically show that data pruning leads to improvements in convergence and reliability of adversarial training, albeit with different levels of utility degradation.
Architectural Backdoors in Neural Networks
Machine learning is vulnerable to adversarial manipulation.
Model Architecture Adaption for Bayesian Neural Networks
Bayesian Neural Networks (BNNs) offer a mathematically grounded framework to quantify the uncertainty of model predictions but come with a prohibitive computation cost for both training and inference.
DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning
First, we introduce a time-adaptive quantization algorithm that increases the quantization level as training progresses.
FedDrop: Trajectory-weighted Dropout for Efficient Federated Learning
Federated learning (FL) enables edge clients to train collaboratively while preserving individual's data privacy.
Rapid Model Architecture Adaption for Meta-Learning
H-Meta-NAS shows a Pareto dominance compared to a variety of NAS and manual baselines in popular few-shot learning benchmarks with various hardware platforms and constraints.
Markpainting: Adversarial Machine Learning meets Inpainting
Inpainting is a learned interpolation technique that is based on generative modeling and used to populate masked or missing pieces in an image; it has wide applications in picture editing and retouching.
Manipulating SGD with Data Ordering Attacks
Machine learning is vulnerable to a wide variety of attacks.
Nudge Attacks on Point-Cloud DNNs
The wide adaption of 3D point-cloud data in safety-critical applications such as autonomous driving makes adversarial samples a real threat.
Learned Low Precision Graph Neural Networks
LPGNAS learns the optimal architecture coupled with the best quantisation strategy for different components in the GNN automatically using back-propagation in a single search round.
Sponge Examples: Energy-Latency Attacks on Neural Networks
The high energy costs of neural network training and inference led to the use of acceleration hardware such as GPUs and TPUs.
Pay Attention to Features, Transfer Learn Faster CNNs
Deep convolutional neural networks are now widely deployed in vision applications, but a limited size of training data can restrict their task performance.
Probabilistic Dual Network Architecture Search on Graphs
We present the first differentiable Network Architecture Search (NAS) for Graph Neural Networks (GNNs).
Towards Certifiable Adversarial Sample Detection
Convolutional Neural Networks (CNNs) are deployed in more and more classification systems, but adversarial samples can be maliciously crafted to trick them, and are becoming a real threat.
Automatic Generation of Multi-precision Multi-arithmetic CNN Accelerators for FPGAs
Furthermore, we show how Tomato produces implementations of networks with various sizes running on single or multiple FPGAs.
Blackbox Attacks on Reinforcement Learning Agents Using Approximated Temporal Information
In this work, we show how such samples can be generalised from White-box and Grey-box attacks to a strong Black-box case, where the attacker has no knowledge of the agents, their training parameters and their training methods.
Focused Quantization for Sparse CNNs
In ResNet-50, we achieved a 18. 08x CR with only 0. 24% loss in top-5 accuracy, outperforming existing compression methods.
Sitatapatra: Blocking the Transfer of Adversarial Samples
Convolutional Neural Networks (CNNs) are widely used to solve classification tasks in computer vision.
The Taboo Trap: Behavioural Detection of Adversarial Samples
Most existing detection mechanisms against adversarial attacksimpose significant costs, either by using additional classifiers to spot adversarial samples, or by requiring the DNN to be restructured.
Dynamic Channel Pruning: Feature Boosting and Suppression
Making deep convolutional neural networks more accurate typically comes at the cost of increased computational and memory resources.
To compress or not to compress: Understanding the Interactions between Adversarial Attacks and Neural Network Compression
We, therefore, investigate the extent to which adversarial samples are transferable between uncompressed and compressed DNNs.
