Methods > Computer Vision > Image Model Blocks

Axial Attention

Introduced by Ho et al. in Axial Attention in Multidimensional Transformers

Axial attention is a simple generalization of self-attention that naturally aligns with the multiple dimensions of the tensors in both the encoding and the decoding settings. The proposed structure of the layers allows for the vast majority of the context to be computed in parallel during decoding without introducing any independence assumptions. It serves as the basic building block for developing self-attention-based autoregressive models for high-dimensional data tensors, e.g., Axial Transformers.

Source: Axial Attention in Multidimensional Transformers

Latest Papers

PAPER DATE
Axial-to-lateral super-resolution for 3D fluorescence microscopy using unsupervised deep learning
Hyoungjun ParkMyeongsu NaBumju KimSoohyun ParkKi Hean KimSunghoe ChangJong Chul Ye
2021-04-19
Population genetics in microchannels
Anzhelika KoldaevaHsieh-Fu TsaiAmy Q. ShenSimone Pigolotti
2021-04-09
Modelling and Analysis of Magnetic Fields from Skeletal Muscle for Valuable Physiological Measurements
Siming ZuoKianoush NazarpourDario FarinaPhilip BroserHadi Heidari
2021-04-05
Physical model simulator-trained neural network for computational 3D phase imaging of multiple-scattering samples
Alex MatlockLei Tian
2021-03-29
Motion Estimation for Optical Coherence Elastography Using Signal Phase and Intensity
Hossein KhodadadiOrcun GokselSabine Kling
2021-03-19
Unsupervised Missing Cone Deep Learning in Optical Diffraction Tomography
Hyungjin ChungJaeyoung HuhGeon KimYong Keun ParkJong Chul Ye
2021-03-16
A learning-based view extrapolation method for axial super-resolution
Zhaolin XiaoJinglei ShiXiaoran JiangChristine Guillemot
2021-03-11
Deep learning can differentiate IDH-mutant from IDH-wild type GBM
Luca PasquiniAntonio NapolitanoEmanuela TaglienteFrancesco DellepianeMartina LucignaniAntonello VidiriGiulio RanazziAntonella StoppacciaroGiulia MoltoniMatteo NicolaiAndrea RomanoAlberto Di NapoliAlessandro Bozzao
2021-02-24
Detection of Winding Axial Deformation in Power Transformers by UWB Radar Imaging
Mohammad S. GolsorkhiM. A. HejaziG. B. GharepetianRazieh Mosayebi
2021-02-21
Detection of Transformer Winding Axial Displacement by Kirchhoff and Delay and sum Radar Imaging Algorithms
Mohammad S. GolsorkhiR. MosayebiM. A. HejaziG. B. GharehpetianH. Sheikhzadeh
2021-02-21
Axial Residual Networks for CycleGAN-based Voice Conversion
Jaeseong YouGyuhyeon NamDalhyun KimGyeongsu Chae
2021-02-16
Unsupervised Domain Adaptation from Axial to Short-Axis Multi-Slice Cardiac MR Images by Incorporating Pretrained Task Networks
| Sven KoehlerTarique HussainZach BlairTyler HuffakerFlorian RitzmannAnimesh TandonThomas PickardtSamir SarikouchHeiner LatusGerald GreilIvo WolfSandy Engelhardt
2021-01-19
Channelized Axial Attention for Semantic Segmentation
| Ye HuangWenjing JiaXiangjian HeLiu LiuYuxin LiDaCheng Tao
2021-01-19
VideoGen: Generative Modeling of Videos using VQ-VAE and Transformers
Anonymous
2021-01-01
A Multi-View Dynamic Fusion Framework: How to Improve the Multimodal Brain Tumor Segmentation from Multi-Views?
Yi DingWei ZhengGuozheng WuJi GengMingsheng CaoZhiguang Qin
2020-12-21
Virtual Source Synthetic Aperture for Accurate Lateral Displacement Estimation in Ultrasound Elastography
Morteza MirzaeiAmir AsifHassan Rivaz
2020-12-19
Deep Dose Plugin Towards Real-time Monte Carlo Dose Calculation Through a Deep Learning based Denoising Algorithm
Ti BaiBiling WangDan NguyenSteve Jiang
2020-11-30
MRI Images Analysis Method for Early Stage Alzheimer's Disease Detection
Achraf Ben MiledTaoufik YefernyAmira ben Rabeh
2020-11-27
Accurate and Rapid Diagnosis of COVID-19 Pneumonia with Batch Effect Removal of Chest CT-Scans and Interpretable Artificial Intelligence
Rassa Ghavami ModeghMehrab HamidiSaeed MasoudianAmir MohseniHamzeh LotfalinezhadMohammad Ali KazemiBehnaz MoradiMahyar GhafooriOmid MotamediOmid PournikKiara Rezaei-KalantariAmirreza ManteghinezhadShaghayegh Haghjooy JavanmardFateme Abdoli NezhadAhmad EnhesariMohammad Saeed KheyrkhahRazieh EghtesadiJavid AzadbakhtAkbar AliasgharzadehMohammad Reza SharifAli KhaleghiAbbas ForoutanHossein GhanaatiHamed DashtiHamid R. Rabiee
2020-11-23
Deep Sequential Learning for Cervical Spine Fracture Detection on Computed Tomography Imaging
Hojjat SalehinejadEdward HoHui-Ming LinPriscila CrivellaroOleksandra SamorodovaMonica Tafur ArciniegasZamir MeraliSuradech SuthiphosuwanAditya BharathaKristen YeomMuhammad MamdaniJefferson WilsonErrol Colak
2020-10-26
Recurrent neural network-based volumetric fluorescence microscopy
Luzhe HuangYilin LuoYair RivensonAydogan Ozcan
2020-10-21
Efficient and high accuracy 3-D OCT angiography motion correction in pathology
Stefan B. PlonerMartin F. KrausEric M. MoultLennart HusvogtJulia SchottenhammlA. Yasin AlibhaiNadia K. WaheedJay S. DukerJames G. FujimotoAndreas K. Maier
2020-10-14
Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy
Kyrollos YannyNick AntipaWilliam LibertiSam DehaeckKristina MonakhovaFanglin Linda LiuKonlin ShenRen NgLaura Waller
2020-10-12
Learning an optimal PSF-pair for ultra-dense 3D localization microscopy
Elias NehmeBoris FerdmanLucien E. WeissTal NaorDaniel FreedmanTomer MichaeliYoav Shechtman
2020-09-29
Anisotropic 3D Multi-Stream CNN for Accurate Prostate Segmentation from Multi-Planar MRI
| Anneke MeyerGrzegorz ChlebusMarko RakDaniel SchindeleMartin SchostakBram van GinnekenAndrea SchenkHans MeineHorst K. HahnAndreas SchreiberChristian Hansen
2020-09-23
Reconstruct high-resolution multi-focal plane images from a single 2D wide field image
Jiabo MaSibo LiuShenghua ChengXiuli LiuLi ChengShaoqun Zeng
2020-09-21
Improving axial resolution in SIM using deep learning
| Miguel BolandEdward A. K. CohenSeth FlaxmanMark A. A. Neil
2020-09-04
Reinforced Axial Refinement Network for Monocular 3D Object Detection
Lijie LiuChufan WuJiwen LuLingxi XieJie zhouQi Tian
2020-08-31
Comparative performance analysis of the ResNet backbones of Mask RCNN to segment the signs of COVID-19 in chest CT scans
Muhammad AleemRahul RajArshad Khan
2020-08-21
Two-Stage Deep Learning for Accelerated 3D Time-of-Flight MRA without Matched Training Data
Hyungjin ChungEunju ChaLeonard SunwooJong Chul Ye
2020-08-04
Polarized Optical-Flow Gyroscope
Masada TzabariYoav Y. Schechner
2020-08-01
MADGAN: unsupervised Medical Anomaly Detection GAN using multiple adjacent brain MRI slice reconstruction
Changhee HanLeonardo RundoKohei MuraoTomoyuki NoguchiYuki ShimaharaZoltan Adam MilacskiSaori KoshinoEvis SalaHideki NakayamaShinichi Satoh
2020-07-24
Multi-Subspace Neural Network for Image Recognition
Chieh-Ning FangChin-Teng Lin
2020-06-17
MetNet: A Neural Weather Model for Precipitation Forecasting
Casper Kaae SønderbyLasse EspeholtJonathan HeekMostafa DehghaniAvital OliverTim SalimansShreya AgrawalJason HickeyNal Kalchbrenner
2020-03-24
Single-shot autofocusing of microscopy images using deep learning
Yilin LuoLuzhe HuangYair RivensonAydogan Ozcan
2020-03-21
A Deep learning Approach to Generate Contrast-Enhanced Computerised Tomography Angiography without the Use of Intravenous Contrast Agents
Anirudh ChandrashekarAshok HandaNatesh ShivakumarPierfrancesco LapollaVicente GrauRegent Lee
2020-03-02
Fully-automated Body Composition Analysis in Routine CT Imaging Using 3D Semantic Segmentation Convolutional Neural Networks
Sven KoitkaLennard KrollEugen MalamutmannArzu OezcelikFelix Nensa
2020-02-25
Automatic Frame Selection using CNN in Ultrasound Elastography
Abdelrahman ZayedGuy CloutierHassan Rivaz
2020-02-17
Fast and robust multiplane single molecule localization microscopy using deep neural network
Toshimitsu AritakeHideitsu HinoShigeyuki NamikiDaisuke AsanumaKenzo HiroseNoboru Murata
2020-01-07
DeepFocus: a Few-Shot Microscope Slide Auto-Focus using a Sample Invariant CNN-based Sharpness Function
| Adrian ShajkofciMichael Liebling
2020-01-02
Improved Spectral Imaging Microscopy for Cultural Heritage through Oblique Illumination
Lindsay OakleyStephanie ZaleskiBillie MalesOllie CossairtMarc Walton
2020-01-01
Volumetric Lung Nodule Segmentation using Adaptive ROI with Multi-View Residual Learning
Muhammad UsmanByoung-Dai LeeShi Sub ByonSung Hyun KimByung-ilLee
2019-12-31
Axial Attention in Multidimensional Transformers
| Jonathan HoNal KalchbrennerDirk WeissenbornTim Salimans
2019-12-20

Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories