Search Results for author: Xiao Xiang Zhu

Found 145 papers, 56 papers with code

True or False? Detecting False Information on Social Media Using Graph Neural Networks

no code implementations COLING (WNUT) 2022 Samyo Rode-Hasinger, Anna Kruspe, Xiao Xiang Zhu

In recent years, false information such as fake news, rumors and conspiracy theories on many relevant issues in society have proliferated.

Misinformation

MaskCD: A Remote Sensing Change Detection Network Based on Mask Classification

1 code implementation18 Apr 2024 Weikang Yu, Xiaokang Zhang, Samiran Das, Xiao Xiang Zhu, Pedram Ghamisi

Change detection (CD) from remote sensing (RS) images using deep learning has been widely investigated in the literature.

Large-scale flood modeling and forecasting with FloodCast

1 code implementation18 Mar 2024 Qingsong Xu, Yilei Shi, Jonathan Bamber, Chaojun Ouyang, Xiao Xiang Zhu

Large-scale hydrodynamic models generally rely on fixed-resolution spatial grids and model parameters as well as incurring a high computational cost.

Change Detection

Causal Graph Neural Networks for Wildfire Danger Prediction

no code implementations13 Mar 2024 Shan Zhao, Ioannis Prapas, Ilektra Karasante, Zhitong Xiong, Ioannis Papoutsis, Gustau Camps-Valls, Xiao Xiang Zhu

In that direction, we propose integrating causality with Graph Neural Networks (GNNs) that explicitly model the causal mechanism among complex variables via graph learning.

Decision Making Graph Learning

AIO2: Online Correction of Object Labels for Deep Learning with Incomplete Annotation in Remote Sensing Image Segmentation

1 code implementation3 Mar 2024 Chenying Liu, Conrad M Albrecht, Yi Wang, Qingyu Li, Xiao Xiang Zhu

AIO2 utilizes a mean teacher model to enhance training robustness with noisy labels to both stabilize the training accuracy curve for fitting in ACT and provide pseudo labels for correction in O2C.

Earth Observation Image Segmentation +1

Task Specific Pretraining with Noisy Labels for Remote sensing Image Segmentation

no code implementations25 Feb 2024 Chenying Liu, Conrad Albrecht, Yi Wang, Xiao Xiang Zhu

In this work, we propose to explore the under-exploited potential of noisy labels for segmentation task specific pretraining, and exam its robustness when confronted with mismatched categories and different decoders during fine-tuning.

Image Segmentation Segmentation +1

Opening the Black-Box: A Systematic Review on Explainable AI in Remote Sensing

no code implementations21 Feb 2024 Adrian Höhl, Ivica Obadic, Miguel Ángel Fernández Torres, Hiba Najjar, Dario Oliveira, Zeynep Akata, Andreas Dengel, Xiao Xiang Zhu

In recent years, black-box machine learning approaches have become a dominant modeling paradigm for knowledge extraction in Remote Sensing.

ChatEarthNet: A Global-Scale Image-Text Dataset Empowering Vision-Language Geo-Foundation Models

no code implementations17 Feb 2024 Zhenghang Yuan, Zhitong Xiong, Lichao Mou, Xiao Xiang Zhu

In this context, we introduce a global-scale, high-quality image-text dataset for remote sensing, providing natural language descriptions for Sentinel-2 data to facilitate the understanding of satellite imagery for common users.

Earth Observation Semantic Segmentation

Efficient Subseasonal Weather Forecast using Teleconnection-informed Transformers

no code implementations31 Jan 2024 Shan Zhao, Zhitong Xiong, Xiao Xiang Zhu

Subseasonal forecasting, which is pivotal for agriculture, water resource management, and early warning of disasters, faces challenges due to the chaotic nature of the atmosphere.

Weather Forecasting

PixelDINO: Semi-Supervised Semantic Segmentation for Detecting Permafrost Disturbances

1 code implementation17 Jan 2024 Konrad Heidler, Ingmar Nitze, Guido Grosse, Xiao Xiang Zhu

To improve model generalization across the Arctic without the need for additional labelled data, we present a semi-supervised learning approach to train semantic segmentation models to detect RTS.

Semi-Supervised Semantic Segmentation

One for All: Toward Unified Foundation Models for Earth Vision

no code implementations15 Jan 2024 Zhitong Xiong, Yi Wang, Fahong Zhang, Xiao Xiang Zhu

Current remote sensing foundation models typically specialize in a single modality or a specific spatial resolution range, limiting their versatility for downstream datasets.

Towards Label Embedding -- Measuring classification difficulty

no code implementations15 Nov 2023 Katharina Hechinger, Christoph Koller, Xiao Xiang Zhu, Göran Kauermann

In supervised learning, uncertainty can already occur in the very first stage of the training process, the labelling step.

Classification Image Classification +1

Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing

1 code implementation28 Oct 2023 Yi Wang, Hugo Hernández Hernández, Conrad M Albrecht, Xiao Xiang Zhu

Self-supervised learning guided by masked image modelling, such as Masked AutoEncoder (MAE), has attracted wide attention for pretraining vision transformers in remote sensing.

Multi-Label Image Classification Self-Supervised Learning

Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology

no code implementations8 Oct 2023 Qingsong Xu, Yilei Shi, Jonathan Bamber, Ye Tuo, Ralf Ludwig, Xiao Xiang Zhu

Specifically, we present a comprehensive review of the physics-aware ML methods, building a structured community (PaML) of existing methodologies that integrate prior physical knowledge or physics-based modeling into ML.

HyperLISTA-ABT: An Ultra-light Unfolded Network for Accurate Multi-component Differential Tomographic SAR Inversion

no code implementations28 Sep 2023 Kun Qian, Yuanyuan Wang, Peter Jung, Yilei Shi, Xiao Xiang Zhu

Deep neural networks based on unrolled iterative algorithms have achieved remarkable success in sparse reconstruction applications, such as synthetic aperture radar (SAR) tomographic inversion (TomoSAR).

3D Reconstruction Computational Efficiency

HTC-DC Net: Monocular Height Estimation from Single Remote Sensing Images

1 code implementation28 Sep 2023 Sining Chen, Yilei Shi, Zhitong Xiong, Xiao Xiang Zhu

To tackle this problem, we propose a method for monocular height estimation from optical imagery, which is currently one of the richest sources of remote sensing data.

regression

Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks

no code implementations26 Sep 2023 Danfeng Hong, Bing Zhang, Hao Li, YuXuan Li, Jing Yao, Chenyu Li, Martin Werner, Jocelyn Chanussot, Alexander Zipf, Xiao Xiang Zhu

Artificial intelligence (AI) approaches nowadays have gained remarkable success in single-modality-dominated remote sensing (RS) applications, especially with an emphasis on individual urban environments (e. g., single cities or regions).

Domain Adaptation Segmentation +1

Self-supervised Domain-agnostic Domain Adaptation for Satellite Images

no code implementations20 Sep 2023 Fahong Zhang, Yilei Shi, Xiao Xiang Zhu

A promising method to address this problem is domain adaptation, where the training and the testing datasets are split into two or multiple domains according to their distributions, and an adaptation method is applied to improve the generalizability of the model on the testing dataset.

Domain Adaptation Image-to-Image Translation

Few-shot Object Detection in Remote Sensing: Lifting the Curse of Incompletely Annotated Novel Objects

1 code implementation19 Sep 2023 Fahong Zhang, Yilei Shi, Zhitong Xiong, Xiao Xiang Zhu

In this context, few-shot object detection (FSOD) has emerged as a promising direction, which aims at enabling the model to detect novel objects with only few of them annotated.

Few-Shot Object Detection object-detection +1

Exploring Geometric Deep Learning For Precipitation Nowcasting

no code implementations11 Sep 2023 Shan Zhao, Sudipan Saha, Zhitong Xiong, Niklas Boers, Xiao Xiang Zhu

Motivated by this, we explore a geometric deep learning-based temporal Graph Convolutional Network (GCN) for precipitation nowcasting.

Deep Semantic Model Fusion for Ancient Agricultural Terrace Detection

1 code implementation4 Aug 2023 Yi Wang, Chenying Liu, Arti Tiwari, Micha Silver, Arnon Karnieli, Xiao Xiang Zhu, Conrad M Albrecht

Discovering ancient agricultural terraces in desert regions is important for the monitoring of long-term climate changes on the Earth's surface.

Segmentation Semantic Segmentation

UCDFormer: Unsupervised Change Detection Using a Transformer-driven Image Translation

1 code implementation2 Aug 2023 Qingsong Xu, Yilei Shi, Jianhua Guo, Chaojun Ouyang, Xiao Xiang Zhu

Specifically, a transformer-driven image translation composed of a light-weight transformer and a domain-specific affinity weight is first proposed to mitigate domain shift between two images with real-time efficiency.

Change Detection Translation

A Deep Active Contour Model for Delineating Glacier Calving Fronts

no code implementations7 Jul 2023 Konrad Heidler, Lichao Mou, Erik Loebel, Mirko Scheinert, Sébastien Lefèvre, Xiao Xiang Zhu

Building on this observation, we completely rephrase the task as a contour tracing problem and propose a model for explicit contour detection that does not incorporate any dense predictions as intermediate steps.

Contour Detection Edge Detection +2

Semi-Supervised Learning for hyperspectral images by non parametrically predicting view assignment

no code implementations19 Jun 2023 Shivam Pande, Nassim Ait Ali Braham, Yi Wang, Conrad M Albrecht, Biplab Banerjee, Xiao Xiang Zhu

Recently, to effectively train the deep learning models with minimal labelled samples, the unlabeled samples are also being leveraged in self-supervised and semi-supervised setting.

Pseudo Label

RRSIS: Referring Remote Sensing Image Segmentation

no code implementations14 Jun 2023 Zhenghang Yuan, Lichao Mou, Yuansheng Hua, Xiao Xiang Zhu

Localizing desired objects from remote sensing images is of great use in practical applications.

Benchmarking Image Segmentation +2

GEO-Bench: Toward Foundation Models for Earth Monitoring

1 code implementation NeurIPS 2023 Alexandre Lacoste, Nils Lehmann, Pau Rodriguez, Evan David Sherwin, Hannah Kerner, Björn Lütjens, Jeremy Andrew Irvin, David Dao, Hamed Alemohammad, Alexandre Drouin, Mehmet Gunturkun, Gabriel Huang, David Vazquez, Dava Newman, Yoshua Bengio, Stefano Ermon, Xiao Xiang Zhu

Recent progress in self-supervision has shown that pre-training large neural networks on vast amounts of unsupervised data can lead to substantial increases in generalization to downstream tasks.

GAMUS: A Geometry-aware Multi-modal Semantic Segmentation Benchmark for Remote Sensing Data

1 code implementation24 May 2023 Zhitong Xiong, Sining Chen, Yi Wang, Lichao Mou, Xiao Xiang Zhu

Towards a fair and comprehensive analysis of existing methods, the proposed benchmark consists of 1) a large-scale dataset including co-registered RGB and nDSM pairs and pixel-wise semantic labels; 2) a comprehensive evaluation and analysis of existing multi-modal fusion strategies for both convolutional and Transformer-based networks on remote sensing data.

Segmentation Semantic Segmentation

Basis Pursuit Denoising via Recurrent Neural Network Applied to Super-resolving SAR Tomography

no code implementations23 May 2023 Kun Qian, Yuanyuan Wang, Peter Jung, Yilei Shi, Xiao Xiang Zhu

An emerging technique known as deep unrolling provided a good combination of the descriptive ability of neural networks, explainable, and computational efficiency for BPDN.

Computational Efficiency Denoising +3

Vision-Language Models in Remote Sensing: Current Progress and Future Trends

2 code implementations9 May 2023 Xiang Li, Congcong Wen, Yuan Hu, Zhenghang Yuan, Xiao Xiang Zhu

Existing AI-related research in remote sensing primarily focuses on visual understanding tasks while neglecting the semantic understanding of the objects and their relationships.

Image Captioning Image Generation +8

High Quality Large-Scale 3-D Urban Mapping with Multi-Master TomoSAR

no code implementations8 May 2023 Yilei Shi, Richard Bamler, Yuanyuan Wang, Xiao Xiang Zhu

Multi-baseline interferometric synthetic aperture radar (InSAR) techniques are effective approaches for retrieving the 3-D information of urban areas.

Deep Unsupervised Learning for 3D ALS Point Cloud Change Detection

1 code implementation5 May 2023 Iris de Gélis, Sudipan Saha, Muhammad Shahzad, Thomas Corpetti, Sébastien Lefèvre, Xiao Xiang Zhu

To circumnavigate this dependence, we propose an unsupervised 3D point cloud change detection method mainly based on self-supervised learning using deep clustering and contrastive learning.

Change Detection Contrastive Learning +2

Multilingual Augmentation for Robust Visual Question Answering in Remote Sensing Images

no code implementations7 Apr 2023 Zhenghang Yuan, Lichao Mou, Xiao Xiang Zhu

With the proposed augmented dataset, we are able to obtain more questions in addition to the original ones with the same meaning.

Contrastive Learning Question Answering +1

Universal Domain Adaptation for Remote Sensing Image Scene Classification

1 code implementation26 Jan 2023 Qingsong Xu, Yilei Shi, Xin Yuan, Xiao Xiang Zhu

Empirical results show that the proposed model is effective and practical for remote sensing image scene classification, regardless of whether the source data is available or not.

Classification Image Classification +3

High-Resolution Cloud Removal with Multi-Modal and Multi-Resolution Data Fusion: A New Baseline and Benchmark

1 code implementation9 Jan 2023 Fang Xu, Yilei Shi, Patrick Ebel, Wen Yang, Xiao Xiang Zhu

In this paper, we introduce Planet-CR, a benchmark dataset for high-resolution cloud removal with multi-modal and multi-resolution data fusion.

Cloud Removal

Geo-Information Harvesting from Social Media Data

no code implementations1 Nov 2022 Xiao Xiang Zhu, Yuanyuan Wang, Mrinalini Kochupillai, Martin Werner, Matthias Häberle, Eike Jens Hoffmann, Hannes Taubenböck, Devis Tuia, Alex Levering, Nathan Jacobs, Anna Kruspe, Karam Abdulahhad

In this article, we address key aspects in the field, including data availability, analysis-ready data preparation and data management, geo-information extraction from social media text messages and images, and the fusion of social media and remote sensing data.

Management

Exploring Self-Attention for Crop-type Classification Explainability

no code implementations24 Oct 2022 Ivica Obadic, Ribana Roscher, Dario Augusto Borges Oliveira, Xiao Xiang Zhu

Using explainable machine learning to reveal the inner workings of these models is an important step towards improving stakeholders' trust and efficient agriculture monitoring.

Classification Time Series Analysis +1

Automatic separation of laminar-turbulent flows on aircraft wings and stabilisers via adaptive attention butterfly network

1 code implementation Experiments in Fluids 2022 Rıdvan Salih Kuzu, Philipp Mühlmann, Xiao Xiang Zhu

Many of the laminar-turbulent flow localisation techniques are strongly dependent upon expert control even-though determining the flow distribution is the prerequisite for analysing the efficiency of wing & stabiliser design in aeronautics.

Laminar-Turbulent Flow Localisation Self-Supervised Learning

EarthNets: Empowering AI in Earth Observation

no code implementations10 Oct 2022 Zhitong Xiong, Fahong Zhang, Yi Wang, Yilei Shi, Xiao Xiang Zhu

Furthermore, a new platform for EO, termed EarthNets, is released to achieve a fair and consistent evaluation of deep learning methods on remote sensing data.

Earth Observation Scene Understanding +1

Anomaly Detection in Aerial Videos with Transformers

1 code implementation25 Sep 2022 Pu Jin, Lichao Mou, Gui-Song Xia, Xiao Xiang Zhu

In this paper, we create a new dataset, named DroneAnomaly, for anomaly detection in aerial videos.

Anomaly Detection

FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification

no code implementations22 Sep 2022 Pu Jin, Lichao Mou, Yuansheng Hua, Gui-Song Xia, Xiao Xiang Zhu

Furthermore, the holistic features are refined by the multi-scale temporal relations in a novel fusion module for yielding more discriminative video representations.

Action Recognition Temporal Action Localization +1

Enabling Country-Scale Land Cover Mapping with Meter-Resolution Satellite Imagery

1 code implementation1 Sep 2022 Xin-Yi Tong, Gui-Song Xia, Xiao Xiang Zhu

To validate the generalizability of our dataset and the proposed approach across different sensors and different geographical regions, we carry out land cover mapping on five megacities in China and six cities in other five Asian countries severally using: PlanetScope (3 m), Gaofen-1 (8 m), and Sentinel-2 (10 m) satellite images.

Land Cover Classification Pseudo Label +5

Doubly Deformable Aggregation of Covariance Matrices for Few-shot Segmentation

1 code implementation30 Jul 2022 Zhitong Xiong, Haopeng Li, Xiao Xiang Zhu

To address this problem, we propose to aggregate the learnable covariance matrices with a deformable 4D Transformer to effectively predict the segmentation map.

Few-Shot Semantic Segmentation Segmentation +2

Self-supervised Learning in Remote Sensing: A Review

2 code implementations27 Jun 2022 Yi Wang, Conrad M Albrecht, Nassim Ait Ali Braham, Lichao Mou, Xiao Xiang Zhu

In deep learning research, self-supervised learning (SSL) has received great attention triggering interest within both the computer vision and remote sensing communities.

Earth Observation Multi-Label Image Classification +1

Monitoring Urban Forests from Auto-Generated Segmentation Maps

no code implementations14 Jun 2022 Conrad M Albrecht, Chenying Liu, Yi Wang, Levente Klein, Xiao Xiang Zhu

We present and evaluate a weakly-supervised methodology to quantify the spatio-temporal distribution of urban forests based on remotely sensed data with close-to-zero human interaction.

Semantic Segmentation

GLF-CR: SAR-Enhanced Cloud Removal with Global-Local Fusion

1 code implementation6 Jun 2022 Fang Xu, Yilei Shi, Patrick Ebel, Lei Yu, Gui-Song Xia, Wen Yang, Xiao Xiang Zhu

The challenge of the cloud removal task can be alleviated with the aid of Synthetic Aperture Radar (SAR) images that can penetrate cloud cover.

Cloud Removal

Semi-Supervised Building Footprint Generation with Feature and Output Consistency Training

no code implementations17 May 2022 Qingyu Li, Yilei Shi, Xiao Xiang Zhu

Considering that rich information is also encoded in feature maps, we propose to integrate the consistency of both features and outputs in the end-to-end network training of unlabeled samples, enabling to impose additional constraints.

Semi-Supervised Semantic Segmentation

Self-supervised Vision Transformers for Joint SAR-optical Representation Learning

2 code implementations11 Apr 2022 Yi Wang, Conrad M Albrecht, Xiao Xiang Zhu

Experimental results employing the BigEarthNet-MM dataset demonstrate the benefits of both, the ViT backbones and the proposed multimodal SSL algorithm DINO-MM.

Data Augmentation Earth Observation +2

So2Sat POP -- A Curated Benchmark Data Set for Population Estimation from Space on a Continental Scale

1 code implementation7 Apr 2022 Sugandha Doda, Yuanyuan Wang, Matthias Kahl, Eike Jens Hoffmann, Kim Ouan, Hannes Taubenböck, Xiao Xiang Zhu

Obtaining a dynamic population distribution is key to many decision-making processes such as urban planning, disaster management and most importantly helping the government to better allocate socio-technical supply.

BIG-bench Machine Learning Decision Making +1

Using Social Media Images for Building Function Classification

no code implementations15 Feb 2022 Eike Jens Hoffmann, Karam Abdulahhad, Xiao Xiang Zhu

To cope with this issue this study proposes a filtering pipeline to yield high quality, ground level imagery from large social media image datasets.

Classification

Landscape of Neural Architecture Search across sensors: how much do they differ ?

no code implementations17 Jan 2022 Kalifou René Traoré, Andrés Camero, Xiao Xiang Zhu

With the rapid rise of neural architecture search, the ability to understand its complexity from the perspective of a search algorithm is desirable.

Image Classification Neural Architecture Search

Disentangled Latent Transformer for Interpretable Monocular Height Estimation

1 code implementation17 Jan 2022 Zhitong Xiong, Sining Chen, Yilei Shi, Xiao Xiang Zhu

Furthermore, a novel unsupervised semantic segmentation task based on height estimation is first introduced in this work.

Unsupervised Semantic Segmentation

THE Benchmark: Transferable Representation Learning for Monocular Height Estimation

no code implementations30 Dec 2021 Zhitong Xiong, Wei Huang, Jingtao Hu, Xiao Xiang Zhu

Therefore, we propose a new benchmark dataset to study the transferability of height estimation models in a cross-dataset setting.

Representation Learning Transfer Learning

$\boldsymbolγ$-Net: Superresolving SAR Tomographic Inversion via Deep Learning

no code implementations8 Dec 2021 Kun Qian, Yuanyuan Wang, Yilei Shi, Xiao Xiang Zhu

This superior performance comes at the cost of extra computational burdens, because of the sparse reconstruction, which cannot be solved analytically and we need to employ computationally expensive iterative solvers.

Compressive Sensing Computational Efficiency +1

Large-scale Building Height Retrieval from Single SAR Imagery based on Bounding Box Regression Networks

no code implementations18 Nov 2021 Yao Sun, Lichao Mou, Yuanyuan Wang, Sina Montazeri, Xiao Xiang Zhu

Building height retrieval from synthetic aperture radar (SAR) imagery is of great importance for urban applications, yet highly challenging owing to the complexity of SAR data.

regression Retrieval

A Data-driven Approach to Neural Architecture Search Initialization

1 code implementation5 Nov 2021 Kalifou René Traoré, Andrés Camero, Xiao Xiang Zhu

First, we perform a calibrated clustering analysis of the search space, and second, we extract the centroids and use them to initialize a NAS algorithm.

Neural Architecture Search

SurroundNet: Towards Effective Low-Light Image Enhancement

1 code implementation11 Oct 2021 Fei Zhou, Xin Sun, Junyu Dong, Haoran Zhao, Xiao Xiang Zhu

Although Convolution Neural Networks (CNNs) has made substantial progress in the low-light image enhancement task, one critical problem of CNNs is the paradox of model complexity and performance.

Low-Light Image Enhancement

Spatial Context Awareness for Unsupervised Change Detection in Optical Satellite Images

no code implementations5 Oct 2021 Lukas Kondmann, Aysim Toker, Sudipan Saha, Bernhard Schölkopf, Laura Leal-Taixé, Xiao Xiang Zhu

It uses this model to analyze differences in the pixel and its spatial context-based predictions in subsequent time periods for change detection.

Change Detection Earth Observation

Reiterative Domain Aware Multi-Target Adaptation

no code implementations26 Aug 2021 Sudipan Saha, Shan Zhao, Nasrullah Sheikh, Xiao Xiang Zhu

Multi-target domain adaptation is a powerful extension in which a single classifier is learned for multiple unlabeled target domains.

Domain Adaptation Multi-target Domain Adaptation

Lessons from the Clustering Analysis of a Search Space: A Centroid-based Approach to Initializing NAS

no code implementations20 Aug 2021 Kalifou Rene Traore, Andrés Camero, Xiao Xiang Zhu

In this study, we propose to accelerate a NAS algorithm using a data-driven initialization technique, leveraging the availability of NAS benchmarks.

Neural Architecture Search

Under the Radar -- Auditing Fairness in ML for Humanitarian Mapping

no code implementations4 Aug 2021 Lukas Kondmann, Xiao Xiang Zhu

Our findings indicate that poverty is systematically overestimated and electricity systematically underestimated for scheduled tribes in comparison to a synthetic counterfactual group of villages.

counterfactual Decision Making +2

Self-supervised Audiovisual Representation Learning for Remote Sensing Data

1 code implementation2 Aug 2021 Konrad Heidler, Lichao Mou, Di Hu, Pu Jin, Guangyao Li, Chuang Gan, Ji-Rong Wen, Xiao Xiang Zhu

By fine-tuning the models on a number of commonly used remote sensing datasets, we show that our approach outperforms existing pre-training strategies for remote sensing imagery.

Cross-Modal Retrieval Representation Learning +1

Segmentation of VHR EO Images using Unsupervised Learning

no code implementations9 Jul 2021 Sudipan Saha, Lichao Mou, Muhammad Shahzad, Xiao Xiang Zhu

The proposed method exploits this property to sample smaller patches from the larger scene and uses deep clustering and contrastive learning to refine the weights of a lightweight deep model composed of a series of the convolution layers along with an embedded channel attention.

Contrastive Learning Deep Clustering +3

A Survey of Uncertainty in Deep Neural Networks

no code implementations7 Jul 2021 Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, JongSeok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, Muhammad Shahzad, Wen Yang, Richard Bamler, Xiao Xiang Zhu

Different examples from the wide spectrum of challenges in different fields give an idea of the needs and challenges regarding uncertainties in practical applications.

Data Augmentation

Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Model

1 code implementation21 May 2021 Danfeng Hong, Jingliang Hu, Jing Yao, Jocelyn Chanussot, Xiao Xiang Zhu

Moreover, to better assess multimodal baselines and the newly-proposed S2FL model, three multimodal RS benchmark datasets, i. e., Houston2013 -- hyperspectral and multispectral data, Berlin -- hyperspectral and synthetic aperture radar (SAR) data, Augsburg -- hyperspectral, SAR, and digital surface model (DSM) data, are released and used for land cover classification.

Land Cover Classification

Aerial Scene Understanding in The Wild: Multi-Scene Recognition via Prototype-based Memory Networks

1 code implementation22 Apr 2021 Yuansheng Hua, Lichao Moua, Jianzhe Lin, Konrad Heidler, Xiao Xiang Zhu

To be more specific, we first learn the prototype representation of each aerial scene from single-scene aerial image datasets and store it in an external memory.

Retrieval Scene Recognition +1

Towards a Collective Agenda on AI for Earth Science Data Analysis

1 code implementation11 Apr 2021 Devis Tuia, Ribana Roscher, Jan Dirk Wegner, Nathan Jacobs, Xiao Xiang Zhu, Gustau Camps-Valls

In the last years we have witnessed the fields of geosciences and remote sensing and artificial intelligence to become closer.

Position

Trusting small training dataset for supervised change detection

no code implementations9 Apr 2021 Sudipan Saha, Biplab Banerjee, Xiao Xiang Zhu

Deep learning (DL) based supervised change detection (CD) models require large labeled training data.

Change Detection

Out-of-distribution detection in satellite image classification

no code implementations9 Apr 2021 Jakob Gawlikowski, Sudipan Saha, Anna Kruspe, Xiao Xiang Zhu

In satellite image analysis, distributional mismatch between the training and test data may arise due to several reasons, including unseen classes in the test data and differences in the geographic area.

Classification General Classification +2

MultiScene: A Large-scale Dataset and Benchmark for Multi-scene Recognition in Single Aerial Images

1 code implementation7 Apr 2021 Yuansheng Hua, Lichao Mou, Pu Jin, Xiao Xiang Zhu

We conduct experiments with extensive baseline models on both MultiScene-Clean and MultiScene to offer benchmarks for multi-scene recognition in single images and learning from noisy labels for this task, respectively.

Learning with noisy labels Scene Recognition

Deep Reinforcement Learning for Band Selection in Hyperspectral Image Classification

1 code implementation15 Mar 2021 Lichao Mou, Sudipan Saha, Yuansheng Hua, Francesca Bovolo, Lorenzo Bruzzone, Xiao Xiang Zhu

To this end, we frame the problem of unsupervised band selection as a Markov decision process, propose an effective method to parameterize it, and finally solve the problem by deep reinforcement learning.

Classification General Classification +4

Interpretable Hyperspectral AI: When Non-Convex Modeling meets Hyperspectral Remote Sensing

no code implementations2 Mar 2021 Danfeng Hong, wei he, Naoto Yokoya, Jing Yao, Lianru Gao, Liangpei Zhang, Jocelyn Chanussot, Xiao Xiang Zhu

Hyperspectral imaging, also known as image spectrometry, is a landmark technique in geoscience and remote sensing (RS).

Adversarial Shape Learning for Building Extraction in VHR Remote Sensing Images

1 code implementation22 Feb 2021 Lei Ding, Hao Tang, Yahui Liu, Yilei Shi, Xiao Xiang Zhu, Lorenzo Bruzzone

To address this issue, we propose an adversarial shape learning network (ASLNet) to model the building shape patterns that improve the accuracy of building segmentation.

Segmentation

Self-Supervised Multisensor Change Detection

no code implementations12 Feb 2021 Sudipan Saha, Patrick Ebel, Xiao Xiang Zhu

In particular, we are interested in the combination of the images acquired by optical and Synthetic Aperture Radar (SAR) sensors.

BIG-bench Machine Learning Change Detection +3

Towards Bridging the gap between Empirical and Certified Robustness against Adversarial Examples

no code implementations9 Feb 2021 Jay Nandy, Sudipan Saha, Wynne Hsu, Mong Li Lee, Xiao Xiang Zhu

In this paper, we propose a novel method, called \emph{Certification through Adaptation}, that transforms an AT model into a randomized smoothing classifier during inference to provide certified robustness for $\ell_2$ norm without affecting their empirical robustness against adversarial attacks.

Adversarial Robustness

Semantic Segmentation of Remote Sensing Images with Sparse Annotations

1 code implementation10 Jan 2021 Yuansheng Hua, Diego Marcos, Lichao Mou, Xiao Xiang Zhu, Devis Tuia

Training Convolutional Neural Networks (CNNs) for very high resolution images requires a large quantity of high-quality pixel-level annotations, which is extremely labor- and time-consuming to produce.

Semantic Segmentation

Exploring Deep 3D Spatial Encodings for Large-Scale 3D Scene Understanding

no code implementations29 Nov 2020 Saqib Ali Khan, Yilei Shi, Muhammad Shahzad, Xiao Xiang Zhu

In this letter, we have proposed an alternative approach to overcome the limitations of CNN based approaches by encoding the spatial features of raw 3D point clouds into undirected symmetrical graph models.

Scene Understanding Semantic Segmentation

Multi-task Learning for Human Settlement Extent Regression and Local Climate Zone Classification

no code implementations23 Nov 2020 Chunping Qiu, Lukas Liebel, Lloyd H. Hughes, Michael Schmitt, Marco Körner, Xiao Xiang Zhu

Human Settlement Extent (HSE) and Local Climate Zone (LCZ) maps are both essential sources, e. g., for sustainable urban development and Urban Heat Island (UHI) studies.

Classification General Classification +2

CG-Net: Conditional GIS-aware Network for Individual Building Segmentation in VHR SAR Images

no code implementations17 Nov 2020 Yao Sun, Yuansheng Hua, Lichao Mou, Xiao Xiang Zhu

Object retrieval and reconstruction from very high resolution (VHR) synthetic aperture radar (SAR) images are of great importance for urban SAR applications, yet highly challenging owing to the complexity of SAR data.

Retrieval Segmentation

A Novel Actor Dual-Critic Model for Remote Sensing Image Captioning

no code implementations5 Oct 2020 Ruchika Chavhan, Biplab Banerjee, Xiao Xiang Zhu, Subhasis Chaudhuri

We deal with the problem of generating textual captions from optical remote sensing (RS) images using the notion of deep reinforcement learning.

Image Captioning Sentence

Joint and Progressive Subspace Analysis (JPSA) with Spatial-Spectral Manifold Alignment for Semi-Supervised Hyperspectral Dimensionality Reduction

1 code implementation21 Sep 2020 Danfeng Hong, Naoto Yokoya, Jocelyn Chanussot, Jian Xu, Xiao Xiang Zhu

Conventional nonlinear subspace learning techniques (e. g., manifold learning) usually introduce some drawbacks in explainability (explicit mapping) and cost-effectiveness (linearization), generalization capability (out-of-sample), and representability (spatial-spectral discrimination).

Dimensionality Reduction

SAR Tomography via Nonlinear Blind Scatterer Separation

no code implementations2 Sep 2020 Yuanyuan Wang, Xiao Xiang Zhu

Layover separation has been fundamental to many synthetic aperture radar applications, such as building reconstruction and biomass estimation.

Cross-language sentiment analysis of European Twitter messages duringthe COVID-19 pandemic

no code implementations27 Aug 2020 Anna Kruspe, Matthias Häberle, Iona Kuhn, Xiao Xiang Zhu

We separate the results by country of origin, and correlate their temporal development with events in those countries.

Sentence Sentence Embeddings +1

Generative Adversarial Networks for Synthesizing InSAR Patches

no code implementations3 Aug 2020 Philipp Sibler, Yuanyuan Wang, Stefan Auer, Mohsin Ali, Xiao Xiang Zhu

Generative Adversarial Networks (GANs) have been employed with certain success for image translation tasks between optical and real-valued SAR intensity imagery.

Translation

Spatial-Spectral Manifold Embedding of Hyperspectral Data

no code implementations17 Jul 2020 Danfeng Hong, Jing Yao, Xin Wu, Jocelyn Chanussot, Xiao Xiang Zhu

In recent years, hyperspectral imaging, also known as imaging spectroscopy, has been paid an increasing interest in geoscience and remote sensing community.

Cross-language sentiment analysis of European Twitter messages during the COVID-19 pandemic

no code implementations ACL 2020 Anna Kruspe, Matthias H{\"a}berle, Iona Kuhn, Xiao Xiang Zhu

In this paper, we analyze Twitter messages (tweets) collected during the first months of the COVID-19 pandemic in Europe with regard to their sentiment.

Sentence Sentence Embeddings +1

Single-Look Multi-Master SAR Tomography: An Introduction

no code implementations29 Jun 2020 Nan Ge, Richard Bamler, Danfeng Hong, Xiao Xiang Zhu

This paper addresses the general problem of single-look multi-master SAR tomography.

On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances and Million-AID

1 code implementation22 Jun 2020 Yang Long, Gui-Song Xia, Shengyang Li, Wen Yang, Michael Ying Yang, Xiao Xiang Zhu, Liangpei Zhang, Deren Li

After reviewing existing benchmark datasets in the research community of RS image interpretation, this article discusses the problem of how to efficiently prepare a suitable benchmark dataset for RS image interpretation.

General Classification Image Classification +1

Deep Learning Meets SAR

no code implementations17 Jun 2020 Xiao Xiang Zhu, Sina Montazeri, Mohsin Ali, Yuansheng Hua, Yuanyuan Wang, Lichao Mou, Yilei Shi, Feng Xu, Richard Bamler

Deep learning in remote sensing has become an international hype, but it is mostly limited to the evaluation of optical data.

Instance segmentation of buildings using keypoints

no code implementations6 Jun 2020 Qingyu Li, Lichao Mou, Yuansheng Hua, Yao Sun, Pu Jin, Yilei Shi, Xiao Xiang Zhu

The detected keypoints are subsequently reformulated as a closed polygon, which is the semantic boundary of the building.

Instance Segmentation Segmentation +1

Multi-level Feature Fusion-based CNN for Local Climate Zone Classification from Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset

1 code implementation16 May 2020 Chunping Qiu, Xiaochong Tong, Michael Schmitt, Benjamin Bechtel, Xiao Xiang Zhu

As a unique classification scheme for urban forms and functions, the local climate zone (LCZ) system provides essential general information for any studies related to urban environments, especially on a large scale.

Classification General Classification

Ambient Sound Helps: Audiovisual Crowd Counting in Extreme Conditions

1 code implementation14 May 2020 Di Hu, Lichao Mou, Qingzhong Wang, Junyu. Gao, Yuansheng Hua, Dejing Dou, Xiao Xiang Zhu

Visual crowd counting has been recently studied as a way to enable people counting in crowd scenes from images.

Crowd Counting

SAR Tomography at the Limit: Building Height Reconstruction Using Only 3-5 TanDEM-X Bistatic Interferograms

no code implementations17 Mar 2020 Yilei Shi, Richard Bamler, Yuanyuan Wang, Xiao Xiang Zhu

Multi-baseline interferometric synthetic aperture radar (InSAR) techniques are effective approaches for retrieving the 3-D information of urban areas.

Weakly Supervised Semantic Segmentation of Satellite Images for Land Cover Mapping -- Challenges and Opportunities

1 code implementation19 Feb 2020 Michael Schmitt, Jonathan Prexl, Patrick Ebel, Lukas Liebel, Xiao Xiang Zhu

Therefore, this paper seeks to make a case for the application of weakly supervised learning strategies to get the most out of available data sources and achieve progress in high-resolution large-scale land cover mapping.

Weakly-supervised Learning Weakly supervised Semantic Segmentation +1

Building Footprint Generation by IntegratingConvolution Neural Network with Feature PairwiseConditional Random Field (FPCRF)

no code implementations11 Feb 2020 Qingyu Li, Yilei Shi, Xin Huang, Xiao Xiang Zhu

Due to the complexity of buildings, the accurate and reliable generation of the building footprint from remote sensing imagery is still a challenging task.

Management

ERA: A Dataset and Deep Learning Benchmark for Event Recognition in Aerial Videos

no code implementations30 Jan 2020 Lichao Mou, Yuansheng Hua, Pu Jin, Xiao Xiang Zhu

In this paper, we introduce a novel problem of event recognition in unconstrained aerial videos in the remote sensing community and present a large-scale, human-annotated dataset, named ERA (Event Recognition in Aerial videos), consisting of 2, 864 videos each with a label from 25 different classes corresponding to an event unfolding 5 seconds.

So2Sat LCZ42: A Benchmark Dataset for Global Local Climate Zones Classification

1 code implementation19 Dec 2019 Xiao Xiang Zhu, Jingliang Hu, Chunping Qiu, Yilei Shi, Jian Kang, Lichao Mou, Hossein Bagheri, Matthias Häberle, Yuansheng Hua, Rong Huang, Lloyd Hughes, Hao Li, Yao Sun, Guichen Zhang, Shiyao Han, Michael Schmitt, Yuanyuan Wang

This is especially true for an automated analysis of remote sensing images on a global scale, which enables us to address global challenges such as urbanization and climate change using state-of-the-art machine learning techniques.

BIG-bench Machine Learning Cultural Vocal Bursts Intensity Prediction +1

Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification

no code implementations18 Dec 2019 Danfeng Hong, Xin Wu, Pedram Ghamisi, Jocelyn Chanussot, Naoto Yokoya, Xiao Xiang Zhu

In this paper, we propose a solution to address this issue by locally extracting invariant features from hyperspectral imagery (HSI) in both spatial and frequency domains, using a method called invariant attribute profiles (IAPs).

Attribute General Classification +1

Learning Shared Cross-modality Representation Using Multispectral-LiDAR and Hyperspectral Data

no code implementations18 Dec 2019 Danfeng Hong, Jocelyn Chanussot, Naoto Yokoya, Jian Kang, Xiao Xiang Zhu

Due to the ever-growing diversity of the data source, multi-modality feature learning has attracted more and more attention.

Building Segmentation through a Gated Graph Convolutional Neural Network with Deep Structured Feature Embedding

no code implementations8 Nov 2019 Yilei Shi, Qingyu Li, Xiao Xiang Zhu

Taking the semantic segmentation of building footprints as a practical example, we compared different feature embedding architectures and graph neural networks.

General Classification Segmentation +1

Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation

no code implementations23 Aug 2019 Yuxing Xie, Jiaojiao Tian, Xiao Xiang Zhu

3D Point Cloud Semantic Segmentation (PCSS) is attracting increasing interest, due to its applicability in remote sensing, computer vision and robotics, and due to the new possibilities offered by deep learning techniques.

Point Cloud Segmentation Segmentation +1

Relation Network for Multi-label Aerial Image Classification

1 code implementation16 Jul 2019 Yuansheng Hua, Lichao Mou, Xiao Xiang Zhu

Particularly, our network consists of three elemental modules: 1) a label-wise feature parcel learning module, 2) an attentional region extraction module, and 3) a label relational inference module.

Classification General Classification +5

SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion

2 code implementations18 Jun 2019 Michael Schmitt, Lloyd Haydn Hughes, Chunping Qiu, Xiao Xiang Zhu

The availability of curated large-scale training data is a crucial factor for the development of well-generalizing deep learning methods for the extraction of geoinformation from multi-sensor remote sensing imagery.

Cloud Computing Scene Classification +1

Fusion of Heterogeneous Earth Observation Data for the Classification of Local Climate Zones

no code implementations29 May 2019 Guichen Zhang, Pedram Ghamisi, Xiao Xiang Zhu

This paper proposes a novel framework for fusing multi-temporal, multispectral satellite images and OpenStreetMap (OSM) data for the classification of local climate zones (LCZs).

Classification Earth Observation +1

Learnable Manifold Alignment (LeMA) : A Semi-supervised Cross-modality Learning Framework for Land Cover and Land Use Classification

no code implementations9 Jan 2019 Danfeng Hong, Naoto Yokoya, Nan Ge, Jocelyn Chanussot, Xiao Xiang Zhu

In this paper, we aim at tackling a general but interesting cross-modality feature learning question in remote sensing community --- can a limited amount of highly-discrimin-ative (e. g., hyperspectral) training data improve the performance of a classification task using a large amount of poorly-discriminative (e. g., multispectral) data?

General Classification

CoSpace: Common Subspace Learning from Hyperspectral-Multispectral Correspondences

no code implementations30 Dec 2018 Danfeng Hong, Naoto Yokoya, Jocelyn Chanussot, Xiao Xiang Zhu

To achieve accurate land cover classification over a large coverage, we propose a cross-modality feature learning framework, called common subspace learning (CoSpace), by jointly considering subspace learning and supervised classification.

Classification General Classification +1

Non-Local Compressive Sensing Based SAR Tomography

no code implementations5 Nov 2018 Yilei Shi, Xiao Xiang Zhu, Richard Bamler

We propose to increase SNR by integrating non-local estimation into the inversion and show that a reasonable reconstruction of buildings from only seven interferograms is feasible.

Compressive Sensing

An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing

no code implementations29 Oct 2018 Danfeng Hong, Naoto Yokoya, Jocelyn Chanussot, Xiao Xiang Zhu

To this end, we propose a novel spectral mixture model, called the augmented linear mixing model (ALMM), to address spectral variability by applying a data-driven learning strategy in inverse problems of hyperspectral unmixing.

Dictionary Learning Hyperspectral Unmixing

Building Footprint Generation Using Improved Generative Adversarial Networks

no code implementations26 Oct 2018 Yilei Shi, Qingyu Li, Xiao Xiang Zhu

The automatic generation of building footprints from satellite images presents a considerable challenge due to the complexity of building shapes.

R$^3$-Net: A Deep Network for Multi-oriented Vehicle Detection in Aerial Images and Videos

no code implementations16 Aug 2018 Qingpeng Li, Lichao Mou, Qizhi Xu, Yun Zhang, Xiao Xiang Zhu

In this paper, we propose a novel deep network, called rotatable region-based residual network (R$^3$-Net), to detect multi-oriented vehicles in aerial images and videos.

Motion Estimation Position +1

Buildings Detection in VHR SAR Images Using Fully Convolution Neural Networks

no code implementations14 Aug 2018 Muhammad Shahzad, Michael Maurer, Friedrich Fraundorfer, Yuanyuan Wang, Xiao Xiang Zhu

This paper addresses the highly challenging problem of automatically detecting man-made structures especially buildings in very high resolution (VHR) synthetic aperture radar (SAR) images.

General Classification Image Classification

Recurrently Exploring Class-wise Attention in A Hybrid Convolutional and Bidirectional LSTM Network for Multi-label Aerial Image Classification

no code implementations30 Jul 2018 Yuansheng Hua, Lichao Mou, Xiao Xiang Zhu

The proposed network consists of three indispensable components: 1) a feature extraction module, 2) a class attention learning layer, and 3) a bidirectional LSTM-based sub-network.

General Classification Image Classification +1

The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusion

no code implementations4 Jul 2018 Michael Schmitt, Lloyd Haydn Hughes, Xiao Xiang Zhu

While deep learning techniques have an increasing impact on many technical fields, gathering sufficient amounts of training data is a challenging problem in remote sensing.

Colorization Image Colorization

RiFCN: Recurrent Network in Fully Convolutional Network for Semantic Segmentation of High Resolution Remote Sensing Images

no code implementations5 May 2018 Lichao Mou, Xiao Xiang Zhu

The former is a classification CNN architecture for feature extraction, which takes an input image and produces multi-level convolutional feature maps from shallow to deep; while in the later, to achieve accurate boundary inference and semantic segmentation, boundary-aware high resolution feature maps in shallower layers and high-level but low-resolution features are recursively embedded into the learning framework (from deep to shallow) to generate a fused feature representation that draws a holistic picture of not only high-level semantic information but also low-level fine-grained details.

Segmentation Semantic Segmentation

Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery

no code implementations7 Mar 2018 Lichao Mou, Lorenzo Bruzzone, Xiao Xiang Zhu

As far as we know, this is the first time that a recurrent convolutional network architecture has been proposed for multitemporal remote sensing image analysis.

Change Detection Earth Observation

IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network

1 code implementation28 Feb 2018 Lichao Mou, Xiao Xiang Zhu

In this paper we tackle a very novel problem, namely height estimation from a single monocular remote sensing image, which is inherently ambiguous, and a technically ill-posed problem, with a large source of uncertainty coming from the overall scale.

Instance Segmentation Semantic Segmentation

Building Instance Classification Using Street View Images

no code implementations25 Feb 2018 Jian Kang, Marco Körner, Yuanyuan Wang, Hannes Taubenböck, Xiao Xiang Zhu

The proposed method is based on Convolutional Neural Networks (CNNs) which classify facade structures from street view images, such as Google StreetView, in addition to remote sensing images which usually only show roof structures.

Classification General Classification

Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN

no code implementations25 Jan 2018 Lloyd H. Hughes, Michael Schmitt, Lichao Mou, Yuanyuan Wang, Xiao Xiang Zhu

In this letter, we propose a pseudo-siamese convolutional neural network (CNN) architecture that enables to solve the task of identifying corresponding patches in very-high-resolution (VHR) optical and synthetic aperture radar (SAR) remote sensing imagery.

Key Point Matching

Deep learning in remote sensing: a review

1 code implementation11 Oct 2017 Xiao Xiang Zhu, Devis Tuia, Lichao Mou, Gui-Song Xia, Liangpei Zhang, Feng Xu, Friedrich Fraundorfer

In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with.

Cannot find the paper you are looking for? You can Submit a new open access paper.