Search Results for author: Devis Tuia

Found 71 papers, 20 papers with code

Multilingual Vision-Language Pre-training for the Remote Sensing Domain

no code implementations30 Oct 2024 João Daniel Silva, Joao Magalhaes, Devis Tuia, Bruno Martins

This work proposes a novel vision-and-language model for the remote sensing domain, exploring the fine-tuning of a multilingual CLIP model and testing the use of a self-supervised method based on aligning local and global representations from individual input images, together with the standard CLIP objective.

Cross-Modal Retrieval Image Classification +4

POLO -- Point-based, multi-class animal detection

no code implementations15 Oct 2024 Giacomo May, Emanuele Dalsasso, Benjamin Kellenberger, Devis Tuia

Automated wildlife surveys based on drone imagery and object detection technology are a powerful and increasingly popular tool in conservation biology.

object-detection Object Detection

Multi-Scale Grouped Prototypes for Interpretable Semantic Segmentation

no code implementations14 Sep 2024 Hugo Porta, Emanuele Dalsasso, Diego Marcos, Devis Tuia

Prototypical part learning is emerging as a promising approach for making semantic segmentation interpretable.

Segmentation Semantic Segmentation

What to align in multimodal contrastive learning?

no code implementations11 Sep 2024 Benoit Dufumier, Javiera Castillo-Navarro, Devis Tuia, Jean-Philippe Thiran

Indeed, by considering each modality as a different view of the same entity, it learns to align features of different modalities in a shared representation space.

Contrastive Learning Self-Supervised Learning

Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants

no code implementations7 Aug 2024 Beatriz Borges, Negar Foroutan, Deniz Bayazit, Anna Sotnikova, Syrielle Montariol, Tanya Nazaretzky, Mohammadreza Banaei, Alireza Sakhaeirad, Philippe Servant, Seyed Parsa Neshaei, Jibril Frej, Angelika Romanou, Gail Weiss, Sepideh Mamooler, Zeming Chen, Simin Fan, Silin Gao, Mete Ismayilzada, Debjit Paul, Alexandre Schöpfer, Andrej Janchevski, Anja Tiede, Clarence Linden, Emanuele Troiani, Francesco Salvi, Freya Behrens, Giacomo Orsi, Giovanni Piccioli, Hadrien Sevel, Louis Coulon, Manuela Pineros-Rodriguez, Marin Bonnassies, Pierre Hellich, Puck van Gerwen, Sankalp Gambhir, Solal Pirelli, Thomas Blanchard, Timothée Callens, Toni Abi Aoun, Yannick Calvino Alonso, Yuri Cho, Alberto Chiappa, Antonio Sclocchi, Étienne Bruno, Florian Hofhammer, Gabriel Pescia, Geovani Rizk, Leello Dadi, Lucas Stoffl, Manoel Horta Ribeiro, Matthieu Bovel, Yueyang Pan, Aleksandra Radenovic, Alexandre Alahi, Alexander Mathis, Anne-Florence Bitbol, Boi Faltings, Cécile Hébert, Devis Tuia, François Maréchal, George Candea, Giuseppe Carleo, Jean-Cédric Chappelier, Nicolas Flammarion, Jean-Marie Fürbringer, Jean-Philippe Pellet, Karl Aberer, Lenka Zdeborová, Marcel Salathé, Martin Jaggi, Martin Rajman, Mathias Payer, Matthieu Wyart, Michael Gastpar, Michele Ceriotti, Ola Svensson, Olivier Lévêque, Paolo Ienne, Rachid Guerraoui, Robert West, Sanidhya Kashyap, Valerio Piazza, Viesturs Simanis, Viktor Kuncak, Volkan Cevher, Philippe Schwaller, Sacha Friedli, Patrick Jermann, Tanja Kaser, Antoine Bosselut

We investigate the potential scale of this vulnerability by measuring the degree to which AI assistants can complete assessment questions in standard university-level STEM courses.

Knowledge-aware Text-Image Retrieval for Remote Sensing Images

no code implementations6 May 2024 Li Mi, Xianjie Dai, Javiera Castillo-Navarro, Devis Tuia

For this reason, as a matching-based task, cross-modal text-image retrieval often suffers from information asymmetry between texts and images.

Diversity Earth Observation +3

ConGeo: Robust Cross-view Geo-localization across Ground View Variations

no code implementations20 Mar 2024 Li Mi, Chang Xu, Javiera Castillo-Navarro, Syrielle Montariol, Wen Yang, Antoine Bosselut, Devis Tuia

Cross-view geo-localization aims at localizing a ground-level query image by matching it to its corresponding geo-referenced aerial view.

geo-localization

Cross-Modal Learning of Housing Quality in Amsterdam

no code implementations13 Mar 2024 Alex Levering, Diego Marcos, Devis Tuia

In our research we test data and models for the recognition of housing quality in the city of Amsterdam from ground-level and aerial imagery.

Imbalance-aware Presence-only Loss Function for Species Distribution Modeling

no code implementations12 Mar 2024 Robin Zbinden, Nina van Tiel, Marc Rußwurm, Devis Tuia

In the face of significant biodiversity decline, species distribution models (SDMs) are essential for understanding the impact of climate change on species habitats by connecting environmental conditions to species occurrences.

ConVQG: Contrastive Visual Question Generation with Multimodal Guidance

no code implementations20 Feb 2024 Li Mi, Syrielle Montariol, Javiera Castillo-Navarro, Xianjie Dai, Antoine Bosselut, Devis Tuia

However, generating focused questions using textual constraints while enforcing a high relevance to the image content remains a challenge, as VQG systems often ignore one or both forms of grounding.

Question Generation Question-Generation

Large Language Models for Captioning and Retrieving Remote Sensing Images

no code implementations9 Feb 2024 João Daniel Silva, João Magalhães, Devis Tuia, Bruno Martins

In this work, we propose RS-CapRet, a Vision and Language method for remote sensing tasks, in particular image captioning and text-image retrieval.

Cross-Modal Retrieval Decoder +6

On the selection and effectiveness of pseudo-absences for species distribution modeling with deep learning

no code implementations3 Jan 2024 Robin Zbinden, Nina van Tiel, Benjamin Kellenberger, Lloyd Hughes, Devis Tuia

Notably, the significant class imbalance between species presences and pseudo-absences is often left unaddressed.

The curse of language biases in remote sensing VQA: the role of spatial attributes, language diversity, and the need for clear evaluation

no code implementations28 Nov 2023 Christel Chappuis, Eliot Walt, Vincent Mendez, Sylvain Lobry, Bertrand Le Saux, Devis Tuia

While new, improved and less-biased datasets appear as a necessity for the development of the promising field of RSVQA, we demonstrate that more informed, relative evaluation metrics remain much needed to transparently communicate results of future RSVQA methods.

Diversity Question Answering +3

High-resolution Population Maps Derived from Sentinel-1 and Sentinel-2

no code implementations23 Nov 2023 Nando Metzger, Rodrigo Caye Daudt, Devis Tuia, Konrad Schindler

With our work we aim to democratize access to up-to-date and high-resolution population maps, recognizing that some regions faced with particularly strong population dynamics may lack the resources for costly micro-census campaigns.

Humanitarian Population Mapping

Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks

1 code implementation10 Oct 2023 Marc Rußwurm, Konstantin Klemmer, Esther Rolf, Robin Zbinden, Devis Tuia

At the same time, little attention has been paid to the exact design of the neural network architectures with which these functional embeddings are combined.

Scalable Semantic 3D Mapping of Coral Reefs with Deep Learning

no code implementations22 Sep 2023 Jonathan Sauder, Guilhem Banc-Prandi, Anders Meibom, Devis Tuia

This paper presents a new paradigm for mapping underwater environments from ego-motion video, unifying 3D mapping systems that use machine learning to adapt to challenging conditions under water, combined with a modern approach for semantic segmentation of images.

Deep Learning Image Segmentation +1

Time Series Analysis of Urban Liveability

no code implementations1 Sep 2023 Alex Levering, Diego Marcos, Devis Tuia

In this paper we explore deep learning models to monitor longitudinal liveability changes in Dutch cities at the neighbourhood level.

Time Series Time Series Analysis

Large-scale Detection of Marine Debris in Coastal Areas with Sentinel-2

1 code implementation5 Jul 2023 Marc Rußwurm, Sushen Jilla Venkatesa, Devis Tuia

Here, remote sensing can provide reliable estimates of plastic pollution by regularly monitoring and detecting marine debris in coastal areas.

Geo-Information Harvesting from Social Media Data

no code implementations1 Nov 2022 Xiao Xiang Zhu, Yuanyuan Wang, Mrinalini Kochupillai, Martin Werner, Matthias Häberle, Eike Jens Hoffmann, Hannes Taubenböck, Devis Tuia, Alex Levering, Nathan Jacobs, Anna Kruspe, Karam Abdulahhad

In this article, we address key aspects in the field, including data availability, analysis-ready data preparation and data management, geo-information extraction from social media text messages and images, and the fusion of social media and remote sensing data.

Management

How to find a good image-text embedding for remote sensing visual question answering?

no code implementations24 Sep 2021 Christel Chappuis, Sylvain Lobry, Benjamin Kellenberger, Bertrand Le Saux, Devis Tuia

Visual question answering (VQA) has recently been introduced to remote sensing to make information extraction from overhead imagery more accessible to everyone.

Question Answering Visual Question Answering

Self-Supervised Pretraining and Controlled Augmentation Improve Rare Wildlife Recognition in UAV Images

1 code implementation17 Aug 2021 Xiaochen Zheng, Benjamin Kellenberger, Rui Gong, Irena Hajnsek, Devis Tuia

In detail, we examine a combination of recent contrastive learning methodologies like Momentum Contrast (MoCo) and Cross-Level Instance-Group Discrimination (CLD) to condition our model on the aerial images without the requirement for labels.

Contrastive Learning

Mapping Vulnerable Populations with AI

no code implementations29 Jul 2021 Benjamin Kellenberger, John E. Vargas-Muñoz, Devis Tuia, Rodrigo C. Daudt, Konrad Schindler, Thao T-T Whelan, Brenda Ayo, Ferda Ofli, Muhammad Imran

Building functions shall be retrieved by parsing social media data like for instance tweets, as well as ground-based imagery, to automatically identify different buildings functions and retrieve further information such as the number of building stories.

Humanitarian Image Segmentation +1

Learning User's confidence for active learning

no code implementations15 Apr 2021 Devis Tuia, Jordi Munoz-Mari

In this paper, we study the applicability of active learning in operative scenarios: more particularly, we consider the well-known contradiction between the active learning heuristics, which rank the pixels according to their uncertainty, and the user's confidence in labeling, which is related to both the homogeneity of the pixel context and user's knowledge of the scene.

Active Learning Pansharpening

Recent Advances in Domain Adaptation for the Classification of Remote Sensing Data

no code implementations15 Apr 2021 Devis Tuia, Claudio Persello, Lorenzo Bruzzone

The success of supervised classification of remotely sensed images acquired over large geographical areas or at short time intervals strongly depends on the representativity of the samples used to train the classification algorithm and to define the model.

Classification Domain Adaptation +2

A survey of active learning algorithms for supervised remote sensing image classification

no code implementations15 Apr 2021 Devis Tuia, Michele Volpi, Loris Copa, Mikhail Kanevski, Jordi Munoz-Mari

Defining an efficient training set is one of the most delicate phases for the success of remote sensing image classification routines.

Active Learning General Classification +2

Semisupervised Manifold Alignment of Multimodal Remote Sensing Images

no code implementations15 Apr 2021 Devis Tuia, Michele Volpi, Maxime Trolliet, Gustau Camps-Valls

We introduce a method for manifold alignment of different modalities (or domains) of remote sensing images.

Image Classification

Towards a Collective Agenda on AI for Earth Science Data Analysis

1 code implementation11 Apr 2021 Devis Tuia, Ribana Roscher, Jan Dirk Wegner, Nathan Jacobs, Xiao Xiang Zhu, Gustau Camps-Valls

In the last years we have witnessed the fields of geosciences and remote sensing and artificial intelligence to become closer.

Position

Self-supervised pre-training enhances change detection in Sentinel-2 imagery

1 code implementation20 Jan 2021 Marrit Leenstra, Diego Marcos, Francesca Bovolo, Devis Tuia

While annotated images for change detection using satellite imagery are scarce and costly to obtain, there is a wealth of unlabeled images being generated every day.

Ranked #5 on Change Detection on OSCD - 13ch (using extra training data)

Change Detection Self-Supervised Learning

Semantic Segmentation of Remote Sensing Images with Sparse Annotations

1 code implementation10 Jan 2021 Yuansheng Hua, Diego Marcos, Lichao Mou, Xiao Xiang Zhu, Devis Tuia

Training Convolutional Neural Networks (CNNs) for very high resolution images requires a large quantity of high-quality pixel-level annotations, which is extremely labor- and time-consuming to produce.

Semantic Segmentation

A deep network approach to multitemporal cloud detection

no code implementations9 Dec 2020 Devis Tuia, Benjamin Kellenberger, Adrian Pérez-Suay, Gustau Camps-Valls

With a single model, we are able to outline clouds along all year and during day and night with high accuracy.

Cloud Detection Time Series +1

Multi-temporal and multi-source remote sensing image classification by nonlinear relative normalization

no code implementations7 Dec 2020 Devis Tuia, Diego Marcos, Gustau Camps-Valls

Remote sensing image classification exploiting multiple sensors is a very challenging problem: data from different modalities are affected by spectral distortions and mis-alignments of all kinds, and this hampers re-using models built for one image to be used successfully in other scenes.

General Classification Image Classification +1

Contextual Semantic Interpretability

1 code implementation18 Sep 2020 Diego Marcos, Ruth Fong, Sylvain Lobry, Remi Flamary, Nicolas Courty, Devis Tuia

Once the attributes are learned, they can be re-combined to reach the final decision and provide both an accurate prediction and an explicit reasoning behind the CNN decision.

Deploying machine learning to assist digital humanitarians: making image annotation in OpenStreetMap more efficient

no code implementations17 Sep 2020 John E. Vargas-Muñoz, Devis Tuia, Alexandre X. Falcão

Locating populations in rural areas of developing countries has attracted the attention of humanitarian mapping projects since it is important to plan actions that affect vulnerable areas.

BIG-bench Machine Learning Humanitarian

OpenStreetMap: Challenges and Opportunities in Machine Learning and Remote Sensing

no code implementations13 Jul 2020 John Vargas, Shivangi Srivastava, Devis Tuia, Alexandre Falcao

OpenStreetMap (OSM) is a community-based, freely available, editable map service that was created as an alternative to authoritative ones.

BIG-bench Machine Learning Earth Observation

Detecting Unsigned Physical Road Incidents from Driver-View Images

no code implementations24 Apr 2020 Alex Levering, Martin Tomko, Devis Tuia, Kourosh Khoshelham

In this paper we propose a system based on an off-the-shelf deep neural network architecture that is able to detect and recognize types of unsigned (non-placarded, such as traffic signs), physical (visible in images) road incidents.

Autonomous Vehicles

RSVQA: Visual Question Answering for Remote Sensing Data

no code implementations16 Mar 2020 Sylvain Lobry, Diego Marcos, Jesse Murray, Devis Tuia

We report the results obtained by applying a model based on Convolutional Neural Networks (CNNs) for the visual part and on a Recurrent Neural Network (RNN) for the natural language part to this task.

Land Cover Classification Object Counting +2

Semantically Interpretable Activation Maps: what-where-how explanations within CNNs

no code implementations18 Sep 2019 Diego Marcos, Sylvain Lobry, Devis Tuia

This gives the user insight into what the model has seen, where, and a final output directly linked to this information in a comprehensive and interpretable way.

Attribute

Adaptive Compression-based Lifelong Learning

no code implementations23 Jul 2019 Shivangi Srivastava, Maxim Berman, Matthew B. Blaschko, Devis Tuia

The latter approach falls under the denomination of lifelong learning, where the model is updated in a way that it performs well on both old and new tasks, without having access to the old task's training samples anymore.

Bayesian Optimization Semantic Segmentation

Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery using Deep CNNs and Active Learning

no code implementations17 Jul 2019 Benjamin Kellenberger, Diego Marcos, Sylvain Lobry, Devis Tuia

We present an Active Learning (AL) strategy for re-using a deep Convolutional Neural Network (CNN)-based object detector on a new dataset.

Active Learning Retrieval

Wasserstein Adversarial Regularization (WAR) on label noise

1 code implementation8 Apr 2019 Kilian Fatras, Bharath Bhushan Damodaran, Sylvain Lobry, Rémi Flamary, Devis Tuia, Nicolas Courty

Noisy labels often occur in vision datasets, especially when they are obtained from crowdsourcing or Web scraping.

Semantic Segmentation

End-to-End Learned Early Classification of Time Series for In-Season Crop Type Mapping

2 code implementations30 Jan 2019 Marc Rußwurm, Nicolas Courty, Rémi Emonet, Sébastien Lefèvre, Devis Tuia, Romain Tavenard

In this work, we present an End-to-End Learned Early Classification of Time Series (ELECTS) model that estimates a classification score and a probability of whether sufficient data has been observed to come to an early and still accurate decision.

Classification Crop Classification +6

Correcting rural building annotations in OpenStreetMap using convolutional neural networks

no code implementations24 Jan 2019 John E. Vargas-Muñoz, Sylvain Lobry, Alexandre X. Falcão, Devis Tuia

Rural building mapping is paramount to support demographic studies and plan actions in response to crisis that affect those areas.

Decision fusion with multiple spatial supports by conditional random fields

no code implementations24 Aug 2018 Devis Tuia, Michele Volpi, Gabriele Moser

In this paper, we follow these two observations and encode them as priors in an energy minimization framework based on conditional random fields (CRFs), where classification results obtained at pixel and region levels are probabilistically fused.

General Classification

Deep multi-task learning for a geographically-regularized semantic segmentation of aerial images

no code implementations23 Aug 2018 Michele Volpi, Devis Tuia

When approaching the semantic segmentation of overhead imagery in the decimeter spatial resolution range, successful strategies usually combine powerful methods to learn the visual appearance of the semantic classes (e. g. convolutional neural networks) with strategies for spatial regularization (e. g. graphical models such as conditional random fields).

Multi-Task Learning Semantic Segmentation

Scale equivariance in CNNs with vector fields

no code implementations31 Jul 2018 Diego Marcos, Benjamin Kellenberger, Sylvain Lobry, Devis Tuia

We study the effect of injecting local scale equivariance into Convolutional Neural Networks.

General Classification

Change Detection between Multimodal Remote Sensing Data Using Siamese CNN

1 code implementation25 Jul 2018 Zhenchao Zhang, George Vosselman, Markus Gerke, Devis Tuia, Michael Ying Yang

Detecting topographic changes in the urban environment has always been an important task for urban planning and monitoring.

Change Detection

Detecting Mammals in UAV Images: Best Practices to address a substantially Imbalanced Dataset with Deep Learning

no code implementations29 Jun 2018 Benjamin Kellenberger, Diego Marcos, Devis Tuia

In this paper, we study how to scale CNNs to large wildlife census tasks and present a number of recommendations to train a CNN on a large UAV dataset.

object-detection Object Detection

DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images

1 code implementation17 May 2018 Ilke Demir, Krzysztof Koperski, David Lindenbaum, Guan Pang, Jing Huang, Saikat Basu, Forest Hughes, Devis Tuia, Ramesh Raskar

We present the DeepGlobe 2018 Satellite Image Understanding Challenge, which includes three public competitions for segmentation, detection, and classification tasks on satellite images.

DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation

4 code implementations ECCV 2018 Bharath Bhushan Damodaran, Benjamin Kellenberger, Rémi Flamary, Devis Tuia, Nicolas Courty

In computer vision, one is often confronted with problems of domain shifts, which occur when one applies a classifier trained on a source dataset to target data sharing similar characteristics (e. g. same classes), but also different latent data structures (e. g. different acquisition conditions).

Unsupervised Domain Adaptation

Learning deep structured active contours end-to-end

2 code implementations CVPR 2018 Diego Marcos, Devis Tuia, Benjamin Kellenberger, Lisa Zhang, Min Bai, Renjie Liao, Raquel Urtasun

The world is covered with millions of buildings, and precisely knowing each instance's position and extents is vital to a multitude of applications.

Instance Segmentation Segmentation +1

Optimal Transport for Multi-source Domain Adaptation under Target Shift

3 code implementations13 Mar 2018 Ievgen Redko, Nicolas Courty, Rémi Flamary, Devis Tuia

In this paper, we propose to tackle the problem of reducing discrepancies between multiple domains referred to as multi-source domain adaptation and consider it under the target shift assumption: in all domains we aim to solve a classification problem with the same output classes, but with labels' proportions differing across them.

Domain Adaptation Image Segmentation +1

Deep learning in remote sensing: a review

1 code implementation11 Oct 2017 Xiao Xiang Zhu, Devis Tuia, Lichao Mou, Gui-Song Xia, Liangpei Zhang, Feng Xu, Friedrich Fraundorfer

In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with.

Deep Learning

Detecting animals in African Savanna with UAVs and the crowds

no code implementations6 Sep 2017 Nicolas Rey, Michele Volpi, Stéphane Joost, Devis Tuia

Unmanned aerial vehicles (UAVs) offer new opportunities for wildlife monitoring, with several advantages over traditional field-based methods.

Management

Towards seamless multi-view scene analysis from satellite to street-level

no code implementations23 May 2017 Sébastien Lefèvre, Devis Tuia, Jan Dirk Wegner, Timothée Produit, Ahmed Samy Nassar

In this paper, we discuss and review how combined multi-view imagery from satellite to street-level can benefit scene analysis.

Change Detection Earth Observation +3

Non-convex regularization in remote sensing

1 code implementation23 Jun 2016 Devis Tuia, Remi Flamary, Michel Barlaud

In this paper, we study the effect of different regularizers and their implications in high dimensional image classification and sparse linear unmixing.

Classification General Classification +1

Multiclass feature learning for hyperspectral image classification: sparse and hierarchical solutions

no code implementations23 Jun 2016 Devis Tuia, Rémi Flamary, Nicolas Courty

In this paper, we tackle the question of discovering an effective set of spatial filters to solve hyperspectral classification problems.

Classification General Classification +1

Geospatial Correspondences for Multimodal Registration

no code implementations CVPR 2016 Diego Marcos, Raffay Hamid, Devis Tuia

The growing availability of very high resolution (<1 m/pixel) satellite and aerial images has opened up unprecedented opportunities to monitor and analyze the evolution of land-cover and land-use across the world.

Change Detection

Learning rotation invariant convolutional filters for texture classification

1 code implementation22 Apr 2016 Diego Marcos, Michele Volpi, Devis Tuia

We present a method for learning discriminative filters using a shallow Convolutional Neural Network (CNN).

Classification General Classification +2

Optimal Transport for Domain Adaptation

no code implementations2 Jul 2015 Nicolas Courty, Rémi Flamary, Devis Tuia, Alain Rakotomamonjy

Domain adaptation from one data space (or domain) to another is one of the most challenging tasks of modern data analytics.

Domain Adaptation

Kernel Manifold Alignment

1 code implementation9 Apr 2015 Devis Tuia, Gustau Camps-Valls

We introduce a kernel method for manifold alignment (KEMA) and domain adaptation that can match an arbitrary number of data sources without needing corresponding pairs, just few labeled examples in all domains.

Computational Efficiency Domain Adaptation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.