Search Results for author: Gengchen Mai

Found 40 papers, 17 papers with code

Examining the Commitments and Difficulties Inherent in Multimodal Foundation Models for Street View Imagery

no code implementations23 Aug 2024 Zhenyuan Yang, Xuhui Lin, Qinyi He, Ziye Huang, Zhengliang Liu, Hanqi Jiang, Peng Shu, Zihao Wu, Yiwei Li, Stephen Law, Gengchen Mai, Tianming Liu, Tao Yang

The emergence of Large Language Models (LLMs) and multimodal foundation models (FMs) has generated heightened interest in their applications that integrate vision and language.

Question Answering Zero-Shot Learning

TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning

1 code implementation21 Jun 2024 Nemin Wu, Qian Cao, Zhangyu Wang, Zeping Liu, Yanlin Qi, Jielu Zhang, Joshua Ni, Xiaobai Yao, Hongxu Ma, Lan Mu, Stefano Ermon, Tanuja Ganu, Akshay Nambi, Ni Lao, Gengchen Mai

To fill this gap, we propose TorchSpatial, a learning framework and benchmark for location (point) encoding, which is one of the most fundamental data types of spatial representation learning.

Fairness Geographic Question Answering +4

MC-GTA: Metric-Constrained Model-Based Clustering using Goodness-of-fit Tests with Autocorrelations

1 code implementation28 May 2024 Zhangyu Wang, Gengchen Mai, Krzysztof Janowicz, Ni Lao

A wide range of (multivariate) temporal (1D) and spatial (2D) data analysis tasks, such as grouping vehicle sensor trajectories, can be formulated as clustering with given metric constraints.

Constrained Clustering

Probing the Information Theoretical Roots of Spatial Dependence Measures

1 code implementation28 May 2024 Zhangyu Wang, Krzysztof Janowicz, Gengchen Mai, Ivan Majic

Intuitively, there is a relation between measures of spatial dependence and information theoretical measures of entropy.

Relation

Img2Loc: Revisiting Image Geolocalization using Multi-modality Foundation Models and Image-based Retrieval-Augmented Generation

1 code implementation28 Mar 2024 Zhongliang Zhou, Jielu Zhang, Zihan Guan, Mengxuan Hu, Ni Lao, Lan Mu, Sheng Li, Gengchen Mai

Geolocating precise locations from images presents a challenging problem in computer vision and information retrieval. Traditional methods typically employ either classification, which dividing the Earth surface into grid cells and classifying images accordingly, or retrieval, which identifying locations by matching images with a database of image-location pairs.

Retrieval Text Generation

TransFlower: An Explainable Transformer-Based Model with Flow-to-Flow Attention for Commuting Flow Prediction

no code implementations23 Feb 2024 Yan Luo, Zhuoyue Wan, Yuzhong Chen, Gengchen Mai, Fu-Lai Chung, Kent Larson

Understanding the link between urban planning and commuting flows is crucial for guiding urban development and policymaking.

Revolutionizing Finance with LLMs: An Overview of Applications and Insights

no code implementations22 Jan 2024 Huaqin Zhao, Zhengliang Liu, Zihao Wu, Yiwei Li, Tianze Yang, Peng Shu, Shaochen Xu, Haixing Dai, Lin Zhao, Gengchen Mai, Ninghao Liu, Tianming Liu

Additionally, we conducted holistic tests on multiple financial tasks through the combination of natural language instructions.

Multimodality of AI for Education: Towards Artificial General Intelligence

no code implementations10 Dec 2023 Gyeong-Geon Lee, Lehong Shi, Ehsan Latif, Yizhu Gao, Arne Bewersdorff, Matthew Nyaaba, Shuchen Guo, Zihao Wu, Zhengliang Liu, Hui Wang, Gengchen Mai, Tiaming Liu, Xiaoming Zhai

This paper presents a comprehensive examination of how multimodal artificial intelligence (AI) approaches are paving the way towards the realization of Artificial General Intelligence (AGI) in educational contexts.

Transformation vs Tradition: Artificial General Intelligence (AGI) for Arts and Humanities

no code implementations30 Oct 2023 Zhengliang Liu, Yiwei Li, Qian Cao, Junwen Chen, Tianze Yang, Zihao Wu, John Hale, John Gibbs, Khaled Rasheed, Ninghao Liu, Gengchen Mai, Tianming Liu

Recent advances in artificial general intelligence (AGI), particularly large language models and creative image generation systems have demonstrated impressive capabilities on diverse tasks spanning the arts and humanities.

Image Generation Marketing

GeoLLM: Extracting Geospatial Knowledge from Large Language Models

1 code implementation10 Oct 2023 Rohin Manvi, Samar Khanna, Gengchen Mai, Marshall Burke, David Lobell, Stefano Ermon

With GeoLLM, we observe that GPT-3. 5 outperforms Llama 2 and RoBERTa by 19% and 51% respectively, suggesting that the performance of our method scales well with the size of the model and its pretraining dataset.

SSIF: Learning Continuous Image Representation for Spatial-Spectral Super-Resolution

no code implementations30 Sep 2023 Gengchen Mai, Ni Lao, Weiwei Sun, Yuchi Ma, Jiaming Song, Chenlin Meng, Hongxu Ma, Jinmeng Rao, Ziyuan Li, Stefano Ermon

Existing digital sensors capture images at fixed spatial and spectral resolutions (e. g., RGB, multispectral, and hyperspectral images), and each combination requires bespoke machine learning models.

Spectral Super-Resolution Super-Resolution

Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Models

no code implementations29 Sep 2023 Jinmeng Rao, Song Gao, Gengchen Mai, Krzysztof Janowicz

Through this vision paper, we hope to draw the attention of researchers and policymakers in geospatial domains to these privacy and security risks inherent in GeoAI foundation models and advocate for the development of privacy-preserving and secure GeoAI foundation models.

Geographic Question Answering Privacy Preserving +1

Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges

no code implementations14 Sep 2023 Fei Dou, Jin Ye, Geng Yuan, Qin Lu, Wei Niu, Haijian Sun, Le Guan, Guoyu Lu, Gengchen Mai, Ninghao Liu, Jin Lu, Zhengliang Liu, Zihao Wu, Chenjiao Tan, Shaochen Xu, Xianqiao Wang, Guoming Li, Lilong Chai, Sheng Li, Jin Sun, Hongyue Sun, Yunli Shao, Changying Li, Tianming Liu, WenZhan Song

Artificial General Intelligence (AGI), possessing the capacity to comprehend, learn, and execute tasks with human cognitive abilities, engenders significant anticipation and intrigue across scientific, commercial, and societal arenas.

Decision Making

Sphere2Vec: A General-Purpose Location Representation Learning over a Spherical Surface for Large-Scale Geospatial Predictions

no code implementations30 Jun 2023 Gengchen Mai, Yao Xuan, Wenyun Zuo, Yutong He, Jiaming Song, Stefano Ermon, Krzysztof Janowicz, Ni Lao

So when applied to large-scale real-world GPS coordinate datasets, which require distance metric learning on the spherical surface, both types of models can fail due to the map projection distortion problem (2D) and the spherical-to-Euclidean distance approximation error (3D).

Image Classification Metric Learning +2

Exploring New Frontiers in Agricultural NLP: Investigating the Potential of Large Language Models for Food Applications

no code implementations20 Jun 2023 Saed Rezayi, Zhengliang Liu, Zihao Wu, Chandra Dhakal, Bao Ge, Haixing Dai, Gengchen Mai, Ninghao Liu, Chen Zhen, Tianming Liu, Sheng Li

ChatGPT has shown to be a strong baseline in many NLP tasks, and we believe it has the potential to improve our model in the task of semantic matching and enhance our model's understanding of food-related concepts and relationships.

Language Modelling Nutrition

AD-AutoGPT: An Autonomous GPT for Alzheimer's Disease Infodemiology

no code implementations16 Jun 2023 Haixing Dai, Yiwei Li, Zhengliang Liu, Lin Zhao, Zihao Wu, Suhang Song, Ye Shen, Dajiang Zhu, Xiang Li, Sheng Li, Xiaobai Yao, Lu Shi, Quanzheng Li, Zhuo Chen, Donglan Zhang, Gengchen Mai, Tianming Liu

In this pioneering study, inspired by AutoGPT, the state-of-the-art open-source application based on the GPT-4 large language model, we develop a novel tool called AD-AutoGPT which can conduct data collection, processing, and analysis about complex health narratives of Alzheimer's Disease in an autonomous manner via users' textual prompts.

Language Modelling Large Language Model

ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs

1 code implementation3 May 2023 Yucheng Shi, Hehuan Ma, Wenliang Zhong, Qiaoyu Tan, Gengchen Mai, Xiang Li, Tianming Liu, Junzhou Huang

To tackle these limitations, we propose a novel framework that leverages the power of ChatGPT for specific tasks, such as text classification, while improving its interpretability.

Decision Making Language Modelling +3

CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations

1 code implementation1 May 2023 Gengchen Mai, Ni Lao, Yutong He, Jiaming Song, Stefano Ermon

To directly leverage the abundant geospatial information associated with images in pre-training, fine-tuning, and inference stages, we present Contrastive Spatial Pre-Training (CSP), a self-supervised learning framework for geo-tagged images.

Contrastive Learning Image Classification +1

AGI: Artificial General Intelligence for Education

no code implementations24 Apr 2023 Ehsan Latif, Gengchen Mai, Matthew Nyaaba, Xuansheng Wu, Ninghao Liu, Guoyu Lu, Sheng Li, Tianming Liu, Xiaoming Zhai

AGI, driven by the recent large pre-trained models, represents a significant leap in the capability of machines to perform tasks that require human-level intelligence, such as reasoning, problem-solving, decision-making, and even understanding human emotions and social interactions.

Decision Making Fairness

Text2Seg: Remote Sensing Image Semantic Segmentation via Text-Guided Visual Foundation Models

1 code implementation20 Apr 2023 Jielu Zhang, Zhongliang Zhou, Gengchen Mai, Mengxuan Hu, Zihan Guan, Sheng Li, Lan Mu

As image databases grow each year, performing automatic segmentation with deep learning models has gradually become the standard approach for processing the data.

Instance Segmentation Segmentation +4

AGI for Agriculture

no code implementations12 Apr 2023 Guoyu Lu, Sheng Li, Gengchen Mai, Jin Sun, Dajiang Zhu, Lilong Chai, Haijian Sun, Xianqiao Wang, Haixing Dai, Ninghao Liu, Rui Xu, Daniel Petti, Tianming Liu, Changying Li

Artificial General Intelligence (AGI) is poised to revolutionize a variety of sectors, including healthcare, finance, transportation, and education.

Decision Making Knowledge Graphs +1

EVKG: An Interlinked and Interoperable Electric Vehicle Knowledge Graph for Smart Transportation System

1 code implementation10 Apr 2023 Yanlin Qi, Gengchen Mai, Rui Zhu, Michael Zhang

Over the past decade, the electric vehicle industry has experienced unprecedented growth and diversification, resulting in a complex ecosystem.

Decision Making Knowledge Graphs

Towards General-Purpose Representation Learning of Polygonal Geometries

1 code implementation29 Sep 2022 Gengchen Mai, Chiyu Jiang, Weiwei Sun, Rui Zhu, Yao Xuan, Ling Cai, Krzysztof Janowicz, Stefano Ermon, Ni Lao

For the spatial domain approach, we propose ResNet1D, a 1D CNN-based polygon encoder, which uses circular padding to achieve loop origin invariance on simple polygons.

Relation Prediction Representation Learning

Sphere2Vec: Multi-Scale Representation Learning over a Spherical Surface for Geospatial Predictions

no code implementations25 Jan 2022 Gengchen Mai, Yao Xuan, Wenyun Zuo, Krzysztof Janowicz, Ni Lao

However, a map projection distortion problem rises when applying location encoding models to large-scale real-world GPS coordinate datasets (e. g., species images taken all over the world) - all current location encoding models are designed for encoding points in a 2D (Euclidean) space but not on a spherical surface, e. g., earth surface.

Image Classification Representation Learning

Narrative Cartography with Knowledge Graphs

1 code implementation2 Dec 2021 Gengchen Mai, Weiming Huang, Ling Cai, Rui Zhu, Ni Lao

With the help of this tool, the retrieved data from KGs are directly materialized in a GIS format which is ready for spatial analysis and mapping.

Knowledge Graphs

Time in a Box: Advancing Knowledge Graph Completion with Temporal Scopes

1 code implementation12 Nov 2021 Ling Cai, Krzysztof Janowic, Bo Yan, Rui Zhu, Gengchen Mai

Hence, knowledge base completion (KBC) on temporal knowledge bases (TKB), where each statement \textit{may} be associated with a temporal scope, has attracted growing attention.

Knowledge Base Completion Knowledge Graph Completion +3

A Review of Location Encoding for GeoAI: Methods and Applications

no code implementations7 Nov 2021 Gengchen Mai, Krzysztof Janowicz, Yingjie Hu, Song Gao, Bo Yan, Rui Zhu, Ling Cai, Ni Lao

A common need for artificial intelligence models in the broader geoscience is to represent and encode various types of spatial data, such as points (e. g., points of interest), polylines (e. g., trajectories), polygons (e. g., administrative regions), graphs (e. g., transportation networks), or rasters (e. g., remote sensing images), in a hidden embedding space so that they can be readily incorporated into deep learning models.

Sphere2Vec: Self-Supervised Location Representation Learning on Spherical Surfaces

no code implementations29 Sep 2021 Gengchen Mai, Yao Xuan, Wenyun Zuo, Yutong He, Stefano Ermon, Jiaming Song, Krzysztof Janowicz, Ni Lao

Location encoding is valuable for a multitude of tasks where both the absolute positions and local contexts (image, text, and other types of metadata) of spatial objects are needed for accurate predictions.

Image Classification Representation Learning +1

Geographic Question Answering: Challenges, Uniqueness, Classification, and Future Directions

no code implementations19 May 2021 Gengchen Mai, Krzysztof Janowicz, Rui Zhu, Ling Cai, Ni Lao

As an important part of Artificial Intelligence (AI), Question Answering (QA) aims at generating answers to questions phrased in natural language.

Classification Geographic Question Answering +1

Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting

no code implementations11 Jun 2020 Ling Cai, Krzysztof Janowicz, Gengchen Mai, Bo Yan, Rui Zhu

In this work we propose a novel deep learning architecture called Traffic Transformer to capture the continuity and periodicity of time series and to model spatial dependency.

Machine Translation Time Series +3

SE-KGE: A Location-Aware Knowledge Graph Embedding Model for Geographic Question Answering and Spatial Semantic Lifting

1 code implementation25 Apr 2020 Gengchen Mai, Krzysztof Janowicz, Ling Cai, Rui Zhu, Blake Regalia, Bo Yan, Meilin Shi, Ni Lao

We also construct a geographic knowledge graph as well as a set of geographic query-answer pairs called DBGeo to evaluate the performance of SE-KGE in comparison to multiple baselines.

Geographic Question Answering Information Retrieval +5

Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells

2 code implementations ICLR 2020 Gengchen Mai, Krzysztof Janowicz, Bo Yan, Rui Zhu, Ling Cai, Ni Lao

The key idea is to use neural networks to convert words in texts to vector space representations based on word positions in a sentence and their contexts, which are suitable for end-to-end training of downstream tasks.

Image Classification Representation Learning +1

TransGCN:Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction

no code implementations1 Oct 2019 Ling Cai, Bo Yan, Gengchen Mai, Krzysztof Janowicz, Rui Zhu

Link prediction is an important and frequently studied task that contributes to an understanding of the structure of knowledge graphs (KGs) in statistical relational learning.

Decoder Entity Embeddings +5

POIReviewQA: A Semantically Enriched POI Retrieval and Question Answering Dataset

no code implementations5 Oct 2018 Gengchen Mai, Krzysztof Janowicz, Cheng He, Sumang Liu, Ni Lao

To test a system's ability to understand the text we adopt an information retrieval evaluation by ranking all the review sentences for a question based on the likelihood that they answer this question.

Information Retrieval Question Answering +4

Cannot find the paper you are looking for? You can Submit a new open access paper.