Search Results for author: Wei-Lun Chao

Found 64 papers, 39 papers with code

Dual-View Visual Contextualization for Web Navigation

no code implementations6 Feb 2024 Jihyung Kil, Chan Hee Song, Boyuan Zheng, Xiang Deng, Yu Su, Wei-Lun Chao

Automatic web navigation aims to build a web agent that can follow language instructions to execute complex and diverse tasks on real-world websites.

Bringing Back the Context: Camera Trap Species Identification as Link Prediction on Multimodal Knowledge Graphs

no code implementations31 Dec 2023 Vardaan Pahuja, Weidi Luo, Yu Gu, Cheng-Hao Tu, Hong-You Chen, Tanya Berger-Wolf, Charles Stewart, Song Gao, Wei-Lun Chao, Yu Su

In this work, we leverage the structured context associated with the camera trap images to improve out-of-distribution generalization for the task of species identification in camera traps.

Knowledge Graphs Link Prediction +1

BioCLIP: A Vision Foundation Model for the Tree of Life

1 code implementation30 Nov 2023 Samuel Stevens, Jiaman Wu, Matthew J Thompson, Elizabeth G Campolongo, Chan Hee Song, David Edward Carlyn, Li Dong, Wasila M Dahdul, Charles Stewart, Tanya Berger-Wolf, Wei-Lun Chao, Yu Su

We then develop BioCLIP, a foundation model for the tree of life, leveraging the unique properties of biology captured by TreeOfLife-10M, namely the abundance and variety of images of plants, animals, and fungi, together with the availability of rich structured biological knowledge.

A Simple Interpretable Transformer for Fine-Grained Image Classification and Analysis

1 code implementation7 Nov 2023 Dipanjyoti Paul, Arpita Chowdhury, Xinqi Xiong, Feng-Ju Chang, David Carlyn, Samuel Stevens, Kaiya Provost, Anuj Karpatne, Bryan Carstens, Daniel Rubenstein, Charles Stewart, Tanya Berger-Wolf, Yu Su, Wei-Lun Chao

Unlike mainstream classifiers that wait until the last fully-connected layer to incorporate class information to make predictions, we investigate a proactive approach, asking each class to search for itself in an image.

Fine-Grained Image Classification

Pre-Training LiDAR-Based 3D Object Detectors Through Colorization

1 code implementation23 Oct 2023 Tai-Yu Pan, Chenyang Ma, Tianle Chen, Cheng Perng Phoo, Katie Z Luo, Yurong You, Mark Campbell, Kilian Q. Weinberger, Bharath Hariharan, Wei-Lun Chao

Accurate 3D object detection and understanding for self-driving cars heavily relies on LiDAR point clouds, necessitating large amounts of labeled data to train.

3D Object Detection Colorization +4

FLEE-GNN: A Federated Learning System for Edge-Enhanced Graph Neural Network in Analyzing Geospatial Resilience of Multicommodity Food Flows

1 code implementation20 Oct 2023 Yuxiao Qu, Jinmeng Rao, Song Gao, Qianheng Zhang, Wei-Lun Chao, Yu Su, Michelle Miller, Alfonso Morales, Patrick Huber

This paper proposes FLEE-GNN, a novel Federated Learning System for Edge-Enhanced Graph Neural Network, designed to overcome these challenges and enhance the analysis of geospatial resilience of multicommodity food flow network, which is one type of spatial networks.

Federated Learning

Towards Open-World Segmentation of Parts

1 code implementation CVPR 2023 Tai-Yu Pan, Qing Liu, Wei-Lun Chao, Brian Price

Second, we introduce a novel approach to improve part segmentation on unseen objects, inspired by an interesting finding -- for unseen objects, the pixel-wise features extracted by the model often reveal high-quality part segments.

Contrastive Learning Segmentation

Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentation

1 code implementation9 May 2023 Tianle Chen, Zheda Mai, Ruiwen Li, Wei-Lun Chao

Weakly supervised semantic segmentation (WSSS) aims to bypass the need for laborious pixel-level annotation by using only image-level annotation.

Object Pseudo Label +2

Unified Out-Of-Distribution Detection: A Model-Specific Perspective

no code implementations ICCV 2023 Reza Averly, Wei-Lun Chao

We show that this framework unifies the detection of OOD examples caused by semantic shift and covariate shift, and closely addresses the concern of applying a machine learning model to uncontrolled environments.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

Unsupervised Adaptation from Repeated Traversals for Autonomous Driving

1 code implementation27 Mar 2023 Yurong You, Cheng Perng Phoo, Katie Z Luo, Travis Zhang, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger

For a self-driving car to operate reliably, its perceptual system must generalize to the end-user's environment -- ideally without additional annotation efforts.

3D Object Detection Autonomous Driving +2

Learning Fractals by Gradient Descent

1 code implementation14 Mar 2023 Cheng-Hao Tu, Hong-You Chen, David Carlyn, Wei-Lun Chao

Fractals are geometric shapes that can display complex and self-similar patterns found in nature (e. g., clouds and plants).

Making Batch Normalization Great in Federated Deep Learning

no code implementations12 Mar 2023 Jike Zhong, Hong-You Chen, Wei-Lun Chao

We reinvestigate factors that are believed to cause this problem, including the mismatch of BN statistics across clients and the deviation of gradients during local training.

Federated Learning

Train-Once-for-All Personalization

no code implementations CVPR 2023 Hong-You Chen, Yandong Li, Yin Cui, Mingda Zhang, Wei-Lun Chao, Li Zhang

We study the problem of how to train a "personalization-friendly" model such that given only the task descriptions, the model can be adapted to different end-users' needs, e. g., for accurately classifying different subsets of objects.

LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models

no code implementations ICCV 2023 Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M. Sadler, Wei-Lun Chao, Yu Su

In this work, we propose a novel method, LLM-Planner, that harnesses the power of large language models to do few-shot planning for embodied agents.

Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning

1 code implementation CVPR 2023 Cheng-Hao Tu, Zheda Mai, Wei-Lun Chao

Through introducing a handful of learnable ``query'' tokens to each layer, VQT leverages the inner workings of Transformers to ``summarize'' rich intermediate features of each layer, which can then be used to train the prediction heads of downstream tasks.

Transfer Learning

Image-to-Image Translation for Autonomous Driving from Coarsely-Aligned Image Pairs

no code implementations23 Sep 2022 Youya Xia, Josephine Monica, Wei-Lun Chao, Bharath Hariharan, Kilian Q Weinberger, Mark Campbell

In this paper, we investigate the idea of turning sensor inputs (i. e., images) captured in an adverse condition into a benign one (i. e., sunny), upon which the downstream tasks (e. g., semantic segmentation) can attain high accuracy.

Autonomous Driving Image-to-Image Translation +4

PreSTU: Pre-Training for Scene-Text Understanding

no code implementations ICCV 2023 Jihyung Kil, Soravit Changpinyo, Xi Chen, Hexiang Hu, Sebastian Goodman, Wei-Lun Chao, Radu Soricut

The ability to recognize and reason about text embedded in visual inputs is often lacking in vision-and-language (V&L) models, perhaps because V&L pre-training methods have often failed to include such an ability in their training objective.

Image Captioning Optical Character Recognition (OCR) +2

Gradual Domain Adaptation without Indexed Intermediate Domains

1 code implementation NeurIPS 2021 Hong-You Chen, Wei-Lun Chao

This coarse domain sequence then undergoes a fine indexing step via a novel cycle-consistency loss, which encourages the next intermediate domain to preserve sufficient discriminative knowledge of the current intermediate domain.

Unsupervised Domain Adaptation

On the Importance and Applicability of Pre-Training for Federated Learning

1 code implementation23 Jun 2022 Hong-You Chen, Cheng-Hao Tu, Ziwei Li, Han-Wei Shen, Wei-Lun Chao

To make our findings applicable to situations where pre-trained models are not directly available, we explore pre-training with synthetic data or even with clients' data in a decentralized manner, and found that they can already improve FL notably.

Federated Learning

Learning with Free Object Segments for Long-Tailed Instance Segmentation

no code implementations22 Feb 2022 Cheng Zhang, Tai-Yu Pan, Tianle Chen, Jike Zhong, WenJin Fu, Wei-Lun Chao

One fundamental challenge in building an instance segmentation model for a large number of classes in complex scenes is the lack of training examples, especially for rare objects.

Instance Segmentation Object +1

One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones

1 code implementation CVPR 2022 Chan Hee Song, Jihyung Kil, Tai-Yu Pan, Brian M. Sadler, Wei-Lun Chao, Yu Su

We study the problem of developing autonomous agents that can follow human instructions to infer and perform a sequence of actions to complete the underlying task.

Vision and Language Navigation

Two-Stage Mesh Deep Learning for Automated Tooth Segmentation and Landmark Localization on 3D Intraoral Scans

no code implementations24 Sep 2021 Tai-Hsien Wu, Chunfeng Lian, Sanghee Lee, Matthew Pastewait, Christian Piers, Jie Liu, Fang Wang, Li Wang, Chiung-Ying Chiu, Wenchi Wang, Christina Jackson, Wei-Lun Chao, Dinggang Shen, Ching-Chang Ko

Our TS-MDL first adopts an end-to-end \emph{i}MeshSegNet method (i. e., a variant of the existing MeshSegNet with both improved accuracy and efficiency) to label each tooth on the downsampled scan.

Code Generation

On Model Calibration for Long-Tailed Object Detection and Instance Segmentation

1 code implementation NeurIPS 2021 Tai-Yu Pan, Cheng Zhang, Yandong Li, Hexiang Hu, Dong Xuan, Soravit Changpinyo, Boqing Gong, Wei-Lun Chao

We propose NorCal, Normalized Calibration for long-tailed object detection and instance segmentation, a simple and straightforward recipe that reweighs the predicted scores of each class by its training sample size.

Instance Segmentation Long-tailed Object Detection +4

On Bridging Generic and Personalized Federated Learning for Image Classification

3 code implementations ICLR 2022 Hong-You Chen, Wei-Lun Chao

On the one hand, we introduce a family of losses that are robust to non-identical class distributions, enabling clients to train a generic predictor with a consistent objective across them.

Classification Image Classification +1

Few-Shot Learning with a Strong Teacher

1 code implementation1 Jul 2021 Han-Jia Ye, Lu Ming, De-Chuan Zhan, Wei-Lun Chao

Many existing works take the meta-learning approach, constructing a few-shot learner that can learn from few-shot examples to generate a classifier.

Few-Shot Learning

How to Train Your MAML to Excel in Few-Shot Classification

1 code implementation ICLR 2022 Han-Jia Ye, Wei-Lun Chao

We find that these permutations lead to a huge variance of accuracy, making MAML unstable in few-shot classification.

Classification Meta-Learning

Procrustean Training for Imbalanced Deep Learning

no code implementations ICCV 2021 Han-Jia Ye, De-Chuan Zhan, Wei-Lun Chao

To correct these wrong predictions, the neural network then must focus on pushing features of minor class data across the decision boundaries between major and minor classes, leading to much larger gradients for features of minor classes.

Attribute

MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection

1 code implementation ICCV 2021 Cheng Zhang, Tai-Yu Pan, Yandong Li, Hexiang Hu, Dong Xuan, Soravit Changpinyo, Boqing Gong, Wei-Lun Chao

Many objects do not appear frequently enough in complex scenes (e. g., certain handbags in living rooms) for training an accurate object detector, but are often found frequently by themselves (e. g., in product images).

Imputation Instance Segmentation +5

FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning

2 code implementations ICLR 2021 Hong-You Chen, Wei-Lun Chao

Federated learning aims to collaboratively train a strong global model by accessing users' locally trained models but not their own data.

Bayesian Inference Federated Learning

Deep Co-Training with Task Decomposition for Semi-Supervised Domain Adaptation

1 code implementation ICCV 2021 Luyu Yang, Yan Wang, Mingfei Gao, Abhinav Shrivastava, Kilian Q. Weinberger, Wei-Lun Chao, Ser-Nam Lim

To integrate the strengths of the two classifiers, we apply the well-established co-training framework, in which the two classifiers exchange their high confident predictions to iteratively "teach each other" so that both classifiers can excel in the target domain.

Semi-supervised Domain Adaptation Unsupervised Domain Adaptation

Interactive Natural Language-based Person Search

1 code implementation19 Feb 2020 Vikram Shree, Wei-Lun Chao, Mark Campbell

In this work, we consider the problem of searching people in an unconstrained environment, with natural language descriptions.

Person Search Question Answering

An Empirical Study of Person Re-Identification with Attributes

1 code implementation25 Jan 2020 Vikram Shree, Wei-Lun Chao, Mark Campbell

Person re-identification aims to identify a person from an image collection, given one image of that person as the query.

Attribute Person Re-Identification

Visual Question Answering on 360° Images

no code implementations10 Jan 2020 Shih-Han Chou, Wei-Lun Chao, Wei-Sheng Lai, Min Sun, Ming-Hsuan Yang

We then study two different VQA models on VQA 360, including one conventional model that takes an equirectangular image (with intrinsic distortion) as input and one dedicated model that first projects a 360 image onto cubemaps and subsequently aggregates the information from multiple spatial resolutions.

Question Answering Visual Question Answering

Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning

1 code implementation6 Jan 2020 Han-Jia Ye, Hong-You Chen, De-Chuan Zhan, Wei-Lun Chao

Classifiers trained with class-imbalanced data are known to perform poorly on test data of the "minor" classes, of which we have insufficient training data.

LDLS: 3-D Object Segmentation Through Label Diffusion From 2-D Images

1 code implementation30 Oct 2019 Brian H. Wang, Wei-Lun Chao, Yan Wang, Bharath Hariharan, Kilian Q. Weinberger, Mark Campbell

We obtain 2-D segmentation predictions by applying Mask-RCNN to the RGB image, and then link this image to a 3-D lidar point cloud by building a graph of connections among 3-D points and 2-D pixels.

Image Segmentation Point Cloud Segmentation +2

A New Defense Against Adversarial Images: Turning a Weakness into a Strength

1 code implementation NeurIPS 2019 Tao Yu, Shengyuan Hu, Chuan Guo, Wei-Lun Chao, Kilian Q. Weinberger

Natural images are virtually surrounded by low-density misclassified regions that can be efficiently discovered by gradient-guided search --- enabling the generation of adversarial images.

Adversarial Defense

An Empirical Study on Leveraging Scene Graphs for Visual Question Answering

no code implementations28 Jul 2019 Cheng Zhang, Wei-Lun Chao, Dong Xuan

Specifically, we investigate the use of scene graphs derived from images for Visual QA: an image is abstractly represented by a graph with nodes corresponding to object entities and edges to object relationships.

Knowledge Graphs Question Answering +1

Classifier and Exemplar Synthesis for Zero-Shot Learning

1 code implementation16 Dec 2018 Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, Fei Sha

Zero-shot learning (ZSL) enables solving a task without the need to see its examples.

Denoising Zero-Shot Learning

Cross-Dataset Adaptation for Visual Question Answering

no code implementations CVPR 2018 Wei-Lun Chao, Hexiang Hu, Fei Sha

Analogous to domain adaptation for visual recognition, this setting is appealing when the target dataset does not have a sufficient amount of labeled data to learn an "in-domain" model.

Domain Adaptation Question Answering +1

Learning Answer Embeddings for Visual Question Answering

no code implementations CVPR 2018 Hexiang Hu, Wei-Lun Chao, Fei Sha

These properties make the approach particularly appealing for transfer learning for open-ended Visual QA, where the source dataset on which the model is learned has limited overlapping with the target dataset in the space of answers.

Question Answering Transfer Learning +1

Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasets

no code implementations NAACL 2018 Wei-Lun Chao, Hexiang Hu, Fei Sha

We apply the procedures to re-construct decoy answers for two popular Visual QA datasets as well as to create a new Visual QA dataset from the Visual Genome project, resulting in the largest dataset for this task.

Multiple-choice Question Answering +1

Video Summarization with Long Short-term Memory

1 code implementation26 May 2016 Ke Zhang, Wei-Lun Chao, Fei Sha, Kristen Grauman

We propose a novel supervised learning technique for summarizing videos by automatically selecting keyframes or key subshots.

Domain Adaptation Structured Prediction +1

Predicting Visual Exemplars of Unseen Classes for Zero-Shot Learning

no code implementations ICCV 2017 Soravit Changpinyo, Wei-Lun Chao, Fei Sha

Leveraging class semantic descriptions and examples of known objects, zero-shot learning makes it possible to train a recognition model for an object class whose examples are not available.

Clustering Object +1

An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild

1 code implementation13 May 2016 Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, Fei Sha

Zero-shot learning (ZSL) methods have been studied in the unrealistic setting where test data are assumed to come from unseen classes only.

Few-Shot Learning Generalized Zero-Shot Learning +1

Summary Transfer: Exemplar-based Subset Selection for Video Summarization

no code implementations CVPR 2016 Ke Zhang, Wei-Lun Chao, Fei Sha, Kristen Grauman

Video summarization has unprecedented importance to help us digest, browse, and search today's ever-growing video collections.

Video Summarization

Synthesized Classifiers for Zero-Shot Learning

2 code implementations CVPR 2016 Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, Fei Sha

Given semantic descriptions of object classes, zero-shot learning aims to accurately recognize objects of the unseen classes, from which no examples are available at the training stage, by associating them to the seen classes, from which labeled examples are provided.

Object Zero-Shot Learning

Large-Margin Determinantal Point Processes

no code implementations6 Nov 2014 Boqing Gong, Wei-Lun Chao, Kristen Grauman, Fei Sha

Extensive empirical studies validate our contributions, including applications on challenging document and video summarization, where flexibility in modeling the kernel matrix and balancing different errors is indispensable.

Point Processes Video Summarization

Cannot find the paper you are looking for? You can Submit a new open access paper.