Search Results for author: Boqing Gong

Found 70 papers, 33 papers with code

CrossVQA: Scalably Generating Benchmarks for Systematically Testing VQA Generalization

no code implementations EMNLP 2021 Arjun Akula, Soravit Changpinyo, Boqing Gong, Piyush Sharma, Song-Chun Zhu, Radu Soricut

One challenge in evaluating visual question answering (VQA) models in the cross-dataset adaptation setting is that the distribution shifts are multi-modal, making it difficult to identify if it is the shifts in visual or language features that play a key role.

Question-Answer-Generation Question Answering +1

Synthesize Policies for Transfer and Adaptation across Tasks and Environments

no code implementations NeurIPS 2018 Hexiang Hu, Liyu Chen, Boqing Gong, Fei Sha

The ability to transfer in reinforcement learning is key towards building an agent of general artificial intelligence.

Anti-Neuron Watermarking: Protecting Personal Data Against Unauthorized Neural Model Training

1 code implementation18 Sep 2021 Zihang Zou, Boqing Gong, Liqiang Wang

By embedding a watermarking signature using specialized linear color transformation to user images, neural models will be imprinted with such a signature if training data include watermarked images.

Federated Multi-Target Domain Adaptation

no code implementations17 Aug 2021 Chun-Han Yao, Boqing Gong, Yin Cui, Hang Qi, Yukun Zhu, Ming-Hsuan Yang

We further take the server-client and inter-client domain shifts into account and pose a domain adaptation problem with one source (centralized server data) and multiple targets (distributed client data).

Domain Adaptation Federated Learning +3

On Model Calibration for Long-Tailed Object Detection and Instance Segmentation

1 code implementation NeurIPS 2021 Tai-Yu Pan, Cheng Zhang, Yandong Li, Hexiang Hu, Dong Xuan, Soravit Changpinyo, Boqing Gong, Wei-Lun Chao

We propose NorCal, Normalized Calibration for long-tailed object detection and instance segmentation, a simple and straightforward recipe that reweighs the predicted scores of each class by its training sample size.

Instance Segmentation Object Detection +1

Bridging the Gap Between Object Detection and User Intent via Query-Modulation

no code implementations18 Jun 2021 Marco Fornoni, Chaochao Yan, Liangchen Luo, Kimberly Wilber, Alex Stark, Yin Cui, Boqing Gong, Andrew Howard

This often leads to incorrect results, such as lack of a high-confidence detection on the object of interest, or detection with a wrong class label.

Object Detection

When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations

2 code implementations3 Jun 2021 Xiangning Chen, Cho-Jui Hsieh, Boqing Gong

Vision Transformers (ViTs) and MLPs signal further efforts on replacing hand-wired features or inductive biases with general-purpose neural architectures.

 Ranked #1 on Domain Generalization on ImageNet-R (Top 1 Accuracy metric)

Domain Generalization Fine-Grained Image Classification +1

2.5D Visual Relationship Detection

1 code implementation26 Apr 2021 Yu-Chuan Su, Soravit Changpinyo, Xiangning Chen, Sathish Thoppay, Cho-Jui Hsieh, Lior Shapira, Radu Soricut, Hartwig Adam, Matthew Brown, Ming-Hsuan Yang, Boqing Gong

To enable progress on this task, we create a new dataset consisting of 220k human-annotated 2. 5D relationships among 512K objects from 11K images.

Depth Estimation Visual Relationship Detection

VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

2 code implementations NeurIPS 2021 Hassan Akbari, Liangzhe Yuan, Rui Qian, Wei-Hong Chuang, Shih-Fu Chang, Yin Cui, Boqing Gong

We train VATT end-to-end from scratch using multimodal contrastive losses and evaluate its performance by the downstream tasks of video action recognition, audio event classification, image classification, and text-to-video retrieval.

 Ranked #1 on Action Classification on Moments in Time (using extra training data)

Action Classification Action Recognition +7

Class-Balanced Distillation for Long-Tailed Visual Recognition

1 code implementation12 Apr 2021 Ahmet Iscen, André Araujo, Boqing Gong, Cordelia Schmid

In this work, we introduce a new framework, by making the key observation that a feature representation learned with instance sampling is far from optimal in a long-tailed setting.

Knowledge Distillation

Robust and Accurate Object Detection via Adversarial Learning

1 code implementation CVPR 2021 Xiangning Chen, Cihang Xie, Mingxing Tan, Li Zhang, Cho-Jui Hsieh, Boqing Gong

Data augmentation has become a de facto component for training high-performance deep image classifiers, but its potential is under-explored for object detection.

AutoML Data Augmentation +2

MoViNets: Mobile Video Networks for Efficient Video Recognition

2 code implementations CVPR 2021 Dan Kondratyuk, Liangzhe Yuan, Yandong Li, Li Zhang, Mingxing Tan, Matthew Brown, Boqing Gong

We present Mobile Video Networks (MoViNets), a family of computation and memory efficient video networks that can operate on streaming video for online inference.

Action Classification Action Recognition +2

MosaicOS: A Simple and Effective Use of Object-Centric Images for Long-Tailed Object Detection

1 code implementation ICCV 2021 Cheng Zhang, Tai-Yu Pan, Yandong Li, Hexiang Hu, Dong Xuan, Soravit Changpinyo, Boqing Gong, Wei-Lun Chao

Many objects do not appear frequently enough in complex scenes (e. g., certain handbags in living rooms) for training an accurate object detector, but are often found frequently by themselves (e. g., in product images).

Imputation Instance Segmentation +2

Large-Scale Meta-Learning with Continual Trajectory Shifting

no code implementations14 Feb 2021 Jaewoong Shin, Hae Beom Lee, Boqing Gong, Sung Ju Hwang

Meta-learning of shared initialization parameters has shown to be highly effective in solving few-shot learning tasks.

Few-Shot Learning Fine-tuning +1

Contrastive Learning for Label Efficient Semantic Segmentation

no code implementations ICCV 2021 Xiangyun Zhao, Raviteja Vemulapalli, Philip Andrew Mansfield, Boqing Gong, Bradley Green, Lior Shapira, Ying Wu

While recent Convolutional Neural Network (CNN) based semantic segmentation approaches have achieved impressive results by using large amounts of labeled training data, their performance drops significantly as the amount of labeled data decreases.

Contrastive Learning Semantic Segmentation

Contrastive Learning for Label-Efficient Semantic Segmentation

no code implementations13 Dec 2020 Xiangyun Zhao, Raviteja Vemulapalli, Philip Mansfield, Boqing Gong, Bradley Green, Lior Shapira, Ying Wu

While recent Convolutional Neural Network (CNN) based semantic segmentation approaches have achieved impressive results by using large amounts of labeled training data, their performance drops significantly as the amount of labeled data decreases.

Contrastive Learning Semantic Segmentation

Ranking Neural Checkpoints

no code implementations CVPR 2021 Yandong Li, Xuhui Jia, Ruoxin Sang, Yukun Zhu, Bradley Green, Liqiang Wang, Boqing Gong

This paper is concerned with ranking many pre-trained deep neural networks (DNNs), called checkpoints, for the transfer learning to a downstream task.

Transfer Learning

Spatiotemporal Contrastive Video Representation Learning

3 code implementations CVPR 2021 Rui Qian, Tianjian Meng, Boqing Gong, Ming-Hsuan Yang, Huisheng Wang, Serge Belongie, Yin Cui

Our representations are learned using a contrastive loss, where two augmented clips from the same short video are pulled together in the embedding space, while clips from different videos are pushed away.

 Ranked #1 on Self-Supervised Action Recognition on Kinetics-400 (using extra training data)

Contrastive Learning Data Augmentation +4

Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds

no code implementations CVPR 2021 Li Yi, Boqing Gong, Thomas Funkhouser

We study an unsupervised domain adaptation problem for the semantic labeling of 3D point clouds, with a particular focus on domain discrepancies induced by different LiDAR sensors.

Semantic Segmentation Unsupervised Domain Adaptation

Smooth Adversarial Training

1 code implementation25 Jun 2020 Cihang Xie, Mingxing Tan, Boqing Gong, Alan Yuille, Quoc V. Le

SAT also works well with larger networks: it helps EfficientNet-L1 to achieve 82. 2% accuracy and 58. 6% robustness on ImageNet, outperforming the previous state-of-the-art defense by 9. 5% for accuracy and 11. 6% for robustness.

Adversarial Defense Adversarial Robustness

When Ensembling Smaller Models is More Efficient than Single Large Models

no code implementations1 May 2020 Dan Kondratyuk, Mingxing Tan, Matthew Brown, Boqing Gong

Ensembling is a simple and popular technique for boosting evaluation performance by training multiple models (e. g., with different initializations) and aggregating their predictions.

Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Model

no code implementations CVPR 2020 Dongdong Wang, Yandong Li, Liqiang Wang, Boqing Gong

The other is that the number of images used for the knowledge distillation should be small; otherwise, it violates our expectation of reducing the dependence on large-scale datasets.

Active Learning Knowledge Distillation

Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective

1 code implementation CVPR 2020 Muhammad Abdullah Jamal, Matthew Brown, Ming-Hsuan Yang, Liqiang Wang, Boqing Gong

Object frequency in the real world often follows a power law, leading to a mismatch between datasets with long-tailed class distributions seen by a machine learning model and our expectation of the model to perform well on all classes.

Domain Adaptation Long-tail Learning +1

MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius

2 code implementations ICLR 2020 Runtian Zhai, Chen Dan, Di He, huan zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, Li-Wei Wang

Adversarial training is one of the most popular ways to learn robust models but is usually attack-dependent and time costly.

Look, Listen, and Act: Towards Audio-Visual Embodied Navigation

1 code implementation25 Dec 2019 Chuang Gan, Yiwei Zhang, Jiajun Wu, Boqing Gong, Joshua B. Tenenbaum

In this paper, we attempt to approach the problem of Audio-Visual Embodied Navigation, the task of planning the shortest path from a random starting location in a scene to the sound source in an indoor environment, given only raw egocentric visual and audio sensory data.

Adversarial Examples Improve Image Recognition

6 code implementations CVPR 2020 Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan Yuille, Quoc V. Le

We show that AdvProp improves a wide range of models on various image recognition tasks and performs better when the models are bigger.

Image Classification

Open Compound Domain Adaptation

no code implementations CVPR 2020 Ziwei Liu, Zhongqi Miao, Xingang Pan, Xiaohang Zhan, Dahua Lin, Stella X. Yu, Boqing Gong

A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e. g., sunny weather) for achieving high performance on the test data in a target domain (e. g., rainy weather).

Domain Adaptation Facial Expression Recognition +1

Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach

1 code implementation ICCV 2019 Qing Lian, Fengmao Lv, Lixin Duan, Boqing Gong

We propose a new approach, called self-motivated pyramid curriculum domain adaptation (PyCDA), to facilitate the adaptation of semantic segmentation neural networks from synthetic source domains to real target domains.

Semantic Segmentation Unsupervised Domain Adaptation

Defending Against Adversarial Attacks Using Random Forests

no code implementations16 Jun 2019 Yifan Ding, Liqiang Wang, huan zhang, Jin-Feng Yi, Deliang Fan, Boqing Gong

As deep neural networks (DNNs) have become increasingly important and popular, the robustness of DNNs is the key to the safety of both the Internet and the physical world.

CAMOU: Learning Physical Vehicle Camouflages to Adversarially Attack Detectors in the Wild

1 code implementation ICLR 2019 Yang Zhang, Hassan Foroosh, Philip David, Boqing Gong

In particular, we learn a camouflage pattern to hide vehicles from being detected by state-of-the-art convolutional neural network based detectors.

Adversarial Attack

DHER: Hindsight Experience Replay for Dynamic Goals

1 code implementation ICLR 2019 Meng Fang, Cheng Zhou, Bei Shi, Boqing Gong, Jia Xu, Tong Zhang

Dealing with sparse rewards is one of the most important challenges in reinforcement learning (RL), especially when a goal is dynamic (e. g., to grasp a moving object).

Object Tracking

NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks

1 code implementation1 May 2019 Yandong Li, Lijun Li, Liqiang Wang, Tong Zhang, Boqing Gong

Powerful adversarial attack methods are vital for understanding how to construct robust deep neural networks (DNNs) and for thoroughly testing defense techniques.

Adversarial Attack

Large-Scale Long-Tailed Recognition in an Open World

1 code implementation CVPR 2019 Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, Stella X. Yu

We define Open Long-Tailed Recognition (OLTR) as learning from such naturally distributed data and optimizing the classification accuracy over a balanced test set which include head, tail, and open classes.

Classification Few-Shot Learning +4

Synthesized Policies for Transfer and Adaptation across Tasks and Environments

2 code implementations NeurIPS 2018 Hexiang Hu, Liyu Chen, Boqing Gong, Fei Sha

The ability to transfer in reinforcement learning is key towards building an agent of general artificial intelligence.

Joint Modeling of Dense and Incomplete Trajectories for Citywide Traffic Volume Inference

no code implementations25 Feb 2019 Xianfeng Tang, Boqing Gong, Yanwei Yu, Huaxiu Yao, Yandong Li, Haiyong Xie, Xiaoyu Wang

In this paper, we propose a novel framework for the citywide traffic volume inference using both dense GPS trajectories and incomplete trajectories captured by camera surveillance systems.

Graph Embedding

Classifier and Exemplar Synthesis for Zero-Shot Learning

1 code implementation16 Dec 2018 Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, Fei Sha

Zero-shot learning (ZSL) enables solving a task without the need to see its examples.

Denoising Zero-Shot Learning

Controllable Image-to-Video Translation: A Case Study on Facial Expression Generation

no code implementations9 Aug 2018 Lijie Fan, Wenbing Huang, Chuang Gan, Junzhou Huang, Boqing Gong

The recent advances in deep learning have made it possible to generate photo-realistic images by using neural networks and even to extrapolate video frames from an input video clip.

Image-to-Image Translation Translation +1

Improving Sequential Determinantal Point Processes for Supervised Video Summarization

no code implementations ECCV 2018 Aidean Sharghi, Ali Borji, Chengtao Li, Tianbao Yang, Boqing Gong

In terms of modeling, we design a new probabilistic distribution such that, when it is integrated into SeqDPP, the resulting model accepts user input about the expected length of the summary.

Point Processes Supervised Video Summarization

Optimize Deep Convolutional Neural Network with Ternarized Weights and High Accuracy

no code implementations20 Jul 2018 Zhezhi He, Boqing Gong, Deliang Fan

Deep convolution neural network has achieved great success in many artificial intelligence applications.

Model Compression

Geometry Guided Convolutional Neural Networks for Self-Supervised Video Representation Learning

no code implementations CVPR 2018 Chuang Gan, Boqing Gong, Kun Liu, Hao Su, Leonidas J. Guibas

In addition, we also find that a progressive training strategy can foster a better neural network for the video recognition task than blindly pooling the distinct sources of geometry cues together.

Action Recognition Representation Learning +4

End-to-End Learning of Motion Representation for Video Understanding

1 code implementation CVPR 2018 Lijie Fan, Wenbing Huang, Chuang Gan, Stefano Ermon, Boqing Gong, Junzhou Huang

Despite the recent success of end-to-end learned representations, hand-crafted optical flow features are still widely used in video analysis tasks.

Action Recognition Optical Flow Estimation +1

End-to-End Video Captioning with Multitask Reinforcement Learning

1 code implementation21 Mar 2018 Lijun Li, Boqing Gong

Although end-to-end (E2E) learning has led to impressive progress on a variety of visual understanding tasks, it is often impeded by hardware constraints (e. g., GPU memory) and is prone to overfitting.

Video Captioning

Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect

1 code implementation ICLR 2018 Xiang Wei, Boqing Gong, Zixia Liu, Wei Lu, Liqiang Wang

Despite being impactful on a variety of problems and applications, the generative adversarial nets (GANs) are remarkably difficult to train.

A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels

no code implementations8 Feb 2018 Yifan Ding, Liqiang Wang, Deliang Fan, Boqing Gong

In the first stage, we identify a small portion of images from the noisy training set of which the labels are correct with a high probability.

Blind Pre-Processing: A Robust Defense Method Against Adversarial Examples

no code implementations5 Feb 2018 Adnan Siraj Rakin, Zhezhi He, Boqing Gong, Deliang Fan

Blind pre-processing improves the white box attack accuracy of MNIST from 94. 3\% to 98. 7\%.

Adversarial Attack

VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentation

1 code implementation ICCV 2017 Chuang Gan, Yandong Li, Haoxiang Li, Chen Sun, Boqing Gong

Many seemingly distant annotations (e. g., semantic segmentation and visual question answering (VQA)) are inherently connected in that they reveal different levels and perspectives of human understandings about the same visual scenes --- and even the same set of images (e. g., of COCO).

Language Modelling Language understanding +3

Query-Focused Video Summarization: Dataset, Evaluation, and A Memory Network Based Approach

no code implementations CVPR 2017 Aidean Sharghi, Jacob S. Laurel, Boqing Gong

However, one of the main obstacles to the research on video summarization is the user subjectivity - users have various preferences over the summaries.

Video Summarization

Infinite-Label Learning with Semantic Output Codes

no code implementations23 Aug 2016 Yang Zhang, Rupam Acharyya, Ji Liu, Boqing Gong

We develop a new statistical machine learning paradigm, named infinite-label learning, to annotate a data point with more than one relevant labels from a candidate set, which pools both the finite labels observed at training and a potentially infinite number of previously unseen labels.

Multi-Label Learning Zero-Shot Learning

Query-Focused Extractive Video Summarization

no code implementations18 Jul 2016 Aidean Sharghi, Boqing Gong, Mubarak Shah

The decision to include a shot in the summary depends on the shot's relevance to the user query and importance in the context of the video, jointly.

Video Summarization

Fast Zero-Shot Image Tagging

no code implementations CVPR 2016 Yang Zhang, Boqing Gong, Mubarak Shah

The well-known word analogy experiments show that the recent word vectors capture fine-grained linguistic regularities in words by linear vector offsets, but it is unclear how well the simple vector offsets can encode visual regularities over words.

Multi-label zero-shot learning

An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild

1 code implementation13 May 2016 Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, Fei Sha

Zero-shot learning (ZSL) methods have been studied in the unrealistic setting where test data are assumed to come from unseen classes only.

Few-Shot Learning Generalized Zero-Shot Learning +1

Learning Attributes Equals Multi-Source Domain Generalization

no code implementations CVPR 2016 Chuang Gan, Tianbao Yang, Boqing Gong

Attributes possess appealing properties and benefit many computer vision problems, such as object recognition, learning with humans in the loop, and image retrieval.

Domain Generalization Image Retrieval +1

Synthesized Classifiers for Zero-Shot Learning

2 code implementations CVPR 2016 Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, Fei Sha

Given semantic descriptions of object classes, zero-shot learning aims to accurately recognize objects of the unseen classes, from which no examples are available at the training stage, by associating them to the seen classes, from which labeled examples are provided.

Zero-Shot Learning

Improved Dropout for Shallow and Deep Learning

no code implementations NeurIPS 2016 Zhe Li, Boqing Gong, Tianbao Yang

To exhibit the optimal dropout probabilities, we analyze the shallow learning with multinomial dropout and establish the risk bound for stochastic optimization.

Stochastic Optimization

Large-Margin Determinantal Point Processes

no code implementations6 Nov 2014 Boqing Gong, Wei-Lun Chao, Kristen Grauman, Fei Sha

Extensive empirical studies validate our contributions, including applications on challenging document and video summarization, where flexibility in modeling the kernel matrix and balancing different errors is indispensable.

Point Processes Video Summarization

Reshaping Visual Datasets for Domain Adaptation

no code implementations NeurIPS 2013 Boqing Gong, Kristen Grauman, Fei Sha

By maximum distinctiveness, we require the underlying distributions of the identified domains to be different from each other; by maximum learnability, we ensure that a strong discriminative model can be learned from the domain.

Activity Recognition Domain Adaptation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.