Search Results for author: Jiebo Luo

Found 191 papers, 57 papers with code

Deep Federated Anomaly Detection for Multivariate Time Series Data

no code implementations9 May 2022 Wei Zhu, Dongjin Song, Yuncong Chen, Wei Cheng, Bo Zong, Takehiko Mizoguchi, Cristian Lumezanu, Haifeng Chen, Jiebo Luo

Specifically, we first design an Exemplar-based Deep Neural network (ExDNN) to learn local time series representations based on their compatibility with an exemplar module which consists of hidden parameters learned to capture varieties of normal patterns on each edge device.

Federated Learning Time Series +1

Localized Adversarial Domain Generalization

1 code implementation9 May 2022 Wei Zhu, Le Lu, Jing Xiao, Mei Han, Jiebo Luo, Adam P. Harrison

Adversarial domain generalization is a popular approach to DG, but conventional approaches (1) struggle to sufficiently align features so that local neighborhoods are mixed across domains; and (2) can suffer from feature space over collapse which can threaten generalization performance.

Domain Generalization

Explainable Fairness in Recommendation

no code implementations24 Apr 2022 Yingqiang Ge, Juntao Tan, Yan Zhu, Yinglong Xia, Jiebo Luo, Shuchang Liu, Zuohui Fu, Shijie Geng, Zelong Li, Yongfeng Zhang

In this paper, we study the problem of explainable fairness, which helps to gain insights about why a system is fair or unfair, and guides the design of fair recommender systems with a more informed and unified methodology.

Fairness Recommendation Systems

CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training

1 code implementation22 Mar 2022 Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Eli Shechtman, Connelly Barnes, Jianming Zhang, Ning Xu, Sohrab Amirghodsi, Jiebo Luo

Recent image inpainting methods have made great progress but often struggle to generate plausible image structures when dealing with large holes in complex images.

Image Inpainting

Breast Cancer Induced Bone Osteolysis Prediction Using Temporal Variational Auto-Encoders

no code implementations20 Mar 2022 Wei Xiong, Neil Yeung, Shubo Wang, Haofu Liao, Liyun Wang, Jiebo Luo

Its ability of predicting the development of bone lesions in cancer-invading bones can assist in assessing the risk of impending fractures and choosing proper treatments in breast cancer bone metastasis.

Computed Tomography (CT)

Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning

no code implementations12 Mar 2022 Kai Zhu, Wei Zhai, Yang Cao, Jiebo Luo, Zheng-Jun Zha

Non-exemplar class-incremental learning is to recognize both the old and new classes when old class samples cannot be saved.

class-incremental learning Incremental Learning +1

RawlsGCN: Towards Rawlsian Difference Principle on Graph Convolutional Network

no code implementations28 Feb 2022 Jian Kang, Yan Zhu, Yinglong Xia, Jiebo Luo, Hanghang Tong

Graph Convolutional Network (GCN) plays pivotal roles in many real-world applications.

Point Cloud Denoising via Momentum Ascent in Gradient Fields

no code implementations21 Feb 2022 Yaping Zhao, Haitian Zheng, Zhongrui Wang, Jiebo Luo, Edmund Y. Lam

To achieve point cloud denoising, traditional methods heavily rely on geometric priors, and most learning-based approaches suffer from outliers and loss of details.


MANet: Improving Video Denoising with a Multi-Alignment Network

no code implementations20 Feb 2022 Yaping Zhao, Haitian Zheng, Zhongrui Wang, Jiebo Luo, Edmund Y. Lam

In video denoising, the adjacent frames often provide very useful information, but accurate alignment is needed before such information can be harnassed.

Denoising Video Denoising

Cross-modal Contrastive Distillation for Instructional Activity Anticipation

no code implementations18 Jan 2022 Zhengyuan Yang, Jingen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, Jiebo Luo

In this study, we aim to predict the plausible future action steps given an observation of the past and study the task of instructional activity anticipation.

Knowledge Distillation

Multi-modal Dependency Tree for Video Captioning

no code implementations NeurIPS 2021 Wentian Zhao, Xinxiao wu, Jiebo Luo

To this end, we propose a novel video captioning method that generates a sentence by first constructing a multi-modal dependency tree and then traversing the constructed tree, where the syntactic structure and semantic relationship in the sentence are represented by the tree topology.

Dependency Parsing Text Generation +1

SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing

no code implementations30 Nov 2021 Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, Chenliang Xu

Recently, large pretrained models (e. g., BERT, StyleGAN, CLIP) have shown great knowledge transfer and generalization capability on various downstream tasks within their domains.

Image-to-Image Translation Transfer Learning

Music Sentiment Transfer

1 code implementation12 Oct 2021 Miles Sigel, Michael Zhou, Jiebo Luo

Results and literature suggest that the task of music sentiment transfer is more difficult than image sentiment transfer because of the temporal characteristics of music and lack of existing datasets.

Style Transfer

Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning

no code implementations ICCV 2021 Jing Bi, Jiebo Luo, Chenliang Xu

In this work, we leverage instructional videos to study humans' decision-making processes, focusing on learning a model to plan goal-directed actions in real-life videos.

Action Recognition Bayesian Inference +2

CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentation

no code implementations30 Sep 2021 Xiao Wang, Jingen Liu, Tao Mei, Jiebo Luo

Unlike the mainstream clustering-based methods, our framework exploits a transformer-based feature reconstruction scheme to detect event boundary by reconstruction errors.

Boundary Detection Event Segmentation +2

Federated Learning of Molecular Properties with Graph Neural Networks in a Heterogeneous Setting

no code implementations15 Sep 2021 Wei Zhu, Jiebo Luo, Andrew White

FLIT(+) can align the local training across heterogeneous clients by improving the performance for uncertain samples.

Federated Learning

Learning to Aggregate and Refine Noisy Labels for Visual Sentiment Analysis

no code implementations15 Sep 2021 Wei Zhu, Zihe Zheng, Haitian Zheng, Hanjia Lyu, Jiebo Luo

The learned prototypes and their labels can be regarded as denoising features and labels for the local regions and can guide the training process to prevent the model from overfitting the noisy cases.

Denoising Learning with noisy labels +1

LibFewShot: A Comprehensive Library for Few-shot Learning

1 code implementation10 Sep 2021 Wenbin Li, Chuanqi Dong, Pinzhuo Tian, Tiexin Qin, Xuesong Yang, Ziyi Wang, Jing Huo, Yinghuan Shi, Lei Wang, Yang Gao, Jiebo Luo

Furthermore, based on LibFewShot, we provide comprehensive evaluations on multiple benchmark datasets with multiple backbone architectures to evaluate common pitfalls and effects of different training tricks.

Data Augmentation Few-Shot Image Classification +1

Learning Fine-Grained Motion Embedding for Landscape Animation

no code implementations6 Sep 2021 Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo

To tackle this problem, we propose a model named FGLA to generate high-quality and realistic videos by learning Fine-Grained motion embedding for Landscape Animation.

Multi-Modulation Network for Audio-Visual Event Localization

no code implementations26 Aug 2021 Hao Wang, Zheng-Jun Zha, Liang Li, Xuejin Chen, Jiebo Luo

We propose a novel MultiModulation Network (M2N) to learn the above correlation and leverage it as semantic guidance to modulate the related auditory, visual, and fused features.

audio-visual event localization

UniFaceGAN: A Unified Framework for Temporally Consistent Facial Video Editing

no code implementations12 Aug 2021 Meng Cao, HaoZhi Huang, Hao Wang, Xuan Wang, Li Shen, Sheng Wang, Linchao Bao, Zhifeng Li, Jiebo Luo

Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.

3D Reconstruction Face Reenactment +3

Boosting Entity-aware Image Captioning with Multi-modal Knowledge Graph

no code implementations26 Jul 2021 Wentian Zhao, Yao Hu, HeDa Wang, Xinxiao wu, Jiebo Luo

Entity-aware image captioning aims to describe named entities and events related to the image by utilizing the background knowledge in the associated article.

Graph Attention Image Captioning

Adaptive Recursive Circle Framework for Fine-grained Action Recognition

no code implementations25 Jul 2021 Hanxi Lin, Xinxiao wu, Jiebo Luo

It inherits the operators and parameters of the original layer but is slightly different in the use of those operators and parameters.

Fine-grained Action Recognition

Triplet is All You Need with Random Mappings for Unsupervised Visual Representation Learning

no code implementations22 Jul 2021 Wenbin Li, Xuesong Yang, Meihao Kong, Lei Wang, Jing Huo, Yang Gao, Jiebo Luo

However, this type of methods, such as SimCLR and MoCo, relies heavily on a large number of negative pairs and thus requires either large batches or memory banks.

Representation Learning Self-Supervised Learning

Improving OCR-Based Image Captioning by Incorporating Geometrical Relationship

no code implementations CVPR 2021 Jing Wang, Jinhui Tang, Mingkun Yang, Xiang Bai, Jiebo Luo

Under the guidance of the geometrical relationship between OCR tokens, our LSTM-R capitalizes on a newly-devised relation-aware pointer network to select OCR tokens from the scene text for OCR-based image captioning.

Image Captioning Optical Character Recognition

Structured Multi-Level Interaction Network for Video Moment Localization via Language Query

no code implementations CVPR 2021 Hao Wang, Zheng-Jun Zha, Liang Li, Dong Liu, Jiebo Luo

In particular, for cross-modal interaction, we interact the sentence-level query with the whole moment while interact the word-level query with content and boundary, as in a coarse-to-fine manner.


How COVID-19 Has Changed Crowdfunding: Evidence From GoFundMe

no code implementations18 Jun 2021 Junda Wang, Xupin Zhang, Jiebo Luo

More importantly, sentiment analysis and the paired sample t-test are performed to examine the differences in crowdfunding campaigns before and after the COVID-19 outbreak that started in March 2020.

Sentiment Analysis

SAT: 2D Semantics Assisted Training for 3D Visual Grounding

1 code implementation ICCV 2021 Zhengyuan Yang, Songyang Zhang, LiWei Wang, Jiebo Luo

3D visual grounding aims at grounding a natural language description about a 3D scene, usually represented in the form of 3D point clouds, to the targeted object region.

Representation Learning Visual Grounding

Few-shot Partial Multi-view Learning

no code implementations5 May 2021 Yuan Zhou, Yanrong Guo, Shijie Hao, Richang Hong, Jiebo Luo

The challenges of this task are twofold: (1) under the interference of the missing views, it is difficult to overcome the negative impact brought by data scarcity; (2) the limited number of data exacerbates information scarcity, thereby making it harder to address the view-missing problem.


Video-aided Unsupervised Grammar Induction

1 code implementation NAACL 2021 Songyang Zhang, Linfeng Song, Lifeng Jin, Kun Xu, Dong Yu, Jiebo Luo

We investigate video-aided grammar induction, which learns a constituency parser from both unlabeled text and its corresponding video.

Optical Character Recognition

Facial Attribute Transformers for Precise and Robust Makeup Transfer

no code implementations7 Apr 2021 Zhaoyi Wan, Haoran Chen, Jielei Zhang, Wentao Jiang, Cong Yao, Jiebo Luo

In this paper, we address the problem of makeup transfer, which aims at transplanting the makeup from the reference face to the source face while preserving the identity of the source.

Face Generation

ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows

1 code implementation CVPR 2021 Jie An, Siyu Huang, Yibing Song, Dejing Dou, Wei Liu, Jiebo Luo

The forward inference projects input images into deep features, while the backward inference remaps deep features back to input images in a lossless and unbiased way.

Style Transfer

Memory Enhanced Embedding Learning for Cross-Modal Video-Text Retrieval

no code implementations29 Mar 2021 Rui Zhao, Kecheng Zheng, Zheng-Jun Zha, Hongtao Xie, Jiebo Luo

The cross-modal memory module is employed to record the instance embeddings of all the datasets for global negative mining.

Video-Text Retrieval

Few-Shot Learning for Video Object Detection in a Transfer-Learning Scheme

no code implementations26 Mar 2021 Zhongjie Yu, Gaoang Wang, Lin Chen, Sebastian Raschka, Jiebo Luo

We employ a transfer-learning framework to effectively train the video object detector on a large number of base-class objects and a few video clips of novel-class objects.

Few-Shot Video Object Detection Transfer Learning +1

Group-aware Label Transfer for Domain Adaptive Person Re-identification

1 code implementation CVPR 2021 Kecheng Zheng, Wu Liu, Lingxiao He, Tao Mei, Jiebo Luo, Zheng-Jun Zha

In this paper, we propose a Group-aware Label Transfer (GLT) algorithm, which enables the online interaction and mutual promotion of pseudo-label prediction and representation learning.

Domain Adaptive Person Re-Identification Online Clustering +2

From Static to Dynamic Prediction: Wildfire Risk Assessment Based on Multiple Environmental Factors

no code implementations14 Mar 2021 Tanqiu Jiang, Sidhant K. Bendre, Hanjia Lyu, Jiebo Luo

Wildfire is one of the biggest disasters that frequently occurs on the west coast of the United States.

Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning

1 code implementation5 Mar 2021 Jinsong Su, Jialong Tang, Hui Jiang, Ziyao Lu, Yubin Ge, Linfeng Song, Deyi Xiong, Le Sun, Jiebo Luo

In aspect-based sentiment analysis (ABSA), many neural models are equipped with an attention mechanism to quantify the contribution of each context word to sentiment prediction.

Aspect-Based Sentiment Analysis

DAIL: Dataset-Aware and Invariant Learning for Face Recognition

no code implementations14 Jan 2021 Gaoang Wang, Lin Chen, Tianqiang Liu, Mingwei He, Jiebo Luo

To solve the first issue of identity overlapping, we propose a dataset-aware loss for multi-dataset training by reducing the penalty when the same person appears in multiple datasets.

Domain Adaptation Face Recognition

Semantic Layout Manipulation with High-Resolution Sparse Attention

1 code implementation14 Dec 2020 Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Jianming Zhang, Ning Xu, Jiebo Luo

A core problem of this task is how to transfer visual details from the input images to the new semantic layout while making the resulting image visually realistic.

TAP: Text-Aware Pre-training for Text-VQA and Text-Caption

1 code implementation CVPR 2021 Zhengyuan Yang, Yijuan Lu, JianFeng Wang, Xi Yin, Dinei Florencio, Lijuan Wang, Cha Zhang, Lei Zhang, Jiebo Luo

Due to this aligned representation learning, even pre-trained on the same downstream task dataset, TAP already boosts the absolute accuracy on the TextVQA dataset by +5. 4%, compared with a non-TAP baseline.

Language Modelling Masked Language Modeling +5

Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language

1 code implementation4 Dec 2020 Songyang Zhang, Houwen Peng, Jianlong Fu, Yijuan Lu, Jiebo Luo

It is a challenging problem because a target moment may take place in the context of other temporal moments in the untrimmed video.

Social Media Study of Public Opinions on Potential COVID-19 Vaccines: Informing Dissent, Disparities, and Dissemination

no code implementations3 Dec 2020 Hanjia Lyu, Wei Wu, Junda Wang, Viet Duong, Xiyang Zhang, Jiebo Luo

People who have the worst personal pandemic experience are more likely to hold the anti-vaccine opinion.

Social and Information Networks

Learning Semantic-aware Normalization for Generative Adversarial Networks

1 code implementation NeurIPS 2020 Heliang Zheng, Jianlong Fu, Yanhong Zeng, Jiebo Luo, Zheng-Jun Zha

Such a model disentangles latent factors according to the semantic of feature channels by channel-/group- wise fusion of latent codes and feature channels.

Image Inpainting Unconditional Image Generation

Slender Object Detection: Diagnoses and Improvements

1 code implementation17 Nov 2020 Zhaoyi Wan, Yimin Chen, Sutao Deng, Kunpeng Chen, Cong Yao, Jiebo Luo

In this paper, we are concerned with the detection of a particular type of objects with extreme aspect ratios, namely \textbf{slender objects}.

Object Detection

Content-based Analysis of the Cultural Differences between TikTok and Douyin

no code implementations3 Nov 2020 Li Sun, Haoqi Zhang, Songyang Zhang, Jiebo Luo

Short-form video social media shifts away from the traditional media paradigm by telling the audience a dynamic story to attract their attention.

Object Detection

Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation

no code implementations30 Oct 2020 Zhengyuan Yang, Amanda Kay, Yuncheng Li, Wendi Cross, Jiebo Luo

We then evaluate the framework on a proposed URMC dataset, which consists of conversations between a standardized patient and a behavioral health professional, along with expert annotations of body language, emotions, and potential psychiatric symptoms.

Action Recognition Emotion Recognition

Region Comparison Network for Interpretable Few-shot Image Classification

1 code implementation8 Sep 2020 Zhiyu Xue, Lixin Duan, Wen Li, Lin Chen, Jiebo Luo

For that, in this work, we propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works as in a neural network as well as to find out specific regions that are related to each other in images coming from the query and support sets.

Classification Few-Shot Image Classification +2

Dynamic Context-guided Capsule Network for Multimodal Machine Translation

1 code implementation4 Sep 2020 Huan Lin, Fandong Meng, Jinsong Su, Yongjing Yin, Zhengyuan Yang, Yubin Ge, Jie zhou, Jiebo Luo

Particularly, we represent the input image with global and regional visual features, we introduce two parallel DCCNs to model multimodal context vectors with visual features at different granularities.

Multimodal Machine Translation Representation Learning +1

Learning to Localize Actions from Moments

1 code implementation ECCV 2020 Fuchen Long, Ting Yao, Zhaofan Qiu, Xinmei Tian, Jiebo Luo, Tao Mei

In this paper, we introduce a new design of transfer learning type to learn action localization for a large set of action categories, but only on action moments from the categories of interest and temporal annotations of untrimmed videos from a small set of action classes.

Action Localization Transfer Learning

A Smartphone-based System for Real-time Early Childhood Caries Diagnosis

no code implementations17 Aug 2020 Yi-Peng Zhang, Haofu Liao, Jin Xiao, Nisreen Al Jallad, Oriana Ly-Mapes, Jiebo Luo

The identification of ECC in an early stage usually requires expertise in the field, and hence is often ignored by parents.

Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification

3 code implementations ECCV 2020 Mang Ye, Jianbing Shen, David J. Crandall, Ling Shao, Jiebo Luo

In this paper, we propose a novel dynamic dual-attentive aggregation (DDAG) learning method by mining both intra-modality part-level and cross-modality graph-level contextual cues for VI-ReID.

Person Re-Identification

Universal Model for Multi-Domain Medical Image Retrieval

no code implementations14 Jul 2020 Yang Feng, Yubao Liu, Jiebo Luo

Usually, one image retrieval model is only trained to handle images from one modality or one source.

Medical Image Retrieval

Task-agnostic Temporally Consistent Facial Video Editing

no code implementations3 Jul 2020 Meng Cao, Hao-Zhi Huang, Hao Wang, Xuan Wang, Li Shen, Sheng Wang, Linchao Bao, Zhifeng Li, Jiebo Luo

Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.

3D Reconstruction Frame

Monitoring Depression Trend on Twitter during the COVID-19 Pandemic

no code implementations1 Jul 2020 Yi-Peng Zhang, Hanjia Lyu, Yubao Liu, Xiyang Zhang, Yu Wang, Jiebo Luo

The COVID-19 pandemic has severely affected people's daily lives and caused tremendous economic loss worldwide.

Global Image Sentiment Transfer

no code implementations22 Jun 2020 Jie An, Tianlang Chen, Songyang Zhang, Jiebo Luo

This work proposes a novel framework consisting of a reference image retrieval step and a global sentiment transfer step to transfer sentiments of images according to a given sentiment tag.

Image Retrieval SSIM +2

Image Sentiment Transfer

no code implementations19 Jun 2020 Tianlang Chen, Wei Xiong, Haitian Zheng, Jiebo Luo

In this paper, we propose an effective and flexible framework that performs image sentiment transfer at the object level.

Disentanglement Image-to-Image Translation +1

Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning

no code implementations16 Jun 2020 Jie An, Tao Li, Hao-Zhi Huang, Li Shen, Xuan Wang, Yongyi Tang, Jinwen Ma, Wei Liu, Jiebo Luo

Extracting effective deep features to represent content and style information is the key to universal style transfer.

Style Transfer

Personalized Fashion Recommendation from Personal Social Media Data: An Item-to-Set Metric Learning Approach

no code implementations25 May 2020 Haitian Zheng, Kefei Wu, Jong-Hwi Park, Wei Zhu, Jiebo Luo

In this work, we study the problem of personalized fashion recommendation from social media data, i. e. recommending new outfits to social media users that fit their fashion preferences.

Metric Learning

On Vocabulary Reliance in Scene Text Recognition

no code implementations CVPR 2020 Zhaoyi Wan, Jielei Zhang, Liang Zhang, Jiebo Luo, Cong Yao

This remedy alleviates the problem of vocabulary reliance and improves the overall scene text recognition performance.

Scene Text Recognition

Unsupervised Low-light Image Enhancement with Decoupled Networks

no code implementations6 May 2020 Wei Xiong, Ding Liu, Xiaohui Shen, Chen Fang, Jiebo Luo

In this paper, we tackle the problem of enhancing real-world low-light images with significant noise in an unsupervised fashion.

Image-to-Image Translation Low-Light Image Enhancement

Alleviating the Incompatibility between Cross Entropy Loss and Episode Training for Few-shot Skin Disease Classification

no code implementations21 Apr 2020 Wei Zhu, Haofu Liao, Wenbin Li, Weijian Li, Jiebo Luo

Inspired by the recent success of Few-Shot Learning (FSL) in natural image classification, we propose to apply FSL to skin disease identification to address the extreme scarcity of training sample problem.

Few-Shot Learning General Classification +2

The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic

no code implementations21 Apr 2020 Viet Duong, Phu Pham, Tongyu Yang, Yu Wang, Jiebo Luo

Recently, the pandemic of the novel Coronavirus Disease-2019 (COVID-19) has presented governments with ultimate challenges.

In the Eyes of the Beholder: Analyzing Social Media Use of Neutral and Controversial Terms for COVID-19

no code implementations21 Apr 2020 Long Chen, Hanjia Lyu, Tongyu Yang, Yu Wang, Jiebo Luo

To model the substantive difference of tweets with controversial terms and those with non-controversial terms, we apply topic modeling and LIWC-based sentiment analysis.

Sentiment Analysis

Unsupervised Learning of Landmarks based on Inter-Intra Subject Consistencies

1 code implementation16 Apr 2020 Weijian Li, Haofu Liao, Shun Miao, Le Lu, Jiebo Luo

To recover from the transformed images back to the original subject, the landmark detector is forced to learn spatial locations that contain the consistent semantic meanings both for the paired intra-subject images and between the paired inter-subject images.

TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images

1 code implementation ECCV 2020 Jianxin Lin, Yingxue Pang, Yingce Xia, Zhibo Chen, Jiebo Luo

With TuiGAN, an image is translated in a coarse-to-fine manner where the generated image is gradually refined from global structures to local details.

Translation Unsupervised Image-To-Image Translation

Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection

no code implementations CVPR 2020 Jie Chen, Zhiheng Li, Jiebo Luo, Chenliang Xu

Instead of blindly trusting quality-inconsistent PAs, WS^2 employs a learning-based selection to select effective PAs and a novel region integrity criterion as a stopping condition for weakly-supervised training.

Action Segmentation Semantic Segmentation +2

Adaptive Offline Quintuplet Loss for Image-Text Matching

1 code implementation ECCV 2020 Tianlang Chen, Jiajun Deng, Jiebo Luo

For each image or text anchor in a training mini-batch, the model is trained to distinguish between a positive and the most confusing negative of the anchor mined from the mini-batch (i. e. online hard negative).

Text Matching

Expressing Objects just like Words: Recurrent Visual Embedding for Image-Text Matching

no code implementations20 Feb 2020 Tianlang Chen, Jiebo Luo

Existing image-text matching approaches typically infer the similarity of an image-text pair by capturing and aggregating the affinities between the text and each independent object of the image.

Text Matching text similarity +1

Asymmetric Distribution Measure for Few-shot Learning

no code implementations1 Feb 2020 Wenbin Li, Lei Wang, Jing Huo, Yinghuan Shi, Yang Gao, Jiebo Luo

Given the natural asymmetric relation between a query image and a support class, we argue that an asymmetric measure is more suitable for metric-based few-shot learning.

Few-Shot Image Classification

Mi YouTube es Su YouTube? Analyzing the Cultures using YouTube Thumbnails of Popular Videos

no code implementations27 Jan 2020 Songyang Zhang, Tolga Aktas, Jiebo Luo

In this study, we explore culture preferences among countries using the thumbnails of YouTube trending videos.

#MeToo on Campus: Studying College Sexual Assault at Scale Using Data Reported on Social Media

no code implementations16 Jan 2020 Viet Duong, Phu Pham, Ritwik Bose, Jiebo Luo

Recently, the emergence of the #MeToo trend on social media has empowered thousands of people to share their own sexual harassment experiences.

Fine-grained Image-to-Image Transformation towards Visual Recognition

no code implementations CVPR 2020 Wei Xiong, Yutong He, Yixuan Zhang, Wenhan Luo, Lin Ma, Jiebo Luo

In this paper, we aim at transforming an image with a fine-grained category to synthesize new images that preserve the identity of the input image, which can thereby benefit the subsequent fine-grained image recognition and few-shot learning tasks.

Few-Shot Learning Fine-Grained Image Recognition

TransMatch: A Transfer-Learning Scheme for Semi-Supervised Few-Shot Learning

no code implementations CVPR 2020 Zhongjie Yu, Lin Chen, Zhongwei Cheng, Jiebo Luo

Under the proposed framework, we develop a novel method for semi-supervised few-shot learning called TransMatch by instantiating the three components with Imprinting and MixMatch.

Few-Shot Learning Transfer Learning

Neural Simile Recognition with Cyclic Multitask Learning and Local Attention

1 code implementation19 Dec 2019 Jiali Zeng, Linfeng Song, Jinsong Su, Jun Xie, Wei Song, Jiebo Luo

Simile recognition is to detect simile sentences and to extract simile components, i. e., tenors and vehicles.

Sentence Classification

Iterative Dual Domain Adaptation for Neural Machine Translation

no code implementations IJCNLP 2019 Jiali Zeng, Yang Liu, Jinsong Su, Yubin Ge, Yaojie Lu, Yongjing Yin, Jiebo Luo

Previous studies on the domain adaptation for neural machine translation (NMT) mainly focus on the one-pass transferring out-of-domain translation knowledge to in-domain NMT model.

Domain Adaptation Knowledge Distillation +3

Graph-based Neural Sentence Ordering

1 code implementation16 Dec 2019 Yongjing Yin, Linfeng Song, Jinsong Su, Jiali Zeng, Chulun Zhou, Jiebo Luo

Sentence ordering is to restore the original paragraph from a set of sentences.

Sentence Ordering


no code implementations13 Dec 2019 Zhengyuan Yang, Tushar Kumar, Tianlang Chen, Jinsong Su, Jiebo Luo

In this paper, we study Tracking by Language that localizes the target box sequence in a video based on a language query.


Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language

3 code implementations8 Dec 2019 Songyang Zhang, Houwen Peng, Jianlong Fu, Jiebo Luo

We address the problem of retrieving a specific moment from an untrimmed video by a query sentence.

Learning Sparse 2D Temporal Adjacent Networks for Temporal Action Localization

2 code implementations8 Dec 2019 Songyang Zhang, Houwen Peng, Le Yang, Jianlong Fu, Jiebo Luo

In this report, we introduce the Winner method for HACS Temporal Action Localization Challenge 2019.

Temporal Action Localization

Ultrafast Photorealistic Style Transfer via Neural Architecture Search

no code implementations5 Dec 2019 Jie An, Haoyi Xiong, Jun Huan, Jiebo Luo

Our method consists of a construction step (C-step) to build a photorealistic stylization network and a pruning step (P-step) for acceleration.

Network Pruning Neural Architecture Search +1

Defensive Few-shot Adversarial Learning

no code implementations16 Nov 2019 Wenbin Li, Lei Wang, Xingxing Zhang, Jing Huo, Yang Gao, Jiebo Luo

In this paper, instead of assuming such a distribution consistency, we propose to make this assumption at a task-level in the episodic training paradigm in order to better transfer the defense knowledge.

Adversarial Defense Few-Shot Learning

Open-Ended Visual Question Answering by Multi-Modal Domain Adaptation

no code implementations Findings of the Association for Computational Linguistics 2020 Yiming Xu, Lin Chen, Zhongwei Cheng, Lixin Duan, Jiebo Luo

A straightforward solution is to fine-tune a pre-trained source model by using those limited labeled target data, but it usually cannot work well due to the considerable difference between the data distributions of the source and target domains.

Domain Adaptation Question Answering +2

Learning Deep Bilinear Transformation for Fine-grained Image Representation

1 code implementation NeurIPS 2019 Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo

However, the computational cost to learn pairwise interactions between deep feature channels is prohibitively expensive, which restricts this powerful transformation to be used in deep neural networks.

Fine-Grained Image Recognition

SMP Challenge: An Overview of Social Media Prediction Challenge 2019

no code implementations4 Oct 2019 Bo Wu, Wen-Huang Cheng, Peiye Liu, Bei Liu, Zhaoyang Zeng, Jiebo Luo

In the SMP Challenge at ACM Multimedia 2019, we introduce a novel prediction task Temporal Popularity Prediction, which focuses on predicting future interaction or attractiveness (in terms of clicks, views or likes etc.)

Unsupervised Pose Flow Learning for Pose Guided Synthesis

no code implementations30 Sep 2019 Haitian Zheng, Lele Chen, Chenliang Xu, Jiebo Luo

Pose guided synthesis aims to generate a new image in an arbitrary target pose while preserving the appearance details from the source image.

Large-scale Tag-based Font Retrieval with Generative Feature Learning

no code implementations ICCV 2019 Tianlang Chen, Zhaowen Wang, Ning Xu, Hailin Jin, Jiebo Luo

In this paper, we address the problem of large-scale tag-based font retrieval which aims to bring semantics to the font selection process and enable people without expert knowledge to use fonts effectively.


Exploiting Temporal Relationships in Video Moment Localization with Natural Language

1 code implementation11 Aug 2019 Songyang Zhang, Jinsong Su, Jiebo Luo

We address the problem of video moment localization with natural language, i. e. localizing a video segment described by a natural language sentence.

Semi-Supervised Adversarial Monocular Depth Estimation

no code implementations6 Aug 2019 Rongrong Ji, Ke Li, Yan Wang, Xiaoshuai Sun, Feng Guo, Xiaowei Guo, Yongjian Wu, Feiyue Huang, Jiebo Luo

In this paper, we address the problem of monocular depth estimation when only a limited number of training image-depth pairs are available.

Monocular Depth Estimation

ADN: Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction

1 code implementation3 Aug 2019 Haofu Liao, Wei-An Lin, S. Kevin Zhou, Jiebo Luo

Current deep neural network based approaches to computed tomography (CT) metal artifact reduction (MAR) are supervised methods that rely on synthesized metal artifacts for training.

Computed Tomography (CT) Disentanglement +4

Weakly Supervised Body Part Segmentation with Pose based Part Priors

no code implementations30 Jul 2019 Zhengyuan Yang, Yuncheng Li, Linjie Yang, Ning Zhang, Jiebo Luo

The core idea is first converting the sparse weak labels such as keypoints to the initial estimate of body part masks, and then iteratively refine the part mask predictions.

Face Parsing Semantic Segmentation

Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichment

no code implementations22 Jul 2019 Jianbo Yuan, Haofu Liao, Rui Luo, Jiebo Luo

In addition, in order to enrich the decoder with descriptive semantics and enforce the correctness of the deterministic medical-related contents such as mentions of organs or diagnoses, we extract medical concepts based on the radiology reports in the training data and fine-tune the encoder to extract the most frequent medical concepts from the x-ray images.

Image Captioning Image Classification

Fast Universal Style Transfer for Artistic and Photorealistic Rendering

no code implementations6 Jul 2019 Jie An, Haoyi Xiong, Jiebo Luo, Jun Huan, Jinwen Ma

Given a pair of images as the source of content and the reference of style, existing solutions usually first train an auto-encoder (AE) to reconstruct the image using deep features and then embeds pre-defined style transfer modules into the AE reconstruction procedure to transfer the style of the reconstructed image through modifying the deep features.

Style Transfer

Uncovering Download Fraud Activities in Mobile App Markets

no code implementations5 Jul 2019 Yingtong Dou, Weijian Li, Zhirong Liu, Zhenhua Dong, Jiebo Luo, Philip S. Yu

To the best of our knowledge, this is the first work that investigates the download fraud problem in mobile App markets.

DuDoNet: Dual Domain Network for CT Metal Artifact Reduction

no code implementations CVPR 2019 Wei-An Lin, Haofu Liao, Cheng Peng, Xiaohang Sun, Jingdan Zhang, Jiebo Luo, Rama Chellappa, Shaohua Kevin Zhou

The linkage between the sigogram and image domains is a novel Radon inversion layer that allows the gradients to back-propagate from the image domain to the sinogram domain during training.

Computed Tomography (CT) Medical Diagnosis +1

Generative Mask Pyramid Network for CT/CBCT Metal Artifact Reduction with Joint Projection-Sinogram Correction

no code implementations29 Jun 2019 Haofu Liao, Wei-An Lin, Zhimin Huo, Levon Vogelsang, William J. Sehnert, S. Kevin Zhou, Jiebo Luo

A conventional approach to computed tomography (CT) or cone beam CT (CBCT) metal artifact reduction is to replace the X-ray projection data within the metal trace with synthesized data.

Computed Tomography (CT) Metal Artifact Reduction

Patch Transformer for Multi-tagging Whole Slide Histopathology Images

no code implementations10 Jun 2019 Weijian Li, Viet-Duy Nguyen, Haofu Liao, Matt Wilder, Ke Cheng, Jiebo Luo

Automated whole slide image (WSI) tagging has become a growing demand due to the increasing volume and diversity of WSIs collected nowadays in histopathology.


StyleNAS: An Empirical Study of Neural Architecture Search to Uncover Surprisingly Fast End-to-End Universal Style Transfer Networks

no code implementations6 Jun 2019 Jie An, Haoyi Xiong, Jinwen Ma, Jiebo Luo, Jun Huan

Finally compared to existing universal style transfer networks for photorealistic rendering such as PhotoWCT that stacks multiple well-trained auto-encoders and WCT transforms in a non-end-to-end manner, the architectures designed by StyleNAS produce better style-transferred images with details preserving, using a tiny number of operators/parameters, and enjoying around 500x inference time speed-up.

Image Classification Neural Architecture Search +3

Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction

1 code implementation5 Jun 2019 Haofu Liao, Wei-An Lin, Jianbo Yuan, S. Kevin Zhou, Jiebo Luo

Extensive experiments show that our method significantly outperforms the existing unsupervised models for image-to-image translation problems, and achieves comparable performance to existing supervised models on a synthesized dataset.

Computed Tomography (CT) Disentanglement +3

Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis

1 code implementation ACL 2019 Jialong Tang, Ziyao Lu, Jinsong Su, Yubin Ge, Linfeng Song, Le Sun, Jiebo Luo

In aspect-level sentiment classification (ASC), it is prevalent to equip dominant neural models with attention mechanisms, for the sake of acquiring the importance of each context word on the given aspect.

Aspect-Based Sentiment Analysis

Relational Reasoning using Prior Knowledge for Visual Captioning

no code implementations4 Jun 2019 Jingyi Hou, Xinxiao Wu, Yayun Qi, Wentian Zhao, Jiebo Luo, Yunde Jia

Extensive experiments on the MS-COCO image captioning benchmark and the MSVD video captioning benchmark validate the superiority of our method on leveraging prior commonsense knowledge to enhance relational reasoning for visual captioning.

Image Captioning Object Detection +2

Spatio-temporal Video Re-localization by Warp LSTM

no code implementations CVPR 2019 Yang Feng, Lin Ma, Wei Liu, Jiebo Luo

The need for efficiently finding the video content a user wants is increasing because of the erupting of user-generated videos on the Web.

Video Retrieval

Human-Centered Emotion Recognition in Animated GIFs

1 code implementation27 Apr 2019 Zhengyuan Yang, Yixuan Zhang, Jiebo Luo

The framework consists of a facial attention module and a hierarchical segment temporal module.

Emotion Recognition Frame

Revisiting Local Descriptor based Image-to-Class Measure for Few-shot Learning

1 code implementation CVPR 2019 Wenbin Li, Lei Wang, Jinglin Xu, Jing Huo, Yang Gao, Jiebo Luo

Its key difference from the literature is the replacement of the image-level feature based measure in the final layer by a local descriptor based image-to-class measure.

Few-Shot Image Classification General Classification

Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods

no code implementations27 Mar 2019 Guo-Jun Qi, Jiebo Luo

Representation learning with small labeled data have emerged in many problems, since the success of deep neural networks often relies on the availability of a huge amount of labeled data that is expensive to collect.

Domain Adaptation Representation Learning +1

Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation ($\text{POINT}^2$)

no code implementations10 Mar 2019 Haofu Liao, Wei-An Lin, Jiarui Zhang, Jingdan Zhang, Jiebo Luo, S. Kevin Zhou

As the POI tracker is shift-invariant, $\text{POINT}^2$ is more robust to the initial pose of the 3D pre-intervention image.

Foreground-aware Image Inpainting

no code implementations CVPR 2019 Wei Xiong, Jiahui Yu, Zhe Lin, Jimei Yang, Xin Lu, Connelly Barnes, Jiebo Luo

We show that by such disentanglement, the contour completion model predicts reasonable contours of objects, and further substantially improves the performance of image inpainting.

Disentanglement Image Inpainting

AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations rather than Data

1 code implementation CVPR 2019 Liheng Zhang, Guo-Jun Qi, Liqiang Wang, Jiebo Luo

The success of deep neural networks often relies on a large amount of labeled examples, which can be difficult to obtain in many real scenarios.

Representation Learning

More Knowledge is Better: Cross-Modality Volume Completion and 3D+2D Segmentation for Intracardiac Echocardiography Contouring

no code implementations9 Dec 2018 Haofu Liao, Yucheng Tang, Gareth Funka-Lea, Jiebo Luo, Shaohua Kevin Zhou

Using catheter ablation to treat atrial fibrillation increasingly relies on intracardiac echocardiography (ICE) for an anatomical delineation of the left atrium and the pulmonary veins that enter the atrium.

Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information

no code implementations9 Dec 2018 Haofu Liao, Addisu Mesfin, Jiebo Luo

For the long-range contextual information, we propose a multi-task bidirectional recurrent neural network (Bi-RNN) to encode the spatial and contextual information among the vertebrae of the visible spine column.

Joint Vertebrae Identification And Localization In Spinal Ct Images

Real-Time Referring Expression Comprehension by Single-Stage Grounding Network

no code implementations9 Dec 2018 Xinpeng Chen, Lin Ma, Jingyuan Chen, Zequn Jie, Wei Liu, Jiebo Luo

Experiments on RefCOCO, RefCOCO+, and RefCOCOg datasets demonstrate that our proposed SSG without relying on any region proposals can achieve comparable performance with other advanced models.

Referring Expression Referring Expression Comprehension

Adversarial Sparse-View CBCT Artifact Reduction

no code implementations9 Dec 2018 Haofu Liao, Zhimin Huo, William J. Sehnert, Shaohua Kevin Zhou, Jiebo Luo

We present an effective post-processing method to reduce the artifacts from sparsely reconstructed cone-beam CT (CBCT) images.

Cbct Artifact Reduction

Face Completion with Semantic Knowledge and Collaborative Adversarial Learning

no code implementations8 Dec 2018 Haofu Liao, Gareth Funka-Lea, Yefeng Zheng, Jiebo Luo, S. Kevin Zhou

Unlike a conventional background inpainting approach that infers a missing area from image patches similar to the background, face completion requires semantic knowledge about the target object for realistic outputs.

Facial Inpainting Semantic Segmentation

Unsupervised Image Captioning

1 code implementation CVPR 2019 Yang Feng, Lin Ma, Wei Liu, Jiebo Luo

Instead of relying on manually labeled image-sentence pairs, our proposed model merely requires an image set, a sentence corpus, and an existing visual concept detector.

Image Captioning

Attentive Relational Networks for Mapping Images to Scene Graphs

no code implementations CVPR 2019 Mengshi Qi, Weijian Li, Zhengyuan Yang, Yunhong Wang, Jiebo Luo

Scene graph generation refers to the task of automatically mapping an image into a semantic structural graph, which requires correctly labeling each extracted object and their interaction relationships.

Graph Generation Object Detection +1

CariGAN: Caricature Generation through Weakly Paired Adversarial Learning

no code implementations1 Nov 2018 Wenbin Li, Wei Xiong, Haofu Liao, Jing Huo, Yang Gao, Jiebo Luo

Furthermore, an attention mechanism is introduced to encourage our model to focus on the key facial parts so that more vivid details in these regions can be generated.


Determining Code Words in Euphemistic Hate Speech Using Word Embedding Networks

no code implementations WS 2018 Rijul Magu, Jiebo Luo

While analysis of online explicit abusive language detection has lately seen an ever-increasing focus, implicit abuse detection remains a largely unexplored space.

Abusive Language Community Detection +2

stagNet: An Attentive Semantic RNN for Group Activity Recognition

no code implementations ECCV 2018 Mengshi Qi, Jie Qin, Annan Li, Yunhong Wang, Jiebo Luo, Luc van Gool

Group activity recognition plays a fundamental role in a variety of applications, e. g. sports video analysis and intelligent surveillance.

Group Activity Recognition

``Factual'' or ``Emotional'': Stylized Image Captioning with Adaptive Learning and Attention

no code implementations ECCV 2018 Tianlang Chen, Zhongping Zhang, Quanzeng You, Chen Fang, Zhaowen Wang, Hailin Jin, Jiebo Luo

It uses two groups of matrices to capture the factual and stylized knowledge, respectively, and automatically learns the word-level weights of the two groups based on previous context.

Image Captioning

Video Re-localization

1 code implementation ECCV 2018 Yang Feng, Lin Ma, Wei Liu, Tong Zhang, Jiebo Luo

We first exploit and reorganize the videos in ActivityNet to form a new dataset for video re-localization research, which consists of about 10, 000 videos of diverse visual appearances associated with localized boundary information.

Copy Detection

Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTM

no code implementations20 Jul 2018 Yuxiao Chen, Jianbo Yuan, Quanzeng You, Jiebo Luo

Sentiment analysis on large-scale social media data is important to bridge the gaps between social media contents and real world activities including political election prediction, individual and public emotional status monitoring and analysis, and so on.

Twitter Sentiment Analysis

"Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention

no code implementations10 Jul 2018 Tianlang Chen, Zhongping Zhang, Quanzeng You, Chen Fang, Zhaowen Wang, Hailin Jin, Jiebo Luo

It uses two groups of matrices to capture the factual and stylized knowledge, respectively, and automatically learns the word-level weights of the two groups based on previous context.

Image Captioning

End-to-End Convolutional Semantic Embeddings

no code implementations CVPR 2018 Quanzeng You, Zhengyou Zhang, Jiebo Luo

Usually, Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are employed for learning image and sentence representations, respectively.

Online Progressive Deep Metric Learning

1 code implementation15 May 2018 Wenbin Li, Jing Huo, Yinghuan Shi, Yang Gao, Lei Wang, Jiebo Luo

Furthermore, in a progressively and nonlinearly learning way, ODML has a stronger learning ability than traditional shallow online metric learning in the case of limited available training data.

Metric Learning

VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions

no code implementations ECCV 2018 Qing Li, Qingyi Tao, Shafiq Joty, Jianfei Cai, Jiebo Luo

Most existing works in visual question answering (VQA) are dedicated to improving the accuracy of predicted answers, while disregarding the explanations.

Frame Multi-Task Learning +3

VizWiz Grand Challenge: Answering Visual Questions from Blind People

no code implementations CVPR 2018 Danna Gurari, Qing Li, Abigale J. Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, Jeffrey P. Bigham

The study of algorithms to automatically answer visual questions currently is motivated by visual question answering (VQA) datasets constructed in artificial VQA settings.

Question Answering Visual Question Answering +1

Action Recognition with Spatio-Temporal Visual Attention on Skeleton Image Sequences

no code implementations31 Jan 2018 Zhengyuan Yang, Yuncheng Li, Jianchao Yang, Jiebo Luo

The attention mechanism is important for skeleton based action recognition because there exist spatio-temporal key stages while the joint predictions can be inaccurate.

Action Recognition Skeleton Based Action Recognition

Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions

no code implementations EMNLP 2018 Qing Li, Jianlong Fu, Dongfei Yu, Tao Mei, Jiebo Luo

Most existing approaches adopt the pipeline of representing an image via pre-trained CNNs, and then using the uninterpretable CNN features in conjunction with the question to predict the answer.

Image Captioning Question Answering +2

End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception

1 code implementation20 Jan 2018 Zhengyuan Yang, Yixuan Zhang, Jerry Yu, Junjie Cai, Jiebo Luo

In this work, we propose a multi-task learning framework to predict the steering angle and speed control simultaneously in an end-to-end manner.

Autonomous Driving Multi-Task Learning +2

Boundary-based Image Forgery Detection by Fast Shallow CNN

1 code implementation20 Jan 2018 Zhongping Zhang, Yixuan Zhang, Zheng Zhou, Jiebo Luo

In this paper, we substantiate that Fast SCNN can detect drastic change of chroma and saturation.


Learning Multi-Attention Convolutional Neural Network for Fine-Grained Image Recognition

4 code implementations ICCV 2017 Heliang Zheng, Jianlong Fu, Tao Mei, Jiebo Luo

Two losses are proposed to guide the multi-task learning of channel grouping and part classification, which encourages MA-CNN to generate more discriminative parts from feature channels and learn better fine-grained features from parts in a mutual reinforced way.

Fine-Grained Image Classification Fine-Grained Image Recognition +2

Cultural Diffusion and Trends in Facebook Photographs

no code implementations24 May 2017 Quanzeng You, Darío García-García, Mahohar Paluri, Jiebo Luo, Jungseock Joo

Online social media is a social vehicle in which people share various moments of their lives with their friends, such as playing sports, cooking dinner or just taking a selfie for fun, via visual means, that is, photographs.

Integrating Scene Text and Visual Appearance for Fine-Grained Image Classification

no code implementations15 Apr 2017 Xiang Bai, Mingkun Yang, Pengyuan Lyu, Yongchao Xu, Jiebo Luo

Then, we combine the word embedding of the recognized words and the deep visual features into a single representation, which is optimized by a convolutional neural network for fine-grained image classification.

Classification Fine-Grained Image Classification +1

Deep Multimodal Representation Learning from Temporal Data

no code implementations CVPR 2017 Xitong Yang, Palghat Ramesh, Radha Chitta, Sriganesh Madhvanath, Edgar A. Bernal, Jiebo Luo

In recent years, Deep Learning has been successfully applied to multimodal learning problems, with the aim of learning useful joint representations in data fusion applications.

Audio-Visual Speech Recognition Representation Learning +2

Improving Pairwise Ranking for Multi-label Image Classification

4 code implementations CVPR 2017 Yuncheng Li, Yale Song, Jiebo Luo

Pairwise ranking, in particular, has been successful in multi-label image classification, achieving state-of-the-art results on various benchmarks.

Classification General Classification +2

A World of Difference: Divergent Word Interpretations among People

no code implementations8 Mar 2017 Tianran Hu, Ruihua Song, Maya Abtahian, Philip Ding, Xing Xie, Jiebo Luo

We propose an approach that quantifies semantic differences in interpretations among different groups of people.

Spice up Your Chat: The Intentions and Sentiment Effects of Using Emoji

no code implementations8 Mar 2017 Tianran Hu, Han Guo, Hao Sun, Thuy-vy Thi Nguyen, Jiebo Luo

Second, from a perspective of message recipients, we further study the sentiment effects of emojis, as well as their duplications, on verbal messages.

Learning from Noisy Labels with Distillation

no code implementations ICCV 2017 Yuncheng Li, Jianchao Yang, Yale Song, Liangliang Cao, Jiebo Luo, Li-Jia Li

The ability of learning from noisy labels is very useful in many visual recognition tasks, as a vast amount of data with noisy labels are relatively easy to obtain.

What the Language You Tweet Says About Your Occupation

no code implementations22 Jan 2017 Tianran Hu, Haoyuan Xiao, Thuy-vy Thi Nguyen, Jiebo Luo

Finally, a classifier is built to predict job types based on the features extracted from tweets.

Job Prediction

Image Based Appraisal of Real Estate Properties

no code implementations28 Nov 2016 Quanzeng You, Ran Pang, Liangliang Cao, Jiebo Luo

Real estate appraisal, which is the process of estimating the price for real estate properties, is crucial for both buys and sellers as the basis for negotiation and transaction.