Search Results for author: Zhenzhong Lan

Found 24 papers, 7 papers with code

S-SimCSE: Sampled Sub-networks for Contrastive Learning of Sentence Embedding

no code implementations23 Nov 2021 Junlei Zhang, Zhenzhong Lan

The corresponding outputs, two sentence embeddings derived from the same sentence with different dropout masks, can be used to build a positive pair.

Contrastive Learning Data Augmentation +1

Dynamic Resolution Network

1 code implementation NeurIPS 2021 Mingjian Zhu, Kai Han, Enhua Wu, Qiulin Zhang, Ying Nie, Zhenzhong Lan, Yunhe Wang

To this end, we propose a novel dynamic-resolution network (DRNet) in which the input resolution is determined dynamically based on each input sample.

Selecting the optimal dialogue response once for all from a panoramic view

no code implementations2 Jun 2021 Chiyu Song, Hongliang He, Haofei Yu, Huachuan Qiu, Zhenzhong Lan

As an essential component of dialogue systems, response selection aims to pick out the optimal response among candidates to continue the dialogue.

Do Transformer Modifications Transfer Across Implementations and Applications?

2 code implementations EMNLP 2021 Sharan Narang, Hyung Won Chung, Yi Tay, William Fedus, Thibault Fevry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel

The research community has proposed copious modifications to the Transformer architecture since it was introduced over three years ago, relatively few of which have seen widespread adoption.

Memformer: The Memory-Augmented Transformer

no code implementations14 Oct 2020 Qingyang Wu, Zhenzhong Lan, Jing Gu, Zhou Yu

To remedy the limitation, we present Memformer, a novel language model that utilizes a single unified memory to encode and retrieve past information.

Language Modelling

Attention that does not Explain Away

no code implementations29 Sep 2020 Nan Ding, Xinjie Fan, Zhenzhong Lan, Dale Schuurmans, Radu Soricut

Models based on the Transformer architecture have achieved better accuracy than the ones based on competing architectures for a large set of tasks.

Talking-Heads Attention

4 code implementations5 Mar 2020 Noam Shazeer, Zhenzhong Lan, Youlong Cheng, Nan Ding, Le Hou

We introduce "talking-heads attention" - a variation on multi-head attention which includes linearprojections across the attention-heads dimension, immediately before and after the softmax operation. While inserting only a small number of additional parameters and a moderate amount of additionalcomputation, talking-heads attention leads to better perplexities on masked language modeling tasks, aswell as better quality when transfer-learning to language comprehension and question answering tasks.

Language Modelling Question Answering +1

Multi-stage Pretraining for Abstractive Summarization

no code implementations23 Sep 2019 Sebastian Goodman, Zhenzhong Lan, Radu Soricut

Neural models for abstractive summarization tend to achieve the best performance in the presence of highly specialized, summarization specific modeling add-ons such as pointer-generator, coverage-modeling, and inferencetime heuristics.

Abstractive Text Summarization

Hidden Two-Stream Convolutional Networks for Action Recognition

3 code implementations2 Apr 2017 Yi Zhu, Zhenzhong Lan, Shawn Newsam, Alexander G. Hauptmann

State-of-the-art action recognition approaches rely on traditional optical flow estimation methods to pre-compute motion information for CNNs.

Action Recognition Optical Flow Estimation

Guided Optical Flow Learning

no code implementations8 Feb 2017 Yi Zhu, Zhenzhong Lan, Shawn Newsam, Alexander G. Hauptmann

We study the unsupervised learning of CNNs for optical flow estimation using proxy ground truth data.

Image Reconstruction Optical Flow Estimation

Deep Local Video Feature for Action Recognition

no code implementations25 Jan 2017 Zhenzhong Lan, Yi Zhu, Alexander G. Hauptmann

We investigate the problem of representing an entire video using CNN features for human action recognition.

Action Recognition

Strategies for Searching Video Content with Text Queries or Video Examples

no code implementations17 Jun 2016 Shoou-I Yu, Yi Yang, Zhongwen Xu, Shicheng Xu, Deyu Meng, Zexi Mao, Zhigang Ma, Ming Lin, Xuanchong Li, Huan Li, Zhenzhong Lan, Lu Jiang, Alexander G. Hauptmann, Chuang Gan, Xingzhong Du, Xiaojun Chang

The large number of user-generated videos uploaded on to the Internet everyday has led to many commercial video search engines, which mainly rely on text metadata for search.

Event Detection Video Retrieval

Improving Human Activity Recognition Through Ranking and Re-ranking

no code implementations11 Dec 2015 Zhenzhong Lan, Shoou-I Yu, Alexander G. Hauptmann

We propose two well-motivated ranking-based methods to enhance the performance of current state-of-the-art human activity recognition systems.

Activity Recognition Curriculum Learning +1

Handcrafted Local Features are Convolutional Neural Networks

no code implementations16 Nov 2015 Zhenzhong Lan, Shoou-I Yu, Ming Lin, Bhiksha Raj, Alexander G. Hauptmann

We approach this problem by first showing that local handcrafted features and Convolutional Neural Networks (CNNs) share the same convolution-pooling network structure.

Action Recognition Optical Flow Estimation

The Best of Both Worlds: Combining Data-independent and Data-driven Approaches for Action Recognition

no code implementations17 May 2015 Zhenzhong Lan, Dezhong Yao, Ming Lin, Shoou-I Yu, Alexander Hauptmann

First, we propose a two-stream Stacked Convolutional Independent Subspace Analysis (ConvISA) architecture to show that unsupervised learning methods can significantly boost the performance of traditional local features extracted from data-independent models.

Action Recognition Multi-class Classification +2

Long-short Term Motion Feature for Action Classification and Retrieval

no code implementations13 Feb 2015 Zhenzhong Lan, Xuanchong Li, Ming Lin, Alexander G. Hauptmann

Therefore, they need to occur frequently enough in the videos and to be be able to tell the difference among different types of motions.

Action Classification General Classification +1

Self-Paced Learning with Diversity

no code implementations NeurIPS 2014 Lu Jiang, Deyu Meng, Shoou-I Yu, Zhenzhong Lan, Shiguang Shan, Alexander Hauptmann

Self-paced learning (SPL) is a recently proposed learning regime inspired by the learning process of humans and animals that gradually incorporates easy to more complex samples into training.

Cannot find the paper you are looking for? You can Submit a new open access paper.