Search Results for author: Ser-Nam Lim

Found 48 papers, 23 papers with code

Rethinking Nearest Neighbors for Visual Classification

1 code implementation15 Dec 2021 Menglin Jia, Bor-Chun Chen, Zuxuan Wu, Claire Cardie, Serge Belongie, Ser-Nam Lim

In this paper, we investigate $k$-Nearest-Neighbor (k-NN) classifiers, a classical model-free learning method from the pre-deep learning era, as an augmentation to modern neural network based approaches.

Object-Centric Unsupervised Image Captioning

no code implementations2 Dec 2021 Zihang Meng, David Yang, Xuefei Cao, Ashish Shah, Ser-Nam Lim

Our work in this paper overcomes this by harvesting objects corresponding to a given sentence from the training set, even if they don't belong to the same image.

Image Captioning

AdaViT: Adaptive Vision Transformers for Efficient Image Recognition

no code implementations30 Nov 2021 Lingchen Meng, Hengduo Li, Bor-Chun Chen, Shiyi Lan, Zuxuan Wu, Yu-Gang Jiang, Ser-Nam Lim

To this end, we introduce AdaViT, an adaptive computation framework that learns to derive usage policies on which patches, self-attention heads and transformer blocks to use throughout the backbone on a per-input basis, aiming to improve inference efficiency of vision transformers with a minimal drop of accuracy for image recognition.

Unsupervised Domain Adaptation: A Reality Check

1 code implementation30 Nov 2021 Kevin Musgrave, Serge Belongie, Ser-Nam Lim

Interest in unsupervised domain adaptation (UDA) has surged in recent years, resulting in a plethora of new algorithms.

Unsupervised Domain Adaptation

Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods

2 code implementations NeurIPS 2021 Derek Lim, Felix Hohne, Xiuyu Li, Sijia Linda Huang, Vaishnavi Gupta, Omkar Bhalerao, Ser-Nam Lim

Many widely used datasets for graph machine learning tasks have generally been homophilous, where nodes with similar labels connect to each other.

Graph Learning

A Frequency Perspective of Adversarial Robustness

no code implementations26 Oct 2021 Shishira R Maiya, Max Ehrlich, Vatsal Agarwal, Ser-Nam Lim, Tom Goldstein, Abhinav Shrivastava

Our analysis shows that adversarial examples are neither in high-frequency nor in low-frequency components, but are simply dataset dependent.

Adversarial Robustness

NeRV: Neural Representations for Videos

1 code implementation NeurIPS 2021 Hao Chen, Bo He, Hanyu Wang, Yixuan Ren, Ser-Nam Lim, Abhinav Shrivastava

In contrast, with NeRV, we can use any neural network compression method as a proxy for video compression, and achieve comparable performance to traditional frame-based video compression approaches (H. 264, HEVC \etc).

Denoising Neural Network Compression +2

MixNorm: Test-Time Adaptation Through Online Normalization Estimation

no code implementations21 Oct 2021 Xuefeng Hu, Gokhan Uzunbas, Sirius Chen, Rui Wang, Ashish Shah, Ram Nevatia, Ser-Nam Lim

We present a simple and effective way to estimate the batch-norm statistics during test time, to fast adapt a source model to target test samples.

Unsupervised Domain Adaptation Zero-Shot Learning

Self-appearance-aided Differential Evolution for Motion Transfer

no code implementations9 Oct 2021 Peirong Liu, Rui Wang, Xuefei Cao, Yipin Zhou, Ashish Shah, Maxime Oquab, Camille Couprie, Ser-Nam Lim

Image animation transfers the motion of a driving video to a static object in a source image, while keeping the source identity unchanged.

Image Animation

Refining Multimodal Representations using a modality-centric self-supervised module

no code implementations29 Sep 2021 Sethuraman Sankaran, David Yang, Ser-Nam Lim

Tasks that rely on multi-modal information typically include a fusion module that combines information from different modalities.

Few-Shot Learning

When in Doubt: Improving Classification Performance with Alternating Normalization

1 code implementation Findings (EMNLP) 2021 Menglin Jia, Austin Reiter, Ser-Nam Lim, Yoav Artzi, Claire Cardie

We introduce Classification with Alternating Normalization (CAN), a non-parametric post-processing step for classification.

Equivariant Manifold Flows

no code implementations NeurIPS 2021 Isay Katsman, Aaron Lou, Derek Lim, Qingxuan Jiang, Ser-Nam Lim, Christopher De Sa

Tractably modelling distributions over manifolds has long been an important goal in the natural sciences.

Edge Proposal Sets for Link Prediction

1 code implementation30 Jun 2021 Abhay Singh, Qian Huang, Sijia Linda Huang, Omkar Bhalerao, Horace He, Ser-Nam Lim, Austin R. Benson

Here, we demonstrate how simply adding a set of edges, which we call a \emph{proposal set}, to the graph as a pre-processing step can improve the performance of several link prediction algorithms.

Experimental Design Link Prediction +1

Cross-Modal Retrieval Augmentation for Multi-Modal Classification

no code implementations Findings (EMNLP) 2021 Shir Gur, Natalia Neverova, Chris Stauffer, Ser-Nam Lim, Douwe Kiela, Austin Reiter

Recent advances in using retrieval components over external knowledge sources have shown impressive results for a variety of downstream tasks in natural language processing.

Cross-Modal Retrieval General Classification +2

Multimodal Fusion Refiner Networks

no code implementations8 Apr 2021 Sethuraman Sankaran, David Yang, Ser-Nam Lim

In this work, we develop a Refiner Fusion Network (ReFNet) that enables fusion modules to combine strong unimodal representation with strong multimodal representations.

New Benchmarks for Learning on Non-Homophilous Graphs

1 code implementation3 Apr 2021 Derek Lim, Xiuyu Li, Felix Hohne, Ser-Nam Lim

Much data with graph structures satisfy the principle of homophily, meaning that connected nodes tend to be similar with respect to a specific attribute.

Fraud Detection Graph Representation Learning +1

THAT: Two Head Adversarial Training for Improving Robustness at Scale

no code implementations25 Mar 2021 Zuxuan Wu, Tom Goldstein, Larry S. Davis, Ser-Nam Lim

Many variants of adversarial training have been proposed, with most research focusing on problems with relatively few classes.

Deep Video Inpainting Detection

no code implementations26 Jan 2021 Peng Zhou, Ning Yu, Zuxuan Wu, Larry S. Davis, Abhinav Shrivastava, Ser-Nam Lim

This paper studies video inpainting detection, which localizes an inpainted region in a video both spatially and temporally.

Video Inpainting

Joint Audio-Visual Deepfake Detection

no code implementations ICCV 2021 Yipin Zhou, Ser-Nam Lim

Deepfakes ("deep learning" + "fake") are synthetically-generated videos from AI algorithms.

DeepFake Detection Face Swapping +2

GTA: Global Temporal Attention for Video Action Understanding

no code implementations15 Dec 2020 Bo He, Xitong Yang, Zuxuan Wu, Hao Chen, Ser-Nam Lim, Abhinav Shrivastava

To this end, we introduce Global Temporal Attention (GTA), which performs global temporal attention on top of spatial attention in a decoupled manner.

Action Recognition Action Understanding

Analyzing and Mitigating JPEG Compression Defects in Deep Learning

no code implementations17 Nov 2020 Max Ehrlich, Larry Davis, Ser-Nam Lim, Abhinav Shrivastava

We show that there is a significant penalty on common performance metrics for high compression.

Intentonomy: a Dataset and Study towards Human Intent Understanding

1 code implementation CVPR 2021 Menglin Jia, Zuxuan Wu, Austin Reiter, Claire Cardie, Serge Belongie, Ser-Nam Lim

Based on our findings, we conduct further study to quantify the effect of attending to object and context classes as well as textual information in the form of hashtags when training an intent classifier.

What makes fake images detectable? Understanding properties that generalize

1 code implementation ECCV 2020 Lucy Chai, David Bau, Ser-Nam Lim, Phillip Isola

The quality of image generation and manipulation is reaching impressive levels, making it increasingly difficult for a human to distinguish between what is real and what is fake.

Image Generation

PyTorch Metric Learning

1 code implementation20 Aug 2020 Kevin Musgrave, Serge Belongie, Ser-Nam Lim

Deep metric learning algorithms have a wide variety of applications, but implementing these algorithms can be tedious and time consuming.

Metric Learning

Deep Co-Training with Task Decomposition for Semi-Supervised Domain Adaptation

1 code implementation ICCV 2021 Luyu Yang, Yan Wang, Mingfei Gao, Abhinav Shrivastava, Kilian Q. Weinberger, Wei-Lun Chao, Ser-Nam Lim

To integrate the strengths of the two classifiers, we apply the well-established co-training framework, in which the two classifiers exchange their high confident predictions to iteratively "teach each other" so that both classifiers can excel in the target domain.

Unsupervised Domain Adaptation

Curriculum Manager for Source Selection in Multi-Source Domain Adaptation

no code implementations ECCV 2020 Luyu Yang, Yogesh Balaji, Ser-Nam Lim, Abhinav Shrivastava

In this paper, we proposed an adversarial agent that learns a dynamic curriculum for source samples, called Curriculum Manager for Source Selection (CMSS).

Multi-Source Unsupervised Domain Adaptation Unsupervised Domain Adaptation

Neural Manifold Ordinary Differential Equations

2 code implementations NeurIPS 2020 Aaron Lou, Derek Lim, Isay Katsman, Leo Huang, Qingxuan Jiang, Ser-Nam Lim, Christopher De Sa

To better conform to data geometry, recent deep generative modelling techniques adapt Euclidean constructions to non-Euclidean spaces.

Density Estimation

Detecting Deep-Fake Videos from Appearance and Behavior

no code implementations29 Apr 2020 Shruti Agarwal, Tarek El-Gaaly, Hany Farid, Ser-Nam Lim

Synthetically-generated audios and videos -- so-called deep fakes -- continue to capture the imagination of the computer-graphics and computer-vision communities.

Metric Learning

Quantization Guided JPEG Artifact Correction

1 code implementation ECCV 2020 Max Ehrlich, Larry Davis, Ser-Nam Lim, Abhinav Shrivastava

The JPEG image compression algorithm is the most popular method of image compression because of its ability for large compression ratios.

JPEG Artifact Correction Quantization

One-Shot Domain Adaptation For Face Generation

no code implementations CVPR 2020 Chao Yang, Ser-Nam Lim

To generate images of the same distribution, we introduce a style-mixing technique that transfers the low-level statistics from the target to faces randomly generated with the model.

Domain Adaptation Face Generation

A Metric Learning Reality Check

4 code implementations ECCV 2020 Kevin Musgrave, Serge Belongie, Ser-Nam Lim

Deep metric learning papers from the past four years have consistently claimed great advances in accuracy, often more than doubling the performance of decade-old methods.

Metric Learning

Better Set Representations For Relational Reasoning

1 code implementation NeurIPS 2020 Qian Huang, Horace He, Abhay Singh, Yan Zhang, Ser-Nam Lim, Austin Benson

Incorporating relational reasoning into neural networks has greatly expanded their capabilities and scope.

Relational Reasoning

Deep Multi-Modal Sets

no code implementations3 Mar 2020 Austin Reiter, Menglin Jia, Pu Yang, Ser-Nam Lim

Most deep learning-based methods rely on a late fusion technique whereby multiple feature types are encoded and concatenated and then a multi layer perceptron (MLP) combines the fused embedding to make predictions.

Differentiating through the Fréchet Mean

2 code implementations ICML 2020 Aaron Lou, Isay Katsman, Qingxuan Jiang, Serge Belongie, Ser-Nam Lim, Christopher De Sa

Recent advances in deep representation learning on Riemannian manifolds extend classical deep learning operations to better capture the geometry of the manifold.

Graph Convolutional Network Representation Learning

On Feature Normalization and Data Augmentation

1 code implementation CVPR 2021 Boyi Li, Felix Wu, Ser-Nam Lim, Serge Belongie, Kilian Q. Weinberger

The moments (a. k. a., mean and standard deviation) of latent features are often removed as noise when training image recognition models, to increase stability and reduce training time.

Data Augmentation Domain Generalization +2

Measuring Dataset Granularity

1 code implementation21 Dec 2019 Yin Cui, Zeqi Gu, Dhruv Mahajan, Laurens van der Maaten, Serge Belongie, Ser-Nam Lim

We also investigate the interplay between dataset granularity with a variety of factors and find that fine-grained datasets are more difficult to learn from, more difficult to transfer to, more difficult to perform few-shot learning with, and more vulnerable to adversarial attacks.

Few-Shot Learning

Unconstrained Facial Expression Transfer using Style-based Generator

1 code implementation12 Dec 2019 Chao Yang, Ser-Nam Lim

Given two face images, our method can create plausible results that combine the appearance of one image and the expression of the other.

Image Manipulation

Fine-grained Synthesis of Unrestricted Adversarial Examples

no code implementations20 Nov 2019 Omid Poursaeed, Tianxing Jiang, Yordanos Goshu, Harry Yang, Serge Belongie, Ser-Nam Lim

We propose a novel approach for generating unrestricted adversarial examples by manipulating fine-grained aspects of image generation.

Image Generation Object Detection +1

Unsupervised Deep Metric Learning via Auxiliary Rotation Loss

no code implementations16 Nov 2019 Xuefei Cao, Bor-Chun Chen, Ser-Nam Lim

In this work, we propose to generate pseudo-labels for deep metric learning directly from clustering assignment and we introduce unsupervised deep metric learning (UDML) regularized by a self-supervision (SS) task.

Image Retrieval Metric Learning +1

Cross-X Learning for Fine-Grained Visual Categorization

no code implementations ICCV 2019 Wei Luo, Xitong Yang, Xianjie Mo, Yuheng Lu, Larry S. Davis, Jun Li, Jian Yang, Ser-Nam Lim

Recognizing objects from subcategories with very subtle differences remains a challenging task due to the large intra-class and small inter-class variation.

Ranked #7 on Fine-Grained Image Classification on NABirds (using extra training data)

Fine-Grained Image Classification Fine-Grained Visual Categorization

Enhancing Adversarial Example Transferability with an Intermediate Level Attack

1 code implementation ICCV 2019 Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, Ser-Nam Lim

We show that we can select a layer of the source model to perturb without any knowledge of the target models while achieving high transferability.

Adversarial Example Decomposition

no code implementations4 Dec 2018 Horace He, Aaron Lou, Qingxuan Jiang, Isay Katsman, Serge Belongie, Ser-Nam Lim

Research has shown that widely used deep neural networks are vulnerable to carefully crafted adversarial perturbations.

Cannot find the paper you are looking for? You can Submit a new open access paper.