Search Results for author: Serena Yeung

Found 39 papers, 15 papers with code

Hyperbolic Deep Learning in Computer Vision: A Survey

no code implementations11 May 2023 Pascal Mettes, Mina Ghadimi Atigh, Martin Keller-Ressel, Jeffrey Gu, Serena Yeung

In this paper, we provide a categorization and in-depth overview of current literature on hyperbolic learning for computer vision.

Representation Learning

Video Pretraining Advances 3D Deep Learning on Chest CT Tasks

1 code implementation2 Apr 2023 Alexander Ke, Shih-Cheng Huang, Chloe P O'Connell, Michal Klimont, Serena Yeung, Pranav Rajpurkar

We demonstrate video pretraining improves the average performance of seven 3D models on two chest CT datasets, regardless of finetuning dataset size, and that video pretraining allows 3D models to outperform 2D baselines.

Image Classification

Diffusion-HPC: Generating Synthetic Images with Realistic Humans

1 code implementation16 Mar 2023 Zhenzhen Weng, Laura Bravo-Sánchez, Serena Yeung

Recent text-to-image generative models have exhibited remarkable abilities in generating high-fidelity and photo-realistic images.

Human Mesh Recovery

Adapting Pre-trained Vision Transformers from 2D to 3D through Weight Inflation Improves Medical Image Segmentation

1 code implementation8 Feb 2023 Yuhui Zhang, Shih-Cheng Huang, Zhengping Zhou, Matthew P. Lungren, Serena Yeung

Given the prevalence of 3D medical imaging technologies such as MRI and CT that are widely used in diagnosing and treating diverse diseases, 3D segmentation is one of the fundamental tasks of medical image analysis.

Image Segmentation Medical Image Segmentation +2

Diagnosing and Rectifying Vision Models using Language

1 code implementation8 Feb 2023 Yuhui Zhang, Jeff Z. HaoChen, Shih-Cheng Huang, Kuan-Chieh Wang, James Zou, Serena Yeung

Our proposed method can discover high-error data slices, identify influential attributes and further rectify undesirable model behaviors, without requiring any visual data.

Contrastive Learning

NeMo: Learning 3D Neural Motion Fields From Multiple Video Instances of the Same Action

no code implementations CVPR 2023 Kuan-Chieh Wang, Zhenzhen Weng, Maria Xenochristou, João Pedro Araújo, Jeffrey Gu, Karen Liu, Serena Yeung

Empirically, we show that NeMo can recover 3D motion in sports using videos from the Penn Action dataset, where NeMo outperforms existing HMR methods in terms of 2D keypoint detection.

3D Reconstruction Human Mesh Recovery +1

NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action

1 code implementation28 Dec 2022 Kuan-Chieh Wang, Zhenzhen Weng, Maria Xenochristou, Joao Pedro Araujo, Jeffrey Gu, C. Karen Liu, Serena Yeung

Empirically, we show that NeMo can recover 3D motion in sports using videos from the Penn Action dataset, where NeMo outperforms existing HMR methods in terms of 2D keypoint detection.

3D Reconstruction Human Mesh Recovery +1

PROB: Probabilistic Objectness for Open World Object Detection

1 code implementation CVPR 2023 Orr Zohar, Kuan-Chieh Wang, Serena Yeung

The resulting Probabilistic Objectness transformer-based open-world detector, PROB, integrates our framework into traditional object detection models, adapting them for the open-world setting.

object-detection Open World Object Detection

Adaptation of Surgical Activity Recognition Models Across Operating Rooms

no code implementations7 Jul 2022 Ali Mottaghi, Aidean Sharghi, Serena Yeung, Omid Mohareri

We propose a new domain adaptation method to improve the performance of the surgical activity recognition model in a new operating room for which we only have unlabeled videos.

Activity Recognition Domain Adaptation

Domain Adaptive 3D Pose Augmentation for In-the-wild Human Mesh Recovery

1 code implementation21 Jun 2022 Zhenzhen Weng, Kuan-Chieh Wang, Angjoo Kanazawa, Serena Yeung

The ability to perceive 3D human bodies from a single image has a multitude of applications ranging from entertainment and robotics to neuroscience and healthcare.

Data Augmentation Domain Adaptation +1

Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning

2 code implementations3 Mar 2022 Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, James Zou

Our systematic analysis demonstrates that this gap is caused by a combination of model initialization and contrastive learning optimization.

Contrastive Learning Fairness +2

FlowVOS: Weakly-Supervised Visual Warping for Detail-Preserving and Temporally Consistent Single-Shot Video Object Segmentation

no code implementations20 Nov 2021 Julia Gong, F. Christopher Holsinger, Serena Yeung

In contrast to prior work that uses full optical flow, we introduce a new foreground-targeted visual warping approach that learns flow fields from VOS data.

Optical Flow Estimation Semantic Segmentation +2

Staying in Shape: Learning Invariant Shape Representations using Contrastive Learning

no code implementations8 Jul 2021 Jeffrey Gu, Serena Yeung

Creating representations of shapes that are invari-ant to isometric or almost-isometric transforma-tions has long been an area of interest in shape anal-ysis, since enforcing invariance allows the learningof more effective and robust shape representations. Most existing invariant shape representations arehandcrafted, and previous work on learning shaperepresentations do not focus on producing invariantrepresentations.

Contrastive Learning

DARCNN: Domain Adaptive Region-based Convolutional Neural Network for Unsupervised Instance Segmentation in Biomedical Images

no code implementations CVPR 2021 Joy Hsu, Wah Chiu, Serena Yeung

In the biomedical domain, there is an abundance of dense, complex data where objects of interest may be challenging to detect or constrained by limits of human knowledge.

Domain Adaptation Instance Segmentation +3

Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision

no code implementations CVPR 2021 Zhenzhen Weng, Mehmet Giray Ogut, Shai Limonchik, Serena Yeung

Instance segmentation is an active topic in computer vision that is usually solved by using supervised learning approaches over very large datasets composed of object level masks.

Instance Segmentation Semantic Segmentation

Personalized Federated Learning with First Order Model Optimization

3 code implementations ICLR 2021 Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, Jose M. Alvarez

While federated learning traditionally aims to train a single global model across decentralized local datasets, one model may not always be ideal for all participating clients.

Model Optimization Personalized Federated Learning

Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations

no code implementations NeurIPS 2021 Joy Hsu, Jeffrey Gu, Gong-Her Wu, Wah Chiu, Serena Yeung

To that end, we consider encoder-decoder architectures with a hyperbolic latent space, to explicitly capture hierarchical relationships present in subvolumes of the data.

Representation Learning

Holistic 3D Human and Scene Mesh Estimation from Single View Images

1 code implementation CVPR 2021 Zhenzhen Weng, Serena Yeung

Indeed, from a single image of a person placed in an indoor scene, we as humans are adept at resolving ambiguities of the human pose and room layout through our knowledge of the physical laws and prior perception of the plausible object and human poses.

Indoor Scene Reconstruction

Medical symptom recognition from patient text: An active learning approach for long-tailed multilabel distributions

no code implementations12 Nov 2020 Ali Mottaghi, Prathusha K Sarma, Xavier Amatriain, Serena Yeung, Anitha Kannan

We study the problem of medical symptoms recognition from patient text, for the purposes of gathering pertinent information from the patient (known as history-taking).

Active Learning

Learning Hyperbolic Representations for Unsupervised 3D Segmentation

no code implementations28 Sep 2020 Joy Hsu, Jeffrey Gu, Gong Her Wu, Wah Chiu, Serena Yeung

There exists a need for unsupervised 3D segmentation on complex volumetric data, particularly when annotation ability is limited or discovery of new categories is desired.

Rapidly Personalizing Mobile Health Treatment Policies with Limited Data

no code implementations23 Feb 2020 Sabina Tomkins, Peng Liao, Predrag Klasnja, Serena Yeung, Susan Murphy

In mobile health (mHealth), reinforcement learning algorithms that adapt to one's context without learning personalized policies might fail to distinguish between the needs of individuals.

Reinforcement Learning (RL)

Adversarial Representation Active Learning

1 code implementation20 Dec 2019 Ali Mottaghi, Serena Yeung

Active learning aims to develop label-efficient algorithms by querying the most informative samples to be labeled by an oracle.

Active Learning

Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference

no code implementations25 Nov 2018 Edward Chou, Josh Beal, Daniel Levy, Serena Yeung, Albert Haque, Li Fei-Fei

Homomorphic encryption enables arbitrary computation over data while it remains encrypted.

Cryptography and Security

Temporal Modular Networks for Retrieving Complex Compositional Activities in Videos

no code implementations ECCV 2018 Bingbin Liu, Serena Yeung, Edward Chou, De-An Huang, Li Fei-Fei, Juan Carlos Niebles

A major challenge in computer vision is scaling activity understanding to the long tail of complex activities without requiring collecting large quantities of data for new actions.

Retrieval Video Retrieval

Neural Graph Matching Networks for Fewshot 3D Action Recognition

no code implementations ECCV 2018 Michelle Guo, Edward Chou, De-An Huang, Shuran Song, Serena Yeung, Li Fei-Fei

We propose Neural Graph Matching (NGM) Networks, a novel framework that can learn to recognize a previous unseen 3D action class with only a few examples.

Few-Shot Learning Graph Matching +1

Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks

no code implementations24 Feb 2018 Amy Jin, Serena Yeung, Jeffrey Jopling, Jonathan Krause, Dan Azagury, Arnold Milstein, Li Fei-Fei

We show that our method both effectively detects the spatial bounds of tools as well as significantly outperforms existing methods on tool presence detection.

Tackling Over-pruning in Variational Autoencoders

no code implementations9 Jun 2017 Serena Yeung, Anitha Kannan, Yann Dauphin, Li Fei-Fei

The so-called epitomes of this model are groups of mutually exclusive latent factors that compete to explain the data.

Learning to Learn from Noisy Web Videos

no code implementations CVPR 2017 Serena Yeung, Vignesh Ramanathan, Olga Russakovsky, Liyue Shen, Greg Mori, Li Fei-Fei

Our method uses Q-learning to learn a data labeling policy on a small labeled training dataset, and then uses this to automatically label noisy web data for new visual concepts.

Action Recognition Q-Learning +1

End-to-end Learning of Action Detection from Frame Glimpses in Videos

1 code implementation CVPR 2016 Serena Yeung, Olga Russakovsky, Greg Mori, Li Fei-Fei

In this work we introduce a fully end-to-end approach for action detection in videos that learns to directly predict the temporal bounds of actions.

Ranked #9 on Temporal Action Localization on THUMOS’14 (mAP IOU@0.2 metric)

Action Detection

VideoSET: Video Summary Evaluation through Text

no code implementations23 Jun 2014 Serena Yeung, Alireza Fathi, Li Fei-Fei

In this paper we present VideoSET, a method for Video Summary Evaluation through Text that can evaluate how well a video summary is able to retain the semantic information contained in its original video.

Cannot find the paper you are looking for? You can Submit a new open access paper.