Search Results for author: Stella X. Yu

Found 65 papers, 31 papers with code

Unsupervised Sketch to Photo Synthesis

no code implementations ECCV 2020 Runtao Liu, Qian Yu, Stella X. Yu

Humans can envision a realistic photo given a free-hand sketch that is not only spatially imprecise and geometrically distorted but also without colors and visual details.

Denoising Retrieval +2

Trajectory Regularization Enhances Self-Supervised Geometric Representation

no code implementations22 Mar 2024 Jiayun Wang, Stella X. Yu, Yubei Chen

To address this gap, we introduce a new pose-estimation benchmark for assessing SSL geometric representations, which demands training without semantic or pose labels and achieving proficiency in both semantic and geometric downstream tasks.

Pose Estimation Representation Learning +1

Debiased Learning for Remote Sensing Data

no code implementations24 Dec 2023 Chun-Hsiao Yeh, Xudong Wang, Stella X. Yu, Charles Hill, Zackery Steck, Scott Kangas, Aaron Reite

Deep learning has had remarkable success at analyzing handheld imagery such as consumer photos due to the availability of large-scale human annotations (e. g., ImageNet).

Zero-shot Building Attribute Extraction from Large-Scale Vision and Language Models

no code implementations19 Dec 2023 Fei Pan, Sangryul Jeon, Brian Wang, Frank Mckenna, Stella X. Yu

The proposed workflow contains two key components: image-level captioning and segment-level captioning for the building images based on the vocabularies pertinent to structural and civil engineering.

Attribute Attribute Extraction +1

How to guess a gradient

no code implementations7 Dec 2023 Utkarsh Singhal, Brian Cheung, Kartik Chandra, Jonathan Ragan-Kelley, Joshua B. Tenenbaum, Tomaso A. Poggio, Stella X. Yu

We study how to narrow the gap in optimization performance between methods that calculate exact gradients and those that use directional derivatives.

Learning to Transform for Generalizable Instance-wise Invariance

1 code implementation ICCV 2023 Utkarsh Singhal, Carlos Esteves, Ameesh Makadia, Stella X. Yu

However, too much or too little invariance can hurt, and the correct amount is unknown a priori and dependent on the instance.

Data Augmentation

VEATIC: Video-based Emotion and Affect Tracking in Context Dataset

no code implementations13 Sep 2023 Zhihang Ren, Jefferson Ortega, Yifan Wang, Zhimin Chen, Yunhui Guo, Stella X. Yu, David Whitney

Along with the dataset, we propose a new computer vision task to infer the affect of the selected character via both context and character information in each video frame.

Coordinate-based neural representations for computational adaptive optics in widefield microscopy

1 code implementation7 Jul 2023 Iksung Kang, Qinrong Zhang, Stella X. Yu, Na Ji

We implemented CoCoA for widefield imaging of mouse brain tissues and validated its performance with direct-wavefront-sensing-based adaptive optics.

Unsupervised Feature Learning with Emergent Data-Driven Prototypicality

no code implementations4 Jul 2023 Yunhui Guo, Youren Zhang, Yubei Chen, Stella X. Yu

With our feature mapper simply trained to spread out training instances in hyperbolic space, we observe that images move closer to the origin with congealing, validating our idea of unsupervised prototypicality discovery.

Metric Learning

Bootstrapping Objectness from Videos by Relaxed Common Fate and Visual Grouping

1 code implementation CVPR 2023 Long Lian, Zhirong Wu, Stella X. Yu

The Gestalt law of common fate, i. e., what move at the same speed belong together, has inspired unsupervised object discovery based on motion segmentation.

Motion Segmentation Object +7

High-fidelity Direct Contrast Synthesis from Magnetic Resonance Fingerprinting

no code implementations21 Dec 2022 Ke Wang, Mariya Doneva, Jakob Meineke, Thomas Amthor, Ekin Karasan, Fei Tan, Jonathan I. Tamir, Stella X. Yu, Michael Lustig

Here we propose a supervised learning-based method that directly synthesizes contrast-weighted images from the MRF data without going through the quantitative mapping and spin-dynamics simulation.

Generative Adversarial Network Magnetic Resonance Fingerprinting +1

Improving Unsupervised Video Object Segmentation with Motion-Appearance Synergy

no code implementations17 Dec 2022 Long Lian, Zhirong Wu, Stella X. Yu

Previous methods in unsupervised video object segmentation (UVOS) have demonstrated the effectiveness of motion as either input or supervision for segmentation.

Misconceptions Object +5

Tracking the Dynamics of the Tear Film Lipid Layer

no code implementations7 Dec 2022 Tejasvi Kothapalli, Charlie Shou, Jennifer Ding, Jiayun Wang, Andrew D. Graham, Tatyana Svitova, Stella X. Yu, Meng C. Lin

Tear film instability is a known factor for DED, and is thought to be regulated in large part by the thin lipid layer that covers and stabilizes the tear film.

Multi-Spectral Image Classification with Ultra-Lean Complex-Valued Models

no code implementations21 Nov 2022 Utkarsh Singhal, Stella X. Yu, Zackery Steck, Scott Kangas, Aaron A. Reite

Multi-spectral imagery is invaluable for remote sensing due to different spectral signatures exhibited by materials that often appear identical in greyscale and RGB imagery.

Data Augmentation Humanitarian +2

CAST: Concurrent Recognition and Segmentation with Adaptive Segment Tokens

no code implementations1 Oct 2022 Tsung-Wei Ke, Stella X. Yu

We are thus inspired to learn image recognition with hierarchical image segmentation based entirely on unlabeled images.

Computational Efficiency Image Segmentation +3

Unsupervised Scene Sketch to Photo Synthesis

1 code implementation6 Sep 2022 Jiayun Wang, Sangryul Jeon, Stella X. Yu, Xi Zhang, Himanshu Arora, Yu Lou

Taking this advantage, we synthesize a photo-realistic image by combining the structure of a sketch and the visual style of a reference photo.

Open Long-Tailed Recognition in a Dynamic World

no code implementations17 Aug 2022 Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, Stella X. Yu

A practical recognition system must balance between majority (head) and minority (tail) classes, generalize across the distribution, and acknowledge novelty upon the instances of unseen classes (open classes).

Active Learning Classification +4

Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformers

1 code implementation CVPR 2022 Tsung-Wei Ke, Jyh-Jing Hwang, Yunhui Guo, Xudong Wang, Stella X. Yu

We enforce spatial consistency of grouping and bootstrap feature learning with co-segmentation among multiple views of the same image, and enforce semantic consistency across the grouping hierarchy with clustering transformers between coarse- and fine-grained features.

Clustering Segmentation +1

Debiased Learning from Naturally Imbalanced Pseudo-Labels

1 code implementation CVPR 2022 Xudong Wang, Zhirong Wu, Long Lian, Stella X. Yu

Our key insight is that pseudo-labels are naturally imbalanced due to intrinsic data similarity, even when a model is trained on balanced source data and evaluated on balanced target data.

 Ranked #1 on Few-Shot Image Classification on ImageNet - 0-Shot (using extra training data)

counterfactual Counterfactual Reasoning +4

Co-domain Symmetry for Complex-Valued Deep Learning

no code implementations CVPR 2022 Utkarsh Singhal, Yifei Xing, Stella X. Yu

We study complex-valued scaling as a type of symmetry natural and unique to complex-valued measurements and representations.

The Emergence of Objectness: Learning Zero-Shot Segmentation from Videos

1 code implementation NeurIPS 2021 Runtao Liu, Zhirong Wu, Stella X. Yu, Stephen Lin

Our model starts with two separate pathways: an appearance pathway that outputs feature-based region segmentation for a single image, and a motion pathway that outputs motion features for a pair of images.

Contrastive Learning Image Segmentation +6

Unsupervised Selective Labeling for More Effective Semi-Supervised Learning

1 code implementation6 Oct 2021 Xudong Wang, Long Lian, Stella X. Yu

Intuitively, no matter what the downstream task is, instances to be labeled must be representative and diverse: The former would facilitate label propagation to unlabeled data, whereas the latter would ensure coverage of the entire dataset.

Active Learning Semi-Supervised Image Classification (Cold Start)

Compact and Optimal Deep Learning with Recurrent Parameter Generators

1 code implementation15 Jul 2021 Jiayun Wang, Yubei Chen, Stella X. Yu, Brian Cheung, Yann Lecun

We propose a drastically different approach to compact and optimal deep learning: We decouple the Degrees of freedom (DoF) and the actual number of parameters of a model, optimize a small DoF with predefined random linear constraints for a large model of arbitrary architecture, in one-stage end-to-end learning.

Ranked #97 on Image Classification on ObjectNet (using extra training data)

Image Classification Model Compression

Unsupervised Discriminative Learning of Sounds for Audio Event Classification

no code implementations19 May 2021 Sascha Hornauer, Ke Li, Stella X. Yu, Shabnam Ghaffarzadegan, Liu Ren

Recent progress in network-based audio event classification has shown the benefit of pre-training models on visual data such as ImageNet.

Classification Transfer Learning

Iterative Human and Automated Identification of Wildlife Images

1 code implementation5 May 2021 Zhongqi Miao, Ziwei Liu, Kaitlyn M. Gaynor, Meredith S. Palmer, Stella X. Yu, Wayne M. Getz

Camera trapping is increasingly used to monitor wildlife, but this technology typically requires extensive data annotation.

Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning

1 code implementation ICLR 2021 Tsung-Wei Ke, Jyh-Jing Hwang, Stella X. Yu

Weakly supervised segmentation requires assigning a label to every pixel based on training instances with partial annotations such as image-level tags, object bounding boxes, labeled points and scribbles.

Contrastive Learning Metric Learning +3

Unsupervised Visual Attention and Invariance for Reinforcement Learning

no code implementations CVPR 2021 Xudong Wang, Long Lian, Stella X. Yu

Existing methods focus on training an RL policy that is universal to changing visual domains, whereas we focus on extracting visual foreground that is universal, feeding clean invariant vision to the RL policy learner.

Domain Generalization Keypoint Detection +2

Long-tailed Recognition by Routing Diverse Distribution-Aware Experts

2 code implementations ICLR 2021 Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, Stella X. Yu

We take a dynamic view of the training data and provide a principled model bias and variance analysis as the training data fluctuates: Existing long-tail classifiers invariably increase the model variance and the head-tail model bias gap remains large, due to more and larger confusion with hard negatives for the tail.

Image Classification imbalanced classification +1

Tied Block Convolution: Leaner and Better CNNs with Shared Thinner Filters

1 code implementation25 Sep 2020 Xudong Wang, Stella X. Yu

The concept of TBC can also be extended to group convolution and fully connected layers, and can be applied to various backbone networks and attention modules.

Instance Segmentation object-detection +2

Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination

2 code implementations CVPR 2021 Xudong Wang, Ziwei Liu, Stella X. Yu

Unsupervised feature learning has made great strides with contrastive learning based on instance discrimination and invariant mapping, as benchmarked on curated class-balanced datasets.

Contrastive Learning Semi-Supervised Image Classification +2

3D Shape Reconstruction from Free-Hand Sketches

1 code implementation17 Jun 2020 Jiayun Wang, Jierui Lin, Qian Yu, Runtao Liu, Yubei Chen, Stella X. Yu

Additionally, we propose a sketch standardization module to handle different sketch distortions and styles.

3D Reconstruction 3D Shape Reconstruction

A Deep Learning Approach for Meibomian Gland Atrophy Evaluation in Meibography Images

1 code implementation Translational Vision Science & Technology 2019 Jiayun Wang, Thao N. Yeh, Rudrasis Chakraborty, Stella X. Yu, Meng C. Lin

The development set was used to train and tune the deep learning model, while the evaluation set was used to evaluate the performance of the model.

Orthogonal Convolutional Neural Networks

1 code implementation CVPR 2020 Jiayun Wang, Yubei Chen, Rudrasis Chakraborty, Stella X. Yu

We develop an efficient approach to impose filter orthogonality on a convolutional layer based on the doubly block-Toeplitz matrix representation of the convolutional kernel instead of using the common kernel orthogonality approach, which we show is only necessary but not sufficient for ensuring orthogonal convolutions.

Image Classification Image Retrieval

POIRot: A rotation invariant omni-directional pointnet

no code implementations29 Oct 2019 Liu Yang, Rudrasis Chakraborty, Stella X. Yu

Our proposed model is rotationally invariant and can preserve geometric shape of a 3D point-cloud.

Data Augmentation Point Cloud Segmentation

SegSort: Segmentation by Discriminative Sorting of Segments

1 code implementation ICCV 2019 Jyh-Jing Hwang, Stella X. Yu, Jianbo Shi, Maxwell D. Collins, Tien-Ju Yang, Xiao Zhang, Liang-Chieh Chen

The proposed SegSort further produces an interpretable result, as each choice of label can be easily understood from the retrieved nearest segments.

Ranked #10 on Unsupervised Semantic Segmentation on PASCAL VOC 2012 val (using extra training data)

Clustering Metric Learning +2

Open Compound Domain Adaptation

no code implementations CVPR 2020 Ziwei Liu, Zhongqi Miao, Xingang Pan, Xiaohang Zhan, Dahua Lin, Stella X. Yu, Boqing Gong

A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e. g., sunny weather) for achieving high performance on the test data in a target domain (e. g., rainy weather).

Domain Adaptation Facial Expression Recognition +2

Spatial Transformer for 3D Point Clouds

1 code implementation26 Jun 2019 Jiayun Wang, Rudrasis Chakraborty, Stella X. Yu

We propose a novel end-to-end approach to learn different non-rigid transformations of the input point cloud so that optimal local neighborhoods can be adopted at each layer.

Semantic Segmentation

SurReal: Fréchet Mean and Distance Transform for Complex-Valued Deep Learning

1 code implementation24 Jun 2019 Rudrasis Chakraborty, Jiayun Wang, Stella X. Yu

On RadioML, our model achieves comparable RF modulation classification accuracy at 10% of the baseline model size.

General Classification

Large-Scale Long-Tailed Recognition in an Open World

2 code implementations CVPR 2019 Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, Stella X. Yu

We define Open Long-Tailed Recognition (OLTR) as learning from such naturally distributed data and optimizing the classification accuracy over a balanced test set which include head, tail, and open classes.

Classification Few-Shot Learning +4

Improving Generalization via Scalable Neighborhood Component Analysis

2 code implementations ECCV 2018 Zhirong Wu, Alexei A. Efros, Stella X. Yu

Current major approaches to visual recognition follow an end-to-end formulation that classifies an input image into one of the pre-determined set of semantic categories.

Unsupervised Feature Learning via Non-Parametric Instance Discrimination

4 code implementations CVPR 2018 Zhirong Wu, Yuanjun Xiong, Stella X. Yu, Dahua Lin

Neural net classifiers trained on data with annotated class labels can also capture apparent visual similarity among categories without being directed to do so.

General Classification object-detection +4

Adversarial Structure Matching for Structured Prediction Tasks

1 code implementation CVPR 2019 Jyh-Jing Hwang, Tsung-Wei Ke, Jianbo Shi, Stella X. Yu

The structure analyzer is trained to maximize the ASM loss, or to emphasize recurring multi-scale hard negative structural mistakes among co-occurring patterns.

Image Classification Monocular Depth Estimation +2

Learning Beyond Human Expertise with Generative Models for Dental Restorations

no code implementations30 Mar 2018 Jyh-Jing Hwang, Sergei Azernikov, Alexei A. Efros, Stella X. Yu

In the dental industry, it takes a technician years of training to design synthetic crowns that restore the function and integrity of missing teeth.

Object Recognition

Adaptive Affinity Fields for Semantic Segmentation

1 code implementation ECCV 2018 Tsung-Wei Ke, Jyh-Jing Hwang, Ziwei Liu, Stella X. Yu

Semantic segmentation has made much progress with increasingly powerful pixel-wise classifiers and incorporating structural priors via Conditional Random Fields (CRF) or Generative Adversarial Networks (GAN).

Segmentation Semantic Segmentation

Successive Embedding and Classification Loss for Aerial Image Classification

1 code implementation5 Dec 2017 Jiayun Wang, Patrick Virtue, Stella X. Yu

To address the overfitting problem in aerial image classification, we consider the neural network as successive transformations of an input image into embedded feature representations and ultimately into a semantic class label, and train neural networks to optimize image representations in the embedded space in addition to optimizing the final classification score.

Classification Clustering +2

Better than Real: Complex-valued Neural Nets for MRI Fingerprinting

no code implementations1 Jul 2017 Patrick Virtue, Stella X. Yu, Michael Lustig

The task of MRI fingerprinting is to identify tissue parameters from complex-valued MRI signals.

Learning Non-Lambertian Object Intrinsics across ShapeNet Categories

1 code implementation CVPR 2017 Jian Shi, Yue Dong, Hao Su, Stella X. Yu

Rendered with realistic environment maps, millions of synthetic images of objects and their corresponding albedo, shading, and specular ground-truth images are used to train an encoder-decoder CNN.


Multigrid Neural Architectures

1 code implementation CVPR 2017 Tsung-Wei Ke, Michael Maire, Stella X. Yu

Most critically, multigrid structure enables networks to learn internal attention and dynamic routing mechanisms, and use them to accomplish tasks on which modern CNNs fail.

Image Classification Semantic Segmentation

Unsupervised Learning of Important Objects from First-Person Videos

1 code implementation ICCV 2017 Gedas Bertasius, Hyun Soo Park, Stella X. Yu, Jianbo Shi

In this work, we show that we can detect important objects in first-person images without the supervision by the camera wearer or even third-person labelers.

Object Segmentation +1

Am I a Baller? Basketball Performance Assessment from First-Person Videos

no code implementations ICCV 2017 Gedas Bertasius, Hyun Soo Park, Stella X. Yu, Jianbo Shi

Finally, we use this feature to learn a basketball assessment model from pairs of labeled first-person basketball videos, for which a basketball expert indicates, which of the two players is better.

Convolutional Random Walk Networks for Semantic Image Segmentation

no code implementations CVPR 2017 Gedas Bertasius, Lorenzo Torresani, Stella X. Yu, Jianbo Shi

It combines these two objectives via a novel random walk layer that enforces consistent spatial grouping in the deep layers of the network.

Image Segmentation Scene Labeling +2

Fine-to-coarse Knowledge Transfer For Low-Res Image Classification

no code implementations21 May 2016 Xingchao Peng, Judy Hoffman, Stella X. Yu, Kate Saenko

We address the difficult problem of distinguishing fine-grained object categories in low resolution images.

Classification General Classification +2

First Person Action-Object Detection with EgoNet

no code implementations15 Mar 2016 Gedas Bertasius, Hyun Soo Park, Stella X. Yu, Jianbo Shi

Unlike traditional third-person cameras mounted on robots, a first-person camera, captures a person's visual sensorimotor object interactions from up close.

Human-Object Interaction Detection Object +2

Affinity CNN: Learning Pixel-Centric Pairwise Relations for Figure/Ground Embedding

no code implementations CVPR 2016 Michael Maire, Takuya Narihira, Stella X. Yu

Spectral embedding provides a framework for solving perceptual organization problems, including image segmentation and figure/ground organization.

Edge Detection Image Segmentation +2

Mapping Images to Sentiment Adjective Noun Pairs with Factorized Neural Nets

no code implementations21 Nov 2015 Takuya Narihira, Damian Borth, Stella X. Yu, Karl Ni, Trevor Darrell

We consider the visual sentiment task of mapping an image to an adjective noun pair (ANP) such as "cute baby".

Image Captioning

FlowWeb: Joint Image Set Alignment by Weaving Consistent, Pixel-Wise Correspondences

no code implementations CVPR 2015 Tinghui Zhou, Yong Jae Lee, Stella X. Yu, Alyosha A. Efros

Given a set of poorly aligned images of the same visual concept without any annotations, we propose an algorithm to jointly bring them into pixel-wise correspondence by estimating a FlowWeb representation of the image set.

Optical Flow Estimation

Learning Lightness From Human Judgement on Relative Reflectance

no code implementations CVPR 2015 Takuya Narihira, Michael Maire, Stella X. Yu

We develop a new approach to inferring lightness, the perceived reflectance of surfaces, from a single image.

Intrinsic Image Decomposition

Reconstructive Sparse Code Transfer for Contour Detection and Semantic Labeling

no code implementations16 Oct 2014 Michael Maire, Stella X. Yu, Pietro Perona

We frame the task of predicting a semantic labeling as a sparse reconstruction procedure that applies a target-specific learned transfer function to a generic deep sparse code representation of an image.

Contour Detection Dictionary Learning

Feature Transitions with Saccadic Search: Size, Color, and Orientation Are Not Alike

no code implementations NeurIPS 2010 Stella X. Yu

Size, color, and orientation have long been considered elementary features whose attributes are extracted in parallel and available to guide the deployment of attention.


Cannot find the paper you are looking for? You can Submit a new open access paper.