Search Results for author: Manmohan Chandraker

Found 107 papers, 26 papers with code

Weakly supervised 3D Reconstruction with Adversarial Constraint

2 code implementations31 May 2017 JunYoung Gwak, Christopher B. Choy, Animesh Garg, Manmohan Chandraker, Silvio Savarese

Supervised 3D reconstruction has witnessed a significant progress through the use of deep neural networks.

3D Reconstruction

Domain Adaptation for Structured Output via Discriminative Patch Representations

8 code implementations ICCV 2019 Yi-Hsuan Tsai, Kihyuk Sohn, Samuel Schulter, Manmohan Chandraker

Predicting structured outputs such as semantic segmentation relies on expensive per-pixel annotations to learn supervised models like convolutional neural networks.

Domain Adaptation Segmentation +2

YMIR: A Rapid Data-centric Development Platform for Vision Applications

1 code implementation19 Nov 2021 Phoenix X. Huang, Wenze Hu, William Brendel, Manmohan Chandraker, Li-Jia Li, Xiaoyu Wang

This paper introduces an open source platform to support the rapid development of computer vision applications at scale.

Active Learning

ALBench: A Framework for Evaluating Active Learning in Object Detection

1 code implementation27 Jul 2022 Zhanpeng Feng, Shiliang Zhang, Rinyoichi Takezoe, Wenze Hu, Manmohan Chandraker, Li-Jia Li, Vijay K. Narayanan, Xiaoyu Wang

To facilitate the research in this field, this paper contributes an active learning benchmark framework named as ALBench for evaluating active learning in object detection.

Active Learning Image Classification +4

Modulated Periodic Activations for Generalizable Local Functional Representations

2 code implementations ICCV 2021 Ishit Mehta, Michaël Gharbi, Connelly Barnes, Eli Shechtman, Ravi Ramamoorthi, Manmohan Chandraker

Our approach produces generalizable functional representations of images, videos and shapes, and achieves higher reconstruction quality than prior works that are optimized for a single signal.

FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation

1 code implementation15 Dec 2020 Tarun Kalluri, Deepak Pathak, Manmohan Chandraker, Du Tran

A majority of methods for video frame interpolation compute bidirectional optical flow between adjacent frames of a video, followed by a suitable warping algorithm to generate the output frames.

Action Recognition Motion Magnification +2

Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image

1 code implementation CVPR 2020 Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, Manmohan Chandraker

Our inverse rendering network incorporates physical insights -- including a spatially-varying spherical Gaussian lighting representation, a differentiable rendering layer to model scene appearance, a cascade structure to iteratively refine the predictions and a bilateral solver for refinement -- allowing us to jointly reason about shape, lighting, and reflectance.

Inverse Rendering

Deep Keypoint-Based Camera Pose Estimation with Geometric Constraints

1 code implementation29 Jul 2020 You-Yi Jau, Rui Zhu, Hao Su, Manmohan Chandraker

Estimating relative camera poses from consecutive frames is a fundamental problem in visual odometry (VO) and simultaneous localization and mapping (SLAM), where classic methods consisting of hand-crafted features and sampling-based outlier rejection have been a dominant choice for over a decade.

Pose Estimation Simultaneous Localization and Mapping +1

Neural Mesh Flow: 3D Manifold Mesh Generation via Diffeomorphic Flows

1 code implementation NeurIPS 2020 Kunal Gupta, Manmohan Chandraker

Applications like rendering, simulations and 3D printing require meshes to be manifold so that they can interact with the world like the real objects they represent.

DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

3 code implementations CVPR 2017 Namhoon Lee, Wongun Choi, Paul Vernaza, Christopher B. Choy, Philip H. S. Torr, Manmohan Chandraker

DESIRE effectively predicts future locations of objects in multiple scenes by 1) accounting for the multi-modal nature of the future prediction (i. e., given the same context, future may vary), 2) foreseeing the potential future outcomes and make a strategic prediction based on that, and 3) reasoning not only from the past motion history, but also from the scene context as well as the interactions among the agents.

Future prediction Multi Future Trajectory Prediction +1

Exploiting Unlabeled Data with Vision and Language Models for Object Detection

1 code implementation18 Jul 2022 Shiyu Zhao, Zhixing Zhang, Samuel Schulter, Long Zhao, Vijay Kumar B. G, Anastasis Stathopoulos, Manmohan Chandraker, Dimitris Metaxas

We propose a novel method that leverages the rich semantics available in recent vision and language models to localize and classify objects in unlabeled images, effectively generating pseudo labels for object detection.

Ranked #14 on Open Vocabulary Object Detection on MSCOCO (using extra training data)

Object object-detection +3

Taming Self-Training for Open-Vocabulary Object Detection

2 code implementations11 Aug 2023 Shiyu Zhao, Samuel Schulter, Long Zhao, Zhixing Zhang, Vijay Kumar B. G, Yumin Suh, Manmohan Chandraker, Dimitris N. Metaxas

This work identifies two challenges of using self-training in OVD: noisy PLs from VLMs and frequent distribution changes of PLs.

Object object-detection +1

Universal Semi-Supervised Semantic Segmentation

1 code implementation ICCV 2019 Tarun Kalluri, Girish Varma, Manmohan Chandraker, C. V. Jawahar

In recent years, the need for semantic segmentation has arisen across several different applications and environments.

Ranked #27 on Semantic Segmentation on DensePASS (using extra training data)

Segmentation Semi-Supervised Semantic Segmentation +1

IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments

2 code implementations26 Nov 2018 Girish Varma, Anbumani Subramanian, Anoop Namboodiri, Manmohan Chandraker, C. V. Jawahar

It also reflects label distributions of road scenes significantly different from existing datasets, with most classes displaying greater within-class diversity.

Autonomous Navigation Domain Adaptation +3

Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation

1 code implementation CVPR 2021 Astuti Sharma, Tarun Kalluri, Manmohan Chandraker

Domain adaptation deals with training models using large scale labeled data from a specific source domain and then adapting the knowledge to certain target domains that have few or no labels.

Clustering Unsupervised Domain Adaptation

TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments

1 code implementation16 Aug 2022 Shubham Dokania, Anbumani Subramanian, Manmohan Chandraker, C. V. Jawahar

We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation, mimicking real scene properties with high-fidelity, along with mechanisms to diversify samples in a physically meaningful way.

Semantic Segmentation Synthetic Data Generation

Single-Stream Multi-Level Alignment for Vision-Language Pretraining

1 code implementation27 Mar 2022 Zaid Khan, Vijay Kumar BG, Xiang Yu, Samuel Schulter, Manmohan Chandraker, Yun Fu

Self-supervised vision-language pretraining from pure images and text with a contrastive loss is effective, but ignores fine-grained alignment due to a dual-stream architecture that aligns image and text representations only on a global level.

Question Answering Referring Expression +4

MemSAC: Memory Augmented Sample Consistency for Large Scale Unsupervised Domain Adaptation

1 code implementation25 Jul 2022 Tarun Kalluri, Astuti Sharma, Manmohan Chandraker

Practical real world datasets with plentiful categories introduce new challenges for unsupervised domain adaptation like small inter-class discriminability, that existing approaches relying on domain invariance alone cannot handle sufficiently well.

Fine-Grained Visual Recognition Unsupervised Domain Adaptation

Gotta Adapt 'Em All: Joint Pixel and Feature-Level Domain Adaptation for Recognition in the Wild

1 code implementation CVPR 2019 Luan Tran, Kihyuk Sohn, Xiang Yu, Xiaoming Liu, Manmohan Chandraker

Recent developments in deep domain adaptation have allowed knowledge transfer from a labeled source domain to an unlabeled target domain at the level of intermediate features or input pixels.

Attribute Domain Adaptation +2

Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image

no code implementations ECCV 2018 Zhengqin Li, Kalyan Sunkavalli, Manmohan Chandraker

We propose a material acquisition approach to recover the spatially-varying BRDF and normal map of a near-planar surface from a single image captured by a handheld mobile phone camera.

Learning to Look around Objects for Top-View Representations of Outdoor Scenes

no code implementations ECCV 2018 Samuel Schulter, Menghua Zhai, Nathan Jacobs, Manmohan Chandraker

Given a single RGB image of a complex outdoor road scene in the perspective view, we address the novel problem of estimating an occlusion-reasoned semantic scene layout in the top-view.

Semantic Segmentation

Memory Warps for Learning Long-Term Online Video Representations

no code implementations28 Mar 2018 Tuan-Hung Vu, Wongun Choi, Samuel Schulter, Manmohan Chandraker

This paper proposes a novel memory-based online video representation that is efficient, accurate and predictive.

object-detection Object Detection

Hierarchical Metric Learning and Matching for 2D and 3D Geometric Correspondences

no code implementations ECCV 2018 Mohammed E. Fathy, Quoc-Huy Tran, M. Zeeshan Zia, Paul Vernaza, Manmohan Chandraker

Further, we propose to use activation maps at different layers of a CNN, as an effective and principled replacement for the multi-resolution image pyramids often used for matching tasks.

Geometric Matching Metric Learning +1

Feature Transfer Learning for Deep Face Recognition with Under-Represented Data

no code implementations23 Mar 2018 Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, Manmohan Chandraker

In this paper, we propose a center-based feature transfer framework to augment the feature space of under-represented subjects from the regular subjects that have sufficiently diverse samples.

Disentanglement Face Recognition +1

Learning random-walk label propagation for weakly-supervised semantic segmentation

no code implementations CVPR 2017 Paul Vernaza, Manmohan Chandraker

Large-scale training for semantic segmentation is challenging due to the expense of obtaining training data for this task relative to other vision tasks.

Segmentation Weakly supervised Semantic Segmentation +1

Deep Supervision with Intermediate Concepts

no code implementations8 Jan 2018 Chi Li, M. Zeeshan Zia, Quoc-Huy Tran, Xiang Yu, Gregory D. Hager, Manmohan Chandraker

In this work, we explore an approach for injecting prior domain structure into neural network training by supervising hidden layers of a CNN with intermediate concepts that normally are not observed in practice.

Image Classification

Towards Large-Pose Face Frontalization in the Wild

no code implementations ICCV 2017 Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, Manmohan Chandraker

Despite recent advances in face recognition using deep learning, severe accuracy drops are observed for large pose variations in unconstrained environments.

3D Reconstruction Face Recognition +1

Reconstruction-Based Disentanglement for Pose-invariant Face Recognition

no code implementations ICCV 2017 Xi Peng, Xiang Yu, Kihyuk Sohn, Dimitris Metaxas, Manmohan Chandraker

Finally, we propose a new feature reconstruction metric learning to explicitly disentangle identity and pose, by demanding alignment between the feature reconstructions through various combinations of identity and pose features, which is obtained from two images of the same subject.

Disentanglement Face Recognition +2

Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos

no code implementations ICCV 2017 Kihyuk Sohn, Sifei Liu, Guangyu Zhong, Xiang Yu, Ming-Hsuan Yang, Manmohan Chandraker

Despite rapid advances in face recognition, there remains a clear gap between the performance of still image-based face recognition and video-based face recognition, due to the vast difference in visual quality between the domains and the difficulty of curating diverse large-scale video datasets.

Data Augmentation Face Recognition +1

Deep Network Flow for Multi-Object Tracking

no code implementations CVPR 2017 Samuel Schulter, Paul Vernaza, Wongun Choi, Manmohan Chandraker

In this work, we demonstrate that it is possible to learn features for network-flow-based data association via backpropagation, by expressing the optimum of a smoothed network flow problem as a differentiable function of the pairwise association costs.

Graph Matching Multi-Object Tracking +1

Person Re-identification in the Wild

no code implementations CVPR 2017 Liang Zheng, Hengheng Zhang, Shaoyan Sun, Manmohan Chandraker, Yi Yang, Qi Tian

Our baselines address three issues: the performance of various combinations of detectors and recognizers, mechanisms for pedestrian detection to help improve overall re-identification accuracy and assessing the effectiveness of different detectors for re-identification.

Benchmarking Pedestrian Detection +2

Universal Correspondence Network

no code implementations NeurIPS 2016 Christopher B. Choy, JunYoung Gwak, Silvio Savarese, Manmohan Chandraker

We present a deep learning framework for accurate visual correspondences and demonstrate its effectiveness for both geometric and semantic matching, spanning across rigid motions to intra-class shape or appearance variations.

Metric Learning Semantic Similarity +1

A 4D Light-Field Dataset and CNN Architectures for Material Recognition

no code implementations24 Aug 2016 Ting-Chun Wang, Jun-Yan Zhu, Ebi Hiroaki, Manmohan Chandraker, Alexei A. Efros, Ravi Ramamoorthi

We introduce a new light-field dataset of materials, and take advantage of the recent success of deep learning to perform material recognition on the 4D light-field.

Image Classification Image Segmentation +4

Deep Deformation Network for Object Landmark Localization

no code implementations3 May 2016 Xiang Yu, Feng Zhou, Manmohan Chandraker

We propose a novel cascaded framework, namely deep deformation network (DDN), for localizing landmarks in non-rigid objects.

Face Alignment Object +1

WarpNet: Weakly Supervised Matching for Single-view Reconstruction

no code implementations CVPR 2016 Angjoo Kanazawa, David W. Jacobs, Manmohan Chandraker

This is in contrast to prior works that require part annotations, since matching objects across class and pose variations is challenging with appearance features alone.

Learning To Simulate

no code implementations ICLR 2019 Nataniel Ruiz, Samuel Schulter, Manmohan Chandraker

Simulation is a useful tool in situations where training data for machine learning models is costly to annotate or even hard to acquire.

Learning Efficient Object Detection Models with Knowledge Distillation

no code implementations NeurIPS 2017 Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, Manmohan Chandraker

In this work, we propose a new framework to learn compact and fast ob- ject detection networks with improved accuracy using knowledge distillation [20] and hint learning [34].

Knowledge Distillation Model Compression +4

Domain Adaptation for Structured Output via Disentangled Patch Representations

no code implementations ICLR 2019 Yi-Hsuan Tsai, Kihyuk Sohn, Samuel Schulter, Manmohan Chandraker

To this end, we propose to learn discriminative feature representations of patches based on label histograms in the source domain, through the construction of a disentangled space.

Domain Adaptation Semantic Segmentation

Unsupervised Domain Adaptation for Distance Metric Learning

no code implementations ICLR 2019 Kihyuk Sohn, Wenling Shang, Xiang Yu, Manmohan Chandraker

Unsupervised domain adaptation is a promising avenue to enhance the performance of deep neural networks on a target domain, using labels only from a source domain.

Face Recognition Metric Learning +1

Dense Object Reconstruction with Semantic Priors

no code implementations CVPR 2013 Sid Yingze Bao, Manmohan Chandraker, Yuanqing Lin, Silvio Savarese

Given multiple images of an unseen instance, we collate information from 2D object detectors to align the structure from motion point cloud with the mean shape, which is subsequently warped and refined to approach the actual shape.

Object object-detection +2

What Object Motion Reveals about Shape with Unknown BRDF and Lighting

no code implementations CVPR 2013 Manmohan Chandraker, Dikpal Reddy, Yizhou Wang, Ravi Ramamoorthi

Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination.

Surface Reconstruction

Robust Scale Estimation in Real-Time Monocular SFM for Autonomous Driving

no code implementations CVPR 2014 Shiyu Song, Manmohan Chandraker

Experiments on the KITTI dataset demonstrate the accuracy of our ground plane estimation, monocular SFM and object localization relative to ground truth, with detailed comparisons to prior art.

Autonomous Driving Object +3

What Camera Motion Reveals About Shape With Unknown BRDF

no code implementations CVPR 2014 Manmohan Chandraker

For the perspective case, we show that three differential motions suffice to yield surface depth for unknown isotropic BRDF and unknown directional lighting, while additional constraints are obtained with restrictions on BRDF or lighting.

Object

Joint SFM and Detection Cues for Monocular 3D Localization in Road Scenes

no code implementations CVPR 2015 Shiyu Song, Manmohan Chandraker

Experiments on the KITTI dataset show the efficacy of our cues, as well as the accuracy and robustness of our 3D object localization relative to ground truth and prior works.

Autonomous Driving Motion Segmentation +5

A Continuous Occlusion Model for Road Scene Understanding

no code implementations CVPR 2016 Vikas Dhiman, Quoc-Huy Tran, Jason J. Corso, Manmohan Chandraker

We present a physically interpretable, continuous 3D model for handling occlusions with applications to road scene understanding.

Motion Segmentation object-detection +3

Robust Energy Minimization for BRDF-Invariant Shape From Light Fields

no code implementations CVPR 2017 Zhengqin Li, Zexiang Xu, Ravi Ramamoorthi, Manmohan Chandraker

On the other hand, recent works have explored PDE invariants for shape recovery with complex BRDFs, but they have not been incorporated into robust numerical optimization frameworks.

Active Adversarial Domain Adaptation

no code implementations16 Apr 2019 Jong-Chyi Su, Yi-Hsuan Tsai, Kihyuk Sohn, Buyu Liu, Subhransu Maji, Manmohan Chandraker

Our approach, active adversarial domain adaptation (AADA), explores a duality between two related problems: adversarial domain alignment and importance sampling for adapting models across domains.

Active Learning Domain Adaptation +3

Adaptation Across Extreme Variations using Unlabeled Domain Bridges

no code implementations5 Jun 2019 Shuyang Dai, Kihyuk Sohn, Yi-Hsuan Tsai, Lawrence Carin, Manmohan Chandraker

We tackle an unsupervised domain adaptation problem for which the domain discrepancy between labeled source and unlabeled target domains is large, due to many factors of inter and intra-domain variation.

Object Recognition Semantic Segmentation +1

Pose-variant 3D Facial Attribute Generation

no code implementations24 Jul 2019 Feng-Ju Chang, Xiang Yu, Ram Nevatia, Manmohan Chandraker

We address the challenging problem of generating facial attributes using a single image in an unconstrained pose.

3D Reconstruction Attribute +1

Adversarial Learning of Privacy-Preserving and Task-Oriented Representations

no code implementations22 Nov 2019 Taihong Xiao, Yi-Hsuan Tsai, Kihyuk Sohn, Manmohan Chandraker, Ming-Hsuan Yang

For instance, there could be a potential privacy risk of machine learning systems via the model inversion attack, whose goal is to reconstruct the input data from the latent representation of deep networks.

Attribute BIG-bench Machine Learning +2

DAVID: Dual-Attentional Video Deblurring

no code implementations7 Dec 2019 Junru Wu, Xiang Yu, Ding Liu, Manmohan Chandraker, Zhangyang Wang

To train and evaluate on more diverse blur severity levels, we propose a Challenging DVD dataset generated from the raw DVD video set by pooling frames with different temporal windows.

Deblurring

Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Prediction

1 code implementation ECCV 2020 Lokender Tiwari, Pan Ji, Quoc-Huy Tran, Bingbing Zhuang, Saket Anand, Manmohan Chandraker

Classical monocular Simultaneous Localization And Mapping (SLAM) and the recently emerging convolutional neural networks (CNNs) for monocular depth prediction represent two largely disjoint approaches towards building a 3D map of the surrounding environment.

Depth Estimation Depth Prediction +1

Improving Face Recognition by Clustering Unlabeled Faces in the Wild

no code implementations ECCV 2020 Aruni RoyChowdhury, Xiang Yu, Kihyuk Sohn, Erik Learned-Miller, Manmohan Chandraker

While deep face recognition has benefited significantly from large-scale labeled data, current research is focused on leveraging unlabeled data to further boost performance, reducing the cost of human annotation.

Clustering Face Clustering +3

SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction

no code implementations ECCV 2020 Sriram N. N, Buyu Liu, Francesco Pittaluga, Manmohan Chandraker

Our second contribution is a novel method that generates diverse predictions while accounting for scene semantics and multi-agent interactions, with constant-time inference independent of the number of agents.

Motion Forecasting Trajectory Forecasting

OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets

no code implementations25 Jul 2020 Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Meng Song, YuHan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh Gundavarapu, Jia Shi, Sai Bi, Zexiang Xu, Hong-Xing Yu, Kalyan Sunkavalli, Miloš Hašan, Ravi Ramamoorthi, Manmohan Chandraker

Finally, we demonstrate that our framework may also be integrated with physics engines, to create virtual robotics environments with unique ground truth such as friction coefficients and correspondence to real scenes.

Friction Inverse Rendering +2

Domain Adaptive Semantic Segmentation Using Weak Labels

no code implementations ECCV 2020 Sujoy Paul, Yi-Hsuan Tsai, Samuel Schulter, Amit K. Roy-Chowdhury, Manmohan Chandraker

In this work, we propose a novel framework for domain adaptation in semantic segmentation with image-level weak labels in the target domain.

Segmentation Semantic Segmentation +1

Object Detection with a Unified Label Space from Multiple Datasets

no code implementations ECCV 2020 Xiangyun Zhao, Samuel Schulter, Gaurav Sharma, Yi-Hsuan Tsai, Manmohan Chandraker, Ying Wu

To address this challenge, we design a framework which works with such partial annotations, and we exploit a pseudo labeling approach that we adapt for our specific case.

Object object-detection +1

Single-Shot Neural Relighting and SVBRDF Estimation

no code implementations ECCV 2020 Shen Sang, Manmohan Chandraker

We present a novel physically-motivated deep network for joint shape and material estimation, as well as relighting under novel illumination conditions, using a single image captured by a mobile phone camera.

Inverse Rendering SVBRDF Estimation

Uncertainty-Aware Physically-Guided Proxy Tasks for Unseen Domain Face Anti-spoofing

no code implementations28 Nov 2020 Junru Wu, Xiang Yu, Buyu Liu, Zhangyang Wang, Manmohan Chandraker

Face anti-spoofing (FAS) seeks to discriminate genuine faces from fake ones arising from any type of spoofing attack.

Attribute Domain Generalization +1

Cross-Domain Similarity Learning for Face Recognition in Unseen Domains

no code implementations CVPR 2021 Masoud Faraki, Xiang Yu, Yi-Hsuan Tsai, Yumin Suh, Manmohan Chandraker

Intuitively, it discriminatively correlates explicit metrics derived from one domain, with triplet samples from another domain in a unified loss function to be minimized within a network, which leads to better alignment of the training domains.

Face Recognition Metric Learning

Weakly But Deeply Supervised Occlusion-Reasoned Parametric Road Layouts

no code implementations CVPR 2022 Buyu Liu, Bingbing Zhuang, Manmohan Chandraker

We propose an end-to-end network that takes a single perspective RGB image of a complex road scene as input, to produce occlusion-reasoned layouts in perspective space as well as a parametric bird's-eye-view (BEV) space.

Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty

no code implementations CVPR 2021 Bingbing Zhuang, Manmohan Chandraker

While we focus on relative pose, we envision that our pipeline is broadly applicable for fusing classical geometry and deep learning.

Pose Estimation

Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction

no code implementations CVPR 2021 Sriram Narayanan, Ramin Moslemi, Francesco Pittaluga, Buyu Liu, Manmohan Chandraker

Our second contribution is a novel trajectory prediction framework called ALAN that uses existing lane centerlines as anchors to provide trajectories constrained to the input lanes.

Autonomous Vehicles Trajectory Prediction

OpenRooms: An Open Framework for Photorealistic Indoor Scene Datasets

no code implementations CVPR 2021 Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Meng Song, YuHan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh Gundavarapu, Jia Shi, Sai Bi, Hong-Xing Yu, Zexiang Xu, Kalyan Sunkavalli, Milos Hasan, Ravi Ramamoorthi, Manmohan Chandraker

Finally, we demonstrate that our framework may also be integrated with physics engines, to create virtual robotics environments with unique ground truth such as friction coefficients and correspondence to real scenes.

Friction Inverse Rendering +1

Learning Semantic Segmentation from Multiple Datasets with Label Shifts

no code implementations28 Feb 2022 Dongwan Kim, Yi-Hsuan Tsai, Yumin Suh, Masoud Faraki, Sparsh Garg, Manmohan Chandraker, Bohyung Han

First, a gradient conflict in training due to mismatched label spaces is identified and a class-independent binary cross-entropy loss is proposed to alleviate such label conflicts.

Semantic Segmentation

On Generalizing Beyond Domains in Cross-Domain Continual Learning

no code implementations CVPR 2022 Christian Simon, Masoud Faraki, Yi-Hsuan Tsai, Xiang Yu, Samuel Schulter, Yumin Suh, Mehrtash Harandi, Manmohan Chandraker

Humans have the ability to accumulate knowledge of new tasks in varying conditions, but deep neural networks often suffer from catastrophic forgetting of previously learned knowledge after learning a new task.

Continual Learning Knowledge Distillation

Controllable Dynamic Multi-Task Architectures

no code implementations CVPR 2022 Dripta S. Raychaudhuri, Yumin Suh, Samuel Schulter, Xiang Yu, Masoud Faraki, Amit K. Roy-Chowdhury, Manmohan Chandraker

In contrast to the existing dynamic multi-task approaches that adjust only the weights within a fixed architecture, our approach affords the flexibility to dynamically control the total computational cost and match the user-preferred task importance better.

Multi-Task Learning

A Level Set Theory for Neural Implicit Evolution under Explicit Flows

no code implementations14 Apr 2022 Ishit Mehta, Manmohan Chandraker, Ravi Ramamoorthi

Our method uses the flow field to deform parametric implicit surfaces by extending the classical theory of level sets.

Inverse Rendering

Physically-Based Editing of Indoor Scene Lighting from a Single Image

no code implementations19 May 2022 Zhengqin Li, Jia Shi, Sai Bi, Rui Zhu, Kalyan Sunkavalli, Miloš Hašan, Zexiang Xu, Ravi Ramamoorthi, Manmohan Chandraker

We tackle this problem using two novel components: 1) a holistic scene reconstruction method that estimates scene reflectance and parametric 3D lighting, and 2) a neural rendering framework that re-renders the scene from our predictions.

Inverse Rendering Lighting Estimation +1

IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes

no code implementations CVPR 2022 Rui Zhu, Zhengqin Li, Janarbek Matai, Fatih Porikli, Manmohan Chandraker

Indoor scenes exhibit significant appearance variations due to myriad interactions between arbitrarily diverse object shapes, spatially-changing materials, and complex lighting.

Inverse Rendering

Cluster-to-adapt: Few Shot Domain Adaptation for Semantic Segmentation across Disjoint Labels

no code implementations4 Aug 2022 Tarun Kalluri, Manmohan Chandraker

Domain adaptation for semantic segmentation across datasets consisting of the same categories has seen several recent successes.

Clustering Domain Adaptation +2

IDD-3D: Indian Driving Dataset for 3D Unstructured Road Scenes

no code implementations23 Oct 2022 Shubham Dokania, A. H. Abdul Hafez, Anbumani Subramanian, Manmohan Chandraker, C. V. Jawahar

Autonomous driving and assistance systems rely on annotated data from traffic and road scenarios to model and learn the various object relations in complex real-world scenarios.

3D Object Detection Autonomous Driving +2

Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport

no code implementations28 Oct 2022 Sriram Narayanan, Dinesh Jayaraman, Manmohan Chandraker

We address key challenges in long-horizon embodied exploration and navigation by proposing a new object transport task and a novel modular framework for temporally extended navigation.

Motion Planning

Open-world Instance Segmentation: Top-down Learning with Bottom-up Supervision

no code implementations9 Mar 2023 Tarun Kalluri, Weiyao Wang, Heng Wang, Manmohan Chandraker, Lorenzo Torresani, Du Tran

Many top-down architectures for instance segmentation achieve significant success when trained and tested on pre-defined closed-world taxonomy.

Open-World Instance Segmentation Segmentation +1

GeoNet: Benchmarking Unsupervised Adaptation across Geographies

no code implementations CVPR 2023 Tarun Kalluri, Wangdong Xu, Manmohan Chandraker

In recent years, several efforts have been aimed at improving the robustness of vision models to domains and environments unseen during training.

Benchmarking Image Classification +2

Real-Time Radiance Fields for Single-Image Portrait View Synthesis

no code implementations3 May 2023 Alex Trevithick, Matthew Chan, Michael Stengel, Eric R. Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano

We present a one-shot method to infer and render a photorealistic 3D representation from a single unposed image (e. g., face portrait) in real-time.

Data Augmentation Novel View Synthesis

Spatiotemporally Consistent HDR Indoor Lighting Estimation

no code implementations7 May 2023 Zhengqin Li, Li Yu, Mikhail Okunev, Manmohan Chandraker, Zhao Dong

For training, we significantly enhance the OpenRooms public dataset of photorealistic synthetic indoor scenes with around 360K HDR environment maps of much higher resolution and 38K video sequences, rendered with GPU-based path tracing.

Lighting Estimation

Tuned Contrastive Learning

no code implementations18 May 2023 Chaitanya Animesh, Manmohan Chandraker

A recent state-of-the-art, supervised contrastive (SupCon) loss, extends self-supervised contrastive learning to supervised setting by generalizing to multiple positives and negatives in a batch and improves upon the cross-entropy loss.

Contrastive Learning Representation Learning +1

Learning Phase Mask for Privacy-Preserving Passive Depth Estimation

no code implementations European Conference on Computer Vision (ECCV) 2022 Zaid Tasneem, Giovanni Milione, Yi-Hsuan Tsai, Xiang Yu, Ashok Veeraraghavan, Manmohan Chandraker, Francesco Pittaluga

With over a billion sold each year, cameras are not only becoming ubiquitous and omnipresent, but are driving progress in a wide range of applications such as augmented/virtual reality, robotics, surveillance, security, autonomous navigation and many others.

Autonomous Navigation Depth Estimation +2

A Theory of Topological Derivatives for Inverse Rendering of Geometry

no code implementations ICCV 2023 Ishit Mehta, Manmohan Chandraker, Ravi Ramamoorthi

We introduce a theoretical framework for differentiable surface evolution that allows discrete topology changes through the use of topological derivatives for variational optimization of image functionals.

3D Reconstruction Image Reconstruction +3

Efficient Controllable Multi-Task Architectures

no code implementations ICCV 2023 Abhishek Aich, Samuel Schulter, Amit K. Roy-Chowdhury, Manmohan Chandraker, Yumin Suh

Further, we present a simple but effective search algorithm that translates user constraints to runtime width configurations of both the shared encoder and task decoders, for sampling the sub-architectures.

Knowledge Distillation

OpEnCam: Lensless Optical Encryption Camera

no code implementations2 Dec 2023 Salman S. Khan, Xiang Yu, Kaushik Mitra, Manmohan Chandraker, Francesco Pittaluga

OpEnCam encrypts the incoming light before capturing it using the modulating ability of optical masks.

Controllable Safety-Critical Closed-loop Traffic Simulation via Guided Diffusion

no code implementations31 Dec 2023 Wei-Jer Chang, Francesco Pittaluga, Masayoshi Tomizuka, Wei Zhan, Manmohan Chandraker

These findings affirm that guided diffusion models provide a robust and versatile foundation for safety-critical, interactive traffic simulation, extending their utility across the broader landscape of autonomous driving.

Autonomous Driving Denoising

LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning

no code implementations30 Dec 2023 S P Sharan, Francesco Pittaluga, Vijay Kumar B G, Manmohan Chandraker

Although planning is a crucial component of the autonomous driving stack, researchers have yet to develop robust planning algorithms that are capable of safely handling the diverse range of possible driving scenarios.

Autonomous Driving Common Sense Reasoning

Generating Enhanced Negatives for Training Language-Based Object Detectors

no code implementations29 Dec 2023 Shiyu Zhao, Long Zhao, Vijay Kumar B. G, Yumin Suh, Dimitris N. Metaxas, Manmohan Chandraker, Samuel Schulter

The recent progress in language-based open-vocabulary object detection can be largely attributed to finding better ways of leveraging large-scale data with free-form text annotations.

Object object-detection +1

What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs

no code implementations4 Jan 2024 Alex Trevithick, Matthew Chan, Towaki Takikawa, Umar Iqbal, Shalini De Mello, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano

3D-aware Generative Adversarial Networks (GANs) have shown remarkable progress in learning to generate multi-view-consistent images and 3D geometries of scenes from collections of 2D images via neural volume rendering.

Neural Rendering Super-Resolution

TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion

no code implementations17 Jan 2024 Yu-Ying Yeh, Jia-Bin Huang, Changil Kim, Lei Xiao, Thu Nguyen-Phuoc, Numair Khan, Cheng Zhang, Manmohan Chandraker, Carl S Marshall, Zhao Dong, Zhengqin Li

In contrast, TextureDreamer can transfer highly detailed, intricate textures from real-world environments to arbitrary objects with only a few casually captured images, potentially significantly democratizing texture creation.

Texture Synthesis

Tell, Don't Show!: Language Guidance Eases Transfer Across Domains in Images and Videos

no code implementations8 Mar 2024 Tarun Kalluri, Bodhisattwa Prasad Majumder, Manmohan Chandraker

We introduce LaGTran, a novel framework that utilizes readily available or easily acquired text descriptions to guide robust transfer of discriminative knowledge from labeled source to unlabeled target data with domain shifts.

Cannot find the paper you are looking for? You can Submit a new open access paper.