no code implementations • ECCV 2020 • Zudi Lin, Donglai Wei, Won-Dong Jang, Siyan Zhou, Xupeng Chen, Xueying Wang, Richard Schalek, Daniel Berger, Brian Matejek, Lee Kamentsky, Adi Peleg, Daniel Haehn, Thouis Jones, Toufiq Parag, Jeff Lichtman, Hanspeter Pfister
As a use case, we build an end-to-end active learning framework with our query suggestion method for 3D synapse detection and mitochondria segmentation in connectomics.
no code implementations • 15 Jun 2022 • Spandan Madan, Li You, Mengmi Zhang, Hanspeter Pfister, Gabriel Kreiman
Here we present SemanticDG (Semantic Domain Generalization): a benchmark with 15 photo-realistic domains with the same geometry, scene layout and camera parameters as the popular 3D ScanNet dataset, but with controlled domain shifts in lighting, materials, and viewpoints.
no code implementations • 9 Jun 2022 • Weikai Yang, Xi Ye, Xingxing Zhang, Lanxi Xiao, Jiazhi Xia, Zhongyuan Wang, Jun Zhu, Hanspeter Pfister, Shixia Liu
The base learners and labeled samples (shots) in an ensemble few-shot classifier greatly affect the model performance.
1 code implementation • 17 Apr 2022 • Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Zhang, Hanspeter Pfister, Radu Timofte, Luc van Gool
Existing leading methods for spectral reconstruction (SR) focus on designing deeper or wider convolutional neural networks (CNNs) to learn the end-to-end mapping from the RGB image to its hyperspectral image (HSI).
no code implementations • 6 Apr 2022 • Leander Lauenburg, Zudi Lin, Ruihan Zhang, Márcia dos Santos, Siyu Huang, Ignacio Arganda-Carreras, Edward S. Boyden, Hanspeter Pfister, Donglai Wei
Instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming.
2 code implementations • NeurIPS 2021 • Yuanhao Cai, Xiaowan Hu, Haoqian Wang, Yulun Zhang, Hanspeter Pfister, Donglai Wei
Simultaneously, PNGAN establishes a pixel-level adversarial training to conduct noise domain alignment.
1 code implementation • 27 Jan 2022 • Zudi Lin, Prateek Garg, Atmadeep Banerjee, Salma Abdel Magid, Deqing Sun, Yulun Zhang, Luc van Gool, Donglai Wei, Hanspeter Pfister
Image super-resolution (SR) is a fast-moving field with novel architectures attracting the spotlight.
no code implementations • CVPR 2022 • Salma Abdel Magid, Zudi Lin, Donglai Wei, Yulun Zhang, Jinjin Gu, Hanspeter Pfister
Our key contribution is to leverage a texture classifier, which enables us to assign patches with semantic labels, to identify the source of SR errors both globally and locally.
no code implementations • CVPR 2022 • Donglai Wei, Siddhant Kharbanda, Sarthak Arora, Roshan Roy, Nishant Jain, Akash Palrecha, Tanav Shah, Shray Mathur, Ritik Mathur, Abhijay Kemkar, Anirudh Chakravarthy, Zudi Lin, Won-Dong Jang, Yansong Tang, Song Bai, James Tompkin, Philip H.S. Torr, Hanspeter Pfister
Many video understanding tasks require analyzing multi-shot videos, but existing datasets for video object segmentation (VOS) only consider single-shot videos.
1 code implementation • 10 Dec 2021 • Zudi Lin, Donglai Wei, Jeff Lichtman, Hanspeter Pfister
We present PyTorch Connectomics (PyTC), an open-source deep-learning framework for the semantic and instance segmentation of volumetric microscopy images, built upon PyTorch.
1 code implementation • 15 Nov 2021 • Anirudh S Chakravarthy, Won-Dong Jang, Zudi Lin, Donglai Wei, Song Bai, Hanspeter Pfister
Motivated by this, we propose a video instance segmentation method that alleviates the problem due to missing detections.
Ranked #16 on
Video Instance Segmentation
on YouTube-VIS validation
1 code implementation • 30 Oct 2021 • Akira Sakai, Taro Sunagawa, Spandan Madan, Kanata Suzuki, Takashi Katoh, Hiromichi Kobashi, Hanspeter Pfister, Pawan Sinha, Xavier Boix, Tomotake Sasaki
While humans have a remarkable capability of recognizing objects in out-of-distribution (OoD) orientations and illuminations, Deep Neural Networks (DNNs) severely suffer in this case, even when large amounts of training examples are available.
2 code implementations • 27 Oct 2021 • Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, Bingbing Ni
We introduce MedMNIST v2, a large-scale MNIST-like dataset collection of standardized biomedical images, including 12 datasets for 2D and 6 datasets for 3D.
no code implementations • 19 Oct 2021 • Hendrik Strobelt, Jambay Kinley, Robert Krueger, Johanna Beyer, Hanspeter Pfister, Alexander M. Rush
These controls allow users to globally constrain model generations, without sacrificing the representation power of the deep learning models.
1 code implementation • 10 Oct 2021 • Jared Jessup, Robert Krueger, Simon Warchol, John Hoffer, Jeremy Muhlich, Cecily C. Ritch, Giorgio Gaglia, Shannon Coy, Yu-An Chen, Jia-Ren Lin, Sandro Santagata, Peter K. Sorger, Hanspeter Pfister
Inspection of tissues using a light microscope is the primary method of diagnosing many diseases, notably cancer.
no code implementations • 29 Sep 2021 • Yuke Li, Kenneth Li, Pin Wang, Donglai Wei, Hanspeter Pfister, Ching-Yao Chan
Non-stationary casual structures are prevalent in real-world physical systems.
no code implementations • 28 Sep 2021 • Avi Cooper, Xavier Boix, Daniel Harari, Spandan Madan, Hanspeter Pfister, Tomotake Sasaki, Pawan Sinha
In contrast, for DNNs, it remains unknown how generalization abilities are distributed among OoD orientations.
1 code implementation • 17 Sep 2021 • Jiancheng Yang, Yi He, Kaiming Kuang, Zudi Lin, Hanspeter Pfister, Bingbing Ni
The proposed A3D consistently outperforms symmetric context fusion operators by considerable margins, and establishes a new \emph{state of the art} on DeepLesion.
1 code implementation • 17 Sep 2021 • Jiancheng Yang, Shixuan Gu, Donglai Wei, Hanspeter Pfister, Bingbing Ni
Manual rib inspections in computed tomography (CT) scans are clinically critical but labor-intensive, as 24 ribs are typically elongated and oblique in 3D volumes.
1 code implementation • 13 Jul 2021 • Zudi Lin, Donglai Wei, Mariela D. Petkova, Yuelong Wu, Zergham Ahmed, Krishna Swaroop K, Silin Zou, Nils Wendt, Jonathan Boulanger-Weill, Xueying Wang, Nagaraju Dhanyasi, Ignacio Arganda-Carreras, Florian Engert, Jeff Lichtman, Hanspeter Pfister
Segmenting 3D cell nuclei from microscopy image volumes is critical for biological and clinical analysis, enabling the study of cellular expression patterns and cell lineages.
no code implementations • 13 Jul 2021 • Stanislav Lukyanenko, Won-Dong Jang, Donglai Wei, Robbert Struyven, Yoon Kim, Brian Leahy, Helen Yang, Alexander Rush, Dalit Ben-Yosef, Daniel Needleman, Hanspeter Pfister
In this work, we propose a two-stream model for developmental stage classification.
1 code implementation • 12 Jul 2021 • Donglai Wei, Kisuk Lee, Hanyu Li, Ran Lu, J. Alexander Bae, Zequan Liu, Lifu Zhang, Márcia dos Santos, Zudi Lin, Thomas Uram, Xueying Wang, Ignacio Arganda-Carreras, Brian Matejek, Narayanan Kasthuri, Jeff Lichtman, Hanspeter Pfister
Electron microscopy (EM) enables the reconstruction of neural circuits at the level of individual synapses, which has been transformative for scientific discoveries.
no code implementations • 30 Jun 2021 • Spandan Madan, Tomotake Sasaki, Tzu-Mao Li, Xavier Boix, Hanspeter Pfister
Despite training with a large-scale (0. 5 million images), unbiased dataset of camera and light variations, in over 71% cases CMA-Search can find camera parameters in the vicinity of a correctly classified image which lead to in-distribution misclassifications with < 3. 6% change in parameters.
no code implementations • 14 May 2021 • Felix Gonda, Xueying Wang, Johanna Beyer, Markus Hadwiger, Jeff W. Lichtman, Hanspeter Pfister
A connectivity graph of neurons at the resolution of single synapses provides scientists with a tool for understanding the nervous system in health and disease.
no code implementations • CVPR 2022 • Zixiang Zhao, Jiangshe Zhang, Shuang Xu, Zudi Lin, Hanspeter Pfister
Guided depth super-resolution (GDSR) is an essential topic in multi-modal image processing, which reconstructs high-resolution (HR) depth maps from low-resolution ones collected with suboptimal conditions with the help of HR RGB images of the same scene.
1 code implementation • 13 Mar 2021 • Mallikarjun B R, Ayush Tewari, Abdallah Dib, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Louis Chevallier, Mohamed Elgharib, Christian Theobalt
We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image.
no code implementations • 1 Feb 2021 • Felix Gonda, Donglai Wei, Hanspeter Pfister
We present a recurrent network for the 3D reconstruction of neurons that sequentially generates binary masks for every object in an image with spatio-temporal consistency.
no code implementations • ICCV 2021 • Salma Abdel Magid, Yulun Zhang, Donglai Wei, Won-Dong Jang, Zudi Lin, Yun Fu, Hanspeter Pfister
Specifically, we propose a dynamic high-pass filtering (HPF) module that locally applies adaptive filter weights for each spatial location and channel group to preserve high-frequency signals.
no code implementations • ICCV 2021 • Yulun Zhang, Donglai Wei, Can Qin, Huan Wang, Hanspeter Pfister, Yun Fu
However, the basic convolutional layer in CNNs is designed to extract local patterns, lacking the ability to model global context.
no code implementations • 1 Jan 2021 • Spandan Madan, Timothy Henry, Jamell Arthur Dozier, Helen Ho, Nishchal Bhandari, Tomotake Sasaki, Fredo Durand, Hanspeter Pfister, Xavier Boix
We find that learning category and viewpoint in separate networks compared to a shared one leads to an increase in selectivity and invariance, as separate networks are not forced to preserve information about both category and viewpoint.
no code implementations • 2 Dec 2020 • Won-Dong Jang, Donglai Wei, Xingxuan Zhang, Brian Leahy, Helen Yang, James Tompkin, Dalit Ben-Yosef, Daniel Needleman, Hanspeter Pfister
To alleviate the problem, we propose to classify input features into intermediate shape codes and recover complete object shapes from them.
1 code implementation • Medical Image Computing and Computer Assisted Intervention 2020 • Donglai Wei, Zudi Lin, Daniel Franco-Barranco, Nils Wendt, Xingyu Liu, Wenjie Yin, Xin Huang, Aarush Gupta, Won-Dong Jang, Xueying Wang, Ignacio Arganda-Carreras, Jeff Lichtman, Hanspeter Pfister
On MitoEM, we find existing instance segmentation methods often fail to correctly segment mitochondria with complex shapes or close contacts with other instances.
Ranked #2 on
3D Instance Segmentation
on MitoEM
(AP75-R-Test metric)
no code implementations • CVPR 2021 • Mallikarjun B R., Ayush Tewari, Tae-Hyun Oh, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, Christian Theobalt
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.
1 code implementation • 15 Jul 2020 • Spandan Madan, Timothy Henry, Jamell Dozier, Helen Ho, Nishchal Bhandari, Tomotake Sasaki, Frédo Durand, Hanspeter Pfister, Xavier Boix
In this paper, we investigate when and how such OOD generalization may be possible by evaluating CNNs trained to classify both object category and 3D viewpoint on OOD combinations, and identifying the neural mechanisms that facilitate such OOD generalization.
no code implementations • 29 May 2020 • Brian D. Leahy, Won-Dong Jang, Helen Y. Yang, Robbert Struyven, Donglai Wei, Zhe Sun, Kylie R. Lee, Charlotte Royston, Liz Cam, Yael Kalma, Foad Azem, Dalit Ben-Yosef, Hanspeter Pfister, Daniel Needleman
A major challenge in clinical In-Vitro Fertilization (IVF) is selecting the highest quality embryo to transfer to the patient in the hopes of achieving a pregnancy.
no code implementations • 27 Apr 2020 • Polina Golland, Jack Gallant, Greg Hager, Hanspeter Pfister, Christos Papadimitriou, Stefan Schaal, Joshua T. Vogelstein
In December 2014, a two-day workshop supported by the Computing Community Consortium (CCC) and the National Science Foundation's Computer and Information Science and Engineering Directorate (NSF CISE) was convened in Washington, DC, with the goal of bringing together computer scientists and brain researchers to explore these new opportunities and connections, and develop a new, modern dialogue between the two research communities.
1 code implementation • 27 Sep 2019 • Abhimanyu Talwar, Zudi Lin, Donglai Wei, Yuesong Wu, Bowen Zheng, Jinglin Zhao, Won-Dong Jang, Xueying Wang, Jeff W. Lichtman, Hanspeter Pfister
Next, we develop nomenclature rules for pyramidal neurons and mitochondria from the reduced graph and finally learn the feature embedding for shape manipulation.
1 code implementation • 13 Sep 2019 • Zudi Lin, Hanspeter Pfister, Ziming Zhang
In this paper, we study the problem of how to defend classifiers against adversarial attacks that fool the classifiers using subtly modified input data.
no code implementations • 29 Jul 2019 • Frederik L. Dennig, Tom Polk, Zudi Lin, Tobias Schreck, Hanspeter Pfister, Michael Behrisch
The detection of interesting patterns in large high-dimensional datasets is difficult because of their dimensionality and pattern complexity.
1 code implementation • 24 Jul 2019 • Sebastian Gehrmann, Hendrik Strobelt, Robert Krüger, Hanspeter Pfister, Alexander M. Rush
Automation of tasks can have critical consequences when humans lose agency over decision processes.
no code implementations • MIDL 2019 • Vincent Casser, Kai Kang, Hanspeter Pfister, Daniel Haehn
High-resolution connectomics data allows for the identification of dysfunctional mitochondria which are linked to a variety of diseases such as autism or bipolar.
no code implementations • WS 2018 • Hendrik Strobelt, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, Alex Rush, er
Neural attention-based sequence-to-sequence models (seq2seq) (Sutskever et al., 2014; Bahdanau et al., 2014) have proven to be accurate and robust for many sequence prediction tasks.
no code implementations • 11 Sep 2018 • Felix Gonda, Donglai Wei, Toufiq Parag, Hanspeter Pfister
For video and volumetric data understanding, 3D convolution layers are widely used in deep learning, however, at the cost of increasing computation and training time.
1 code implementation • 27 Jul 2018 • Spandan Madan, Zoya Bylinskii, Matthew Tancik, Adrià Recasens, Kimberli Zhong, Sami Alsheikh, Hanspeter Pfister, Aude Oliva, Fredo Durand
While automatic text extraction works well on infographics, computer vision approaches trained on natural images fail to identify the stand-alone visual elements in infographics, or `icons'.
1 code implementation • 8 Jul 2018 • Toufiq Parag, Daniel Berger, Lee Kamentsky, Benedikt Staffler, Donglai Wei, Moritz Helmstaedter, Jeff W. Lichtman, Hanspeter Pfister
The few methods that computes direction along with contact location have only been demonstrated to work on either dyadic (most common in vertebrate brain) or polyadic (found in fruit fly brain) synapses, but not on both types.
1 code implementation • 25 Apr 2018 • Hendrik Strobelt, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, Alexander M. Rush
In this work, we present a visual analysis tool that allows interaction with a trained sequence-to-sequence model through each stage of the translation process.
no code implementations • ICCV 2017 • Xiangyu Xu, Deqing Sun, Jinshan Pan, Yu-Jin Zhang, Hanspeter Pfister, Ming-Hsuan Yang
We present an algorithm to directly restore a clear high-resolution image from a blurry low-resolution input.
1 code implementation • 26 Sep 2017 • Zoya Bylinskii, Sami Alsheikh, Spandan Madan, Adria Recasens, Kimberli Zhong, Hanspeter Pfister, Fredo Durand, Aude Oliva
And second, we use these predicted text tags as a supervisory signal to localize the most diagnostic visual elements from within the infographic i. e. visual hashtags.
1 code implementation • 8 Aug 2017 • Zoya Bylinskii, Nam Wook Kim, Peter O'Donovan, Sami Alsheikh, Spandan Madan, Hanspeter Pfister, Fredo Durand, Bryan Russell, Aaron Hertzmann
Our models are neural networks trained on human clicks and importance annotations on hundreds of designs.
no code implementations • 27 Jul 2017 • Toufiq Parag, Fabian Tschopp, William Grisaitis, Srinivas C. Turaga, Xuewen Zhang, Brian Matejek, Lee Kamentsky, Jeff W. Lichtman, Hanspeter Pfister
The field of connectomics has recently produced neuron wiring diagrams from relatively large brain regions from multiple animals.
no code implementations • CVPR 2017 • Ajjen Joshi, Soumya Ghosh, Margrit Betke, Stan Sclaroff, Hanspeter Pfister
Leveraging recent work on learning Bayesian neural networks, we build fast, scalable algorithms for inferring the posterior distribution over all network weights in the hierarchy.
no code implementations • 12 Jun 2017 • James Tompkin, Kwang In Kim, Hanspeter Pfister, Christian Theobalt
Large databases are often organized by hand-labeled metadata, or criteria, which are expensive to collect.
no code implementations • 30 May 2017 • David Rolnick, Yaron Meirovitch, Toufiq Parag, Hanspeter Pfister, Viren Jain, Jeff W. Lichtman, Edward S. Boyden, Nir Shavit
Deep learning algorithms for connectomics rely upon localized classification, rather than overall morphology.
no code implementations • CVPR 2018 • Daniel Haehn, Verena Kaynig, James Tompkin, Jeff W. Lichtman, Hanspeter Pfister
Automatic cell image segmentation methods in connectomics produce merge and split errors, which require correction through proofreading.
no code implementations • 16 Feb 2017 • Nam Wook Kim, Zoya Bylinskii, Michelle A. Borkin, Krzysztof Z. Gajos, Aude Oliva, Fredo Durand, Hanspeter Pfister
In this paper, we present BubbleView, an alternative methodology for eye tracking using discrete mouse clicks to measure which information people consciously choose to examine.
no code implementations • 7 Dec 2016 • Yaron Meirovitch, Alexander Matveev, Hayk Saribekyan, David Budden, David Rolnick, Gergely Odor, Seymour Knowles-Barley, Thouis Raymond Jones, Hanspeter Pfister, Jeff William Lichtman, Nir Shavit
The field of connectomics faces unprecedented "big data" challenges.
no code implementations • 21 Nov 2016 • Seymour Knowles-Barley, Verena Kaynig, Thouis Ray Jones, Alyssa Wilson, Joshua Morgan, Dongil Lee, Daniel Berger, Narayanan Kasthuri, Jeff W. Lichtman, Hanspeter Pfister
The best segmentation results obtained gave $V^\text{Info}_\text{F-score}$ scores of 0. 9054 and 09182 for the cortex datasets, 0. 9438 for LGN, and 0. 9150 for Cerebellum.
no code implementations • 27 Oct 2016 • Felix Gonda, Verena Kaynig, Ray Thouis, Daniel Haehn, Jeff Lichtman, Toufiq Parag, Hanspeter Pfister
We present an interactive approach to train a deep neural network pixel classifier for the segmentation of neuronal structures.
1 code implementation • 23 Jun 2016 • Hendrik Strobelt, Sebastian Gehrmann, Hanspeter Pfister, Alexander M. Rush
In this work, we present LSTMVIS, a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics.
no code implementations • CVPR 2016 • Jinshan Pan, Deqing Sun, Hanspeter Pfister, Ming-Hsuan Yang
Therefore, enforcing the sparsity of the dark channel helps blind deblurring on various scenarios, including natural, face, text, and low-illumination images.
Ranked #6 on
Deblurring
on RealBlur-R (trained on GoPro)
no code implementations • ICCV 2015 • Kwang In Kim, James Tompkin, Hanspeter Pfister, Christian Theobalt
Existing approaches for diffusion on graphs, e. g., for label propagation, are mainly focused on isotropic diffusion, which is induced by the commonly-used graph Laplacian regularizer.
no code implementations • CVPR 2015 • Kwang In Kim, James Tompkin, Hanspeter Pfister, Christian Theobalt
The iterated graph Laplacian enables high-order regularization, but it has a high computational complexity and so cannot be applied to large problems.
no code implementations • CVPR 2015 • Kwang In Kim, James Tompkin, Hanspeter Pfister, Christian Theobalt
In many learning tasks, the structure of the target space of a function holds rich information about the relationships between evaluations of functions on different data points.
no code implementations • CVPR 2015 • Deqing Sun, Erik B. Sudderth, Hanspeter Pfister
As consumer depth sensors become widely available, estimating scene flow from RGBD sequences has received increasing attention.
no code implementations • CVPR 2014 • Deqing Sun, Ce Liu, Hanspeter Pfister
To handle such situations, we propose a local layering model where motion and occlusion relationships are inferred jointly.
no code implementations • 14 Mar 2014 • William Gray Roncal, Michael Pekala, Verena Kaynig-Fittkau, Dean M. Kleissas, Joshua T. Vogelstein, Hanspeter Pfister, Randal Burns, R. Jacob Vogelstein, Mark A. Chevillet, Gregory D. Hager
An open challenge problem at the forefront of modern neuroscience is to obtain a comprehensive mapping of the neural pathways that underlie human brain function; an enhanced understanding of the wiring diagram of the brain promises to lead to new breakthroughs in diagnosing and treating neurological disorders.
no code implementations • CVPR 2013 • Engin Turetken, Fethallah Benmansour, Bjoern Andres, Hanspeter Pfister, Pascal Fua
We propose a novel approach to automated delineation of linear structures that form complex and potentially loopy networks.
no code implementations • CVPR 2013 • Deqing Sun, Jonas Wulff, Erik B. Sudderth, Hanspeter Pfister, Michael J. Black
Layered models allow scene segmentation and motion estimation to be formulated together and to inform one another.
no code implementations • 28 Mar 2013 • Verena Kaynig, Amelio Vazquez-Reina, Seymour Knowles-Barley, Mike Roberts, Thouis R. Jones, Narayanan Kasthuri, Eric Miller, Jeff Lichtman, Hanspeter Pfister
Segmentation of large-scale electron microscopy data is the main bottleneck in the analysis of these data sets.