Search Results for author: Hanspeter Pfister

Found 86 papers, 34 papers with code

Structure-Preserving Instance Segmentation via Skeleton-Aware Distance Transform

no code implementations8 Oct 2023 Zudi Lin, Donglai Wei, Aarush Gupta, Xingyu Liu, Deqing Sun, Hanspeter Pfister

Objects with complex structures pose significant challenges to existing instance segmentation methods that rely on boundary or affinity maps, which are vulnerable to small errors around contacting pixels that cause noticeable connectivity change.

Image Segmentation Instance Segmentation +2

Sound Source Localization is All about Cross-Modal Alignment

no code implementations ICCV 2023 Arda Senocak, Hyeonggon Ryu, Junsik Kim, Tae-Hyun Oh, Hanspeter Pfister, Joon Son Chung

However, prior arts and existing benchmarks do not account for a more important aspect of the problem, cross-modal semantic understanding, which is essential for genuine sound source localization.

Cross-Modal Retrieval Retrieval

CLIPTrans: Transferring Visual Knowledge with Pre-trained Models for Multimodal Machine Translation

1 code implementation ICCV 2023 Devaansh Gupta, Siddhant Kharbanda, Jiawei Zhou, Wanhua Li, Hanspeter Pfister, Donglai Wei

Simultaneously, there has been an influx of multilingual pre-trained models for NMT and multimodal pre-trained models for vision-language tasks, primarily in English, which have shown exceptional generalisation ability.

Image Captioning Multimodal Machine Translation +2

To reverse engineer an entire nervous system

no code implementations12 Aug 2023 Gal Haspel, Edward S Boyden, Jeffrey Brown, George Church, Netta Cohen, Christopher Fang-Yen, Steven Flavell, Miriam B Goodman, Anne C Hart, Oliver Hobert, Konstantinos Kagias, Shawn Lockery, Yangning Lu, Adam Marblestone, Jordan Matelsky, Hanspeter Pfister, Horacio G Rotstein, Monika Scholz, Eli Shlizerman, Quilee Simeon, Michael A Skuhersky, Vivek Venkatachalam, Guangyu Robert Yang, Eviatar Yemini, Manuel Zimmer, Konrad P Kording

Here we argue that the time is ripe for systems neuroscience to embark on a concerted effort to reverse engineer a smaller system and that Caenorhabditis elegans is the ideal candidate system as the established optophysiology techniques can capture and control each neuron's activity and scale to hundreds of thousands of experiments.

Domain-Scalable Unpaired Image Translation via Latent Space Anchoring

1 code implementation26 Jun 2023 Siyu Huang, Jie An, Donglai Wei, Zudi Lin, Jiebo Luo, Hanspeter Pfister

However, given a UNIT model trained on certain domains, it is difficult for current methods to incorporate new domains because they often need to train the full model on both existing and new domains.

Image-to-Image Translation Translation

Masked Image Training for Generalizable Deep Image Denoising

1 code implementation CVPR 2023 Haoyu Chen, Jinjin Gu, Yihao Liu, Salma Abdel Magid, Chao Dong, Qiong Wang, Hanspeter Pfister, Lei Zhu

To address this issue, we present a novel approach to enhance the generalization performance of denoising networks, known as masked training.

Image Denoising

An Out-of-Domain Synapse Detection Challenge for Microwasp Brain Connectomes

no code implementations1 Feb 2023 Jingpeng Wu, Yicong Li, Nishika Gupta, Kazunori Shinomiya, Pat Gunn, Alexey Polilov, Hanspeter Pfister, Dmitri Chklovskii, Donglai Wei

The size of image stacks in connectomics studies now reaches the terabyte and often petabyte scales with a great diversity of appearance across brain regions and samples.

Domain Adaptation

QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity

1 code implementation CVPR 2023 Siyu Huang, Jie An, Donglai Wei, Jiebo Luo, Hanspeter Pfister

The mechanism of existing style transfer algorithms is by minimizing a hybrid loss function to push the generated image toward high similarities in both content and style.

Quantization Style Transfer +1

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task

1 code implementation24 Oct 2022 Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, Martin Wattenberg

Language models show a surprising range of capabilities, but the source of their apparent competence is unclear.

RibSeg v2: A Large-scale Benchmark for Rib Labeling and Anatomical Centerline Extraction

1 code implementation18 Oct 2022 Liang Jin, Shixuan Gu, Donglai Wei, Jason Ken Adhinarta, Kaiming Kuang, Yongjie Jessica Zhang, Hanspeter Pfister, Bingbing Ni, Jiancheng Yang, Ming Li

Based on the RibSeg v2, we develop a pipeline including deep learning-based methods for rib labeling, and a skeletonization-based method for centerline extraction.


Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models

no code implementations16 Aug 2022 Hendrik Strobelt, Albert Webson, Victor Sanh, Benjamin Hoover, Johanna Beyer, Hanspeter Pfister, Alexander M. Rush

State-of-the-art neural language models can now be used to solve ad-hoc language tasks through zero-shot prompting without the need for supervised training.

Prompt Engineering

What makes domain generalization hard?

no code implementations15 Jun 2022 Spandan Madan, Li You, Mengmi Zhang, Hanspeter Pfister, Gabriel Kreiman

Here we present SemanticDG (Semantic Domain Generalization): a benchmark with 15 photo-realistic domains with the same geometry, scene layout and camera parameters as the popular 3D ScanNet dataset, but with controlled domain shifts in lighting, materials, and viewpoints.

Domain Generalization

Diagnosing Ensemble Few-Shot Classifiers

no code implementations9 Jun 2022 Weikai Yang, Xi Ye, Xingxing Zhang, Lanxi Xiao, Jiazhi Xia, Zhongyuan Wang, Jun Zhu, Hanspeter Pfister, Shixia Liu

The base learners and labeled samples (shots) in an ensemble few-shot classifier greatly affect the model performance.

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction

3 code implementations17 Apr 2022 Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Zhang, Hanspeter Pfister, Radu Timofte, Luc van Gool

Existing leading methods for spectral reconstruction (SR) focus on designing deeper or wider convolutional neural networks (CNNs) to learn the end-to-end mapping from the RGB image to its hyperspectral image (HSI).

Spectral Reconstruction Spectral Super-Resolution

Instance Segmentation of Unlabeled Modalities via Cyclic Segmentation GAN

no code implementations6 Apr 2022 Leander Lauenburg, Zudi Lin, Ruihan Zhang, Márcia dos Santos, Siyu Huang, Ignacio Arganda-Carreras, Edward S. Boyden, Hanspeter Pfister, Donglai Wei

Instance segmentation for unlabeled imaging modalities is a challenging but essential task as collecting expert annotation can be expensive and time-consuming.

Image Segmentation Instance Segmentation +3

Texture-Based Error Analysis for Image Super-Resolution

no code implementations CVPR 2022 Salma Abdel Magid, Zudi Lin, Donglai Wei, Yulun Zhang, Jinjin Gu, Hanspeter Pfister

Our key contribution is to leverage a texture classifier, which enables us to assign patches with semantic labels, to identify the source of SR errors both globally and locally.

Image Super-Resolution SSIM

PyTorch Connectomics: A Scalable and Flexible Segmentation Framework for EM Connectomics

1 code implementation10 Dec 2021 Zudi Lin, Donglai Wei, Jeff Lichtman, Hanspeter Pfister

We present PyTorch Connectomics (PyTC), an open-source deep-learning framework for the semantic and instance segmentation of volumetric microscopy images, built upon PyTorch.

Instance Segmentation Segmentation +1

Three approaches to facilitate DNN generalization to objects in out-of-distribution orientations and illuminations

1 code implementation30 Oct 2021 Akira Sakai, Taro Sunagawa, Spandan Madan, Kanata Suzuki, Takashi Katoh, Hiromichi Kobashi, Hanspeter Pfister, Pawan Sinha, Xavier Boix, Tomotake Sasaki

While humans have a remarkable capability of recognizing objects in out-of-distribution (OoD) orientations and illuminations, Deep Neural Networks (DNNs) severely suffer in this case, even when large amounts of training examples are available.

MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification

3 code implementations27 Oct 2021 Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, Bingbing Ni

We introduce MedMNIST v2, a large-scale MNIST-like dataset collection of standardized biomedical images, including 12 datasets for 2D and 6 datasets for 3D.

AutoML Image Classification

GenNI: Human-AI Collaboration for Data-Backed Text Generation

no code implementations19 Oct 2021 Hendrik Strobelt, Jambay Kinley, Robert Krueger, Johanna Beyer, Hanspeter Pfister, Alexander M. Rush

These controls allow users to globally constrain model generations, without sacrificing the representation power of the deep learning models.

Descriptive Text Generation

Emergent Neural Network Mechanisms for Generalization to Objects in Novel Orientations

no code implementations28 Sep 2021 Avi Cooper, Xavier Boix, Daniel Harari, Spandan Madan, Hanspeter Pfister, Tomotake Sasaki, Pawan Sinha

The capability of Deep Neural Networks (DNNs) to recognize objects in orientations outside the distribution of the training data is not well understood.

RibSeg Dataset and Strong Point Cloud Baselines for Rib Segmentation from CT Scans

1 code implementation17 Sep 2021 Jiancheng Yang, Shixuan Gu, Donglai Wei, Hanspeter Pfister, Bingbing Ni

Manual rib inspections in computed tomography (CT) scans are clinically critical but labor-intensive, as 24 ribs are typically elongated and oblique in 3D volumes.

Computed Tomography (CT) Segmentation

Asymmetric 3D Context Fusion for Universal Lesion Detection

1 code implementation17 Sep 2021 Jiancheng Yang, Yi He, Kaiming Kuang, Zudi Lin, Hanspeter Pfister, Bingbing Ni

The proposed A3D consistently outperforms symmetric context fusion operators by considerable margins, and establishes a new \emph{state of the art} on DeepLesion.

Computed Tomography (CT) Lesion Detection +1

Adversarial examples within the training distribution: A widespread challenge

1 code implementation30 Jun 2021 Spandan Madan, Tomotake Sasaki, Hanspeter Pfister, Tzu-Mao Li, Xavier Boix

This result provides evidence supporting theories attributing adversarial examples to the proximity of data to ground-truth class boundaries, and calls into question other theories which do not account for this more stringent definition of adversarial attacks.

Object Recognition Open-Ended Question Answering

VICE: Visual Identification and Correction of Neural Circuit Errors

no code implementations14 May 2021 Felix Gonda, Xueying Wang, Johanna Beyer, Markus Hadwiger, Jeff W. Lichtman, Hanspeter Pfister

A connectivity graph of neurons at the resolution of single synapses provides scientists with a tool for understanding the nervous system in health and disease.

Clustering Image Segmentation +2

Discrete Cosine Transform Network for Guided Depth Map Super-Resolution

2 code implementations CVPR 2022 Zixiang Zhao, Jiangshe Zhang, Shuang Xu, Zudi Lin, Hanspeter Pfister

Guided depth super-resolution (GDSR) is an essential topic in multi-modal image processing, which reconstructs high-resolution (HR) depth maps from low-resolution ones collected with suboptimal conditions with the help of HR RGB images of the same scene.

Depth Map Super-Resolution

Consistent Recurrent Neural Networks for 3D Neuron Segmentation

no code implementations1 Feb 2021 Felix Gonda, Donglai Wei, Hanspeter Pfister

We present a recurrent network for the 3D reconstruction of neurons that sequentially generates binary masks for every object in an image with spatio-temporal consistency.

3D Reconstruction

Dynamic High-Pass Filtering and Multi-Spectral Attention for Image Super-Resolution

no code implementations ICCV 2021 Salma Abdel Magid, Yulun Zhang, Donglai Wei, Won-Dong Jang, Zudi Lin, Yun Fu, Hanspeter Pfister

Specifically, we propose a dynamic high-pass filtering (HPF) module that locally applies adaptive filter weights for each spatial location and channel group to preserve high-frequency signals.

Image Super-Resolution

Context Reasoning Attention Network for Image Super-Resolution

no code implementations ICCV 2021 Yulun Zhang, Donglai Wei, Can Qin, Huan Wang, Hanspeter Pfister, Yun Fu

However, the basic convolutional layer in CNNs is designed to extract local patterns, lacking the ability to model global context.

Image Super-Resolution

On the Capability of CNNs to Generalize to Unseen Category-Viewpoint Combinations

no code implementations1 Jan 2021 Spandan Madan, Timothy Henry, Jamell Arthur Dozier, Helen Ho, Nishchal Bhandari, Tomotake Sasaki, Fredo Durand, Hanspeter Pfister, Xavier Boix

We find that learning category and viewpoint in separate networks compared to a shared one leads to an increase in selectivity and invariance, as separate networks are not forced to preserve information about both category and viewpoint.

Object Recognition Viewpoint Estimation

Monocular Reconstruction of Neural Face Reflectance Fields

no code implementations CVPR 2021 Mallikarjun B R., Ayush Tewari, Tae-Hyun Oh, Tim Weyrich, Bernd Bickel, Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, Christian Theobalt

The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.

Monocular Reconstruction

When and how CNNs generalize to out-of-distribution category-viewpoint combinations

2 code implementations15 Jul 2020 Spandan Madan, Timothy Henry, Jamell Dozier, Helen Ho, Nishchal Bhandari, Tomotake Sasaki, Frédo Durand, Hanspeter Pfister, Xavier Boix

In this paper, we investigate when and how such OOD generalization may be possible by evaluating CNNs trained to classify both object category and 3D viewpoint on OOD combinations, and identifying the neural mechanisms that facilitate such OOD generalization.

Object Recognition Viewpoint Estimation

A New Age of Computing and the Brain

no code implementations27 Apr 2020 Polina Golland, Jack Gallant, Greg Hager, Hanspeter Pfister, Christos Papadimitriou, Stefan Schaal, Joshua T. Vogelstein

In December 2014, a two-day workshop supported by the Computing Community Consortium (CCC) and the National Science Foundation's Computer and Information Science and Engineering Directorate (NSF CISE) was convened in Washington, DC, with the goal of bringing together computer scientists and brain researchers to explore these new opportunities and connections, and develop a new, modern dialogue between the two research communities.

A Topological Nomenclature for 3D Shape Analysis in Connectomics

1 code implementation27 Sep 2019 Abhimanyu Talwar, Zudi Lin, Donglai Wei, Yuesong Wu, Bowen Zheng, Jinglin Zhao, Won-Dong Jang, Xueying Wang, Jeff W. Lichtman, Hanspeter Pfister

Next, we develop nomenclature rules for pyramidal neurons and mitochondria from the reduced graph and finally learn the feature embedding for shape manipulation.

3D Shape Classification 3D Shape Retrieval +1

White-Box Adversarial Defense via Self-Supervised Data Estimation

1 code implementation13 Sep 2019 Zudi Lin, Hanspeter Pfister, Ziming Zhang

In this paper, we study the problem of how to defend classifiers against adversarial attacks that fool the classifiers using subtly modified input data.

Adversarial Defense Self-Supervised Learning +1

FDive: Learning Relevance Models using Pattern-based Similarity Measures

no code implementations29 Jul 2019 Frederik L. Dennig, Tom Polk, Zudi Lin, Tobias Schreck, Hanspeter Pfister, Michael Behrisch

The detection of interesting patterns in large high-dimensional datasets is difficult because of their dimensionality and pattern complexity.

Active Learning feature selection

Fast Mitochondria Detection for Connectomics

no code implementations MIDL 2019 Vincent Casser, Kai Kang, Hanspeter Pfister, Daniel Haehn

High-resolution connectomics data allows for the identification of dysfunctional mitochondria which are linked to a variety of diseases such as autism or bipolar.

Debugging Sequence-to-Sequence Models with Seq2Seq-Vis

no code implementations WS 2018 Hendrik Strobelt, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, Alex Rush, er

Neural attention-based sequence-to-sequence models (seq2seq) (Sutskever et al., 2014; Bahdanau et al., 2014) have proven to be accurate and robust for many sequence prediction tasks.


Parallel Separable 3D Convolution for Video and Volumetric Data Understanding

no code implementations11 Sep 2018 Felix Gonda, Donglai Wei, Toufiq Parag, Hanspeter Pfister

For video and volumetric data understanding, 3D convolution layers are widely used in deep learning, however, at the cost of increasing computation and training time.

Action Recognition Brain Segmentation +3

Synthetically Trained Icon Proposals for Parsing and Summarizing Infographics

1 code implementation27 Jul 2018 Spandan Madan, Zoya Bylinskii, Matthew Tancik, Adrià Recasens, Kimberli Zhong, Sami Alsheikh, Hanspeter Pfister, Aude Oliva, Fredo Durand

While automatic text extraction works well on infographics, computer vision approaches trained on natural images fail to identify the stand-alone visual elements in infographics, or `icons'.

Synthetic Data Generation Test

Detecting Synapse Location and Connectivity by Signed Proximity Estimation and Pruning with Deep Nets

1 code implementation8 Jul 2018 Toufiq Parag, Daniel Berger, Lee Kamentsky, Benedikt Staffler, Donglai Wei, Moritz Helmstaedter, Jeff W. Lichtman, Hanspeter Pfister

The few methods that computes direction along with contact location have only been demonstrated to work on either dyadic (most common in vertebrate brain) or polyadic (found in fruit fly brain) synapses, but not on both types.

Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models

1 code implementation25 Apr 2018 Hendrik Strobelt, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, Alexander M. Rush

In this work, we present a visual analysis tool that allows interaction with a trained sequence-to-sequence model through each stage of the translation process.


Understanding Infographics through Textual and Visual Tag Prediction

1 code implementation26 Sep 2017 Zoya Bylinskii, Sami Alsheikh, Spandan Madan, Adria Recasens, Kimberli Zhong, Hanspeter Pfister, Fredo Durand, Aude Oliva

And second, we use these predicted text tags as a supervisory signal to localize the most diagnostic visual elements from within the infographic i. e. visual hashtags.


Personalizing Gesture Recognition Using Hierarchical Bayesian Neural Networks

no code implementations CVPR 2017 Ajjen Joshi, Soumya Ghosh, Margrit Betke, Stan Sclaroff, Hanspeter Pfister

Leveraging recent work on learning Bayesian neural networks, we build fast, scalable algorithms for inferring the posterior distribution over all network weights in the hierarchy.

Active Learning Gesture Recognition +1

Criteria Sliders: Learning Continuous Database Criteria via Interactive Ranking

no code implementations12 Jun 2017 James Tompkin, Kwang In Kim, Hanspeter Pfister, Christian Theobalt

Large databases are often organized by hand-labeled metadata, or criteria, which are expensive to collect.

Guided Proofreading of Automatic Segmentations for Connectomics

no code implementations CVPR 2018 Daniel Haehn, Verena Kaynig, James Tompkin, Jeff W. Lichtman, Hanspeter Pfister

Automatic cell image segmentation methods in connectomics produce merge and split errors, which require correction through proofreading.

Image Segmentation Segmentation +1

BubbleView: an interface for crowdsourcing image importance maps and tracking visual attention

no code implementations16 Feb 2017 Nam Wook Kim, Zoya Bylinskii, Michelle A. Borkin, Krzysztof Z. Gajos, Aude Oliva, Fredo Durand, Hanspeter Pfister

In this paper, we present BubbleView, an alternative methodology for eye tracking using discrete mouse clicks to measure which information people consciously choose to examine.

RhoanaNet Pipeline: Dense Automatic Neural Annotation

no code implementations21 Nov 2016 Seymour Knowles-Barley, Verena Kaynig, Thouis Ray Jones, Alyssa Wilson, Joshua Morgan, Dongil Lee, Daniel Berger, Narayanan Kasthuri, Jeff W. Lichtman, Hanspeter Pfister

The best segmentation results obtained gave $V^\text{Info}_\text{F-score}$ scores of 0. 9054 and 09182 for the cortex datasets, 0. 9438 for LGN, and 0. 9150 for Cerebellum.

Segmentation Test

LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks

1 code implementation23 Jun 2016 Hendrik Strobelt, Sebastian Gehrmann, Hanspeter Pfister, Alexander M. Rush

In this work, we present LSTMVIS, a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics.

Blind Image Deblurring Using Dark Channel Prior

no code implementations CVPR 2016 Jinshan Pan, Deqing Sun, Hanspeter Pfister, Ming-Hsuan Yang

Therefore, enforcing the sparsity of the dark channel helps blind deblurring on various scenarios, including natural, face, text, and low-illumination images.

Blind Image Deblurring Image Deblurring

Context-guided diffusion for label propagation on graphs

no code implementations ICCV 2015 Kwang In Kim, James Tompkin, Hanspeter Pfister, Christian Theobalt

Existing approaches for diffusion on graphs, e. g., for label propagation, are mainly focused on isotropic diffusion, which is induced by the commonly-used graph Laplacian regularizer.

Semi-supervised Learning with Explicit Relationship Regularization

no code implementations CVPR 2015 Kwang In Kim, James Tompkin, Hanspeter Pfister, Christian Theobalt

In many learning tasks, the structure of the target space of a function holds rich information about the relationships between evaluations of functions on different data points.

Constrained Clustering Dimensionality Reduction +1

Local High-order Regularization on Data Manifolds

no code implementations CVPR 2015 Kwang In Kim, James Tompkin, Hanspeter Pfister, Christian Theobalt

The iterated graph Laplacian enables high-order regularization, but it has a high computational complexity and so cannot be applied to large problems.

Dimensionality Reduction Vocal Bursts Intensity Prediction

Layered RGBD Scene Flow Estimation

no code implementations CVPR 2015 Deqing Sun, Erik B. Sudderth, Hanspeter Pfister

As consumer depth sensors become widely available, estimating scene flow from RGBD sequences has received increasing attention.

Optical Flow Estimation Scene Flow Estimation +1

Local Layering for Joint Motion Estimation and Occlusion Detection

no code implementations CVPR 2014 Deqing Sun, Ce Liu, Hanspeter Pfister

To handle such situations, we propose a local layering model where motion and occlusion relationships are inferred jointly.

Motion Estimation Optical Flow Estimation

VESICLE: Volumetric Evaluation of Synaptic Interfaces using Computer vision at Large Scale

no code implementations14 Mar 2014 William Gray Roncal, Michael Pekala, Verena Kaynig-Fittkau, Dean M. Kleissas, Joshua T. Vogelstein, Hanspeter Pfister, Randal Burns, R. Jacob Vogelstein, Mark A. Chevillet, Gregory D. Hager

An open challenge problem at the forefront of modern neuroscience is to obtain a comprehensive mapping of the neural pathways that underlie human brain function; an enhanced understanding of the wiring diagram of the brain promises to lead to new breakthroughs in diagnosing and treating neurological disorders.

object-detection Object Detection

Reconstructing Loopy Curvilinear Structures Using Integer Programming

no code implementations CVPR 2013 Engin Turetken, Fethallah Benmansour, Bjoern Andres, Hanspeter Pfister, Pascal Fua

We propose a novel approach to automated delineation of linear structures that form complex and potentially loopy networks.

Cannot find the paper you are looking for? You can Submit a new open access paper.