Search Results for author: Simon Lucey

Found 85 papers, 30 papers with code

On progressive sharpening, flat minima and generalisation

no code implementations24 May 2023 Lachlan Ewen MacDonald, Jack Valmadre, Simon Lucey

We present a new approach to understanding the relationship between loss curvature and generalisation in deep learning.

Curvature-Aware Training for Coordinate Networks

no code implementations15 May 2023 Hemanth Saratchandran, Shin-Fang Chng, Sameera Ramasinghe, Lachlan MacDonald, Simon Lucey

Coordinate networks are widely used in computer vision due to their ability to represent signals as compressed, continuous entities.

Fast Neural Scene Flow

1 code implementation18 Apr 2023 Xueqian Li, Jianqiao Zheng, Francesco Ferroni, Jhony Kaesemodel Pontes, Simon Lucey

Neural Scene Flow Prior (NSFP) is of significant interest to the vision community due to its inherent robustness to out-of-distribution (OOD) effects and its ability to deal with dense lidar points.

Autonomous Driving Scene Flow Estimation

On the effectiveness of neural priors in modeling dynamical systems

no code implementations10 Mar 2023 Sameera Ramasinghe, Hemanth Saratchandran, Violetta Shevchenko, Simon Lucey

Modelling dynamical systems is an integral component for understanding the natural world.

A global analysis of global optimisation

no code implementations10 Oct 2022 Lachlan Ewen MacDonald, Hemanth Saratchandran, Jack Valmadre, Simon Lucey

We introduce a general theoretical framework, designed for the study of gradient optimisation of deep neural networks, that encompasses ubiquitous architectural choices including batch normalisation, weight normalisation and skip connections.

MBW: Multi-view Bootstrapping in the Wild

2 code implementations4 Oct 2022 Mosam Dabhi, Chaoyang Wang, Tim Clifford, Laszlo Attila Jeni, Ian R. Fasel, Simon Lucey

Our Multi-view Bootstrapping in the Wild (MBW) approach demonstrates impressive results on standard human datasets, as well as tigers, cheetahs, fish, colobus monkeys, chimpanzees, and flamingos from videos captured casually in a zoo.

3D Reconstruction Semi-supervised 2D and 3D landmark labeling +1

On Quantizing Implicit Neural Representations

no code implementations1 Sep 2022 Cameron Gordon, Shin-Fang Chng, Lachlan MacDonald, Simon Lucey

The role of quantization within implicit/coordinate neural networks is still not fully understood.

Image Reconstruction Quantization

How You Start Matters for Generalization

no code implementations17 Jun 2022 Sameera Ramasinghe, Lachlan MacDonald, Moshiur Farazi, Hemanth Saratchandran, Simon Lucey

Characterizing the remarkable generalization properties of over-parameterized neural networks remains an open problem.

Trading Positional Complexity vs. Deepness in Coordinate Networks

1 code implementation18 May 2022 Jianqiao Zheng, Sameera Ramasinghe, Xueqian Li, Simon Lucey

It is well noted that coordinate-based MLPs benefit -- in terms of preserving high-frequency information -- through the encoding of coordinate positions as an array of Fourier features.

GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimation

1 code implementation12 Apr 2022 Shin-Fang Chng, Sameera Ramasinghe, Jamie Sherrah, Simon Lucey

Despite Neural Radiance Fields (NeRF) showing compelling results in photorealistic novel views synthesis of real-world scenes, most existing approaches require accurate prior camera poses.

Pose Estimation

On Regularizing Coordinate-MLPs

no code implementations1 Feb 2022 Sameera Ramasinghe, Lachlan MacDonald, Simon Lucey

We show that typical implicit regularization assumptions for deep neural networks (for regression) do not hold for coordinate-MLPs, a family of MLPs that are now ubiquitous in computer vision for representing high-frequency signals.


Neural Prior for Trajectory Estimation

no code implementations CVPR 2022 Chaoyang Wang, Xueqian Li, Jhony Kaesemodel Pontes, Simon Lucey

Here, we propose a neural trajectory prior to capture continuous spatio-temporal information without the need for offline data.

Image Denoising Super-Resolution

Learning Positional Embeddings for Coordinate-MLPs

no code implementations21 Dec 2021 Sameera Ramasinghe, Simon Lucey

We propose a novel method to enhance the performance of coordinate-MLPs by learning instance-specific positional embeddings.


Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPs

no code implementations30 Nov 2021 Sameera Ramasinghe, Simon Lucey

Coordinate-MLPs are emerging as an effective tool for modeling multidimensional continuous signals, overcoming many drawbacks associated with discrete grid-based approximations.

Enabling equivariance for arbitrary Lie groups

1 code implementation CVPR 2022 Lachlan Ewen MacDonald, Sameera Ramasinghe, Simon Lucey

Our framework enables the implementation of group convolutions over any finite-dimensional Lie group.

Neural Scene Flow Prior

1 code implementation NeurIPS 2021 Xueqian Li, Jhony Kaesemodel Pontes, Simon Lucey

A central innovation here is the inclusion of a neural scene flow prior, which uses the architecture of neural networks as a new type of implicit regularizer.

Autonomous Driving Scene Flow Estimation

High Fidelity 3D Reconstructions with Limited Physical Views

no code implementations22 Oct 2021 Mosam Dabhi, Chaoyang Wang, Kunal Saluja, Laszlo Jeni, Ian Fasel, Simon Lucey

Multi-view triangulation is the gold standard for 3D reconstruction from 2D correspondences given known calibration and sufficient views.

3D Reconstruction Vocal Bursts Intensity Prediction

Rethinking Positional Encoding

1 code implementation6 Jul 2021 Jianqiao Zheng, Sameera Ramasinghe, Simon Lucey

It is well noted that coordinate based MLPs benefit -- in terms of preserving high-frequency information -- through the encoding of coordinate positions as an array of Fourier features.

On the Bias Against Inductive Biases

no code implementations28 May 2021 George Cazenavette, Simon Lucey

Borrowing from the transformer models that revolutionized the field of natural language processing, self-supervised feature learning for visual tasks has also seen state-of-the-art success using these extremely deep, isotropic networks.

Neural Trajectory Fields for Dynamic Novel View Synthesis

no code implementations12 May 2021 Chaoyang Wang, Ben Eckart, Simon Lucey, Orazio Gallo

Recent approaches to render photorealistic views from a limited set of photographs have pushed the boundaries of our interactions with pictures of static scenes.

Novel View Synthesis

Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization

1 code implementation CVPR 2022 Damien Teney, Ehsan Abbasnejad, Simon Lucey, Anton Van Den Hengel

The method - the first to evade the simplicity bias - highlights the need for a better understanding and control of inductive biases in deep learning.

Model Selection

BARF: Bundle-Adjusting Neural Radiance Fields

4 code implementations ICCV 2021 Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, Simon Lucey

In this paper, we propose Bundle-Adjusting Neural Radiance Fields (BARF) for training NeRF from imperfect (or even unknown) camera poses -- the joint problem of learning neural 3D representations and registering camera frames.

Visual Localization

PAUL: Procrustean Autoencoder for Unsupervised Lifting

no code implementations CVPR 2021 Chaoyang Wang, Simon Lucey

Recent success in casting Non-rigid Structure from Motion (NRSfM) as an unsupervised deep learning problem has raised fundamental questions about what novelty in NRSfM prior could the deep learning offer.

Reframing Neural Networks: Deep Structure in Overcomplete Representations

no code implementations10 Mar 2021 Calvin Murdock, George Cazenavette, Simon Lucey

In comparison to classical shallow representation learning techniques, deep neural networks have achieved superior performance in nearly every application benchmark.

Adversarial Robustness Model Selection +1

Architectural Adversarial Robustness: The Case for Deep Pursuit

no code implementations CVPR 2021 George Cazenavette, Calvin Murdock, Simon Lucey

Despite their unmatched performance, deep neural networks remain susceptible to targeted attacks by nearly imperceptible levels of adversarial noise.

Adversarial Robustness

Scene Flow from Point Clouds with or without Learning

no code implementations31 Oct 2020 Jhony Kaesemodel Pontes, James Hays, Simon Lucey

Our proposed objective function can be used with or without learning---as a self-supervisory signal to learn scene flow representations, or as a non-learning-based method in which the scene flow is optimized during runtime.

Motion Segmentation Self-Supervised Learning

SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images

1 code implementation NeurIPS 2020 Chen-Hsuan Lin, Chaoyang Wang, Simon Lucey

Dense 3D object reconstruction from a single image has recently witnessed remarkable advances, but supervising neural networks with ground-truth 3D shapes is impractical due to the laborious process of creating paired image-shape datasets.

3D Object Reconstruction From A Single Image 3D Reconstruction

Joint Pose and Shape Estimation of Vehicles from LiDAR Data

no code implementations8 Sep 2020 Hunter Goforth, Xiaoyan Hu, Michael Happold, Simon Lucey

We address the problem of estimating the pose and shape of vehicles from LiDAR scans, a common problem faced by the autonomous vehicle community.

PointNetLK Revisited

1 code implementation CVPR 2021 Xueqian Li, Jhony Kaesemodel Pontes, Simon Lucey

We address the generalization ability of recent learning-based point cloud registration methods.

Point Cloud Registration

Dataless Model Selection with the Deep Frame Potential

no code implementations CVPR 2020 Calvin Murdock, Simon Lucey

Choosing a deep neural network architecture is a fundamental problem in applications that require balancing performance and parameter efficiency.

Model Selection

When to Use Convolutional Neural Networks for Inverse Problems

no code implementations CVPR 2020 Nathaniel Chodosh, Simon Lucey

In this work we argue that for some types of inverse problems the CNN approximation breaks down leading to poor performance.

Image Denoising Super-Resolution

Deep NRSfM++: Towards Unsupervised 2D-3D Lifting in the Wild

1 code implementation27 Jan 2020 Chaoyang Wang, Chen-Hsuan Lin, Simon Lucey

The recovery of 3D shape and pose from 2D landmarks stemming from a large ensemble of images can be viewed as a non-rigid structure from motion (NRSfM) problem.

3D Reconstruction

One Framework to Register Them All: PointNet Encoding for Point Cloud Alignment

1 code implementation12 Dec 2019 Vinit Sarode, Xueqian Li, Hunter Goforth, Yasuhiro Aoki, Animesh Dhagat, Rangaprasad Arun Srivatsan, Simon Lucey, Howie Choset

We perform extensive simulation and real-world experiments to validate the efficacy of our approach and compare the performance with state-of-art approaches.

3D Reconstruction object-detection +2

Argoverse: 3D Tracking and Forecasting with Rich Maps

2 code implementations CVPR 2019 Ming-Fang Chang, John Lambert, Patsorn Sangkloy, Jagjeet Singh, Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, James Hays

In our baseline experiments, we illustrate how detailed map information such as lane direction, driveable area, and ground height improves the accuracy of 3D object tracking and motion forecasting.

3D Object Tracking Autonomous Vehicles +3

PCRNet: Point Cloud Registration Network using PointNet Encoding

5 code implementations21 Aug 2019 Vinit Sarode, Xueqian Li, Hunter Goforth, Yasuhiro Aoki, Rangaprasad Arun Srivatsan, Simon Lucey, Howie Choset

PointNet has recently emerged as a popular representation for unstructured point cloud data, allowing application of deep learning to tasks such as object detection, segmentation and shape completion.

3D Reconstruction object-detection +3

Deep Non-Rigid Structure from Motion

no code implementations ICCV 2019 Chen Kong, Simon Lucey

Current non-rigid structure from motion (NRSfM) algorithms are mainly limited with respect to: (i) the number of images, and (ii) the type of shape variability they can handle.

Dictionary Learning

Deep Non-Rigid Structure from Motion with Missing Data

no code implementations30 Jul 2019 Chen Kong, Simon Lucey

Non-Rigid Structure from Motion (NRSfM) refers to the problem of reconstructing cameras and the 3D point cloud of a non-rigid object from an ensemble of images with 2D correspondences.

Matrix Completion

Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency

1 code implementation7 May 2019 Tejas Khot, Shubham Agrawal, Shubham Tulsiani, Christoph Mertz, Simon Lucey, Martial Hebert

We demonstrate our ability to learn MVS without 3D supervision using a real dataset, and show that each component of our proposed robust loss results in a significant improvement.

Depth Estimation Depth Prediction

Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes

no code implementations25 Apr 2019 Chaoyang Wang, Simon Lucey, Federico Perazzi, Oliver Wang

We present a fully data-driven method to compute depth from diverse monocular video sequences that contain large amounts of non-rigid objects, e. g., people.

Depth Estimation Depth Prediction

Deep Interpretable Non-Rigid Structure from Motion

1 code implementation28 Feb 2019 Chen Kong, Simon Lucey

All current non-rigid structure from motion (NRSfM) algorithms are limited with respect to: (i) the number of images, and (ii) the type of shape variability they can handle.

Dictionary Learning

Deep Convolutional Compressed Sensing for LiDAR Depth Completion

no code implementations23 Mar 2018 Nathaniel Chodosh, Chaoyang Wang, Simon Lucey

In this paper we consider the problem of estimating a dense depth map from a set of sparse LiDAR points.

Depth Completion

Aligning Across Large Gaps in Time

no code implementations22 Mar 2018 Hunter Goforth, Simon Lucey

We present a method of temporally-invariant image registration for outdoor scenes, with invariance across time of day, across seasonal variations, and across decade-long periods, for low- and high-texture scenes.

3D Reconstruction Image Registration

Deep Component Analysis via Alternating Direction Neural Networks

1 code implementation ECCV 2018 Calvin Murdock, Ming-Fang Chang, Simon Lucey

Despite a lack of theoretical understanding, deep neural networks have achieved unparalleled performance in a wide range of applications.

Depth Estimation Depth Prediction +1

ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing

2 code implementations CVPR 2018 Chen-Hsuan Lin, Ersin Yumer, Oliver Wang, Eli Shechtman, Simon Lucey

We address the problem of finding realistic geometric corrections to a foreground object such that it appears natural when composited into a background image.

Take it in your stride: Do we need striding in CNNs?

no code implementations7 Dec 2017 Chen Kong, Simon Lucey

Since their inception, CNNs have utilized some type of striding operator to reduce the overlap of receptive fields and spatial dimensions.

CNNs are Globally Optimal Given Multi-Layer Support

no code implementations7 Dec 2017 Chen Huang, Chen Kong, Simon Lucey

Stochastic Gradient Descent (SGD) is the central workhorse for training modern CNNs.

Learning Depth from Monocular Videos using Direct Methods

1 code implementation CVPR 2018 Chaoyang Wang, Jose Miguel Buenaposada, Rui Zhu, Simon Lucey

The ability to predict depth from a single image - using recent advances in CNNs - is of increasing interest to the vision community.

Depth And Camera Motion Visual Odometry

Semantic Photometric Bundle Adjustment on Natural Sequences

no code implementations30 Nov 2017 Rui Zhu, Chaoyang Wang, Chen-Hsuan Lin, Ziyan Wang, Simon Lucey

More recently, excellent results have been attained through the application of photometric bundle adjustment (PBA) methods -- which directly minimize the photometric error across frames.

Object Reconstruction

Image2Mesh: A Learning Framework for Single Image 3D Reconstruction

1 code implementation29 Nov 2017 Jhony K. Pontes, Chen Kong, Sridha Sridharan, Simon Lucey, Anders Eriksson, Clinton Fookes

One challenge that remains open in 3D deep learning is how to efficiently represent 3D data to feed deep networks.

3D Reconstruction

Object-Centric Photometric Bundle Adjustment with Deep Shape Prior

no code implementations4 Nov 2017 Rui Zhu, Chaoyang Wang, Chen-Hsuan Lin, Ziyan Wang, Simon Lucey

Reconstructing 3D shapes from a sequence of images has long been a problem of interest in computer vision.

Learning Policies for Adaptive Tracking with Deep Feature Cascades

no code implementations ICCV 2017 Chen Huang, Simon Lucey, Deva Ramanan

Our fundamental insight is to take an adaptive approach, where easy frames are processed with cheap features (such as pixel values), while challenging frames are processed with invariant but expensive deep features.

Decision Making Visual Object Tracking

Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image

no code implementations15 Jul 2017 Rui Zhu, Hamed Kiani Galoogahi, Chaoyang Wang, Simon Lucey

An emerging problem in computer vision is the reconstruction of 3D shape and pose of an object from a single image.

Using Locally Corresponding CAD Models for Dense 3D Reconstructions From a Single Image

no code implementations CVPR 2017 Chen Kong, Chen-Hsuan Lin, Simon Lucey

A common strategy in dictionary learning to encourage generalization is to allow for linear combinations of dictionary elements.

Dictionary Learning Graph Embedding

Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction

3 code implementations21 Jun 2017 Chen-Hsuan Lin, Chen Kong, Simon Lucey

Conventional methods of 3D object generative modeling learn volumetric predictions using deep networks with 3D convolutional operations, which are direct analogies to classical 2D ones.

3D Object Reconstruction Point Cloud Generation

Joint Max Margin and Semantic Features for Continuous Event Detection in Complex Scenes

no code implementations13 Jun 2017 Iman Abbasnejad, Sridha Sridharan, Simon Denman, Clinton Fookes, Simon Lucey

In this paper the problem of complex event detection in the continuous domain (i. e. events with unknown starting and ending locations) is addressed.

Action Recognition Event Detection +1

Deep-LK for Efficient Adaptive Object Tracking

no code implementations19 May 2017 Chaoyang Wang, Hamed Kiani Galoogahi, Chen-Hsuan Lin, Simon Lucey

In this paper we present a new approach for efficient regression based object tracking which we refer to as Deep- LK.

Object Tracking regression

Proxy Templates for Inverse Compositional Photometric Bundle Adjustment

no code implementations23 Apr 2017 Christopher Ham, Simon Lucey, Surya Singh

In this paper, we show that an ordinary inverse compositional formulation does not work for warps of this type of parameterization due to ill-conditioning of its partial derivatives.

Need for Speed: A Benchmark for Higher Frame Rate Object Tracking

1 code implementation ICCV 2017 Hamed Kiani Galoogahi, Ashton Fagg, Chen Huang, Deva Ramanan, Simon Lucey

In this paper, we propose the first higher frame rate video dataset (called Need for Speed - NfS) and benchmark for visual object tracking.

Visual Object Tracking

Learning Background-Aware Correlation Filters for Visual Tracking

1 code implementation ICCV 2017 Hamed Kiani Galoogahi, Ashton Fagg, Simon Lucey

Correlation Filters (CFs) have recently demonstrated excellent performance in terms of rapidly tracking objects under challenging photometric and geometric variations.

Visual Tracking

Fast, Dense Feature SDM on an iPhone

no code implementations16 Dec 2016 Ashton Fagg, Simon Lucey, Sridha Sridharan

In this paper, we present our method for enabling dense SDM to run at over 90 FPS on a mobile device.


Inverse Compositional Spatial Transformer Networks

1 code implementation CVPR 2017 Chen-Hsuan Lin, Simon Lucey

In this paper, we establish a theoretical connection between the classical Lucas & Kanade (LK) algorithm and the emerging topic of Spatial Transformer Networks (STNs).

Classification General Classification

Photometric Bundle Adjustment for Vision-Based SLAM

no code implementations5 Aug 2016 Hatem Alismail, Brett Browning, Simon Lucey

We propose a novel algorithm for the joint refinement of structure and motion parameters from image data directly without relying on fixed and known correspondences.

Prior-Less Compressible Structure From Motion

no code implementations CVPR 2016 Chen Kong, Simon Lucey

Many non-rigid 3D structures are not modelled well through a low-rank subspace assumption.

Dictionary Learning

Direct Visual Odometry using Bit-Planes

no code implementations4 Apr 2016 Hatem Alismail, Brett Browning, Simon Lucey

Our approach utilizes an efficient binary descriptor, which we call Bit-Planes, and show how it can be used in the gradient-based optimization required by direct methods.

Visual Odometry

The Conditional Lucas & Kanade Algorithm

1 code implementation29 Mar 2016 Chen-Hsuan Lin, Rui Zhu, Simon Lucey

In this paper, we present a new approach, referred to as the Conditional LK algorithm, which: (i) directly learns linear models that predict geometric displacement as a function of appearance, and (ii) employs a novel strategy for ensuring that the generative pixel independence assumption can still be taken advantage of.

Bit-Planes: Dense Subpixel Alignment of Binary Descriptors

no code implementations31 Jan 2016 Hatem Alismail, Brett Browning, Simon Lucey

Binary descriptors have been instrumental in the recent evolution of computationally efficient sparse image alignment algorithms.

Learning Temporal Alignment Uncertainty for Efficient Event Detection

no code implementations4 Sep 2015 Iman Abbasnejad, Sridha Sridharan, Simon Denman, Clinton Fookes, Simon Lucey

A popular approach in this regard is to represent a sequence using a bag of words (BOW) representation due to its: (i) fixed dimensionality irrespective of the sequence length, and (ii) its ability to compactly model the statistics in the sequence.

Event Detection

Dense Semantic Correspondence where Every Pixel is a Classifier

no code implementations ICCV 2015 Hilton Bristow, Jack Valmadre, Simon Lucey

Determining dense semantic correspondences across objects and scenes is a difficult problem that underpins many higher-level computer vision algorithms.

Object Recognition Semantic correspondence

Regression-Based Image Alignment for General Object Categories

no code implementations8 Jul 2014 Hilton Bristow, Simon Lucey

Gradient-descent methods have exhibited fast and reliable performance for image alignment in the facial domain, but have largely been ignored by the broader vision community.


Why do linear SVMs trained on HOG features perform so well?

1 code implementation10 Jun 2014 Hilton Bristow, Simon Lucey

Linear Support Vector Machines trained on HOG features are now a de facto standard across many visual perception tasks.

Pedestrian Detection

Optimization Methods for Convolutional Sparse Coding

no code implementations10 Jun 2014 Hilton Bristow, Simon Lucey

We discuss a range of optimization methods for solving the convolutional sparse coding objective, and the properties that make each method suitable for different applications.

Complex Non-Rigid Motion 3D Reconstruction by Union of Subspaces

no code implementations CVPR 2014 Yingying Zhu, Dong Huang, Fernando de la Torre, Simon Lucey

The task of estimating complex non-rigid 3D motion through a monocular camera is of increasing interest to the wider scientific community.

3D Reconstruction

Correlation Filters with Limited Boundaries

no code implementations CVPR 2015 Hamed Kiani Galoogahi, Terence Sim, Simon Lucey

In this paper, we propose a novel approach to correlation filter estimation that: (i) takes advantage of inherent computational redundancies in the frequency domain, and (ii) dramatically reduces boundary effects.

Object Tracking

Learning detectors quickly using structured covariance matrices

no code implementations28 Mar 2014 Jack Valmadre, Sridha Sridharan, Simon Lucey

Computer vision is increasingly becoming interested in the rapid estimation of object detectors.

Fast Convolutional Sparse Coding

no code implementations CVPR 2013 Hilton Bristow, Anders Eriksson, Simon Lucey

Sparse coding has become an increasingly popular method in learning and vision for a variety of classification, reconstruction and coding tasks.

General Classification

Cannot find the paper you are looking for? You can Submit a new open access paper.