Search Results for author: Dennis Decoste

Found 12 papers, 3 papers with code

SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models

no code implementations18 Mar 2023 Vithursan Thangarasa, Abhay Gupta, William Marshall, Tianda Li, Kevin Leong, Dennis Decoste, Sean Lie, Shreyas Saxena

In this work, we show the benefits of using unstructured weight sparsity to train only a subset of weights during pre-training (Sparse Pre-training) and then recover the representational capacity by allowing the zeroed weights to learn (Dense Fine-tuning).

Text Generation Text Summarization

RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network

1 code implementation28 Jun 2022 Vitaliy Chiley, Vithursan Thangarasa, Abhay Gupta, Anshul Samar, Joel Hestness, Dennis Decoste

However, training them requires substantial accelerator memory for saving large, multi-resolution activations.

Ranked #313 on Image Classification on ImageNet (using extra training data)

General Classification Image Classification +2

Training With Data Dependent Dynamic Learning Rates

no code implementations27 May 2021 Shreyas Saxena, Nidhi Vyas, Dennis Decoste

This setting is widely adopted under the assumption that loss functions for each instance are similar in nature, and hence, a common learning rate can be used.

Image Classification

Memory Efficient 3D U-Net with Reversible Mobile Inverted Bottlenecks for Brain Tumor Segmentation

no code implementations19 Apr 2021 Mihir Pendse, Vithursan Thangarasa, Vitaliy Chiley, Ryan Holmdahl, Joel Hestness, Dennis Decoste

The inverted residual bottleneck block uses lightweight depthwise separable convolutions to reduce computation by decomposing convolutions into a pointwise convolution and a depthwise convolution.

Brain Tumor Segmentation Tumor Segmentation

Data Parameters: A New Family of Parameters for Learning a Differentiable Curriculum

1 code implementation NeurIPS 2019 Shreyas Saxena, Oncel Tuzel, Dennis Decoste

To the best of our knowledge, our work is the first curriculum learning method to show gains on large scale image classification and detection tasks.

General Classification Image Classification +2

Hot Swapping for Online Adaptation of Optimization Hyperparameters

no code implementations20 Dec 2014 Kevin Bache, Dennis Decoste, Padhraic Smyth

We describe a general framework for online adaptation of optimization hyperparameters by `hot swapping' their values during learning.

Fast Approximate Matching of Cell-Phone Videos for Robust Background Subtraction

no code implementations22 Apr 2014 Raffay Hamid, Atish Das Sarma, Dennis Decoste, Neel Sundaresan

We identify a novel instance of the background subtraction problem that focuses on extracting near-field foreground objects captured using handheld cameras.

Object

Compact Random Feature Maps

no code implementations17 Dec 2013 Raffay Hamid, Ying Xiao, Alex Gittens, Dennis Decoste

Kernel approximation using randomized feature maps has recently gained a lot of interest.

Dense Non-rigid Point-Matching Using Random Projections

no code implementations CVPR 2013 Raffay Hamid, Dennis Decoste, Chih-Jen Lin

We present a robust and efficient technique for matching dense sets of points undergoing non-rigid spatial transformations.

Cannot find the paper you are looking for? You can Submit a new open access paper.