1 code implementation • 26 May 2018 • Christopher A. Metzler, Ali Mousavi, Reinhard Heckel, Richard G. Baraniuk
We show that, in the context of image recovery, SURE and its generalizations can be used to train convolutional neural networks (CNNs) for a range of image denoising and recovery problems without any ground truth data.
1 code implementation • NeurIPS 2017 • Christopher A. Metzler, Ali Mousavi, Richard G. Baraniuk
The LDAMP network is easy to train, can be applied to a variety of different measurement matrices, and comes with a state-evolution heuristic that accurately predicts its performance.
2 code implementations • 16 Jun 2014 • Christopher A. Metzler, Arian Maleki, Richard G. Baraniuk
A key element in D-AMP is the use of an appropriate Onsager correction term in its iterations, which coerces the signal perturbation at each iteration to be very close to the white Gaussian noise that denoisers are typically designed to remove.
2 code implementations • 26 Sep 2019 • Haoran You, Chaojian Li, Pengfei Xu, Yonggan Fu, Yue Wang, Xiaohan Chen, Richard G. Baraniuk, Zhangyang Wang, Yingyan Lin
In this paper, we discover for the first time that the winning tickets can be identified at the very early training stage, which we term as early-bird (EB) tickets, via low-cost training schemes (e. g., early stopping and low-precision training) at large learning rates.
1 code implementation • ICLR 2020 • Haoran You, Chaojian Li, Pengfei Xu, Yonggan Fu, Yue Wang, Xiaohan Chen, Richard G. Baraniuk, Zhangyang Wang, Yingyan Lin
Finally, we leverage the existence of EB tickets and the proposed mask distance to develop efficient training methods, which are achieved by first identifying EB tickets via low-cost schemes, and then continuing to train merely the EB tickets towards the target accuracy.
1 code implementation • 7 Feb 2022 • Vishwanath Saragadam, Jasper Tan, Guha Balakrishnan, Richard G. Baraniuk, Ashok Veeraraghavan
We introduce a new neural signal model designed for efficient high-resolution representation of large-scale signals.
1 code implementation • ICML 2018 • Christopher A. Metzler, Philip Schniter, Ashok Veeraraghavan, Richard G. Baraniuk
Phase retrieval algorithms have become an important component in many modern computational imaging systems.
1 code implementation • 22 May 2023 • Shashank Sonkar, Naiming Liu, Debshila Basu Mallick, Richard G. Baraniuk
We present a design framework called Conversational Learning with Analytical Step-by-Step Strategies (CLASS) for building advanced Intelligent Tutoring Systems (ITS) powered by high-performance Large Language Models (LLMs).
1 code implementation • 7 Feb 2024 • Shashank Sonkar, Kangqi Ni, Sapana Chaudhary, Richard G. Baraniuk
Building on this perspective, we propose a novel approach for constructing a reward dataset specifically designed for the pedagogical alignment of LLMs.
1 code implementation • 24 Feb 2020 • Bao Wang, Tan M. Nguyen, Andrea L. Bertozzi, Richard G. Baraniuk, Stanley J. Osher
Nesterov accelerated gradient (NAG) improves the convergence rate of gradient descent (GD) for convex optimization using a specially designed momentum; however, it accumulates error when an inexact gradient is used (such as in SGD), slowing convergence at best and diverging at worst.
1 code implementation • 16 Oct 2021 • Tam Nguyen, Tan M. Nguyen, Dung D. Le, Duy Khuong Nguyen, Viet-Anh Tran, Richard G. Baraniuk, Nhat Ho, Stanley J. Osher
Inspired by this observation, we propose Transformer with a Mixture of Gaussian Keys (Transformer-MGK), a novel transformer architecture that replaces redundant heads in transformers with a mixture of keys at each head.
1 code implementation • 21 Feb 2022 • Naiming Liu, Zichao Wang, Richard G. Baraniuk, Andrew Lan
In education applications, knowledge tracing refers to the problem of estimating students' time-varying concept/skill mastery level from their past responses to questions and predicting their future performance.
2 code implementations • NeurIPS 2020 • Tan M. Nguyen, Richard G. Baraniuk, Andrea L. Bertozzi, Stanley J. Osher, Bao Wang
Designing deep neural networks is an art that often involves an expensive search over candidate architectures.
1 code implementation • 15 Apr 2021 • Shashank Sonkar, Arzoo Katiyar, Richard G. Baraniuk
Knowledge graphs link entities through relations to provide a structured representation of real world facts.
Ranked #11 on Link Prediction on FB15k-237
1 code implementation • 12 Jun 2018 • Amirali Aghazadeh, Ryan Spring, Daniel LeJeune, Gautam Dasarathy, Anshumali Shrivastava, Richard G. Baraniuk
We demonstrate that MISSION accurately and efficiently performs feature selection on real-world, large-scale datasets with billions of dimensions.
1 code implementation • 27 Mar 2015 • Azalia Mirhoseini, Eva L. Dyer, Ebrahim. M. Songhori, Richard G. Baraniuk, Farinaz Koushanfar
This paper introduces RankMap, a platform-aware end-to-end framework for efficient execution of a broad class of iterative learning algorithms for massive and dense datasets.
1 code implementation • 14 Jan 2017 • Ali Mousavi, Richard G. Baraniuk
The promise of compressive sensing (CS) has been offset by two significant challenges.
1 code implementation • 21 Sep 2023 • Shashank Sonkar, MyCo Le, Xinghe Chen, Naiming Liu, Debshila Basu Mallick, Richard G. Baraniuk
Our approach notably enhances the quality of synthetic conversation datasets, especially for subjects that are calculation-intensive.
1 code implementation • 25 Feb 2019 • Daniel LeJeune, Richard G. Baraniuk, Reinhard Heckel
Algorithms often carry out equally many computations for "easy" and "hard" problem instances.
1 code implementation • 1 Nov 2022 • Daniel LeJeune, Lorenzo Luzi, Ali Siahkoohi, Sina AlEMohammad, Vishwanath Saragadam, Hossein Babaei, Naiming Liu, Zichao Wang, Richard G. Baraniuk
We study the interpolation capabilities of implicit neural representations (INRs) of images.
1 code implementation • 2 Apr 2015 • Ankit B. Patel, Tan Nguyen, Richard G. Baraniuk
A grand challenge in machine learning is the development of computational algorithms that match or outperform humans in perceptual inference tasks that are complicated by nuisance variation.
1 code implementation • 10 Oct 2019 • Daniel LeJeune, Hamid Javadi, Richard G. Baraniuk
Ensemble methods that average over a collection of independent predictors that are each limited to a subsampling of both the examples and features of the training data command a significant presence in machine learning, such as the ever-popular random forest, yet the nature of the subsampling effect, particularly of the features, is not well understood.
1 code implementation • 15 Mar 2021 • Pavan K. Kota, Daniel LeJeune, Rebekah A. Drezek, Richard G. Baraniuk
Here, we present the first exploration of the MMV problem where signals are independently drawn from a sparse, multivariate Poisson distribution.
1 code implementation • 2 Feb 2022 • Jasper Tan, Blake Mason, Hamid Javadi, Richard G. Baraniuk
A surprising phenomenon in modern machine learning is the ability of a highly overparameterized model to generalize well (small error on the test data) even when it is trained to memorize the training data (zero error on the training data).
1 code implementation • CVPR 2023 • Vishwanath Saragadam, Daniel LeJeune, Jasper Tan, Guha Balakrishnan, Ashok Veeraraghavan, Richard G. Baraniuk
Implicit neural representations (INRs) have recently advanced numerous vision-related areas.
no code implementations • 12 Nov 2017 • Randall Balestriero, Vincent Roger, Herve G. Glotin, Richard G. Baraniuk
We exploit a recently derived inversion scheme for arbitrary deep neural networks to develop a new semi-supervised learning framework that applies to a wide range of systems and problems.
no code implementations • 11 Jul 2017 • Ali Mousavi, Gautam Dasarathy, Richard G. Baraniuk
In this paper we develop a novel computational sensing framework for sensing and recovering structured signals.
no code implementations • 24 Mar 2017 • Joshua J. Michalenko, Andrew S. Lan, Richard G. Baraniuk
An important, yet largely unstudied, problem in student data analysis is to detect misconceptions from students' responses to open-response questions.
no code implementations • 6 Dec 2016 • Tan Nguyen, Wanjia Liu, Ethan Perez, Richard G. Baraniuk, Ankit B. Patel
Semi-supervised learning algorithms reduce the high cost of acquiring labeled training data by using both labeled and unlabeled data during learning.
no code implementations • NeurIPS 2016 • Ankit B. Patel, Tan Nguyen, Richard G. Baraniuk
We develop a probabilistic framework for deep learning based on the Deep Rendering Mixture Model (DRMM), a new generative probabilistic model that explicitly capture variations in data due to latent task nuisance variables.
no code implementations • 5 Aug 2014 • Eric C. Chi, Genevera I. Allen, Richard G. Baraniuk
In the biclustering problem, we seek to simultaneously group observations and features.
no code implementations • 3 Nov 2015 • Ali Mousavi, Arian Maleki, Richard G. Baraniuk
For instance the following basic questions have not yet been studied in the literature: (i) How does the size of the active set $\|\hat{\beta}^\lambda\|_0/p$ behave as a function of $\lambda$?
no code implementations • 17 Aug 2015 • Ali Mousavi, Ankit B. Patel, Richard G. Baraniuk
In this paper, we develop a new framework for sensing and recovering structured signals.
no code implementations • 9 Mar 2015 • Aswin C. Sankaranarayanan, Lina Xu, Christoph Studer, Yun Li, Kevin Kelly, Richard G. Baraniuk
In this paper, we propose the CS multi-scale video (CS-MUVI) sensing and recovery framework for high-quality video acquisition and recovery using SMCs.
no code implementations • 19 May 2015 • Raajen Patel, Thomas A. Goldstein, Eva L. Dyer, Azalia Mirhoseini, Richard G. Baraniuk
Kernel matrices (e. g. Gram or similarity matrices) are essential for many state-of-the-art approaches to classification, clustering, and dimensionality reduction.
no code implementations • 4 May 2015 • Eva L. Dyer, Tom A. Goldstein, Raajen Patel, Konrad P. Kording, Richard G. Baraniuk
Classical approaches discover such structure by learning a basis that can efficiently express the collection.
no code implementations • 18 Jan 2015 • Andrew S. Lan, Divyanshu Vats, Andrew E. Waters, Richard G. Baraniuk
Our data-driven framework for mathematical language processing (MLP) leverages solution data from a large number of learners to evaluate the correctness of their solutions, assign partial-credit scores, and provide feedback to each learner on the likely locations of any errors.
no code implementations • 12 Jan 2015 • Ryan Ning, Andrew E. Waters, Christoph Studer, Richard G. Baraniuk
In this work, we propose a novel methodology for unordered categorical IRT that we call SPRITE (short for stochastic polytomous response item model) that: (i) analyzes both ordered and unordered categories, (ii) offers interpretable outputs, and (iii) provides improved data fitting compared to existing models.
no code implementations • 18 Dec 2014 • Andrew S. Lan, Christoph Studer, Richard G. Baraniuk
The recently proposed SPARse Factor Analysis (SPARFA) framework for personalized learning performs factor analysis on ordinal or binary-valued (e. g., correct/incorrect) graded learner responses to questions.
no code implementations • 18 Dec 2014 • Andrew S. Lan, Christoph Studer, Andrew E. Waters, Richard G. Baraniuk
SPARse Factor Analysis (SPARFA) is a novel framework for machine learning-based learning analytics, which estimates a learner's knowledge of the concepts underlying a domain, and content analytics, which estimates the relationships among a collection of questions and those concepts.
no code implementations • 15 Apr 2014 • Jianing V. Shi, Yangyang Xu, Richard G. Baraniuk
In this paper, we introduce the concept of sparse bilinear logistic regression for decision problems involving explanatory variables that are two-dimensional matrices.
no code implementations • 13 Apr 2014 • Divyanshu Vats, Robert D. Nowak, Richard G. Baraniuk
This paper studies graphical model selection, i. e., the problem of estimating a graph of statistical relationships among a collection of random variables.
no code implementations • 23 Feb 2014 • Divyanshu Vats, Richard G. Baraniuk
In this paper, we address the challenging problem of selecting tuning parameters for high-dimensional sparse regression.
no code implementations • 5 Dec 2013 • Divyanshu Vats, Richard G. Baraniuk
We consider the high-dimensional sparse linear regression problem of accurately estimating a sparse vector using a small number of linear measurements that are contaminated by noise.
no code implementations • 30 Jan 2014 • Jianing V. Shi, Wotao Yin, Aswin C. Sankaranarayanan, Richard G. Baraniuk
We apply this framework to accelerate the acquisition process of dynamic MRI and show it achieves the best reconstruction accuracy with the least computational time compared with existing algorithms in the literature.
no code implementations • 19 Dec 2013 • Andrew S. Lan, Christoph Studer, Richard G. Baraniuk
We propose SPARFA-Trace, a new machine learning-based framework for time-varying learning and content analytics for education applications.
no code implementations • 31 Oct 2013 • Ali Mousavi, Arian Maleki, Richard G. Baraniuk
In particular, both the final reconstruction error and the convergence rate of the algorithm crucially rely on how the threshold parameter is set at each step of the algorithm.
no code implementations • 23 Sep 2013 • Ali Mousavi, Arian Maleki, Richard G. Baraniuk
This paper concerns the performance of the LASSO (also knows as basis pursuit denoising) for recovering sparse signals from undersampled, randomized, noisy measurements.
no code implementations • 22 Mar 2013 • Andrew S. Lan, Andrew E. Waters, Christoph Studer, Richard G. Baraniuk
We estimate these factors given the graded responses to a collection of questions.
no code implementations • 19 Mar 2013 • Eva L. Dyer, Aswin C. Sankaranarayanan, Richard G. Baraniuk
To learn a union of subspaces from a collection of data, sets of signals in the collection that belong to the same subspace must be identified in order to obtain accurate estimates of the subspace structures present in the data.
no code implementations • 23 Jan 2012 • Aswin C. Sankaranarayanan, Pavan K Turaga, Rama Chellappa, Richard G. Baraniuk
Compressive sensing (CS) is a new approach for the acquisition and recovery of sparse signals and images that enables sampling rates significantly below the classical Nyquist rate.
no code implementations • 8 May 2013 • Andrew S. Lan, Christoph Studer, Andrew E. Waters, Richard G. Baraniuk
In order to better interpret the estimated latent concepts, SPARFA relies on a post-processing step that utilizes user-defined tags (e. g., topics or keywords) available for each question.
no code implementations • ICLR 2019 • Randall Balestriero, Richard G. Baraniuk
We show that, under a GMM, piecewise affine, convex nonlinearities like ReLU, absolute value, and max-pooling can be interpreted as solutions to certain natural "hard" VQ inference problems, while sigmoid, hyperbolic tangent, and softmax can be interpreted as solutions to corresponding "soft" VQ inference problems.
no code implementations • ICLR 2019 • Nhat Ho, Tan Nguyen, Ankit B. Patel, Anima Anandkumar, Michael. I. Jordan, Richard G. Baraniuk
The conjugate prior yields a new regularizer for learning based on the paths rendered in the generative model for training CNNs–the Rendering Path Normalization (RPN).
no code implementations • 18 Feb 2019 • Benjamin Coleman, Richard G. Baraniuk, Anshumali Shrivastava
We present the first sublinear memory sketch that can be queried to find the nearest neighbors in a dataset.
no code implementations • ICLR 2019 • Ali Mousavi, Gautam Dasarathy, Richard G. Baraniuk
In this paper, we focus on two challenges which offset the promise of sparse signal representation, sensing, and recovery.
no code implementations • 27 Feb 2019 • Joshua J. Michalenko, Ameesh Shah, Abhinav Verma, Richard G. Baraniuk, Swarat Chaudhuri, Ankit B. Patel
We investigate the internal representations that a recurrent neural network (RNN) uses while learning to recognize a regular formal language.
1 code implementation • 22 May 2019 • Daniel LeJeune, Gautam Dasarathy, Richard G. Baraniuk
The main goal is to efficiently identify a subset of arms in a multi-armed bandit problem whose means are above a specified threshold.
no code implementations • 21 May 2019 • Indu Manickam, Andrew S. Lan, Gautam Dasarathy, Richard G. Baraniuk
We apply this framework to the last two months of the election period for a group of 47508 Twitter users and demonstrate that both liberal and conservative users became more polarized over time.
no code implementations • 28 May 2019 • Daniel LeJeune, Randall Balestriero, Hamid Javadi, Richard G. Baraniuk
Deep (neural) networks have been applied productively in a wide range of supervised and unsupervised learning tasks.
no code implementations • 10 Jul 2019 • Yujia Huang, Sihui Dai, Tan Nguyen, Richard G. Baraniuk, Anima Anandkumar
Our results show that when trained on CIFAR-10, lower likelihood (of latent variables) is assigned to SVHN images.
no code implementations • 9 Dec 2019 • Tan M. Nguyen, Animesh Garg, Richard G. Baraniuk, Anima Anandkumar
Continuous Normalizing Flows (CNFs) have emerged as promising deep generative models for a wide range of tasks thanks to their invertibility and exact likelihood estimation.
no code implementations • 1 Nov 2018 • Tan Nguyen, Nhat Ho, Ankit Patel, Anima Anandkumar, Michael. I. Jordan, Richard G. Baraniuk
This conjugate prior yields a new regularizer based on paths rendered in the generative model for training CNNs-the Rendering Path Normalization (RPN).
no code implementations • NeurIPS 2019 Workshop Neuro AI 2019 • Yujia Huang, Sihui Dai, Tan Nguyen, Pinglei Bao, Doris Y. Tsao, Richard G. Baraniuk, Anima Anandkumar
Primates have a remarkable ability to correctly classify images even in the presence of significant noise and degradation.
no code implementations • ICML 2020 • Yehuda Dar, Paul Mayer, Lorenzo Luzi, Richard G. Baraniuk
We study the linear subspace fitting problem in the overparameterized setting, where the estimated subspace can perfectly interpolate the training examples.
no code implementations • 12 Mar 2020 • Zichao Wang, Sebastian Tschiatschek, Simon Woodhead, Jose Miguel Hernandez-Lobato, Simon Peyton Jones, Richard G. Baraniuk, Cheng Zhang
Online education platforms enable teachers to share a large number of educational resources such as questions to form exercises and quizzes for students.
no code implementations • 12 May 2020 • Gregory Ongie, Ajil Jalal, Christopher A. Metzler, Richard G. Baraniuk, Alexandros G. Dimakis, Rebecca Willett
Recent work in machine learning shows that deep neural networks can be used to solve a wide variety of inverse problems arising in computational imaging.
no code implementations • 25 May 2020 • Shashank Sonkar, Andrew E. Waters, Andrew S. Lan, Phillip J. Grimaldi, Richard G. Baraniuk
Knowledge tracing (KT) models, e. g., the deep knowledge tracing (DKT) model, track an individual learner's acquisition of skills over time by examining the learner's performance on questions related to those skills.
no code implementations • COLING 2020 • Shashank Sonkar, Andrew E. Waters, Richard G. Baraniuk
Word embedding models learn semantically rich vector representations of words and are widely used to initialize natural processing language (NLP) models.
no code implementations • 12 Jun 2020 • Yehuda Dar, Richard G. Baraniuk
We analytically characterize the generalization error of the target task in terms of the salient factors in the transfer learning architecture, i. e., the number of examples available, the number of (free) parameters in each of the tasks, the number of parameters transferred from the source to target task, and the relation between the two tasks.
no code implementations • 13 Jun 2020 • Randall Balestriero, Herve Glotin, Richard G. Baraniuk
We develop an interpretable and learnable Wigner-Ville distribution that produces a super-resolved quadratic signal representation for time-series analysis.
no code implementations • 12 Jun 2020 • Weili Nie, Zichao Wang, Ankit B. Patel, Richard G. Baraniuk
Learning interpretable and disentangled representations is a crucial yet challenging task in representation learning.
no code implementations • NeurIPS 2020 • Randall Balestriero, Sebastien Paris, Richard G. Baraniuk
Deep Generative Networks (DGNs) with probabilistic modeling of their output and latent space are currently trained via Variational Autoencoders (VAEs).
no code implementations • 25 Jun 2020 • Lorenzo Luzi, Randall Balestriero, Richard G. Baraniuk
They can be represented in two ways: With an ensemble of networks or with a single network with truncated latent space.
no code implementations • 23 Jul 2020 • Zichao Wang, Angus Lamb, Evgeny Saveliev, Pashmina Cameron, Yordan Zaykov, José Miguel Hernández-Lobato, Richard E. Turner, Richard G. Baraniuk, Craig Barton, Simon Peyton Jones, Simon Woodhead, Cheng Zhang
In this competition, participants will focus on the students' answer records to these multiple-choice diagnostic questions, with the aim of 1) accurately predicting which answers the students provide; 2) accurately predicting which questions have high quality; and 3) determining a personalized sequence of questions for each student that best predicts the student's answers.
1 code implementation • 27 Oct 2020 • Sina AlEMohammad, Hossein Babaei, Randall Balestriero, Matt Y. Cheung, Ahmed Imtiaz Humayun, Daniel LeJeune, Naiming Liu, Lorenzo Luzi, Jasper Tan, Zichao Wang, Richard G. Baraniuk
High dimensionality poses many challenges to the use of data, from visualization and interpretation, to prediction and storage for historical preservation.
no code implementations • 9 Mar 2021 • Yehuda Dar, Daniel LeJeune, Richard G. Baraniuk
We define a transfer learning approach to the target task as a linear regression optimization with a regularization on the distance between the to-be-learned target parameters and the already-learned source parameters.
1 code implementation • NeurIPS 2021 • Daniel LeJeune, Hamid Javadi, Richard G. Baraniuk
Among the most successful methods for sparsifying deep (neural) networks are those that adaptively mask the network weights throughout training.
no code implementations • 6 Sep 2021 • Yehuda Dar, Vidya Muthukumar, Richard G. Baraniuk
The rapid recent progress in machine learning (ML) has raised a number of scientific questions that challenge the longstanding dogma of the field.
no code implementations • EMNLP 2021 • Zichao Wang, Andrew S. Lan, Richard G. Baraniuk
We study the problem of generating arithmetic math word problems (MWPs) given a math equation that specifies the mathematical computation and a context that specifies the problem scenario.
1 code implementation • 6 Oct 2021 • Fernando Gama, Nicolas Zilberstein, Richard G. Baraniuk, Santiago Segarra
Particle filtering is used to compute good nonlinear estimates of complex systems.
1 code implementation • 11 Oct 2021 • Sina AlEMohammad, Hossein Babaei, CJ Barberan, Naiming Liu, Lorenzo Luzi, Blake Mason, Richard G. Baraniuk
To further contribute interpretability with respect to classification and the layers, we develop a new network as a combination of multiple neural tangent kernels, one to model each layer of the deep neural network individually as opposed to past work which attempts to represent the entire network via a single neural tangent kernel.
no code implementations • 8 Oct 2021 • Lorenzo Luzi, Carlos Ortiz Marrero, Nile Wynar, Richard G. Baraniuk, Michael J. Henry
We define a performance measure, which we call WaM, on two sets of images by using Inception-v3 (or another classifier) to featurize the images, estimate two GMMs, and use the restricted $2$-Wasserstein distance to compare the GMMs.
no code implementations • 15 Oct 2021 • CJ Barberan, Randall Balestriero, Richard G. Baraniuk
Each member of the family is derived from a standard DN architecture by vector quantizing the unit output values and feeding them into a global linear classifier.
no code implementations • 25 Sep 2019 • Tan M. Nguyen, Animesh Garg, Richard G. Baraniuk, Anima Anandkumar
Continuous Normalizing Flows (CNFs) have emerged as promising deep generative models for a wide range of tasks thanks to their invertibility and exact likelihood estimation.
no code implementations • 22 Feb 2022 • T. Mitchell Roddenberry, Fernando Gama, Richard G. Baraniuk, Santiago Segarra
Leveraging this, we are able to seamlessly compare graphs of different sizes and coming from different models, yielding results on the convergence of spectral densities, transferability of filters across arbitrary graphs, and continuity of graph signal properties with respect to the distribution of local substructures.
no code implementations • 23 Feb 2022 • CJ Barberan, Sina AlEMohammad, Naiming Liu, Randall Balestriero, Richard G. Baraniuk
A key interpretability issue with RNNs is that it is not clear how each hidden state per time step contributes to the decision-making process in a quantitative manner.
no code implementations • 7 Mar 2022 • Rudolf H. Riedi, Randall Balestriero, Richard G. Baraniuk
Building on our earlier work connecting deep networks with continuous piecewise-affine splines, we develop an exact local linear representation of a deep network layer for a family of modern deep networks that includes ConvNets at one end of a spectrum and ResNets, DenseNets, and other networks with skip connections at the other.
no code implementations • 7 Apr 2022 • Vishwanath Saragadam, Randall Balestriero, Ashok Veeraraghavan, Richard G. Baraniuk
DeepTensor is a computationally efficient framework for low-rank decomposition of matrices and tensors using deep generative networks.
no code implementations • 27 May 2022 • Jasper Tan, Daniel LeJeune, Blake Mason, Hamid Javadi, Richard G. Baraniuk
Is overparameterization a privacy liability?
no code implementations • 1 Aug 2022 • Tan Nguyen, Richard G. Baraniuk, Robert M. Kirby, Stanley J. Osher, Bao Wang
Transformers have achieved remarkable success in sequence modeling and beyond but suffer from quadratic computational and memory complexities with respect to the length of the input sequence.
no code implementations • 29 Sep 2022 • Randall Balestriero, Richard G. Baraniuk
A critically important, ubiquitous, and yet poorly understood ingredient in modern deep networks (DNs) is batch normalization (BN), which centers and normalizes the feature maps.
no code implementations • 22 Oct 2022 • Shashank Sonkar, Naiming Liu, Richard G. Baraniuk
Transformer models trained on massive text corpora have become the de facto models for a wide range of natural language processing tasks.
no code implementations • 20 Nov 2022 • Yehuda Dar, Lorenzo Luzi, Richard G. Baraniuk
We study how the generalization behavior of transfer learning is affected by the dataset size in the source and target tasks, the number of transferred layers that are kept frozen in the target DNN training, and the similarity between the source and target tasks.
no code implementations • 13 Dec 2022 • Vishwanath Saragadam, Zheyi Han, Vivek Boominathan, Luocheng Huang, Shiyu Tan, Johannes E. Fröch, Karl F. Böhringer, Richard G. Baraniuk, Arka Majumdar, Ashok Veeraraghavan
A computational backend then utilizes a deep image prior to separate the resultant multiplexed image or video into a foveated image consisting of a high-resolution center and a lower-resolution large field of view context.
no code implementations • 19 Dec 2022 • Shashank Sonkar, Zichao Wang, Richard G. Baraniuk
MANER re-purposes the <mask> token for NER prediction.
no code implementations • 22 May 2023 • Shashank Sonkar, Richard G. Baraniuk
This paper investigates the key role of Feed-Forward Networks (FFNs) in transformer models by utilizing the Parallel Attention and Feed-Forward Net Design (PAF) architecture, and comparing it to their Series Attention and Feed-Forward Net Design (SAF) counterparts.
no code implementations • 23 May 2023 • Shashank Sonkar, Richard G. Baraniuk
We explore whether Large Language Models (LLMs) are capable of logical reasoning with distorted facts, which we call Deduction under Perturbed Evidence (DUPE).
no code implementations • 4 Jul 2023 • Sina AlEMohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, Richard G. Baraniuk
Seismic advances in generative AI algorithms for imagery, text, and other data types has led to the temptation to use synthetic data to train next-generation models.
no code implementations • 1 Oct 2023 • T. Mitchell Roddenberry, Vishwanath Saragadam, Maarten V. de Hoop, Richard G. Baraniuk
Implicit neural representations (INRs) have arisen as useful methods for representing signals on Euclidean domains.
no code implementations • 3 Oct 2023 • Naiming Liu, Shashank Sonkar, Zichao Wang, Simon Woodhead, Richard G. Baraniuk
We propose novel evaluations for mathematical reasoning capabilities of Large Language Models (LLMs) based on mathematical misconceptions.
no code implementations • 1 Dec 2023 • Tam Nguyen, Tan M. Nguyen, Richard G. Baraniuk
Transformers have achieved remarkable success in a wide range of natural language processing and computer vision applications.
no code implementations • 24 Jan 2024 • Josue Casco-Rodriguez, Caleb Kemere, Richard G. Baraniuk
Kalman filters provide a straightforward and interpretable means to estimate hidden or latent variables, and have found numerous applications in control, robotics, signal processing, and machine learning.
no code implementations • 25 Feb 2024 • Tam Nguyen, César A. Uribe, Tan M. Nguyen, Richard G. Baraniuk
Motivated by this control framework, we derive a novel class of transformers, PID-controlled Transformer (PIDformer), aimed at improving robustness and mitigating the rank-collapse issue inherent in softmax transformers.