Search Results for author:
Found 11 papers, 2 papers with code
FFN Fusion: Rethinking Sequential Computation in Large Language Models
We introduce FFN Fusion, an architectural optimization technique that reduces sequential computation in large language models by identifying and exploiting natural opportunities for parallelization.
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs
We demonstrate the real-world impact of our framework through Llama-3. 1-Nemotron-51B-Instruct (Nemotron-51B), a publicly available model derived from Llama-3. 1-70B-Instruct.
Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum
We leverage the duality between wide neural networks and Neural Tangent Kernels and propose a preconditioned gradient descent method, which alters the trajectory of GD.
A Kernel Perspective of Skip Connections in Convolutional Networks
Over-parameterized residual networks (ResNets) are amongst the most successful convolutional neural architectures for image processing.
On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels
We study the properties of various over-parametrized convolutional neural architectures through their respective Gaussian process and neural tangent kernels.
Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networks
Deep residual network architectures have been shown to achieve superior accuracy over classical feed-forward networks, yet their success is still not fully understood.
On the Similarity between the Laplace and Neural Tangent Kernels
Experiments show that these kernel methods perform similarly to real neural networks.
Frequency Bias in Neural Networks for Input of Non-Uniform Density
Recent works have partly attributed the generalization ability of over-parameterized neural networks to frequency bias -- networks trained with gradient descent on data drawn from a uniform distribution find a low frequency fit before high frequency ones.
Averaging Essential and Fundamental Matrices in Collinear Camera Settings
Global methods to Structure from Motion have gained popularity in recent years.
Algebraic Characterization of Essential Matrices and Their Averaging in Multiview Settings
A common approach to essential matrix averaging is to separately solve for camera orientations and subsequently for camera positions.
GPSfM: Global Projective SFM Using Algebraic Constraints on Multi-View Fundamental Matrices
First, given ${n \choose 2}$ fundamental matrices computed for $n$ images, we provide a complete algebraic characterization in the form of conditions that are both necessary and sufficient to enabling the recovery of camera matrices.
