Search Results for author:
Found 54 papers, 35 papers with code
Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks
Large language models (LLMs) have shown remarkable advancements in enabling language agents to tackle simple tasks.
ETS: Efficient Tree Search for Inference-Time Scaling
ETS incorporates a linear programming cost model to promote KV cache sharing by penalizing the number of nodes retained, while incorporating a semantic coverage term into the cost model to ensure that we retain trajectories which are semantically different.
QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache
Large Language Models (LLMs) are increasingly being deployed on edge devices for long-context settings, creating a growing need for fast and efficient long-context inference.
Squeezed Attention: Accelerating Long Context Length LLM Inference
During inference, we compare query tokens from the user input with the centroids to predict which of the keys from the fixed context are semantically relevant and need to be loaded during inference.
Efficient and Scalable Estimation of Tool Representations in Vector Space
To address those challenges, we present a novel framework for generating synthetic data for tool retrieval applications and an efficient data-driven tool retrieval strategy using small encoder models.
TinyAgent: Function Calling at the Edge
Recent large language models (LLMs) have enabled the development of advanced agentic systems that can integrate various tools and APIs to fulfill user queries through function calling.
Characterizing Prompt Compression Methods for Long Context Inference
However, there has been little work on comparing the different proposed methods across different tasks through a standardized analysis.
Reliable edge machine learning hardware for scientific applications
Extreme data rate scientific experiments create massive amounts of data that require efficient ML edge processing.
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement
To address this, we propose LLM2LLM, a targeted and iterative data augmentation strategy that uses a teacher LLM to enhance a small seed dataset by augmenting additional data that can be used for fine-tuning on a specific task.
AI and Memory Wall
The availability of unprecedented unsupervised training data, along with neural scaling laws, has resulted in an unprecedented surge in model size and compute requirements for serving/training LLMs.
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
LLMs are seeing growing use for applications which require large context windows, and with these large context windows KV cache activations surface as the dominant contributor to memory consumption during inference.
An LLM Compiler for Parallel Function Calling
To address this, we introduce LLMCompiler, which executes functions in parallel to efficiently orchestrate multiple function calls.
SPEED: Speculative Pipelined Execution for Efficient Decoding
For Transformer decoders that employ parameter sharing, the memory operations for the tokens executing in parallel can be amortized, which allows us to accelerate generative LLM inference.
SqueezeLLM: Dense-and-Sparse Quantization
When applied to the LLaMA models, our 3-bit quantization significantly reduces the perplexity gap from the FP16 baseline by up to 2. 1x as compared to the state-of-the-art methods with the same memory requirement.
End-to-end codesign of Hessian-aware quantized neural networks for FPGAs and ASICs
We develop an end-to-end workflow for the training and implementation of co-designed neural networks (NNs) for efficient field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) hardware.
Full Stack Optimization of Transformer Inference: a Survey
In this work, we survey different approaches for efficient Transformer inference, including: (i) analysis and profiling of the bottlenecks in existing Transformer architectures and their similarities and differences with previous convolutional models; (ii) implications of Transformer architecture on hardware, including the impact of non-linear operations such as Layer Normalization, Softmax, and GELU, as well as linear operations, on hardware design; (iii) approaches for optimizing a fixed Transformer architecture; (iv) challenges in finding the right mapping and scheduling of operations for Transformer models; and (v) approaches for optimizing Transformer models by adapting the architecture using neural architecture search.
Adaptive Self-supervision Algorithms for Physics-informed Neural Networks
We find that training vanilla PINNs for these problems can result in up to 70% prediction error in the solution, especially in the regime of low collocation points.
Squeezeformer: An Efficient Transformer for Automatic Speech Recognition
After re-examining the design choices for both the macro and micro-architecture of Conformer, we propose Squeezeformer which consistently outperforms the state-of-the-art ASR models under the same training schemes.
Ranked #38 on
Speech Recognition
on LibriSpeech test-clean
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
A Fast Post-Training Pruning Framework for Transformers
To address this, we propose a fast post-training pruning framework for Transformers that does not require any retraining.
Applications and Techniques for Fast Machine Learning in Science
In this community review report, we discuss applications and techniques for fast machine learning (ML) in science -- the concept of integrating power ML methods into the real-time experimental data processing loop to accelerate scientific discovery.
Characterizing possible failure modes in physics-informed neural networks
We provide evidence that the soft regularization in PINNs, which involves PDE-based differential operators, can introduce a number of subtle problems, including making the problem more ill-conditioned.
Learned Token Pruning for Transformers
We extensively test the performance of LTP on GLUE tasks and show that our method outperforms the prior state-of-the-art token pruning methods by up to ~2. 5% higher accuracy with the same amount of FLOPs.
Integer-only Zero-shot Quantization for Efficient Speech Recognition
End-to-end neural network models achieve improved performance on various automatic speech recognition (ASR) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Survey of Quantization Methods for Efficient Neural Network Inference
Thus, it is not surprising that quantization has emerged recently as an important and very active sub-area of research in the efficient implementation of computations associated with Neural Networks.
Hessian-Aware Pruning and Optimal Neural Implant
To address this problem, we introduce a new Hessian Aware Pruning (HAP) method coupled with a Neural Implant approach that uses second-order sensitivity as a metric for structured pruning.
I-BERT: Integer-only BERT Quantization
Transformer based models, like BERT and RoBERTa, have achieved state-of-the-art results in many Natural Language Processing tasks.
Natural Language Inference
Natural Language Understanding
+1
HAWQV3: Dyadic Neural Network Quantization
Current low-precision quantization algorithms often have the hidden cost of conversion back and forth from floating point to quantized integer values.
Boundary thickness and robustness in learning models
Using these observations, we show that noise-augmentation on mixup training further increases boundary thickness, thereby combating vulnerability to various forms of adversarial attacks and OOD transforms.
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
We introduce ADAHESSIAN, a second order stochastic optimization algorithm which dynamically incorporates the curvature of the loss function via ADAptive estimates of the HESSIAN.
PowerNorm: Rethinking Batch Normalization in Transformers
To address this, we propose Power Normalization (PN), a novel normalization scheme that resolves this issue by (i) relaxing zero-mean normalization in BN, (ii) incorporating a running quadratic mean instead of per batch statistics to stabilize fluctuations, and (iii) using an approximate backpropagation for incorporating the running statistics in the forward pass.
Ranked #12 on
Machine Translation
on WMT2014 English-German
ZeroQ: A Novel Zero Shot Quantization Framework
Importantly, ZeroQ has a very low computational overhead, and it can finish the entire quantization process in less than 30s (0. 5\% of one epoch training time of ResNet50 on ImageNet).
Ranked #1 on
Data Free Quantization
on CIFAR10
(CIFAR-10 W8A8 Top-1 Accuracy metric)
PyHessian: Neural Networks Through the Lens of the Hessian
To illustrate this, we analyze the effect of residual connections and Batch Normalization layers on the trainability of neural networks.
ANODEV2: A Coupled Neural ODE Framework
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE).
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
However, the search space for a mixed-precision quantization is exponential in the number of layers.
Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
We formalize the problem of trading-off DNN training time and memory requirements as the tensor rematerialization optimization problem, a generalization of prior checkpointing strategies.
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
In particular, we propose a new group-wise quantization scheme, and we use a Hessian based mix-precision method to compress the model further.
Ranked #13 on
Semantic Textual Similarity
on STS Benchmark
ANODEV2: A Coupled Neural ODE Evolution Framework
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE).
HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
Another challenge is a similar factorial complexity for determining block-wise fine-tuning order when quantizing the model to a target precision.
Inefficiency of K-FAC for Large Batch Size Training
In stochastic optimization, using large batch sizes during training can leverage parallel resources to produce faster wall-clock training times per training epoch.
ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs
ANODE has a memory footprint of O(L) + O(N_t), with the same computational cost as reversing ODE solve.
Clustering Multivariate Time Series
Multivariate Time Series Imputation
Trust Region Based Adversarial Attack on Neural Networks
To address this problem, we present a new family of trust region based adversarial attacks, with the goal of computing adversarial perturbations efficiently.
Parameter Re-Initialization through Cyclical Batch Size Schedules
We demonstrate the ability of our method to improve language modeling performance by up to 7. 91 perplexity and reduce training iterations by up to $61\%$, in addition to its flexibility in enabling snapshot ensembling and use with adversarial training.
Ranked #51 on
Natural Language Inference
on SNLI
On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent
Increasing the mini-batch size for stochastic gradient descent offers significant opportunities to reduce wall-clock training time, but there are a variety of theoretical and systems challenges that impede the widespread success of this technique.
Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge
This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i. e., 2012-2018.
A Novel Domain Adaptation Framework for Medical Image Segmentation
Our biophysics based domain adaptation achieves better results, as compared to the existing state-of-the-art GAN model used to create synthetic data for training.
Large batch size training of neural networks with adversarial training and second-order information
Our method exceeds the performance of existing solutions in terms of both accuracy and the number of SGD iterations (up to 1\% and $5\times$, respectively).
CLAIRE: A distributed-memory solver for constrained large deformation diffeomorphic image registration
We present the formulation, discuss algorithmic features, describe the software package, and introduce an improved preconditioner for the reduced space Hessian to speed up the convergence of our solver.
SqueezeNext: Hardware-Aware Neural Network Design
One of the main barriers for deploying neural networks on embedded systems has been large memory and power consumption of existing neural networks.
PDE-constrained optimization in medical image analysis
We review related literature and give examples on the formulation, discretization, and numerical solution of PDE-constrained optimization problems for medical imaging.
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries
Extensive experiments on multiple networks show that saddle-points are not the cause for generalization gap of large batch size training, and the results consistently show that large batch converges to points with noticeably higher Hessian spectrum.
Integrated Model, Batch and Domain Parallelism in Training Neural Networks
We propose a new integrated method of exploiting model, batch and domain parallelism for the training of deep neural networks (DNNs) on large distributed-memory computers using minibatch stochastic gradient descent (SGD).
Distributed-memory large deformation diffeomorphic 3D image registration
We are able to solve the registration problem for images of this size in less than five seconds on 64 x86 nodes of TACC's "Maverick" system.
AccFFT: A library for distributed-memory FFT on CPU and GPU architectures
We present a new library for parallel distributed Fast Fourier Transforms (FFT).
Distributed, Parallel, and Cluster Computing
