Search Results for author:
Found 15 papers, 10 papers with code
Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization
This work suggests fundamentally rethinking the current practice of pruning large language models (LLMs).
Critical Influence of Overparameterization on Sharpness-aware Minimization
Training an overparameterized neural network can yield minimizers of different generalization capabilities despite the same level of training loss.
FedFwd: Federated Learning without Backpropagation
In federated learning (FL), clients with limited resources can disrupt the training efficiency.
JaxPruner: A concise library for sparsity research
This paper introduces JaxPruner, an open-source JAX-based pruning and sparse training library for machine learning research.
A Closer Look at the Intervention Procedure of Concept Bottleneck Models
Concept bottleneck models (CBMs) are a class of interpretable neural network models that predict the target response of a given input based on its high-level concepts.
The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers
Knowledge distillation is an effective method for training lightweight vision models.
SpReME: Sparse Regression for Multi-Environment Dynamic Systems
Learning dynamical systems is a promising avenue for scientific discoveries.
Meta-Learning Sparse Implicit Neural Representations
Implicit neural representations are a promising new avenue of representing general signals by learning a continuous function that, parameterized as a neural network, maps the domain of a signal to its codomain; the mapping from spatial coordinates of an image to its pixel values, for example.
Understanding the Effects of Data Parallelism and Sparsity on Neural Network Training
As a result, we find across various workloads of data set, network model, and optimization algorithm that there exists a general scaling trend between batch size and number of training steps to convergence for the effect of data parallelism, and further, difficulty of training under sparsity.
A Signal Propagation Perspective for Pruning Neural Networks at Initialization
Alternatively, a recent approach shows that pruning can be done at initialization prior to training, based on a saliency criterion called connection sensitivity.
SNIP: Single-shot Network Pruning based on Connection Sensitivity
To achieve this, we introduce a saliency criterion based on connection sensitivity that identifies structurally important connections in the network for the given task.
Learn To Pay Attention
We propose an end-to-end-trainable attention module for convolutional neural network (CNN) architectures built for image classification.
DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
DESIRE effectively predicts future locations of objects in multiple scenes by 1) accounting for the multi-modal nature of the future prediction (i. e., given the same context, future may vary), 2) foreseeing the potential future outcomes and make a strategic prediction based on that, and 3) reasoning not only from the past motion history, but also from the scene context as well as the interactions among the agents.
Ranked #1 on
Trajectory Prediction
on PAID
Visual Compiler: Synthesizing a Scene-Specific Pedestrian Detector and Pose Estimator
We introduce the concept of a Visual Compiler that generates a scene specific pedestrian detector and pose estimator without any pedestrian observations.
Forecasting Interactive Dynamics of Pedestrians with Fictitious Play
We develop predictive models of pedestrian dynamics by encoding the coupled nature of multi-pedestrian interaction using game theory, and deep learning-based visual analysis to estimate person-specific behavior parameters.
