Search Results for author:
Found 10 papers, 3 papers with code
Introducing v0.5 of the AI Safety Benchmark from MLCommons
We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models
Large language models (LLMs) are typically trained on general source data for various domains, but a recent surge in domain-specific LLMs has shown their potential to outperform general-purpose models in domain-specific tasks (e. g., biomedicine).
Ranked #10 on
Question Answering
on PubMedQA
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency
Recent research has focused on weight sparsity in neural network training to reduce FLOPs, aiming for improved efficiency (test accuracy w. r. t training FLOPs).
SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models
In this work, we show the benefits of using unstructured weight sparsity to train only a subset of weights during pre-training (Sparse Pre-training) and then recover the representational capacity by allowing the zeroed weights to learn (Dense Fine-tuning).
RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network
However, training them requires substantial accelerator memory for saving large, multi-resolution activations.
Ranked #310 on
Image Classification
on ImageNet
(using extra training data)
Memory Efficient 3D U-Net with Reversible Mobile Inverted Bottlenecks for Brain Tumor Segmentation
The inverted residual bottleneck block uses lightweight depthwise separable convolutions to reduce computation by decomposing convolutions into a pointwise convolution and a depthwise convolution.
Enabling Continual Learning with Differentiable Hebbian Plasticity
Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge.
Differentiable Hebbian Consolidation for Continual Learning
Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge.
Self-Paced Learning with Adaptive Deep Visual Embeddings
Selecting the most appropriate data examples to present a deep neural network (DNN) at different stages of training is an unsolved challenge.
LEAP: Learning Embeddings for Adaptive Pace
The \textit{student} CNN classifier dynamically selects samples to form a mini-batch based on the \textit{easiness} from cross-entropy losses and \textit{true diverseness} of examples from the representation space sculpted by the \textit{embedding} CNN.
