Search Results for author:
Found 8 papers, 3 papers with code
Named Entity Recognition in Historic Legal Text: A Transformer and State Machine Ensemble Method
Older legal texts are often scanned and digitized via Optical Character Recognition (OCR), which results in numerous errors.
Flash Inference: Near Linear Time Inference for Long Convolution Sequence Models and Beyond
While transformers have been at the core of most recent advancements in sequence generative models, their computational cost remains quadratic in sequence length.
TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training
By stacking training optimizations, we demonstrate accelerations of 65. 08% with 1D parallelism at the 128-GPU scale (Llama 3. 1 8B), an additional 12. 59% with 2D parallelism at the 256-GPU scale (Llama 3. 1 70B), and an additional 30% with 3D parallelism at the 512-GPU scale (Llama 3. 1 405B) on NVIDIA H100 GPUs over optimized baselines.
Proteus: A Self-Designing Range Filter
We introduce Proteus, a novel self-designing approximate range filter, which configures itself based on sampled data in order to optimize its false positive rate (FPR) for a given space requirement.
More or Less: When and How to Build Neural Network Ensembles
We identify a critical part of this design space that is not well-understood: That is how to decide between the alternatives of expanding a single network model or increasing the number of networks and using them together in an ensemble.
Learning Key-Value Store Design
The critical insight and potential long-term impact is that such unifying models 1) render what we consider up to now as fundamentally different data structures to be seen as views of the very same overall design space, and 2) allow seeing new data structure designs with performance properties that are not feasible by existing designs.
MotherNets: Rapid Deep Ensemble Learning
We propose MotherNets to enable higher accuracy and practical training cost for large and diverse neural network ensembles: A MotherNet captures the structural similarity across some or all members of a deep neural network ensemble which allows us to share data movement and computation costs across these networks.
Stochastic Database Cracking: Towards Robust Adaptive Indexing in Main-Memory Column-Stores
Stochastic cracking also uses each query as a hint on how to reorganize data, but not blindly so; it gains resilience and avoids performance bottlenecks by deliberately applying certain arbitrary choices in its decision-making.
