Search Results for author:
Found 30 papers, 13 papers with code
Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems
We find empirically that across multiple language tasks, surprisingly, Voting Inference Systems' performance first increases but then decreases as a function of the number of LLM calls.
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Autoregressive decoding of large language models (LLMs) is memory bandwidth bounded, resulting in high latency and significant wastes of the parallel processing power of modern accelerators.
Online Speculative Decoding
We develop a prototype of online speculative decoding based on online knowledge distillation and evaluate it using both synthetic and real query data on several popular LLMs.
Proof: Accelerating Approximate Aggregation Queries with Expensive Predicates
Given a dataset $\mathcal{D}$, we are interested in computing the mean of a subset of $\mathcal{D}$ which matches a predicate.
Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Training
Self-training is a standard approach to semi-supervised learning where the learner's own predictions on unlabeled data are used as supervision during training.
Leveraging Organizational Resources to Adapt Models to New Data Modalities
As applications in large organizations evolve, the machine learning (ML) models that power them must adapt the same predictive tasks to newly arising data modalities (e. g., a new video content launch in a social media application requires existing text or image models to extend to video).
Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics
This runtime engine a) efficiently pipelines preprocessing and DNN execution for inference, b) places preprocessing operations on the CPU or GPU in a hardware- and input-aware manner, and c) efficiently manages memory and threading for high throughput execution.
Similarity Search for Efficient Active Learning and Search of Rare Concepts
Many active learning and search approaches are intractable for large-scale industrial settings with billions of unlabeled examples.
Chromatic Learning for Sparse Datasets
By leveraging the structural properties of the co-occurrence graph, CL can compress sparse datasets, such as KDD Cup 2012, that contain over 50M features down to 1024, using an order of magnitude fewer features than frequency-based truncation and the hashing trick while maintaining the same test error for linear models.
Model Assertions for Monitoring and Improving ML Models
We propose methods of using model assertions at all stages of ML system deployment, including runtime monitoring, validating labels, and continuously improving ML models.
MLPerf Training Benchmark
Machine learning (ML) needs industry-standard performance benchmarks to support design and competitive evaluation of the many emerging software and hardware solutions for ML.
LIT: Learned Intermediate Representation Training for Model Compression
In this work, we introduce Learned Intermediate representation Training (LIT), a novel model compression technique that outperforms a range of recent model compression techniques by leveraging the highly repetitive structure of modern DNNs (e. g., ResNet).
Selection via Proxy: Efficient Data Selection for Deep Learning
By removing hidden layers from the target model, using smaller architectures, and training for fewer epochs, we create proxies that are an order of magnitude faster to train.
Willump: A Statistically-Aware End-to-end Optimizer for Machine Learning Inference
First, Willump automatically cascades feature computation for classification queries: Willump classifies most data inputs using only high-value, low-cost features selected through empirical observations of ML model performance, improving query performance by up to 5x without statistically significant accuracy loss.
CrossTrainer: Practical Domain Adaptation with Loss Reweighting
Domain adaptation provides a powerful set of model training techniques given domain-specific training data and supplemental data with unknown relevance.
Select Via Proxy: Efficient Data Selection For Training Deep Networks
In our approach, we first train a small proxy model quickly, which we then use to estimate the utility of individual training data points, and then select the most informative ones for training the large target model.
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
Equivariant Transformer Networks
How can prior knowledge on the transformation invariances of a domain be incorporated into the architecture of a neural network?
Block-wise Intermediate Representation Training for Model Compression
Hint training (i. e., FitNets) extends KD by regressing a student model’s intermediate representation (IR) to a teacher model’s IR.
LIT: Block-wise Intermediate Representation Training for Model Compression
Knowledge distillation (KD) is a popular method for reducing the computational overhead of deep network inference, in which the output of a teacher model is used to train a smaller, faster student model.
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark
In this work, we analyze the entries from DAWNBench, which received optimized submissions from multiple industrial groups, to investigate the behavior of TTA as a metric as well as trends in the best-performing entries.
BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics
We introduce two new query optimization techniques in BlazeIt that are not supported by prior work.
Databases
Model Specialization for Inference Via End-to-End Distillation, Pruning, and Cascades
Thus, simply fine-tuning or transfer learn- ing from a general-purpose network inherits a large computational cost that may not be necessary for a given task.
Sketching Linear Classifiers over Data Streams
We introduce a new sub-linear space sketch---the Weight-Median Sketch---for learning compressed linear classifiers over data streams while supporting the efficient recovery of large-magnitude weights in the model.
Compressed Factorization: Fast and Accurate Low-Rank Factorization of Compressively-Sensed Data
What learning algorithms can be run directly on compressively-sensed data?
To Index or Not to Index: Optimizing Exact Maximum Inner Product Search
The brute-force approach to solving exact MIPS is computationally expensive, thus spurring recent development of novel indexes and pruning techniques for this task.
Infrastructure for Usable Machine Learning: The Stanford DAWN Project
Despite incredible recent advances in machine learning, building machine learning applications remains prohibitively time-consuming and expensive for all but the best-trained, best-funded engineering organizations.
NoScope: Optimizing Neural Network Queries over Video at Scale
Given a target video, object to detect, and reference neural network, NoScope automatically searches for and trains a sequence, or cascade, of models that preserves the accuracy of the reference network but is specialized to the target video and are therefore far less computationally expensive.
The Missing Piece in Complex Analytics: Low Latency, Scalable Model Management and Serving with Velox
In this work, we present Velox, a new component of the Berkeley Data Analytics Stack.
Databases
Highly Available Transactions: Virtues and Limitations (Extended Version)
To minimize network latency and remain online during server failures and network partitions, many modern distributed data storage systems eschew transactional functionality, which provides strong semantic guarantees for groups of multiple operations over multiple data items.
Databases
