Search Results for author:
Found 17 papers, 11 papers with code
Scale MLPerf-0.6 models on Google TPU-v3 Pods
The recent submission of Google TPU-v3 Pods to the industry wide MLPerf v0. 6 training benchmark demonstrates the scalability of a suite of industry relevant ML models.
MLPerf Training Benchmark
Machine learning (ML) needs industry-standard performance benchmarks to support design and competitive evaluation of the many emerging software and hardware solutions for ML.
MLPerf Mobile Inference Benchmark
This paper presents the first industry-standard open-source machine learning (ML) benchmark to allow perfor mance and accuracy evaluation of mobile devices with different AI chips and software stacks.
Data Engineering for Everyone
This article shows that open-source data sets are the rocket fuel for research and innovation at even some of the largest AI organizations.
GaNDLF: A Generally Nuanced Deep Learning Framework for Scalable End-to-End Clinical Workflows in Medical Imaging
Deep Learning (DL) has the potential to optimize machine learning in both the scientific and clinical communities.
Common Limitations of Image Processing Metrics: A Picture Story
While the importance of automatic image analysis is continuously increasing, recent meta-research revealed major flaws with respect to algorithm validation.
MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluation
Medical AI has tremendous potential to advance healthcare by supporting the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving provider and patient experience.
MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems
Scientific communities are increasingly adopting machine learning and deep learning models in their applications to accelerate scientific insights.
Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks
We introduce Dynatask: an open source system for setting up custom NLP tasks that aims to greatly lower the technical knowledge and effort required for hosting and evaluating state-of-the-art NLP models, as well as for conducting model in the loop data collection with crowdworkers.
Metrics reloaded: Recommendations for image analysis validation
The framework was developed in a multi-stage Delphi process and is based on the novel concept of a problem fingerprint - a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), data set and algorithm output.
DataPerf: Benchmarks for Data-Centric AI Development
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithfulness of the underlying problems.
Understanding metric-related pitfalls in image analysis validation
Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice.
Benchmarking Neural Network Training Algorithms
In order to address these challenges, we introduce a new, competitive, time-to-result benchmark using multiple workloads running on fixed hardware, the AlgoPerf: Training Algorithms benchmark.
DMLR: Data-centric Machine Learning Research -- Past, Present and Future
Drawing from discussions at the inaugural DMLR workshop at ICML 2023 and meetings prior, in this report we outline the relevance of community engagement and infrastructure development for the creation of next-generation public datasets that will advance machine learning science.
Croissant: A Metadata Format for ML-Ready Datasets
Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point.
Introducing v0.5 of the AI Safety Benchmark from MLCommons
We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.
