Search Results for author:
Found 12 papers, 7 papers with code
Flex Attention: A Programming Model for Generating Optimized Attention Kernels
We introduce FlexAttention, a novel compiler-driven programming model that allows implementing the majority of attention variants in a few lines of idiomatic PyTorch code.
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
We introduce GPT-NeoX-20B, a 20 billion parameter autoregressive language model trained on the Pile, whose weights will be made freely and openly available to the public through a permissive license.
Ranked #95 on
Multi-task Language Understanding
on MMLU
Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python
Modern deep learning frameworks provide imperative, eager execution programming interfaces embedded in Python to provide a productive development experience.
Edge Proposal Sets for Link Prediction
Here, we demonstrate how simply adding a set of edges, which we call a \emph{proposal set}, to the graph as a pre-processing step can improve the performance of several link prediction algorithms.
Ranked #1 on
Link Property Prediction
on ogbl-ddi
Measuring Coding Challenge Competence With APPS
Recent models such as GPT-Neo can pass approximately 20% of the test cases of introductory problems, so we find that machine learning models are now beginning to learn how to code.
Ranked #10 on
Code Generation
on APPS
The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Recent work has demonstrated that increased training dataset diversity improves general cross-domain knowledge and downstream generalization capability for large-scale language models.
Ranked #13 on
Language Modelling
on The Pile
Value Function Based Performance Optimization of Deep Learning Workloads
As machine learning techniques become ubiquitous, the efficiency of neural network implementations is becoming correspondingly paramount.
Combining Label Propagation and Simple Models Out-performs Graph Neural Networks
Graph Neural Networks (GNNs) are the predominant technique for learning over graphs.
Node Classification on Non-Homophilic (Heterophilic) Graphs
Node Property Prediction
Better Set Representations For Relational Reasoning
Incorporating relational reasoning into neural networks has greatly expanded their capabilities and scope.
Enhancing Adversarial Example Transferability with an Intermediate Level Attack
We show that we can select a layer of the source model to perturb without any knowledge of the target models while achieving high transferability.
Adversarial Example Decomposition
Research has shown that widely used deep neural networks are vulnerable to carefully crafted adversarial perturbations.
Intermediate Level Adversarial Attack for Enhanced Transferability
Neural networks are vulnerable to adversarial examples, malicious inputs crafted to fool trained models.
