Search Results for author: William Marshall

Found 7 papers, 2 papers with code

Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster

2 code implementations6 Apr 2023 Nolan Dey, Gurpreet Gosal, Zhiming, Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Marvin Tom, Joel Hestness

We study recent research advances that improve large language models through efficient pre-training and scaling, and open datasets and tools.

SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models

no code implementations18 Mar 2023 Vithursan Thangarasa, Abhay Gupta, William Marshall, Tianda Li, Kevin Leong, Dennis Decoste, Sean Lie, Shreyas Saxena

In this work, we show the benefits of using unstructured weight sparsity to train only a subset of weights during pre-training (Sparse Pre-training) and then recover the representational capacity by allowing the zeroed weights to learn (Dense Fine-tuning).

Text Generation Text Summarization

System Integrated Information

no code implementations30 Dec 2022 William Marshall, Matteo Grasso, William GP Mayner, Alireza Zaeemzadeh, Leonardo S Barbosa, Erick Chastain, Graham Findlay, Shuntaro Sasai, Larissa Albantakis, Giulio Tononi

Integrated information theory (IIT) starts from consciousness itself and identifies a set of properties (axioms) that are true of every conceivable experience.

PyPhi: A toolbox for integrated information theory

1 code implementation27 Dec 2017 William G. P. Mayner, William Marshall, Larissa Albantakis, Graham Findlay, Robert Marchman, Giulio Tononi

Integrated information theory provides a mathematical framework to fully characterize the cause-effect structure of a physical system.

Cannot find the paper you are looking for? You can Submit a new open access paper.