Search Results for author:
Found 37 papers, 18 papers with code
Massively Multilingual Shallow Fusion with Large Language Models
In this work, we propose to train a single multilingual language model (LM) for shallow fusion in multiple languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Mind's Eye: Grounded Language Model Reasoning through Simulation
By training solely on written text, current language models (LMs) miss the grounded experience of humans in the real-world -- their failure to relate language to the physical world causes knowledge to be misrepresented and obvious mistakes in their reasoning.
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Auto Debugging
on Big-bench Lite
Co-training Transformer with Videos and Images Improves Action Recognition
We term this approach as Co-training Videos and Images for Action Recognition (CoVeR).
Ranked #6 on
Action Classification
on Moments in Time
(using extra training data)
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Scaling language models with more data, compute and parameters has driven significant progress in natural language processing.
Ranked #5 on
Common Sense Reasoning
on ARC (Easy)
(using extra training data)
Finetuned Language Models Are Zero-Shot Learners
We show that instruction tuning -- finetuning language models on a collection of tasks described via instructions -- substantially improves zero-shot performance on unseen tasks.
Ranked #1 on
Question Answering
on StoryCloze
BEDS-Bench: Behavior of EHR-models under Distributional Shift--A Benchmark
Most ML approaches focus on generalization performance on unseen data that are similar to the training data (In-Distribution, or IND).
MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records
One important challenge of applying deep learning to electronic health records (EHR) is the complexity of their multimodal structure.
Learning to Select Best Forecast Tasks for Clinical Outcome Prediction
The paradigm of pretraining' from a set of relevant auxiliary tasks and thenfinetuning' on a target task has been successfully applied in many different domains.
Learnability and Complexity of Quantum Samples
Given a quantum circuit, a quantum computer can sample the output distribution exponentially faster in the number of bits than classical computers.
Training independent subnetworks for robust prediction
Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network.
Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss
When training the parameters of a linear dynamical model, the gradient descent algorithm is likely to fail to converge if the squared-error loss is used as the training loss function.
Flow Contrastive Estimation of Energy-Based Models
(2) The update of the flow model approximately minimizes the Jensen-Shannon divergence between the flow model and the data distribution.
Modelling EHR timeseries by restricting feature interaction
Time series data are prevalent in electronic health records, mostly in the form of physiological parameters such as vital signs and lab tests.
Federated and Differentially Private Learning for Electronic Health Records
The use of collaborative and decentralized machine learning techniques such as federated learning have the potential to enable the development and deployment of clinical risk predictions models in low-resource settings without requiring sensitive data be shared or stored in a central repository.
Capacity, Bandwidth, and Compositionality in Emergent Language Learning
In this paper, we investigate the learning biases that affect the efficacy and compositionality of emergent languages.
Learning an Adaptive Learning Rate Schedule
The learning rate is one of the most important hyper-parameters for model training and generalization.
Improved Hierarchical Patient Classification with Language Model Pretraining over Clinical Notes
Clinical notes in electronic health records contain highly heterogeneous writing styles, including non-standard terminology or abbreviations.
Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer
A recent study showed that using the graphical structure underlying EHR data (e. g. relationship between diagnoses and treatments) improves the performance of prediction tasks such as heart failure prediction.
Analyzing the Role of Model Uncertainty for Electronic Health Records
We further show that RNNs with only Bayesian embeddings can be a more efficient way to capture model uncertainty compared to ensembles, and we analyze how model uncertainty is impacted across individual input features and patient subgroups.
Natural Questions: a Benchmark for Question Answering Research
The public release consists of 307, 373 training examples with single annotations, 7, 830 examples with 5-way annotations for development data, and a further 7, 842 examples 5-way annotated sequestered as test data.
Ranked #7 on
Question Answering
on Natural Questions (long)
Gmail Smart Compose: Real-Time Assisted Writing
In this paper, we present Smart Compose, a novel system for generating interactive, real-time suggestions in Gmail that assists users in writing mails by reducing repetitive typing.
Music Transformer
This is impractical for long sequences such as musical compositions since their memory complexity for intermediate relative information is quadratic in the sequence length.
Ranked #3 on
Music Modeling
on JSB Chorales
Peptide-Spectra Matching from Weak Supervision
As in many other scientific domains, we face a fundamental problem when using machine learning to identify proteins from mass spectrometry data: large ground truth datasets mapping inputs to correct outputs are extremely difficult to obtain.
Embedding Text in Hyperbolic Spaces
Ideally, we could incorporate our prior knowledge of this hierarchical structure into unsupervised learning algorithms that work on text data.
Learning Longer-term Dependencies in RNNs with Auxiliary Losses
Despite recent advances in training recurrent neural networks (RNNs), capturing long-term dependencies in sequences remains a fundamental challenge.
Ranked #10 on
Sequential Image Classification
on Sequential CIFAR-10
Scalable and accurate deep learning for electronic health records
Predictive modeling with electronic health record (EHR) data is anticipated to drive personalized medicine and improve healthcare quality.
MaskGAN: Better Text Generation via Filling in the______
Additionally, these models are typically trained via maxi- mum likelihood and teacher forcing.
A Goal-oriented Neural Conversation Model by Self-Play
One challenge in applying such techniques to building goal-oriented conversation models is that maximum likelihood-based models are not optimized toward accomplishing goals.
MaskGAN: Better Text Generation via Filling in the _______
Neural autoregressive and seq2seq models that generate text by sampling words sequentially, with each word conditioned on the previous model, are state-of-the-art for several machine translation and summarization benchmarks.
Ranked #4 on
Multivariate Time Series Imputation
on PEMS-SF
Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost.
Who Said What: Modeling Individual Labelers Improves Classification
We also show that our method performs better than competing algorithms by Welinder and Perona (2010), and by Mnih and Hinton (2012).
Adversarial Training Methods for Semi-Supervised Text Classification
We extend adversarial and virtual adversarial training to the text domain by applying perturbations to the word embeddings in a recurrent neural network rather than to the original input itself.
Ranked #16 on
Sentiment Analysis
on IMDb
General Classification
Semi-Supervised Text Classification
+2
Generating Sentences from a Continuous Space
The standard recurrent neural network language model (RNNLM) generates sentences one word at a time and does not work from an explicit global sentence representation.
Semi-supervised Sequence Learning
In our experiments, we find that long short term memory recurrent networks after being pretrained with the two approaches are more stable and generalize better.
Document Embedding with Paragraph Vectors
Paragraph Vectors has been recently proposed as an unsupervised method for learning distributed representations for pieces of texts.
The supervised hierarchical Dirichlet process
However, until now, Hierarchical Dirichlet Process (HDP) mixtures have not seen significant use in supervised problems with grouped data since a straightforward application of the HDP on the grouped data results in learnt clusters that are not predictive of the responses.
