Search Results for author:
Found 42 papers, 21 papers with code
The supervised hierarchical Dirichlet process
However, until now, Hierarchical Dirichlet Process (HDP) mixtures have not seen significant use in supervised problems with grouped data since a straightforward application of the HDP on the grouped data results in learnt clusters that are not predictive of the responses.
Document Embedding with Paragraph Vectors
Paragraph Vectors has been recently proposed as an unsupervised method for learning distributed representations for pieces of texts.
Semi-supervised Sequence Learning
In our experiments, we find that long short term memory recurrent networks after being pretrained with the two approaches are more stable and generalize better.
Generating Sentences from a Continuous Space
The standard recurrent neural network language model (RNNLM) generates sentences one word at a time and does not work from an explicit global sentence representation.
Adversarial Training Methods for Semi-Supervised Text Classification
We extend adversarial and virtual adversarial training to the text domain by applying perturbations to the word embeddings in a recurrent neural network rather than to the original input itself.
Ranked #21 on
Sentiment Analysis
on IMDb
General Classification
Semi-Supervised Text Classification
+2
Who Said What: Modeling Individual Labelers Improves Classification
We also show that our method performs better than competing algorithms by Welinder and Perona (2010), and by Mnih and Hinton (2012).
Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost.
MaskGAN: Better Text Generation via Filling in the _______
Neural autoregressive and seq2seq models that generate text by sampling words sequentially, with each word conditioned on the previous model, are state-of-the-art for several machine translation and summarization benchmarks.
Ranked #4 on
Multivariate Time Series Imputation
on PEMS-SF
A Goal-oriented Neural Conversation Model by Self-Play
One challenge in applying such techniques to building goal-oriented conversation models is that maximum likelihood-based models are not optimized toward accomplishing goals.
MaskGAN: Better Text Generation via Filling in the______
Additionally, these models are typically trained via maxi- mum likelihood and teacher forcing.
Scalable and accurate deep learning for electronic health records
Predictive modeling with electronic health record (EHR) data is anticipated to drive personalized medicine and improve healthcare quality.
Learning Longer-term Dependencies in RNNs with Auxiliary Losses
Despite recent advances in training recurrent neural networks (RNNs), capturing long-term dependencies in sequences remains a fundamental challenge.
Ranked #13 on
Sequential Image Classification
on Sequential CIFAR-10
Embedding Text in Hyperbolic Spaces
Ideally, we could incorporate our prior knowledge of this hierarchical structure into unsupervised learning algorithms that work on text data.
Peptide-Spectra Matching from Weak Supervision
As in many other scientific domains, we face a fundamental problem when using machine learning to identify proteins from mass spectrometry data: large ground truth datasets mapping inputs to correct outputs are extremely difficult to obtain.
Music Transformer
This is impractical for long sequences such as musical compositions since their memory complexity for intermediate relative information is quadratic in the sequence length.
Ranked #3 on
Music Modeling
on JSB Chorales
Gmail Smart Compose: Real-Time Assisted Writing
In this paper, we present Smart Compose, a novel system for generating interactive, real-time suggestions in Gmail that assists users in writing mails by reducing repetitive typing.
Natural Questions: a Benchmark for Question Answering Research
The public release consists of 307, 373 training examples with single annotations, 7, 830 examples with 5-way annotations for development data, and a further 7, 842 examples 5-way annotated sequestered as test data.
Ranked #7 on
Question Answering
on Natural Questions (long)
Analyzing the Role of Model Uncertainty for Electronic Health Records
We further show that RNNs with only Bayesian embeddings can be a more efficient way to capture model uncertainty compared to ensembles, and we analyze how model uncertainty is impacted across individual input features and patient subgroups.
Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer
A recent study showed that using the graphical structure underlying EHR data (e. g. relationship between diagnoses and treatments) improves the performance of prediction tasks such as heart failure prediction.
Improved Hierarchical Patient Classification with Language Model Pretraining over Clinical Notes
Clinical notes in electronic health records contain highly heterogeneous writing styles, including non-standard terminology or abbreviations.
Learning an Adaptive Learning Rate Schedule
The learning rate is one of the most important hyper-parameters for model training and generalization.
Capacity, Bandwidth, and Compositionality in Emergent Language Learning
In this paper, we investigate the learning biases that affect the efficacy and compositionality of emergent languages.
Federated and Differentially Private Learning for Electronic Health Records
The use of collaborative and decentralized machine learning techniques such as federated learning have the potential to enable the development and deployment of clinical risk predictions models in low-resource settings without requiring sensitive data be shared or stored in a central repository.
Modelling EHR timeseries by restricting feature interaction
Time series data are prevalent in electronic health records, mostly in the form of physiological parameters such as vital signs and lab tests.
Flow Contrastive Estimation of Energy-Based Models
(2) The update of the flow model approximately minimizes the Jensen-Shannon divergence between the flow model and the data distribution.
Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss
When training the parameters of a linear dynamical model, the gradient descent algorithm is likely to fail to converge if the squared-error loss is used as the training loss function.
Training independent subnetworks for robust prediction
Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network.
Learnability and Complexity of Quantum Samples
Given a quantum circuit, a quantum computer can sample the output distribution exponentially faster in the number of bits than classical computers.
Learning to Select Best Forecast Tasks for Clinical Outcome Prediction
The paradigm of pretraining' from a set of relevant auxiliary tasks and thenfinetuning' on a target task has been successfully applied in many different domains.
MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records
One important challenge of applying deep learning to electronic health records (EHR) is the complexity of their multimodal structure.
BEDS-Bench: Behavior of EHR-models under Distributional Shift--A Benchmark
Most ML approaches focus on generalization performance on unseen data that are similar to the training data (In-Distribution, or IND).
Finetuned Language Models Are Zero-Shot Learners
We show that instruction tuning -- finetuning language models on a collection of tasks described via instructions -- substantially improves zero-shot performance on unseen tasks.
Ranked #1 on
Question Answering
on OBQA
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Scaling language models with more data, compute and parameters has driven significant progress in natural language processing.
Ranked #10 on
Language Modelling
on LAMBADA
Co-training Transformer with Videos and Images Improves Action Recognition
We term this approach as Co-training Videos and Images for Action Recognition (CoVeR).
Ranked #8 on
Action Classification
on MiT
(using extra training data)
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Coreference Resolution
on Winograd Schema Challenge
Mind's Eye: Grounded Language Model Reasoning through Simulation
By training solely on written text, current language models (LMs) miss the grounded experience of humans in the real-world -- their failure to relate language to the physical world causes knowledge to be misrepresented and obvious mistakes in their reasoning.
Massively Multilingual Shallow Fusion with Large Language Models
In this work, we propose to train a single multilingual language model (LM) for shallow fusion in multiple languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
PaLM 2 Technical Report
Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM.
Ranked #1 on
Question Answering
on StrategyQA
Training Socially Aligned Language Models on Simulated Social Interactions
Social alignment in AI systems aims to ensure that these models behave according to established societal values.
Order Matters in the Presence of Dataset Imbalance for Multilingual Learning
In this paper, we empirically study the optimization dynamics of multi-task learning, particularly focusing on those that govern a collection of tasks with significant data imbalance.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
Best Practices and Lessons Learned on Synthetic Data for Language Models
The success of AI models relies on the availability of large, diverse, and high-quality datasets, which can be challenging to obtain due to data scarcity, privacy concerns, and high costs.
