Search Results for author:
Found 313 papers, 149 papers with code
Contextualized Role Interaction for Neural Machine Translation
Word inputs tend to be represented as single continuous vectors in deep neural networks.
Connecting the Dots: Event Graph Schema Induction with Path Language Modeling
Event schemas can guide our understanding and ability to make predictions with respect to what might happen next.
Log-Linear Reformulation of the Noisy Channel Model for Document-Level Neural Machine Translation
We seek to maximally use various data sources, such as parallel and monolingual data, to build an effective and efficient document-level translation system.
Black Box Causal Inference: Effect Estimation via Meta Prediction
Causal inference and the estimation of causal effects plays a central role in decision-making across many areas, including healthcare and economics.
An Overview of Large Language Models for Statisticians
Large Language Models (LLMs) have emerged as transformative tools in artificial intelligence (AI), exhibiting remarkable capabilities across diverse tasks such as text generation, reasoning, and decision-making.
NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions
Scaling reasoning capabilities beyond traditional domains such as math and coding is hindered by the lack of diverse and high-quality questions.
Meta-Statistical Learning: Supervised Learning of Statistical Inference
These tasks encompass statistical inference problems such as parameter estimation, hypothesis testing, or mutual information estimation.
Supervised Contrastive Block Disentanglement
We address this by learning two embeddings to independently represent the phenomena of interest and the spurious correlations.
Cost-Efficient Continual Learning with Sufficient Exemplar Memory
Continual learning (CL) research typically assumes highly constrained exemplar memory resources.
The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models
Decoder-only language models have the ability to dynamically switch between various computational tasks based on input prompts.
HERITAGE: An End-to-End Web Platform for Processing Korean Historical Documents in Hanja
First, anyone interested in these documents can get a general understanding from the model predictions and the interactive glossary, especially MT outputs in Korean and English.
How To Think About End-To-End Encryption and AI: Training, Processing, Disclosure, and Consent
End-to-end encryption (E2EE) has become the gold standard for securing communications, bringing strong confidentiality and privacy guarantees to billions of users worldwide.
HIST-AID: Leveraging Historical Patient Reports for Enhanced Multi-Modal Automatic Diagnosis
Chest X-ray imaging is a widely accessible and non-invasive diagnostic tool for detecting thoracic abnormalities.
Language Models as Causal Effect Generators
In particular, we define a procedure for turning any language model and any directed acyclic graph (DAG) into a sequence-driven structural causal model (SD-SCM).
Concept Bottleneck Language Models For protein design
We introduce Concept Bottleneck Protein Language Models (CB-pLM), a generative masked language model with a layer where each neuron corresponds to an interpretable concept.
Aioli: A Unified Optimization Framework for Language Model Data Mixing
Finally, we leverage the insights from our framework to derive a new online method named Aioli, which directly estimates the mixing law parameters throughout training and uses them to dynamically adjust proportions.
When Does Classical Chinese Help? Quantifying Cross-Lingual Transfer in Hanja and Kanbun
Historical and linguistic connections within the Sinosphere have led researchers to use Classical Chinese resources for cross-lingual transfer when processing historical documents from Korea and Japan.
Semiparametric conformal prediction
Many risk-sensitive applications require well-calibrated prediction sets over multiple, potentially correlated target variables, for which the prediction algorithm may report correlated errors.
Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Tasks
When combined with a novel preference learning loss, we find LLOME can not only learn to solve some Ehrlich functions, but can even outperform LaMBO-2 on moderately difficult Ehrlich variants.
Generalizing to any diverse distribution: uniformity, gentle finetuning and rebalancing
Various approaches like domain adaptation, domain generalization, and robust optimization attempt to address the out-of-distribution challenge by posing assumptions about the relation between training and test distribution.
Using Deep Autoregressive Models as Causal Inference Engines
We propose an autoregressive (AR) CI framework capable of handling complex confounders and sequential actions common in modern applications.
On the design space between molecular mechanics and machine learning force fields
A force field as accurate as quantum mechanics (QM) and as fast as molecular mechanics (MM), with which one can simulate a biomolecular system efficiently enough and meaningfully enough to get quantitative insights, is among the most ardent dreams of biophysicists -- a dream, nevertheless, not to be fulfilled any time soon.
Targeted Cause Discovery with Data-Driven Learning
We propose a novel machine learning approach for inferring causal variables of a target variable from observations.
Analysis of the ICML 2023 Ranking Data: Can Authors' Opinions of Their Own Papers Assist Peer Review in Machine Learning?
We focus on the Isotonic Mechanism, which calibrates raw review scores using author-provided rankings.
Non-convolutional Graph Neural Networks
Rethink convolution-based graph neural networks (GNN) -- they characteristically suffer from limited expressiveness, over-smoothing, and over-squashing, and require specialized sparse kernels for efficient computation.
$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs
Based on this observation, we revise the standard contrastive loss to explicitly encode how a sample relates to others.
Antibody DomainBed: Out-of-Distribution Generalization in Therapeutic Protein Design
The model predictions are used to determine which designs to evaluate in the lab, and the model is updated on the new measurements to inform the next cycle of decisions.
MentalAgora: A Gateway to Advanced Personalized Care in Mental Health through Multi-Agent Debating and Attribute Control
As mental health issues globally escalate, there is a tremendous need for advanced digital support systems.
Closed-Form Test Functions for Biophysical Sequence Optimization Algorithms
There is a growing body of work seeking to replicate the success of machine learning (ML) on domains like computer vision (CV) and natural language processing (NLP) to applications involving biophysical data.
Following Length Constraints in Instructions
Aligned instruction following models can better fulfill user requests than their unaligned counterparts.
Training Greedy Policy for Proposal Batch Selection in Expensive Multi-Objective Combinatorial Optimization
Active learning is increasingly adopted for expensive multi-objective combinatorial optimization problems, but it involves a challenging subset selection problem, optimizing the batch acquisition score that quantifies the goodness of a batch for evaluation.
A Progressive Risk Formulation for Enhanced Deep Learning based Total Knee Replacement Prediction in Knee Osteoarthritis
We developed deep learning models for predicting Total Knee Replacement (TKR) need within various time horizons in knee osteoarthritis patients, with a novel capability: the models can perform TKR prediction using a single scan, and furthermore when a previous scan is available, they leverage a progressive risk formulation to improve their predictions.
Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task
Transformers have revolutionized machine learning across diverse domains, yet understanding their behavior remains crucial, particularly in high-stakes applications.
Preference Learning Algorithms Do Not Learn Preference Rankings
Preference learning algorithms (e. g., RLHF and DPO) are frequently used to steer LLMs to produce generations that are more preferred by humans, but our understanding of their inner workings is still limited.
Implicitly Guided Design with PropEn: Match your Data to Follow the Gradient
Matching, combined with an encoder-decoder architecture, forms a domain-agnostic generative framework for property enhancement.
Jointly Modeling Inter- & Intra-Modality Dependencies for Multi-modal Learning
Previous studies in this field have concentrated on capturing in isolation either the inter-modality dependencies (the relationships between different modalities and the label) or the intra-modality dependencies (the relationships within a single modality and the label).
A Brief Introduction to Causal Inference in Machine Learning
This is a lecture note produced for DS-GA 3001. 003 "Special Topics in DS - Causal Inference in Machine Learning" at the Center for Data Science, New York University in Spring, 2024.
MR-Transformer: Vision Transformer for Total Knee Replacement Prediction Using Magnetic Resonance Imaging
A transformer-based deep learning model, MR-Transformer, was developed for total knee replacement (TKR) prediction using magnetic resonance imaging (MRI).
Estimation of Time-to-Total Knee Replacement Surgery
A survival analysis model for predicting time-to-total knee replacement (TKR) was developed using features from medical images and clinical measurements.
Generalization Measures for Zero-Shot Cross-Lingual Transfer
A model's capacity to generalize its knowledge to interpret unseen inputs with different characteristics is crucial to build robust and reliable machine learning systems.
HyperCLOVA X Technical Report
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding.
Hyperparameters in Continual Learning: A Reality Check
Based on this, we propose a revised two-phase evaluation protocol consisting of a hyperparameter tuning phase and an evaluation phase.
Self-Rewarding Language Models
We posit that to achieve superhuman agents, future models require superhuman feedback in order to provide an adequate training signal.
Let's Go Shopping (LGS) -- Web-Scale Image-Text Dataset for Visual Concept Understanding
Vision and vision-language applications of neural networks, such as image classification and captioning, rely on large-scale annotated datasets that require non-trivial data-collecting processes.
Perspectives on the State and Future of Deep Learning -- 2023
The goal of this series is to chronicle opinions and issues in the field of machine learning as they stand today and as they change over time.
Show Your Work with Confidence: Confidence Bands for Tuning Curves
We present the first method to construct valid confidence bands for tuning curves.
First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models
Many NLP researchers are experiencing an existential crisis triggered by the astonishing success of ChatGPT and other systems based on large language models (LLMs).
xVal: A Continuous Numerical Tokenization for Scientific Language Models
Due in part to their discontinuous and discrete default encodings for numbers, Large Language Models (LLMs) have not yet been commonly used to process numerically-dense scientific datasets.
Multiple Physics Pretraining for Physical Surrogate Models
We introduce multiple physics pretraining (MPP), an autoregressive task-agnostic pretraining approach for physical surrogate modeling of spatiotemporal systems with transformers.
AstroCLIP: A Cross-Modal Foundation Model for Galaxies
These embeddings can then be used - without any model fine-tuning - for a variety of downstream tasks including (1) accurate in-modality and cross-modality semantic similarity search, (2) photometric redshift estimation, (3) galaxy property estimation from both images and spectra, and (4) morphology classification.
Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs
Most interpretability research in NLP focuses on understanding the behavior and features of a fully trained model.
Blind Biological Sequence Denoising with Self-Supervised Set Learning
This set embedding represents the "average" of the subreads and can be decoded into a prediction of the clean sequence.
Active and Passive Causal Inference Learning
This paper serves as a starting point for machine learning researchers, engineers and students who are interested in but not yet familiar with causal inference.
Latent State Models of Training Dynamics
We use the HMM representation to study phase transitions and identify latent "detour" states that slow down convergence.
Improving Joint Speech-Text Representations Without Alignment
The last year has seen astonishing progress in text-prompted image generation premised on the idea of a cross-modal representation space in which the text and image domains are represented jointly.
AbDiffuser: Full-Atom Generation of in vitro Functioning Antibodies
We introduce AbDiffuser, an equivariant and physics-informed diffusion model for the joint generation of antibody 3D structures and sequences.
Leveraging Implicit Feedback from Deployment Data in Dialogue
We study improving social conversational agents by learning from natural dialogue between users and a deployed model, without extra annotations.
Making the Most Out of the Limited Context Length: Predictive Power Varies with Clinical Note Type and Note Section
Recent advances in large language models have led to renewed interest in natural language processing in healthcare using the free text of clinical notes.
System-Level Natural Language Feedback
Natural language (NL) feedback offers rich insights into user experience.
On Sensitivity and Robustness of Normalization Schemes to Input Distribution Shifts in Automatic MR Image Diagnosis
However, the image reconstruction process within the MRI pipeline, which requires the use of complex hardware and adjustment of a large number of scanner parameters, is highly susceptible to noise of various forms, resulting in arbitrary artifacts within the images.
Regularizing with Pseudo-Negatives for Continual Self-Supervised Learning
We introduce a novel Pseudo-Negative Regularization (PNR) framework for effective continual self-supervised learning (CSSL).
Ranked #1 on
Image Classification
on ImageNet-100 (Class-IL, 5T)
Protein Discovery with Discrete Walk-Jump Sampling
We resolve difficulties in training and sampling from a discrete generative model by learning a smoothed energy function, sampling from the smoothed data manifold with Langevin Markov chain Monte Carlo (MCMC), and projecting back to the true data manifold with one-step denoising.
BOtied: Multi-objective Bayesian optimization with tied multivariate ranks
Motivated by this link, we propose the Pareto-compliant CDF indicator and the associated acquisition function, BOtied.
Protein Design with Guided Discrete Diffusion
A popular approach to protein design is to combine a generative model with a discriminative model for conditional sampling.
Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs
Large language models (LLMs) have achieved widespread success on a variety of in-context few-shot tasks, but this success is typically evaluated via correctness rather than consistency.
Towards Understanding and Improving GFlowNet Training
We investigate how to learn better flows, and propose (i) prioritized replay training of high-reward $x$, (ii) relative edge flow policy parametrization, and (iii) a novel guided trajectory balance objective, and show how it can solve a substructure credit assignment problem.
A Comparison of Semi-Supervised Learning Techniques for Streaming ASR at Scale
Unpaired text and audio injection have emerged as dominant methods for improving ASR performance in the absence of a large labeled corpus.
Training Language Models with Language Feedback at Scale
Third, finetuning the language model to maximize the likelihood of the chosen refinement given the input.
Improving Code Generation by Training with Natural Language Feedback
The potential for pre-trained large language models (LLMs) to use natural language feedback at inference time has been an exciting recent development.
Unsupervised Learning of Initialization in Deep Neural Networks via Maximum Mean Discrepancy
We first notice that each parameter configuration in the parameter space corresponds to one particular downstream task of d-way classification.
Dual Learning for Large Vocabulary On-Device ASR
Dual learning is a paradigm for semi-supervised machine learning that seeks to leverage unsupervised data by solving two opposite tasks at once.
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation
In this work, we explore a useful but often neglected methodology for robustness analysis of text generation evaluation metrics: stress tests with synthetic data.
Can Current Task-oriented Dialogue Models Automate Real-world Scenarios in the Wild?
Task-oriented dialogue (TOD) systems are mainly based on the slot-filling-based TOD (SF-TOD) framework, in which dialogues are broken down into smaller, controllable units (i. e., slots) to fulfill a specific task.
Joint Embedding Predictive Architectures Focus on Slow Features
Many common methods for learning a world model for pixel-based environments use generative architectures trained with pixel-level reconstruction objectives.
Language Model Classifier Aligns Better with Physician Word Sensitivity than XGBoost on Readmission Prediction
We assess the sensitivity score on a set of representative words in the test set using two classifiers trained for hospital readmission classification with similar performance statistics.
A Pareto-optimal compositional energy-based model for sampling and optimization of protein sequences
Deep generative models have emerged as a popular machine learning-based approach for inverse design problems in the life sciences.
HUE: Pretrained Model and Dataset for Understanding Hanja Documents of Ancient Korea
Historical records in Korea before the 20th century were primarily written in Hanja, an extinct language based on Chinese characters and not understood by modern Korean or Chinese speakers.
PropertyDAG: Multi-objective Bayesian optimization of partially ordered, mixed-variable properties for biological sequence design
Bayesian optimization offers a sample-efficient framework for navigating the exploration-exploitation trade-off in the vast design space of biological sequences.
A Non-monotonic Self-terminating Language Model
We then propose a non-monotonic self-terminating language model, which significantly relaxes the constraint of monotonically increasing termination probability in the originally proposed self-terminating language model by Welleck et al. (2020), to address the issue of non-terminating sequences when using incomplete probable decoding algorithms.
Towards Disentangled Speech Representations
In this paper, we construct a representation learning task based on joint modeling of ASR and TTS, and seek to learn a representation of audio that disentangles that part of the speech signal that is relevant to transcription from that part which is not.
Predicting Out-of-Domain Generalization with Neighborhood Invariance
Developing and deploying machine learning models safely depends on the ability to characterize and compare their abilities to generalize to new environments.
Endowing Language Models with Multimodal Knowledge Graph Representations
We use the recently released VisualSem KG as our external knowledge repository, which covers a subset of Wikipedia and WordNet entities, and compare a mix of tuple-based and graph-based algorithms to learn entity and relation representations that are grounded on the KG multimodal information.
Multilingual Named Entity Recognition
named-entity-recognition
+2
Linear Connectivity Reveals Generalization Strategies
It is widely accepted in the mode connectivity literature that when two neural networks are trained similarly on the same data, they are connected by a path through parameter space over which test set accuracy is maintained.
Translating Hanja Historical Documents to Contemporary Korean and English
Built on top of multilingual neural machine translation, H2KE learns to translate a historical document written in Hanja, from both a full dataset of outdated Korean translation and a small dataset of more recently translated contemporary Korean and English.
Multi-segment preserving sampling for deep manifold sampler
We introduce an alternative approach to this guided sampling procedure, multi-segment preserving sampling, that enables the direct inclusion of domain-specific knowledge by designating preserved and non-preserved segments along the input sequence, thereby restricting variation to only select regions.
Training Language Models with Language Feedback
We learn from language feedback on model outputs using a three-step learning algorithm.
On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model
Many recent studies on large-scale language models have reported successful in-context zero- and few-shot learning ability.
Translation between Molecules and Natural Language
We present $\textbf{MolT5}$ $-$ a self-supervised learning framework for pretraining models on a vast amount of unlabeled natural language text and molecule strings.
Ranked #2 on
Molecule Captioning
on L+M-24
Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks
We propose an algorithm to balance the conditional learning speeds between modalities during training and demonstrate that it indeed addresses the issue of greedy learning.
Causal Scene BERT: Improving object detection by searching for challenging groups of data
Our main contribution is a pseudo-automatic method to discover such groups in foresight by performing causal interventions on simulated scenes.
Generative multitask learning mitigates target-causing confounding
We propose generative multitask learning (GMTL), a simple and scalable approach to causal representation learning for multitask learning.
LINDA: Unsupervised Learning to Interpolate in Natural Language Processing
Despite the success of mixup in data augmentation, its applicability to natural language processing (NLP) tasks has been limited due to the discrete and variable-length nature of natural languages.
Amortized Noisy Channel Neural Machine Translation
For all three approaches, the generated translations fail to achieve rewards comparable to BSR, but the translation quality approximated by BLEU and BLEURT is similar to the quality of BSR-produced translations.
Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling
From these observations, we conclude that the high degree of oversmoothing is the main reason behind the degenerate case of overly probable short sequences in a neural autoregressive model.
Causal Effect Variational Autoencoder with Uniform Treatment
In this paper, we introduce uniform treatment variational autoencoders (UTVAE) that are trained with uniform treatment distribution using importance sampling and show that using uniform treatment over observational treatment distribution leads to better causal inference by mitigating the distribution shift that occurs from training to test time.
DEEP: DEnoising Entity Pre-training for Neural Machine Translation
It has been shown that machine translation models usually generate poor translations for named entities that are infrequent in the training corpus.
AlphaD3M: Machine Learning Pipeline Synthesis
We introduce AlphaD3M, an automatic machine learning (AutoML) system based on meta reinforcement learning using sequence models with self play.
Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinement
Recent work in simultaneous machine translation is often trained with conventional full sentence translation corpora, leading to either excessive latency or necessity to anticipate as-yet-unarrived words, when dealing with a language pair whose word orders significantly differ.
AAVAE: Augmentation-Augmented Variational Autoencoders
We empirically evaluate the proposed AAVAE on image classification, similar to how recent contrastive and non-contrastive learning algorithms have been evaluated.
Recognizing and overcoming the greedy nature of learning in multi-modal deep neural networks
We refer to this gain as the conditional utilization rate of the modality.
Causal Scene BERT: Improving object detection by searching for challenging groups
We verify that the prioritized groups found via intervention are challenging for the object detector and show that retraining with data collected from these groups helps inordinately compared to adding more IID data.
Chemical-Reaction-Aware Molecule Representation Learning
Molecule representation learning (MRL) methods aim to embed molecules into a real vector space.
Stereo Video Reconstruction Without Explicit Depth Maps for Endoscopic Surgery
In the first reader study, a variant of the U-Net that takes as input multiple consecutive video frames and outputs the missing view performs best.
An Empirical Study on Few-shot Knowledge Probing for Pretrained Language Models
Prompt-based knowledge probing for 1-hop relations has been used to measure how much world knowledge is stored in pretrained language models.
Meta-repository of screening mammography classifiers
Artificial intelligence (AI) is showing promise in improving clinical diagnosis.
AASAE: Augmentation-Augmented Stochastic Autoencoders
We empirically evaluate the proposed AASAE on image classification, similar to how recent contrastive and non-contrastive learning algorithms have been evaluated.
Mode recovery in neural autoregressive sequence modeling
We propose to study these phenomena by investigating how the modes, or local maxima, of a distribution are maintained throughout the full learning chain of the ground-truth, empirical, learned and decoding-induced distributions, via the newly proposed mode recovery cost.
Comparing Test Sets with Item Response Theory
Recent years have seen numerous NLP datasets introduced to evaluate the performance of fine-tuned models on natural language understanding tasks.
True Few-Shot Learning with Language Models
Here, we evaluate the few-shot ability of LMs when such held-out examples are unavailable, a setting we call true few-shot learning.
KLUE: Korean Language Understanding Evaluation
We introduce Korean Language Understanding Evaluation (KLUE) benchmark.
The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction
We introduce a new concept of Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations.
Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models
We find that mix-review effectively regularizes the finetuning process, and the forgetting problem is alleviated to some extent.
NaturalProofs: Mathematical Theorem Proving in Natural Language
Understanding and creating mathematics using natural mathematical language - the mixture of symbolic and natural language used by humans - is a challenging and important problem for driving progress in machine learning.
Rissanen Data Analysis: Examining Dataset Characteristics via Description Length
We introduce a method to determine if a certain capability helps to achieve an accurate model of given data.
Online hyperparameter optimization by real-time recurrent learning
Conventional hyperparameter optimization methods are computationally intensive and hard to generalize to scenarios that require dynamically adapting hyperparameters, such as life-long learning.
Self-Supervised Equivariant Scene Synthesis from Video
We propose a self-supervised framework to learn scene representations from video that are automatically delineated into background, characters, and their animations.
Decoupled Exploration and Exploitation Policies for Sample-Efficient Reinforcement Learning
This causes BBE to be actively detrimental to policy learning in many control tasks.
Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization
The early phase of training a deep neural network has a dramatic effect on the local curvature of the loss function.
A Study on the Autoregressive and non-Autoregressive Multi-label Learning
These tasks are hard because the label space is usually (i) very large, e. g. thousands or millions of labels, (ii) very sparse, i. e. very few labels apply to each input document, and (iii) highly correlated, meaning that the existence of one label changes the likelihood of predicting all other labels.
Differences between human and machine perception in medical diagnosis
We compare the two with respect to their robustness to Gaussian low-pass filtering, performing a subgroup analysis on microcalcifications and soft tissue lesions.
Learned Equivariant Rendering without Transformation Supervision
We propose a self-supervised framework to learn scene representations from video that are automatically delineated into objects and background.
Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning
The interaction of conversational systems with users poses an exciting opportunity for improving them after deployment, but little evidence has been provided of its feasibility.
Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search
We then conduct a multi-objective evolutionary search to find a length configuration that maximizes the accuracy and minimizes the efficiency metric under any given computational budget.
SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness
Models that perform well on a training domain often fail to generalize to out-of-domain (OOD) examples.
Reducing false-positive biopsies with deep neural networks that utilize local and global information in screening mammograms
Breast cancer is the most common cancer in women, and hundreds of thousands of unnecessary biopsies are done around the world at a tremendous cost.
Generative Language-Grounded Policy in Vision-and-Language Navigation with Bayes' Rule
Vision-and-language navigation (VLN) is a task in which an agent is embodied in a realistic 3D environment and follows an instruction to reach the goal node.
Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation
Given a continuous latent variable model for machine translation (Shu et al., 2020), we train an inference network to approximate the gradient of the marginal log probability of the target sentence, using only the latent variable as input.
Evaluating representations by the complexity of learning low-loss predictors
We consider the problem of evaluating representations of data for use in solving a downstream task.
A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation
On the other hand, we find that the set of sampling algorithms that satisfies these properties performs on par with the existing sampling algorithms.
A Framework For Contrastive Self-Supervised Learning And Designing A New Approach
Contrastive self-supervised learning (CSL) is an approach to learn useful representations by solving a pretext task that selects and compares anchor, negative and positive (APN) features from an unlabeled dataset.
Ranked #25 on
Image Classification
on STL-10
VisualSem: A High-quality Knowledge Graph for Vision and Language
We also release a neural multi-modal retrieval model that can use images or sentences as inputs and retrieves entities in the KG.
AdapterHub: A Framework for Adapting Transformers
We propose AdapterHub, a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages.
Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset
We present Covidex, a search engine that exploits the latest neural ranking models to provide information access to the COVID-19 Open Research Dataset curated by the Allen Institute for AI.
Compositionality and Capacity in Emergent Languages
Our hypothesis is that there should be a specific range of model capacity and channel bandwidth that induces compositional structure in the resulting language and consequently encourages systematic generalization.
Rapidly Deploying a Neural Search Engine for the COVID-19 Open Research Dataset
The Neural Covidex is a search engine that exploits the latest neural ranking architectures to provide information access to the COVID-19 Open Research Dataset (CORD-19) curated by the Allen Institute for AI.
MLE-guided parameter search for task loss minimization in neural sequence modeling
Typical approaches to directly optimizing the task loss such as policy gradient and minimum risk training are based around sampling in the sequence space to obtain candidate update directions that are scored based on the loss of a single sequence.
AdapterFusion: Non-Destructive Task Composition for Transfer Learning
We show that by separating the two stages, i. e., knowledge extraction and knowledge composition, the classifier can effectively exploit the representations learned from multiple tasks in a non-destructive manner.
Learning Non-Monotonic Automatic Post-Editing of Translations from Human Orderings
Recent research in neural machine translation has explored flexible generation orders, as an alternative to left-to-right generation.
Learning to Learn Morphological Inflection for Resource-Poor Languages
We propose to cast the task of morphological inflection - mapping a lemma to an indicated inflected form - for resource-poor languages as a meta-learning problem.
Rapidly Bootstrapping a Question Answering Dataset for COVID-19
We present CovidQA, the beginnings of a question answering dataset specifically designed for COVID-19, built by hand from knowledge gathered from Kaggle's COVID-19 Open Research Dataset Challenge.
Rapidly Deploying a Neural Search Engine for the COVID-19 Open Research Dataset: Preliminary Thoughts and Lessons Learned
We present the Neural Covidex, a search engine that exploits the latest neural ranking architectures to provide information access to the COVID-19 Open Research Dataset curated by the Allen Institute for AI.
Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
QAGS is based on the intuition that if we ask questions about a summary and its source, we will receive similar answers if the summary is factually consistent with the source.
Understanding the robustness of deep neural network classifiers for breast cancer screening
Deep neural networks (DNNs) show promise in breast cancer screening, but their robustness to input perturbations must be better understood before they can be clinically implemented.
Unsupervised Question Decomposition for Question Answering
We aim to improve question answering (QA) by decomposing hard questions into simpler sub-questions that existing QA systems are capable of answering.
The Break-Even Point on Optimization Trajectories of Deep Neural Networks
We argue for the existence of the "break-even" point on this trajectory, beyond which the curvature of the loss surface and noise in the gradient are implicitly regularized by SGD.
On the Discrepancy between Density Estimation and Sequence Generation
In this paper, by comparing several density estimators on five machine translation tasks, we find that the correlation between rankings of models based on log-likelihood and BLEU varies significantly depending on the range of the model families being compared.
An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization
In this work, we extend the globally-aware multiple instance classifier, a framework we proposed to address these unique properties of medical images.
Consistency of a Recurrent Language Model With Respect to Incomplete Decoding
Despite strong performance on a variety of tasks, neural sequence models trained with maximum likelihood have been shown to exhibit issues such as length bias and degenerate repetition.
Improving the Ability of Deep Neural Networks to Use Information From Multiple Views in Breast Cancer Screening
In breast cancer screening, radiologists make the diagnosis based on images that are taken from two angles.
Navigation-Based Candidate Expansion and Pretrained Language Models for Citation Recommendation
Citation recommendation systems for the scientific literature, to help authors find papers that should be cited, have the potential to speed up discoveries and uncover new routes for scientific exploration.
Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training
Generative dialogue models currently suffer from a number of problems which standard maximum likelihood training does not address.
Neural Unsupervised Parsing Beyond English
Recently, neural network models which automatically infer syntactic structure from raw text have started to achieve promising results.
Finding Generalizable Evidence by Learning to Convince Q\&A Models
We propose a system that finds the strongest supporting evidence for a given answer to a question, using passage-based question-answering (QA) as a testbed.
Multi-Stage Document Ranking with BERT
The advent of deep neural networks pre-trained via language modeling tasks has spurred a number of successful applications in natural language processing.
Capacity, Bandwidth, and Compositionality in Emergent Language Learning
In this paper, we investigate the learning biases that affect the efficacy and compositionality of emergent languages.
Analyzing the Forgetting Problem in the Pretrain-Finetuning of Dialogue Response Models
We find that mix-review effectively regularizes the finetuning process, and the forgetting problem is alleviated to some extent.
Generalized Inner Loop Meta-Learning
Many (but not all) approaches self-qualifying as "meta-learning" in deep learning and reinforcement learning fit a common pattern of approximating the solution to a nested optimization problem.
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models
We empirically evaluate the proposed mixout and its variants on finetuning a pretrained language model on downstream tasks.
Inducing Constituency Trees through Neural Machine Translation
Latent tree learning(LTL) methods learn to parse sentences using only indirect supervision from a downstream task.
Finding Generalizable Evidence by Learning to Convince Q&A Models
We propose a system that finds the strongest supporting evidence for a given answer to a question, using passage-based question-answering (QA) as a testbed.
Evaluating biological plausibility of learning algorithms the lazy way
To which extent can successful machine learning inform our understanding of biological learning?
Countering Language Drift via Visual Grounding
Emergent multi-agent communication protocols are very different from natural language and not easily interpretable by humans.
Neural Machine Translation with Byte-Level Subwords
Representing text at the level of bytes and using the 256 byte set as vocabulary is a potential solution to this issue.
Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Set
Here, we aim to answer the following questions: Does using a development set for early stopping in the low-resource setting influence results as compared to a more realistic alternative, where the number of training epochs is tuned on development languages?
Dynamics-aware Embeddings
In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL).
Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior
By decoding multiple initial latent variables in parallel and rescore using a teacher model, the proposed model further brings the gap down to 1. 0 BLEU point on WMT'14 En-De task with 6. 8x speedup.
Neural Text Generation with Unlikelihood Training
Neural text generation is a key tool in natural language applications, but it is well known there are major problems at its core.
Improving localization-based approaches for breast cancer screening exam classification
We trained and evaluated a localization-based deep CNN for breast cancer screening exam classification on over 200, 000 exams (over 1, 000, 000 images).
Screening Mammogram Classification with Prior Exams
Radiologists typically compare a patient's most recent breast cancer screening exam to their previous ones in making informed diagnoses.
Can Unconditional Language Models Recover Arbitrary Sentences?
We then investigate the conditions under which a language model can be made to generate a sentence through the identification of a point in such a space and find that it is possible to recover arbitrary sentences nearly perfectly with language models and representations of moderate size without modifying any model parameters.
A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks
We present a framework for compactly summarizing many recent results in efficient and/or biologically plausible online training of recurrent neural networks (RNN).
Generating Diverse Translations with Sentence Codes
In this work, we attempt to obtain diverse translations by using sentence codes to condition the sentence generation.
Deep Unsupervised Drum Transcription
We introduce DrummerNet, a drum transcription system that is trained in an unsupervised manner.
Sound Audio and Speech Processing
Globally-Aware Multiple Instance Classifier for Breast Cancer Screening
Moreover, both the global structure and local details play important roles in medical image analysis tasks.
Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations
Zero-shot translation, translating between language pairs on which a Neural Machine Translation (NMT) system has never been trained, is an emergent property when training the system in multilingual settings.
Multi-Turn Beam Search for Neural Dialogue Modeling
We propose a novel approach for conversation-level inference by explicitly modeling the dialogue partner and running beam search across multiple conversation turns.
A Generalized Framework of Sequence Generation with Application to Undirected Sequence Models
We investigate this problem by proposing a generalized model of sequence generation that unifies decoding in directed and undirected models.
Using local plasticity rules to train recurrent neural networks
To learn useful dynamics on long time scales, neurons must use plasticity rules that account for long-term, circuit-wide effects of synaptic changes.
Sequential Graph Dependency Parser
We propose a method for non-projective dependency parsing by incrementally predicting a set of edges.
Automatic Machine Learning by Pipeline Synthesis using Model-Based Reinforcement Learning and a Grammar
Automatic machine learning is an important problem in the forefront of machine learning.
Task-Driven Data Verification via Gradient Descent
We introduce a novel algorithm for the detection of possible sample corruption such as mislabeled samples in a training dataset given a small clean validation set.
Gradient-based learning for F-measure and other performance metrics
Many important classification performance metrics, e. g. $F$-measure, are non-differentiable and non-decomposable, and are thus unfriendly to gradient descent algorithm.
Backplay: 'Man muss immer umkehren'
Our contributions are that we analytically characterize the types of environments where Backplay can improve training speed, demonstrate the effectiveness of Backplay both in large grid worlds and a complex four player zero-sum game (Pommerman), and show that Backplay compares favorably to other competitive methods known to improve sample efficiency.
Advancing GraphSAGE with A Data-Driven Node Sampling
As an efficient and scalable graph neural network, GraphSAGE has enabled an inductive capability for inferring unseen nodes or graphs by aggregating subsampled local neighborhoods and by learning in a mini-batch gradient descent fashion.
Document Expansion by Query Prediction
One technique to improve the retrieval effectiveness of a search engine is to expand documents with terms that are related or representative of the documents' content. From the perspective of a question answering system, this might comprise questions the document can potentially answer.
Ranked #1 on
Passage Re-Ranking
on TREC-PM
Molecular geometry prediction using a deep generative graph neural network
Conventional conformation generation methods minimize hand-designed molecular force field energy functions that are often not well correlated with the true energy function of a molecule observed in nature.
Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening
We present a deep convolutional neural network for breast cancer screening exam classification, trained and evaluated on over 200, 000 exams (over 1, 000, 000 images).
Context-Aware Learning for Neural Machine Translation
By comparing performance using actual and random contexts, we show that a model trained with the proposed algorithm is more sensitive to the additional context.
Continual Learning via Neural Pruning
We introduce Continual Learning via Neural Pruning (CLNP), a new method aimed at lifelong learning in fixed capacity models based on neuronal model sparsification.
Augmentation for small object detection
We evaluate different pasting augmentation strategies, and ultimately, we achieve 9. 7\% relative improvement on the instance segmentation and 7. 1\% on the object detection of small objects, compared to the current state of the art method on
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model
We show that BERT (Devlin et al., 2018) is a Markov random field language model.
Non-Monotonic Sequential Text Generation
Standard sequential generation methods assume a pre-specified generation order, such as text generation methods which generate words from left to right.
Insertion-based Decoding with automatically Inferred Generation Order
Conventional neural autoregressive decoding commonly assumes a fixed left-to-right generation order, which may be sub-optimal.
Emergent Linguistic Phenomena in Multi-Agent Communication Games
In this work, we propose a computational framework in which agents equipped with communication capabilities simultaneously play a series of referential games, where agents are trained using deep reinforcement learning.
Passage Re-ranking with BERT
Recently, neural models pretrained on a language modeling task, such as ELMo (Peters et al., 2017), OpenAI GPT (Radford et al., 2018), and BERT (Devlin et al., 2018), have achieved impressive results on various natural language processing tasks such as question-answering and natural language inference.
Ranked #3 on
Passage Re-Ranking
on MS MARCO
(using extra training data)
Importance of Search and Evaluation Strategies in Neural Dialogue Modeling
We investigate the impact of search strategies in neural dialogue modeling.
Conditional Word Embedding and Hypothesis Testing via Bayes-by-Backprop
Conventional word embedding models do not leverage information from document meta-data, and they do not model uncertainty.
