Search Results for author:
Found 31 papers, 12 papers with code
t-logistic regression
We extend logistic regression by using t-exponential families which were introduced recently in statistical physics.
t-divergence Based Approximate Inference
Approximate inference is an important technique for dealing with large, intractable graphical models based on the exponential family of distributions.
Construction of non-convex polynomial loss functions for training a binary classifier with quantum annealing
Quantum annealing is a heuristic quantum algorithm which exploits quantum resources to minimize an objective function embedded as the energy levels of a programmable physical system.
Bayesian Sampling Using Stochastic Gradient Thermostats
To remedy this problem, we show that one can leverage a small number of additional variables in order to stabilize momentum fluctuations induced by the unknown noise.
Probabilistic Label Relation Graphs with Ising Models
In this paper, we extend the HEX model to allow for soft or probabilistic relations between labels, which is useful when there is uncertainty about the relationship between two labels (e. g., an antelope is "sort of" furry, but not to the same degree as a grizzly bear).
Totally Corrective Boosting with Cardinality Penalization
If actual quantum optimization were to be used with this algorithm in the future, we would expect equivalent or superior results at much smaller time and energy costs during training.
Embedding Inference for Structured Multilabel Prediction
A key bottleneck in structured output prediction is the need for inference during training and testing, usually requiring some form of dynamic programming.
Characterizing Quantum Supremacy in Near-Term Devices
We study the task of sampling from the output distributions of (pseudo-)random quantum circuits, a natural task for benchmarking quantum computers.
Quantum Physics
Stochastic Gradient MCMC with Stale Gradients
In this paper we develop theory to show that while the bias and MSE of an SG-MCMC algorithm depend on the staleness of stochastic gradients, its estimation variance (relative to the expected estimate, based on a prescribed number of samples) is independent of it.
On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators
Our theoretical results show faster convergence rates and more accurate invariant measures for SG-MCMCs with higher-order integrators.
Building Large Machine Reading-Comprehension Datasets using Paragraph Vectors
We present a dual contribution to the task of machine reading-comprehension: a technique for creating large-sized machine-comprehension (MC) datasets using paragraph-vector models; and a novel, hybrid neural-network architecture that combines the representation power of recurrent neural networks with the discriminative power of fully-connected multi-layered networks.
Multilingual Word Embeddings using Multigraphs
We present a family of neural-network--inspired models for computing continuous word representations, specifically designed to exploit both monolingual and multilingual text.
Understanding Image and Text Simultaneously: a Dual Vision-Language Machine Comprehension Task
We introduce a new multi-modal task for computer systems, posed as a combined vision-language comprehension challenge: identifying the most suitable text describing a scene, given several similar options.
Cold-Start Reinforcement Learning with Softmax Policy Gradient
Policy-gradient approaches to reinforcement learning have two common and undesirable overhead procedures, namely warm-start training and sample variance reduction.
SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation
Supervised training of abstractive language generation models results in learning conditional probabilities over language sequences based on the supervised training signal.
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
We present a new dataset of image caption annotations, Conceptual Captions, which contains an order of magnitude more images than the MS-COCO dataset (Lin et al., 2014) and represents a wider variety of both images and image caption styles.
iqiyi Submission to ActivityNet Challenge 2019 Kinetics-700 challenge: Hierarchical Group-wise Attention
The standard non-local (NL) module is effective in aggregating frame-level features on the task of video classification but presents low parameters efficiency and high computational cost.
LOGAN: High-Performance GPU-Based X-Drop Long-Read Alignment
To highlight the impact of our work on a real-world application, we couple LOGAN with a many-to-many long-read alignment software called BELLA, and demonstrate that our implementation improves the overall BELLA runtime by up to 10. 6x.
Talking-Heads Attention
We introduce "talking-heads attention" - a variation on multi-head attention which includes linearprojections across the attention-heads dimension, immediately before and after the softmax operation. While inserting only a small number of additional parameters and a moderate amount of additionalcomputation, talking-heads attention leads to better perplexities on masked language modeling tasks, aswell as better quality when transfer-learning to language comprehension and question answering tasks.
Attention that does not Explain Away
Models based on the Transformer architecture have achieved better accuracy than the ones based on competing architectures for a large set of tasks.
TeaForN: Teacher-Forcing with N-grams
Sequence generation models trained with teacher-forcing suffer from issues related to exposure bias and lack of differentiability across timesteps.
Improving Text Generation Evaluation with Batch Centering and Tempered Word Mover Distance
Recent advances in automatic evaluation metrics for text have shown that deep contextualized word representations, such as those generated by BERT encoders, are helpful for designing metrics that correlate well with human judgements.
Multi-wavelength Selected Compton-thick AGNs in Chandra Deep Field-South Survey
We also find that CT AGNs have a higher Eddington ratio than non-CT AGNs, and that both CT AGNs and non-CT AGNs show similar properties of host galaxies.
Astrophysics of Galaxies High Energy Astrophysical Phenomena
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
The availability of large-scale image captioning and visual question answering datasets has contributed significantly to recent successes in vision-and-language pre-training.
Ranked #9 on
Image Captioning
on nocaps-val-out-domain
Do Transformer Modifications Transfer Across Implementations and Applications?
The research community has proposed copious modifications to the Transformer architecture since it was introduced over three years ago, relatively few of which have seen widespread adoption.
Bridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learning
Despite recent advances in its theoretical understanding, there still remains a significant gap in the ability of existing PAC-Bayesian theories on meta-learning to explain performance improvements in the few-shot learning setting, where the number of training examples in the target tasks is severely limited.
PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Tasks
With the increasing abundance of pretrained models in recent years, the problem of selecting the best pretrained checkpoint for a particular downstream classification task has been gaining increased attention.
Ranked #5 on
Transferability
on classification benchmark
All You May Need for VQA are Image Captions
Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation.
PaLI: A Jointly-Scaled Multilingual Language-Image Model
PaLI generates text based on visual and textual inputs, and with this interface performs many vision, language, and multimodal tasks, in many languages.
Improving Robust Generalization by Direct PAC-Bayesian Bound Minimization
Recent research in robust optimization has shown an overfitting-like phenomenon in which models trained against adversarial attacks exhibit higher robustness on the training set compared to the test set.
CausalLM is not optimal for in-context learning
Recent empirical evidence indicates that transformer based in-context learning performs better when using a prefix language model (prefixLM), in which in-context samples can all attend to each other, compared to causal language models (causalLM), which use auto-regressive attention that prohibits in-context samples to attend to future samples.
