Search Results for author:
Found 24 papers, 8 papers with code
The Impact of Positional Encoding on Length Generalization in Transformers
In this paper, we conduct a systematic empirical study comparing the length generalization performance of decoder-only Transformers with five different position encoding approaches including Absolute Position Embedding (APE), T5's Relative PE, ALiBi, and Rotary, in addition to Transformers without positional encoding (NoPE).
Auditing and Generating Synthetic Data with Controllable Trust Trade-offs
We present an auditing framework that offers a holistic assessment of synthetic datasets and AI models trained on them, centered around bias and discrimination prevention, fidelity to the real data, utility, robustness, and privacy preservation.
Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refitting
The dropping of training points is done in principle, but in practice does not require the model to be refit.
Reprogramming Large Pretrained Language Models for Antibody Sequence Infilling
We introduce Reprogramming for Protein Sequence Infilling, a framework in which pretrained natural language models are repurposed for protein sequence infilling via reprogramming, to infill protein sequence templates as a method of novel protein generation.
Cloud-Based Real-Time Molecular Screening Platform with MolFormer
With the prospect of automating a number of chemical tasks with high fidelity, chemical language processing models are emerging at a rapid speed.
Accelerating Material Design with the Generative Toolkit for Scientific Discovery
With the growing availability of data within various scientific domains, generative models hold enormous potential to accelerate scientific discovery.
ReGen: Reinforcement Learning for Text and Knowledge Base Generation using Pretrained Language Models
Automatic construction of relevant Knowledge Bases (KBs) from text, and generation of semantically meaningful text from KBs are both long-standing goals in Machine Learning.
Large-Scale Chemical Language Representations Capture Molecular Structure and Properties
Models based on machine learning can enable accurate and fast molecular property predictions, which is of interest in drug discovery and material design.
Image Captioning as an Assistive Technology: Lessons Learned from VizWiz 2020 Challenge
Image captioning has recently demonstrated impressive progress largely owing to the introduction of neural network algorithms trained on curated dataset like MS-COCO.
Alleviating Noisy Data in Image Captioning with Cooperative Distillation
Image captioning systems have made substantial progress, largely due to the availability of curated datasets like Microsoft COCO or Vizwiz that have accurate descriptions of their corresponding images.
Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text
Aligned with this, we propose a framework GYC, to generate a set of counterfactual text samples, which are crucial for testing these ML systems.
Tabular Transformers for Modeling Multivariate Time Series
This results in two architectures for tabular time series: one for learning representations that is analogous to BERT and can be pre-trained end-to-end and used in downstream tasks, and one that is akin to GPT and can be used for generation of realistic synthetic tabular sequences.
DualTKB: A Dual Learning Bridge between Text and Knowledge Base
In this work, we present a dual learning approach for unsupervised text to path and path to text transfers in Commonsense Knowledge Bases (KBs).
Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics
De novo therapeutic design is challenged by a vast chemical repertoire and multiple constraints, e. g., high broad-spectrum potency and low toxicity.
Learning Implicit Text Generation via Feature Matching
Generative feature matching network (GFMN) is an approach for training implicit generative models for images by performing moment matching on features from pre-trained neural networks.
CogMol: Target-Specific and Selective Drug Design for COVID-19 Using Deep Generative Models
CogMol also includes insilico screening for assessing toxicity of parent molecules and their metabolites with a multi-task toxicity classifier, synthetic feasibility with a chemical retrosynthesis predictor, and target structure binding with docking simulations.
Sobolev Independence Criterion
In the kernel version we show that SIC can be cast as a convex optimization problem by introducing auxiliary variables that play an important role in feature selection as they are normalized feature importance scores.
Generative Feature Matching Networks
We propose a non-adversarial feature matching-based approach to train generative models.
Learning Implicit Generative Models by Matching Perceptual Features
Perceptual features (PFs) have been used with great success in tasks such as transfer learning, style transfer, and super-resolution.
Interactive Visual Exploration of Latent Space (IVELS) for peptide auto-encoder model selection
We present the pipeline in an interactive visual tool to enable the exploration of the metrics, analysis of the learned latent space, and selection of the best model for a given task.
PepCVAE: Semi-Supervised Targeted Design of Antimicrobial Peptide Sequences
Our model learns a rich latent space of the biological peptide context by taking advantage of abundant, unlabeled peptide sequences.
Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer
We introduce a new approach to tackle the problem of offensive language in online social media.
Improved Neural Text Attribute Transfer with Non-parallel Data
Text attribute transfer using non-parallel data requires methods that can perform disentanglement of content and linguistic attributes.
