Search Results for author:
Found 52 papers, 13 papers with code
CTRL: A Conditional Transformer Language Model for Controllable Generation
Large-scale language models show promising text generation capabilities, but users cannot easily control particular aspects of the generated text.
Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity
Experiments show that for low values of k and p in top-k and top-p sampling, perplexity drops significantly with generated text length, which is also correlated with excessive repetitions in the text (the boredom trap).
BERTology Meets Biology: Interpreting Attention in Protein Language Models
Transformer architectures have proven to learn useful representations for protein classification and generation tasks.
RecipeGPT: Generative Pre-training Based Cooking Recipe Generation and Evaluation System
Interests in the automatic generation of cooking recipes have been growing steadily over the past few years thanks to a large amount of online cooking recipes.
Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding
Inflectional variation is a common feature of World Englishes such as Colloquial Singapore English and African American Vernacular English.
GAEA: Graph Augmentation for Equitable Access via Reinforcement Learning
Similarly, social networks within universities and organizations may enable certain groups to more easily access people with valuable information or influence.
Evaluating State-of-the-Art Classification Models Against Bayes Optimality
Moreover, we show that by varying the temperature of the learned flow models, we can generate synthetic datasets that closely resemble standard benchmark datasets, but with almost any desired Bayes error.
Autoequivariant Network Search via Group Decomposition
We address these problems by proving a new group-theoretic result in the context of equivariant neural networks that shows that a network is equivariant to a large group if and only if it is equivariant to smaller groups from which it is constructed.
Estimating Glycemic Impact of Cooking Recipes via Online Crowdsourcing and Machine Learning
Consumption of diets with low glycemic impact is highly recommended for diabetics and pre-diabetics as it helps maintain their blood glucose levels.
Universal and Succinct Source Coding of Deep Neural Networks
Deep neural networks have shown incredible performance for inference tasks in a variety of domains.
Probabilistic Rule Realization and Selection
Abstraction and realization are bilateral processes that are key in deriving intelligence and creativity.
Universal Joint Image Clustering and Registration using Partition Information
We consider the problem of universal joint clustering and registration of images and define algorithms using multivariate information functionals.
A neural network system for transformation of regional cuisine style
We propose a novel system which can transform a recipe into any selected regional style (e. g., Japanese, Mediterranean, or Italian).
Learning Interpretable Musical Compositional Rules and Traces
Throughout music history, theorists have identified and documented interpretable rules that capture the decisions of composers.
Multi-object Classification via Crowdsourcing with a Reject Option
We present an aggregation approach using a weighted majority voting rule, where each worker's response is assigned an optimized weight to maximize the crowd's classification performance.
Noise Facilitation in Associative Memories of Exponential Capacity
More surprisingly, we show that internal noise actually improves the performance of the recall phase while the pattern retrieval capacity remains intact, i. e., the number of stored patterns does not reduce with noise (up to a threshold).
A Big Data Approach to Computational Creativity
Computational creativity is an emerging branch of artificial intelligence that places computers in the center of the creative process.
A Group-Theoretic Approach to Computational Abstraction: Symmetry-Driven Hierarchical Clustering
Abstraction plays a key role in concept learning and knowledge discovery; this paper is concerned with computational abstraction.
Noise-Enhanced Associative Memories
More surprisingly, we show that internal noise actually improves the performance of the recall phase.
Neural Reconstruction with Approximate Message Passing (NeuRAMP)
Many functional descriptions of spiking neurons assume a cascade structure where inputs are passed through an initial linear filtering stage that produces a low-dimensional signal that drives subsequent nonlinear stages.
Limits of Deepfake Detection: A Robust Estimation Viewpoint
Deepfake detection is formulated as a hypothesis testing problem to classify an image as genuine or GAN-generated.
Accelerated Discovery of Sustainable Building Materials
Concrete is the most widely used engineered material in the world with more than 10 billion tons produced annually.
Beliefs in Decision-Making Cascades
In addition, the agents have their own beliefs instead of the true prior, and have nonidentical noise variances in the private signal.
A Near-Optimal Change-Detection Based Algorithm for Piecewise-Stationary Combinatorial Semi-Bandits
Compared to the original combinatorial semi-bandit problem, our setting assumes the reward distributions of base arms may change in a piecewise-stationary manner at unknown time steps.
Pretrained AI Models: Performativity, Mobility, and Change
The paradigm of pretrained deep learning models has recently emerged in artificial intelligence practice, allowing deployment in numerous societal settings with limited computational resources, but also embedding biases and enabling unintended negative uses.
Boosting Classifiers with Noisy Inference
Suppose that the base classifiers' outputs are noisy or communicated over noisy channels; these noisy outputs will degrade the final classification accuracy.
Nearly Optimal Algorithms for Piecewise-Stationary Cascading Bandits
Cascading bandit (CB) is a popular model for web search and online advertising, where an agent aims to learn the $K$ most attractive items out of a ground set of size $L$ during the interaction with a user.
Short and Wide Network Paths
We show this notion of pipelined network flow is optimized using network paths that are both short and wide, and develop efficient algorithms to compute such paths for given pairs of nodes and for all-pairs.
Limits of Detecting Text Generated by Large-Scale Language Models
Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns.
Human Evaluation of Interpretability: The Case of AI-Generated Music Knowledge
Interpretability of machine learning models has gained more and more attention among researchers in the artificial intelligence (AI) and human-computer interaction (HCI) communities.
Planting trees at the right places: Recommending suitable sites for growing trees using algorithm fusion
Large-scale planting of trees has been proposed as a low-cost natural solution for carbon mitigation, but is hampered by poor selection of plantation sites, especially in developing countries.
Information Lattice Learning
ILL focuses on explainability and generalizability from "small data", and aims for rules akin to those humans distill from experience (rather than a representation optimized for a specific task like classification).
Nonstationary Reinforcement Learning with Linear Function Approximation
We consider reinforcement learning (RL) in episodic Markov decision processes (MDPs) with linear function approximation under drifting environment.
Explaining Creative Artifacts
Human creativity is often described as the mental process of combining associative elements into a new form, but emerging computational creativity algorithms may not operate in this manner.
Social Bubbles and Superspreaders: Source Identification for Contagion Processes on Hypertrees
Previous work has shown that for contagion processes on extended star networks (trees with exactly one node of degree > 2), there is a simple, closed-form expression for a highly accurate approximation to the maximum likelihood infection source.
Adversarial Linear Contextual Bandits with Graph-Structured Side Observations
The second algorithm, \texttt{EXP3-LGC-IX}, is developed for a special class of problems, for which the regret is reduced to $\mathcal{O}(\sqrt{\alpha(G)dT\log{K}\log(KT)})$ for both directed as well as undirected feedback graphs.
The Twelvefold Way of Non-Sequential Lossless Compression
Many information sources are not just sequences of distinguishable symbols but rather have invariances governed by alternative counting paradigms such as permutations, combinations, and partitions.
Information Theory Information Theory
Succinct Source Coding of Deep Neural Networks
Deep neural networks have shown incredible performance for inference tasks in a variety of domains.
Learning from One and Only One Shot
Using simply the nearest-neighbor classifier on this similarity space, we achieve human-level character recognition using only 1--10 examples per class and nothing else (no pre-training).
Advanced Methods for Connectome-Based Predictive Modeling of Human Intelligence: A Novel Approach Based on Individual Differences in Cortical Topography
Investigating the proposed modeling framework's efficacy, we find that advanced connectome-based predictive modeling generates neuroscience predictions that account for a significantly greater proportion of variance in general intelligence scores than previously established methods, advancing our scientific understanding of the network architecture that underlies human intelligence.
Accelerated Design and Deployment of Low-Carbon Concrete for Data Centers
Finally, we report on how these formulations were used in the construction of buildings and structures in a Meta data center in DeKalb, IL, USA.
Debiased Large Language Models Still Associate Muslims with Uniquely Violent Acts
Recent work demonstrates a bias in the GPT-3 model towards generating violent text completions when prompted about Muslims, compared with Christians and Hindus.
Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models
We also provide a novel group-theoretic definition for fairness in NLG.
Learning Optimal Features via Partial Invariance
Learning models that are robust to distribution shifts is a key concern in the context of their real-life applicability.
Designing Discontinuities
To do so, we propose a quantization-theoretic approach to optimize the effect of interest, first learning the causal effect size of a given discontinuous variable and then applying dynamic programming for optimal quantization design of discontinuities to balance the gain and loss in that effect size.
Transformers are Universal Predictors
We find limits to the Transformer architecture for language modeling and show it has a universal prediction property in an information-theoretic sense.
A Meta-Learning Perspective on Transformers for Causal Language Modeling
Focused on the training process, here we establish a meta-learning view of the Transformer architecture when trained for the causal language modeling task, by explicating an inner optimization process within the Transformer.
Efficient Model-Agnostic Multi-Group Equivariant Networks
Inspired by the first design, we use the notion of the IS property to design a second efficient model-agnostic equivariant design for large product groups acting on a single input.
Muslim-Violence Bias Persists in Debiased GPT Models
Abid et al. (2021) showed a tendency in GPT-3 to generate mostly violent completions when prompted about Muslims, compared with other religions.
Language Grounded QFormer for Efficient Vision Language Understanding
Large-scale pretraining and instruction tuning have been successful for training general-purpose language models with broad competencies.
Transformer-based Causal Language Models Perform Clustering
Even though large language models (LLMs) have demonstrated remarkable capability in solving various natural language tasks, the capability of an LLM to follow human instructions is still a concern.
Federated Learning via Lattice Joint Source-Channel Coding
This paper introduces a universal federated learning framework that enables over-the-air computation via digital communications, using a new joint source-channel coding scheme.
