Search Results for author:
Found 73 papers, 38 papers with code
Knowledge-Enriched Natural Language Generation
Knowledge-enriched text generation poses unique challenges in modeling and learning, driving active research in several core directions, ranging from integrated modeling of neural representations and symbolic information in the sequential/hierarchical/graphical structures, learning without direct supervisions due to the cost of structured annotation, efficient optimization and inference with massive and global constraints, to language grounding on multiple modalities, and generative reasoning with implicit commonsense knowledge and background knowledge.
Semantic Aligned Multi-modal Transformer for Vision-LanguageUnderstanding: A Preliminary Study on Visual QA
Recent vision-language understanding approaches adopt a multi-modal transformer pre-training and finetuning paradigm.
Don’t Take It Literally: An Edit-Invariant Sequence Loss for Text Generation
Such training objective is sub-optimal when the target sequence is not perfect, e. g., when the target sequence is corrupted with noises, or when only weak sequence supervision is available.
UOUO: Uncontextualized Uncommon Objects for Measuring Knowledge Horizons of Vision Language Models
Smaller-scale Vision-Langauge Models (VLMs) often claim to perform on par with larger models in general-domain visual grounding and question-answering benchmarks while offering advantages in computational efficiency and storage.
Pandora: Towards General World Model with Natural Language Actions and Video States
This paper makes a step towards building a general world model by introducing Pandora, a hybrid autoregressive-diffusion model that simulates world states by generating videos and allows real-time control with free-text actions.
LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models
(2) We develop LLM Reasoners, a library for standardized modular implementation of existing and new reasoning algorithms, under a unified formulation of the search, reward, and world model components.
Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding
The vast applications of deep generative models are anchored in three core capabilities -- generating new instances, reconstructing inputs, and learning compact representations -- across various data types, such as discrete text/protein sequences and continuous images.
MMToM-QA: Multimodal Theory of Mind Question Answering
To engineer multimodal ToM capacity, we propose a novel method, BIP-ALM (Bayesian Inverse Planning Accelerated by Language Models).
LLM360: Towards Fully Transparent Open-Source LLMs
The recent surge in open-source Large Language Models (LLMs), such as LLaMA, Falcon, and Mistral, provides diverse options for AI practitioners and researchers.
Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning
Despite their tremendous success in many applications, large language models often fall short of consistent reasoning and planning in various (language, embodied, and social) scenarios, due to inherent limitations in their inference, learning, and modeling capabilities.
RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs
In this work, we present RedCoast (Redco), a lightweight and user-friendly tool crafted to automate distributed training and inference for LLMs, as well as to simplify ML pipeline development.
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization
Highly effective, task-specific prompts are often heavily engineered by experts to integrate detailed instructions and domain insights based on a deep understanding of both instincts of large language models (LLMs) and the intricacies of the target task.
AlignScore: Evaluating Factual Consistency with a Unified Alignment Function
AlignScore is based on a general function of information alignment between two arbitrary text pieces.
Reasoning with Language Model is Planning with World Model
RAP on LLAMA-33B surpasses CoT on GPT-4 with 33% relative improvement in a plan generation setting.
ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings
Augmenting large language models (LLMs) with external tools has emerged as a promising approach to solving complex problems.
Language Models Meet World Models: Embodied Experiences Enhance Language Models
While large language models (LMs) have shown remarkable capabilities across numerous tasks, they often struggle with simple reasoning and planning in physical environments, such as understanding object permanence or planning household activities.
ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models
In the data disambiguation stage, we employ the prompted GPT-3 model to understand possibly ambiguous triples from the input data and convert each into a short sentence with reduced ambiguity.
Composable Text Controls in Latent Space with ODEs
This paper proposes a new efficient approach for composable text operations in the compact latent space of text.
Ranked #2 on
Unsupervised Text Style Transfer
on Yelp
BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models
The resulting KGs as a symbolic interpretation of the source LMs also reveal new insights into the LMs' knowledge capacities.
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
RLPrompt formulates a parameter-efficient policy network that generates the desired discrete prompt after training with reward.
elBERto: Self-supervised Commonsense Learning for Question Answering
Commonsense question answering requires reasoning about everyday situations and causes and effects implicit in context.
A Causal Lens for Controllable Text Generation
Controllable text generation concerns two fundamental tasks of wide applications, namely generating text of given attributes (i. e., attribute-conditional generation), and minimally editing existing text to possess desired attributes (i. e., text attribute transfer).
Text Generation with Efficient (Soft) $Q$-Learning
We apply the approach to a wide range of text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation
Based on the nature of information change from input to output, we classify NLG tasks into compression (e. g., summarization), transduction (e. g., text rewriting), and creation (e. g., dialog).
Toward a `Standard Model' of Machine Learning
Machine learning (ML) is about computational methods that enable machines to learn concepts from experience.
Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation
Such training objective is sub-optimal when the target sequence is not perfect, e. g., when the target sequence is corrupted with noises, or when only weak sequence supervision is available.
Efficient (Soft) Q-Learning for Text Generation with Limited Good Data
We apply the approach to a wide range of novel text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
A Data-Centric Framework for Composable NLP Workflows
Empirical natural language processing (NLP) systems in application domains (e. g., healthcare, finance, education) involve interoperation among multiple components, ranging from data ingestion, human annotation, to text retrieval, analysis, generation, and visualization.
Deep Learning for Text Style Transfer: A Survey
Text style transfer is an important task in natural language generation, which aims to control certain attributes in the generated text, such as politeness, emotion, humor, and many others.
Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach
Given a document and a target aspect (e. g., a topic of interest), aspect-based abstractive summarization attempts to generate a summary with respect to the aspect.
A Survey of Knowledge-Enhanced Text Generation
To address this issue, researchers have considered incorporating various forms of knowledge beyond the input text into the generation models.
Progressive Generation of Long Text with Pretrained Language Models
However, as our systematic examination reveals, it is still challenging for such models to generate coherent long passages of text (e. g., 1000 tokens), especially when the models are fine-tuned to the target domain on a small corpus.
Improving GAN Training with Probability Ratio Clipping and Sample Reweighting
Despite success on a wide range of problems related to vision, generative adversarial networks (GANs) often suffer from inferior performance due to unstable training, especially for text generation.
Ranked #2 on
Text Generation
on EMNLP2017 WMT
Learning Data Manipulation for Augmentation and Weighting
In this work, we propose a new method that supports learning different manipulation schemes with the same gradient-based algorithm.
Target-Guided Open-Domain Conversation
We study the problem of imposing conversational goals on open-domain chat agents.
Graph Transformer
Graph neural networks (GNN) have gained increasing research interests as a mean to the challenging goal of robust and universal graph learning.
Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generation
Generating long and semantic-coherent reports to describe medical images poses great challenges towards bridging visual and linguistic modalities, incorporating medical domain knowledge, and generating realistic and accurate descriptions.
Learning Hierarchical Representations of Electronic Health Records for Clinical Outcome Prediction
Our model learns hierarchical representationsof event sequences, to adaptively distinguish between short-range and long-range events, and accurately capture coretemporal dependencies.
Data-to-Text Generation with Style Imitation
That is, the model learns to imitate the writing style of any given exemplar sentence, with automatic adaptions to faithfully describe the content record.
Symbolic Graph Reasoning Meets Convolutions
To cooperate with local convolutions, each SGR is constituted by three modules: a) a primal local-to-semantic voting module where the features of all symbolic nodes are generated by voting from local representations; b) a graph reasoning module propagates information over knowledge graph to achieve global semantic coherency; c) a dual semantic-to-local mapping module learns new associations of the evolved symbolic nodes with local representations, and accordingly enhances local features.
Ranked #81 on
Semantic Segmentation
on ADE20K val
Connecting the Dots Between MLE and RL for Sequence Prediction
Reinforcement learning such as policy gradient addresses the issue but can have prohibitively poor exploration efficiency.
Structured Content Preservation for Unsupervised Text Style Transfer
Text style transfer aims to modify the style of a sentence while keeping its content unchanged.
AutoLoss: Learning Discrete Schedules for Alternate Optimization
Many machine learning problems involve iteratively and alternately optimizing different task objectives with respect to different sets of parameters.
AutoLoss: Learning Discrete Schedule for Alternate Optimization
Many machine learning problems involve iteratively and alternately optimizing different task objectives with respect to different sets of parameters.
Differentiable Expected BLEU for Text Generation
Neural text generation models such as recurrent networks are typically trained by maximizing data log-likelihood based on cross entropy.
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
The versatile toolkit also fosters technique sharing across different text generation tasks.
Texar: A Modularized, Versatile, and Extensible Toolbox for Text Generation
The features make Texar particularly suitable for technique sharing and generalization across different text generation applications.
Deep Generative Models with Learnable Knowledge Constraints
The broad set of deep generative models (DGMs) has achieved remarkable advances.
Unsupervised Text Style Transfer using Language Models as Discriminators
Binary classifiers are often employed as discriminators in GAN-based unsupervised style transfer systems to ensure that transferred sentences are similar to sentences in the target domain.
Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
Experiments show that our approach achieves the state-of-the-art results on two medical report datasets, generating well-balanced structured sentences with robust coverage of heterogeneous medical report contents.
Automatic Article Commenting: the Task and Dataset
Comments of online articles provide extended views and improve user engagement.
Towards Automated ICD Coding Using Deep Learning
Considering the complicated and dedicated process to assign correct codes to each patient admission based on overall diagnosis, we propose a hierarchical deep learning model with attention mechanism which can automatically assign ICD diagnostic codes given written diagnosis.
Efficient Correlated Topic Modeling with Topic Embedding
Correlated topic modeling has been limited to small model and problem sizes due to their high computational cost and poor scaling.
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters
We show that Poseidon enables Caffe and TensorFlow to achieve 15. 5x speed-up on 16 single-GPU machines, even with limited bandwidth (10GbE) and the challenging VGG19-22K network for image classification.
On Unifying Deep Generative Models
Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), as emerging families for generative model learning, have largely been considered as two distinct paradigms and received extensive independent studies respectively.
Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification
Implicit discourse relation classification is of great challenge due to the lack of connectives as strong linguistic cues, which motivates the use of annotated implicit connectives to improve the recognition.
Nonparametric Variational Auto-encoders for Hierarchical Representation Learning
In this work, we propose hierarchical nonparametric variational autoencoders, which combines tree-structured Bayesian nonparametric priors with VAEs, to enable infinite flexibility of the latent representation space.
Recurrent Topic-Transition GAN for Visual Paragraph Generation
The proposed Recurrent Topic-Transition Generative Adversarial Network (RTT-GAN) builds an adversarial framework between a structured paragraph generator and multi-level paragraph discriminators.
Generative Adversarial Network
Image Paragraph Captioning
+1
Toward Controlled Generation of Text
Generic generation and manipulation of text is challenging and has limited success compared to recent deep generative modeling in visual domain.
Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
Recent work on generative modeling of text has found that variational auto-encoders (VAE) incorporating LSTM decoders perform worse than simpler LSTM language models (Bowman et al., 2015).
Ranked #3 on
Text Generation
on Yahoo Questions
Stochastic Variational Deep Kernel Learning
We propose a novel deep kernel learning model and stochastic variational inference procedure which generalizes deep kernel learning approaches to enable classification, multi-task learning, additive covariance structures, and stochastic gradient training.
Learning Scalable Deep Kernels with Recurrent Structure
To model such structure, we propose expressive closed-form kernel functions for Gaussian processes.
Dropout with Expectation-linear Regularization
Algorithmically, we show that our proposed measure of the inference gap can be used to regularize the standard dropout training objective, resulting in an \emph{explicit} control of the gap.
Joint Embedding of Hierarchical Categories and Entities for Concept Categorization and Dataless Classification
Due to the lack of structured knowledge applied in learning distributed representation of cate- gories, existing work cannot incorporate category hierarchies into entity information.
Neural Machine Translation with Recurrent Attention Modeling
Knowing which words have been attended to in previous time steps while generating a translation is a rich source of information for predicting what words will be attended to in the future.
Learning Concept Taxonomies from Multi-modal Data
We study the problem of automatically building hypernym taxonomies from textual and visual data.
Joint Embeddings of Hierarchical Categories and Entities
Due to the lack of structured knowledge applied in learning distributed representation of categories, existing work cannot incorporate category hierarchies into entity information.~We propose a framework that embeds entities and categories into a semantic space by integrating structured knowledge and taxonomy hierarchy from large knowledge bases.
Harnessing Deep Neural Networks with Logic Rules
Combining deep neural networks with structured logic rules is desirable to harness flexibility and reduce uninterpretability of the neural models.
Ranked #65 on
Sentiment Analysis
on SST-2 Binary classification
Poseidon: A System Architecture for Efficient GPU-based Deep Learning on Multiple Machines
To investigate how to adapt existing frameworks to efficiently support distributed GPUs, we propose Poseidon, a scalable system architecture for distributed inter-machine communication in existing DL frameworks.
Deep Kernel Learning
We introduce scalable deep kernels, which combine the structural properties of deep learning architectures with the non-parametric flexibility of kernel methods.
