Search Results for author:
Found 45 papers, 37 papers with code
Compression Represents Intelligence Linearly
We open-source our compression datasets as well as our data collection pipelines to facilitate future researchers to assess compression properly.
In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation
Large language models (LLMs) frequently hallucinate and produce factual errors, yet our understanding of why they make these errors remains limited.
Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models
In the face of uncertainty, the ability to seek information is of fundamental importance.
AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
Evaluating large language models (LLMs) as general-purpose agents is essential for understanding their capabilities and facilitating their integration into practical applications.
GeoGalactica: A Scientific Large Language Model in Geoscience
To our best knowledge, it is the largest language model for the geoscience domain.
What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning
We present deita (short for Data-Efficient Instruction Tuning for Alignment), a series of models fine-tuned from LLaMA and Mistral models using data samples automatically selected with our proposed approach.
A Survey of Reasoning with Foundation Models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation.
Prompt Optimization via Adversarial In-Context Learning
We propose a new method, Adversarial In-Context Learning (adv-ICL), to optimize prompt for in-context learning (ICL) by employing one LLM as a generator, another as a discriminator, and a third as a prompt modifier.
InstructCoder: Instruction Tuning Large Language Models for Code Editing
In this work, we explore the use of Large Language Models (LLMs) to edit code based on user instructions.
FELM: Benchmarking Factuality Evaluation of Large Language Models
In this benchmark, we collect responses generated from LLMs and annotate factuality labels in a fine-grained manner.
SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning
More recently, with the rapid development of language models (LMs), researchers have focused on leveraging LMs to facilitate the learning of TGs, either by jointly training them in a computationally intensive framework (merging the two stages), or designing complex self-supervised training tasks for feature extraction (enhancing the first stage).
Ranked #1 on
Node Property Prediction
on ogbn-arxiv
FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios
With the above challenges in mind, in this paper, we propose FacTool, a task and domain agnostic framework for detecting factual errors of texts generated by large language models (e. g., ChatGPT).
Composing Parameter-Efficient Modules with Arithmetic Operations
In this paper, we propose to compose these parameter-efficient modules through linear arithmetic operations in the weight space, thereby integrating different module capabilities.
Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs
To better break down the problem, we define a systematic framework with three components: prompting strategies for eliciting verbalized confidence, sampling methods for generating multiple responses, and aggregation techniques for computing consistency.
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization
Large language models (LLMs) have achieved great success in general domains of natural language processing.
Contrastive Learning of Sentence Embeddings from Scratch
Contrastive learning has been the dominant approach to train state-of-the-art sentence embeddings.
Evaluating Factual Consistency of Summaries with Large Language Models
Detecting factual errors in summaries has been an important and challenging subject in summarization research.
CodeInstruct: Empowering Language Models to Edit Code
In this work, we explore the use of large language models (LLMs) to edit code based on user instructions, covering a broad range of implicit tasks such as comment insertion, code optimization, and code refactoring.
Automatic Model Selection with Large Language Models for Reasoning
Chain-of-Thought (CoT) and Program-Aided Language Models (PAL) represent two distinct reasoning methods, each with its own strengths.
Ranked #1 on
Math Word Problem Solving
on SVAMP
C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models
We present C-Eval, the first comprehensive Chinese evaluation suite designed to assess advanced knowledge and reasoning abilities of foundation models in a Chinese context.
Self-Evaluation Guided Beam Search for Reasoning
Stochastic beam search balances exploitation and exploration of the search space with temperature-controlled randomness.
Mega: Moving Average Equipped Gated Attention
The design choices in the Transformer attention mechanism, including weak inductive bias and quadratic computational complexity, have limited its application for modeling long sequences.
Ranked #1 on
Long-range modeling
on LRA
Non-Parametric Temporal Adaptation for Social Media Topic Classification
User-generated social media data is constantly changing as new trends influence online discussion and personal information is deleted due to privacy concerns.
Prompt Consistency for Zero-Shot Task Generalization
One of the most impressive results of recent NLP history is the ability of pre-trained language models to solve new tasks in a zero-shot setting.
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval
Retrieval-based language models (R-LM) model the probability of natural language text by combining a standard language model (LM) with examples retrieved from an external datastore at test time.
Towards a Unified View of Parameter-Efficient Transfer Learning
Furthermore, our unified framework enables the transfer of design elements across different approaches, and as a result we are able to instantiate new parameter-efficient fine-tuning methods that tune less parameters than previous methods while being more effective, achieving comparable results to fine-tuning all parameters on all four tasks.
Capturing Structural Locality in Non-parametric Language Models
Structural locality is a ubiquitous feature of real-world datasets, wherein data points are organized into local hierarchies.
Dependency Induction Through the Lens of Visual Perception
Our experiments find that concreteness is a strong indicator for learning dependency grammars, improving the direct attachment score (DAS) by over 50\% as compared to state-of-the-art models trained on pure text.
Efficient Nearest Neighbor Language Models
Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore, which allows them to learn through explicitly memorizing the training datapoints.
CTRLsum: Towards Generic Controllable Text Summarization
Our approach enables users to control multiple aspects of generated summaries by interacting with the summarization system through textual input in the form of a set of keywords or descriptive prompts.
On the Sentence Embeddings from Pre-trained Language Models
Pre-trained contextual representations like BERT have achieved great success in natural language processing.
Ranked #16 on
Semantic Textual Similarity
on STS16
Learning Sparse Prototypes for Text Generation
While effective, these methods are inefficient at test time as a result of needing to store and index the entire training corpus.
A Probabilistic Formulation of Unsupervised Text Style Transfer
Across all style transfer tasks, our approach yields substantial gains over state-of-the-art non-generative baselines, including the state-of-the-art unsupervised machine translation techniques that our approach generalizes.
Revisiting Self-Training for Neural Sequence Generation
In this work, we first empirically show that self-training is able to decently improve the supervised baseline on neural sequence generation tasks.
The Source-Target Domain Mismatch Problem in Machine Translation
While we live in an increasingly interconnected world, different places still exhibit strikingly different cultures and many events we experience in our every day life pertain only to the specific place we live in.
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text
In this paper, we investigate a simple fix for posterior collapse which yields surprisingly effective results.
Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections
The parameters of source model and target model are softly shared through a regularized log likelihood objective.
Choosing Transfer Languages for Cross-Lingual Learning
Cross-lingual transfer, where a high-resource transfer language is used to improve the accuracy of a low-resource task language, is now an invaluable tool for improving performance of natural language processing (NLP) on low-resource languages.
Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
The variational autoencoder (VAE) is a popular combination of deep latent variable model and accompanying variational learning technique.
Ranked #1 on
Text Generation
on Yahoo Questions
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
The versatile toolkit also fosters technique sharing across different text generation tasks.
Unsupervised Learning of Syntactic Structure with Invertible Neural Projections
In this work, we propose a novel generative model that jointly learns discrete syntactic structure and continuous word representations in an unsupervised fashion by cascading an invertible neural network with a structured generative prior.
Texar: A Modularized, Versatile, and Extensible Toolbox for Text Generation
The features make Texar particularly suitable for technique sharing and generalization across different text generation applications.
StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsing
Semantic parsing is the task of transducing natural language (NL) utterances into formal meaning representations (MRs), commonly represented as tree structures.
Efficient Correlated Topic Modeling with Topic Embedding
Correlated topic modeling has been limited to small model and problem sizes due to their high computational cost and poor scaling.
Text Network Exploration via Heterogeneous Web of Topics
A text network refers to a data type that each vertex is associated with a text document and the relationship between documents is represented by edges.
