Search Results for author:
Found 51 papers, 33 papers with code
MIMIR: A Streamlined Platform for Personalized Agent Tuning in Domain Expertise
Recently, large language models (LLMs) have evolved into interactive agents, proficient in planning, tool use, and task execution across a wide variety of tasks.
CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models
The advancement of large language models (LLMs) has enhanced the ability to generalize across a wide range of unseen natural language processing (NLP) tasks through instruction-following.
LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild
Low-Rank Adaptation (LoRA) provides an effective yet efficient solution for fine-tuning large language models (LLM).
Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science
Intelligent agents powered by large language models (LLMs) have demonstrated substantial promise in autonomously conducting experiments and facilitating scientific discoveries across various disciplines.
Weaver: Foundation Models for Creative Writing
Weaver is pre-trained on a carefully selected corpus that focuses on improving the writing capabilities of large language models.
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
To help the open-source community have a better understanding of Mixture-of-Experts (MoE) based large language models (LLMs), we train and release OpenMoE, a series of fully open-sourced and reproducible decoder-only MoE LLMs, ranging from 650M to 34B parameters and trained on up to over 1T tokens.
AUTOACT: Automatic Agent Learning from Scratch via Self-Planning
Further analysis demonstrates the effectiveness of the division-of-labor strategy, with the trajectory quality generated by AutoAct significantly outperforming that of others.
How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs
Different from prior studies, we shift our focus from evaluating standard performance to introducing a comprehensive safety evaluation suite, covering both out-of-distribution (OOD) generalization and adversarial robustness.
ML-Bench: Evaluating Large Language Models for Code Generation in Repository-Level Machine Learning Tasks
While Large Language Models (LLMs) have demonstrated proficiency in code generation benchmarks, translating these results into practical development scenarios - where leveraging existing repository-level libraries is the norm - remains challenging.
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
We show that MechanisticProbe is able to detect the information of the reasoning tree from the model's attentions for most examples, suggesting that the LM indeed is going through a process of multi-step reasoning within its architecture in many cases.
Evaluating Large Language Models on Controlled Generation Tasks
While recent studies have looked into the abilities of large language models in various benchmark tasks, including question generation, reading comprehension, multilingual and etc, there have been few studies looking into the controllability of large language models on generation tasks.
Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models
*Data Synthesis* is a promising way to train a small model with very little labeled data.
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters.
Mixup Your Own Pairs
In representation learning, regression has traditionally received less attention than classification.
Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?
Despite the remarkable capabilities of Large Language Models (LLMs) like GPT-4, producing complex, structured tabular data remains challenging.
Agents: An Open-source Framework for Autonomous Language Agents
Recent advances on large language models (LLMs) enable researchers and developers to build autonomous language agents that can automatically solve various tasks and interact with environments, humans, and other agents using natural language interfaces.
Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference
Modular Transformers train modularized layers that have the same function of two or more consecutive layers in the original model via module replacing and knowledge distillation.
Commonsense Knowledge Transfer for Pre-trained Language Models
In this work, we introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained language model.
SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models
Despite achieving remarkable performance on various vision-language tasks, Transformer-based Vision-Language Models (VLMs) suffer from redundancy in inputs and parameters, significantly hampering their efficiency in real-world applications.
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
In addition to producing AI-generated content (AIGC), we also demonstrate the possibility of using RecurrentGPT as an interactive fiction that directly interacts with consumers.
Interactive Natural Language Processing
Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP, aimed at addressing limitations in existing frameworks while aligning with the ultimate goals of artificial intelligence.
Efficient Prompting via Dynamic In-Context Learning
To achieve this, we train a meta controller that predicts the number of in-context examples suitable for the generalist model to make a good prediction based on the performance-efficiency trade-off for a specific input.
Controlled Text Generation with Natural Language Instructions
Large language models generate fluent texts and can follow natural language instructions to solve a wide range of tasks without task-specific training.
Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference
Machine translation quality estimation (QE) predicts human judgements of a translation hypothesis without seeing the reference.
X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks
Vision language pre-training aims to learn alignments between vision and language from a large amount of data.
Ranked #1 on
Cross-Modal Retrieval
on Flickr30k
(using extra training data)
Efficiently Tuned Parameters are Task Embeddings
Thus, we propose to exploit these efficiently tuned parameters as off-the-shelf task embeddings for the efficient selection of source datasets for intermediate-task transfer.
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning
Pre-trained vision-language models (VLMs) have achieved impressive results in a range of vision-language tasks.
Write and Paint: Generative Vision-Language Models are Unified Modal Learners
In this work, we disclose the potential of symmetric generative vision-language pre-training in learning to write and paint concurrently, and propose a new unified modal model, named DaVinci, trained with prefix language modeling and prefix image modeling, a simple generative self-supervised objective on image-text pairs.
Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training
To this end, the cross-view language modeling framework considers both multi-modal data (i. e., image-caption pairs) and multi-lingual data (i. e., parallel sentence pairs) as two different views of the same object, and trains the model to align the two views by maximizing the mutual information between them with conditional masked language modeling and contrastive learning.
VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models
We release the VLUE benchmark to promote research on building vision-language models that generalize well to more diverse images and concepts unseen during pre-training, and are practical in terms of efficiency-performance trade-off.
Vietnamese Language Models
Vietnamese Natural Language Understanding
+1
Contextual Representation Learning beyond Masked Language Modeling
How do masked language models (MLMs) such as BERT learn contextual representations?
Learning to Predict Persona Information forDialogue Personalization without Explicit Persona Description
Personalizing dialogue agents is important for dialogue systems to generate more specific, consistent, and engaging responses.
A Survey on Green Deep Learning
In recent years, larger and deeper models are springing up and continuously pushing state-of-the-art (SOTA) results across various fields like natural language processing (NLP) and computer vision (CV).
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
Recent studies on compression of pretrained language models (e. g., BERT) usually use preserved accuracy as the metric for evaluation.
BERT Learns to Teach: Knowledge Distillation with Meta Learning
We present Knowledge Distillation with Meta Learning (MetaDistil), a simple yet effective alternative to traditional knowledge distillation (KD) methods where the teacher model is fixed during training.
Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training
By giving higher rewards for responses whose output probability reduces more significantly when dialogue history is perturbed, the model is encouraged to generate more diverse and consistent responses.
Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge
Cant is important for understanding advertising, comedies and dog-whistle politics.
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting
In this paper, we generalize text infilling (e. g., masked language models) by proposing Sequence Span Rewriting (SSR) as a self-supervised sequence-to-sequence (seq2seq) pre-training objective.
Pre-training Text-to-Text Transformers to Write and Reason with Concepts
To augment PTLMs with common sense, we propose generative and contrastive objectives as intermediate self-supervised pre-training tasks between general pre-training and downstream task-specific fine-tuning.
Pre-training Text-to-Text Transformers for Concept-centric Common Sense
Pre-trained language models (PTLM) have achieved impressive results in a range of natural language understanding (NLU) and generation (NLG) tasks.
Towards Interpretable Natural Language Understanding with Explanations as Latent Variables
In this paper, we develop a general framework for interpretable natural language understanding that requires only a small set of human annotated explanations for training.
Connecting the Dots Between Fact Verification and Fake News Detection
Fact verification models have enjoyed a fast advancement in the last two years with the development of pre-trained language models like BERT and the release of large scale datasets such as FEVER.
BERT Loses Patience: Fast and Robust Inference with Early Exit
In this paper, we propose Patience-based Early Exit, a straightforward yet effective inference method that can be used as a plug-and-play technique to simultaneously improve the efficiency and robustness of a pretrained language model (PLM).
Scheduled DropHead: A Regularization Method for Transformer Models
In this paper, we introduce DropHead, a structured dropout method specifically designed for regularizing the multi-head attention mechanism, which is a key component of transformer, a state-of-the-art model for various NLP tasks.
Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models
While able to be trained in a fully self-supervised fashion, our model can be further fine-tuned with a little amount of human preference annotation to better imitate human judgment.
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Our approach first divides the original BERT into several modules and builds their compact substitutes.
Self-Adversarial Learning with Comparative Discrimination for Text Generation
Conventional Generative Adversarial Networks (GANs) for text generation tend to have issues of reward sparsity and mode collapse that affect the quality and diversity of generated samples.
Pseudo-Bidirectional Decoding for Local Sequence Transduction
PBD copies the corresponding representation of source tokens to the decoder as pseudo future context to enable the decoder to attends to its bi-directional context.
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning
In this paper, we present a constrained text generation task, CommonGen associated with a benchmark dataset, to explicitly test machines for the ability of generative commonsense reasoning.
Ranked #1 on
Text Generation
on CommonGen
Improving Grammatical Error Correction with Machine Translation Pairs
The poor translation model resembles the ESL (English as a second language) learner and tends to generate translations of low quality in terms of fluency and grammatical correctness, while the good translation model generally generates fluent and grammatically correct translations.
BERT-based Lexical Substitution
Our approach first applies dropout to the target word{'}s embedding for partially masking the word, allowing BERT to take balanced consideration of the target word{'}s semantics and contexts for proposing substitute candidates, and then validates the candidates based on their substitution{'}s influence on the global contextualized representation of the sentence.
