Search Results for author:
Found 33 papers, 18 papers with code
Attribute Injection for Pretrained Language Models: A New Benchmark and an Efficient Method
Metadata attributes (e. g., user and product IDs from reviews) can be incorporated as additional inputs to neural-based NLP models, by expanding the architecture of the models to improve performance.
Aligning Language Models to Explicitly Handle Ambiguity
However, conversational agents built upon even the most recent large language models (LLMs) face challenges in processing ambiguous inputs, primarily due to the following two hurdles: (1) LLMs are not directly trained to handle inputs that are too ambiguous to be properly managed; (2) the degree of ambiguity in an input can vary according to the intrinsic knowledge of the LLMs, which is difficult to investigate.
HyperCLOVA X Technical Report
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding.
KMMLU: Measuring Massive Multitask Language Understanding in Korean
We propose KMMLU, a new Korean benchmark with 35, 030 expert-level multiple-choice questions across 45 subjects ranging from humanities to STEM.
Aligning Large Language Models by On-Policy Self-Judgment
Existing approaches for aligning large language models with human preferences face a trade-off that requires a separate reward model (RM) for on-policy learning.
Unified Speech-Text Pretraining for Spoken Dialog Modeling
While recent work shows promising results in expanding the capabilities of large language models (LLM) to directly understand and synthesize speech, an LLM-based strategy for modeling spoken dialogs remains elusive and calls for further investigation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Model
An exciting advancement in the field of multilingual models is the emergence of autoregressive models with zero- and few-shot capabilities, a phenomenon widely reported in large-scale language models.
Universal Domain Adaptation for Robust Handling of Distributional Shifts in NLP
When deploying machine learning systems to the wild, it is highly desirable for them to effectively leverage prior knowledge to the unfamiliar domain while also firing alarms to anomalous inputs.
Instruction Tuning with Human Curriculum
In this work, we (1) introduce Curriculum Instruction Tuning, (2) explore the potential advantages of employing diverse curriculum strategies, and (3) delineate a synthetic instruction-response generation framework that complements our theoretical approach.
Aligning Large Language Models through Synthetic Feedback
In human evaluation, our model is preferred to Alpaca and Dolly-v2, 55. 0% and 58. 5% of the time, respectively.
Probing Out-of-Distribution Robustness of Language Models with Parameter-Efficient Transfer Learning
As the size of the pre-trained language model (PLM) continues to increase, numerous parameter-efficient transfer learning methods have been proposed recently to compensate for the tremendous cost of fine-tuning.
Prompt-Augmented Linear Probing: Scaling beyond the Limit of Few-shot In-Context Learners
Through in-context learning (ICL), large-scale language models are effective few-shot learners without additional model fine-tuning.
Critic-Guided Decoding for Controlled Text Generation
In this work, we propose a novel critic decoding method for controlled language generation (CriticControl) that combines the strengths of reinforcement learning and weighted decoding.
Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble
Out-of-distribution (OOD) detection aims to discern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience.
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models
To combine parameter-efficient adaptation and model compression, we propose AlphaTuning consisting of post-training quantization of the pre-trained language model and fine-tuning only some parts of quantized parameters for a target task.
Continuous Decomposition of Granularity for Neural Paraphrase Generation
While Transformers have had significant success in paragraph generation, they treat sentences as linear sequences of tokens and often neglect their hierarchical information.
Self-Generated In-Context Learning: Leveraging Auto-regressive Language Models as a Demonstration Generator
Large-scale pre-trained language models (PLMs) are well-known for being capable of solving a task simply by conditioning a few input-label pairs dubbed demonstrations on a prompt without being explicitly tuned for the desired downstream task.
Mutual Information Divergence: A Unified Metric for Multimodal Generative Models
Based on a recent trend that multimodal generative evaluations exploit a vison-and-language pre-trained model, we propose the negative Gaussian cross-mutual information using the CLIP features as a unified metric, coined by Mutual Information Divergence (MID).
Ranked #1 on
Human Judgment Classification
on Pascal-50S
Hallucination Pair-wise Detection (1-ref)
Hallucination Pair-wise Detection (4-ref)
+5
Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations
Despite recent explosion of interests in in-context learning, the underlying mechanism and the precise impact of the quality of demonstrations remain elusive.
Generating Information-Seeking Conversations from Unlabeled Documents
In this paper, we introduce a novel framework, SIMSEEK, (Simulating information-Seeking conversation from unlabeled documents), and compare its two variants.
Masked Summarization to Generate Factually Inconsistent Summaries for Improved Factual Consistency Checking
To this end, the latest approach is to train a factual consistency classifier on factually consistent and inconsistent summaries.
Response Generation with Context-Aware Prompt Learning
Pre-trained language models (PLM) have marked a huge leap in neural dialogue modeling.
Efficient Attribute Injection for Pretrained Language Models
Metadata attributes (e. g., user and product IDs from reviews) can be incorporated as additional inputs to neural-based NLP models, by modifying the architecture of the models, in order to improve their performance.
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers
GPT-3 shows remarkable in-context learning ability of large-scale language models (LMs) trained on hundreds of billion scale data.
Self-Guided Contrastive Learning for BERT Sentence Representations
In this work, we propose a contrastive learning method that utilizes self-guidance for improving the quality of BERT sentence representations.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Large-scale language models such as GPT-3 are excellent few-shot learners, allowing them to be controlled via natural text prompts.
Reward Optimization for Neural Machine Translation with Learned Metrics
Results show that the reward optimization with BLEURT is able to increase the metric scores by a large margin, in contrast to limited gain when training with smoothed BLEU.
DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterances
Recent advances in pre-trained language models have significantly improved neural response generation.
Variational Hierarchical Dialog Autoencoder for Dialog State Tracking Data Augmentation
Recent works have shown that generative data augmentation, where synthetic samples generated from deep generative models complement the training dataset, benefit NLP tasks.
Don't Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja
We propose a simple yet effective approach for improving Korean word representations using additional linguistic annotation (i. e. Hanja).
Data Augmentation for Spoken Language Understanding via Joint Variational Generation
Data scarcity is one of the main obstacles of domain adaptation in spoken language understanding (SLU) due to the high cost of creating manually tagged SLU datasets.
Improving Visually Grounded Sentence Representations with Self-Attention
Sentence representation models trained only on language could potentially suffer from the grounding problem.
Learning to Compose Task-Specific Tree Structures
For years, recursive neural networks (RvNNs) have been shown to be suitable for representing text into fixed-length vectors and achieved good performance on several natural language processing tasks.
Ranked #62 on
Natural Language Inference
on SNLI
