Search Results for author:
Found 58 papers, 35 papers with code
Plan-CVAE: A Planning-based Conditional Variational Autoencoder for Story Generation
Story generation is a challenging task of automatically creating natural languages to describe a sequence of events, which requires outputting text with not only a consistent topic but also novel wordings.
KeyB2: Selecting Key Blocks is Also Important for Long Document Ranking with Large Language Models
Comprehensive experiments on long-document datasets, including TREC 2019 DL, Robust04, and MLDR-zh, show that KeyB2 outperforms baselines like RankLLaMA and KeyB by reducing reranking time and GPU memory usage while enhancing retrieval performance, achieving new SOTA results on TREC 2019 DL with higher NDCG@10 and MAP scores.
LOGO -- Long cOntext aliGnment via efficient preference Optimization
To overcome the GPU memory-bound issue caused by the long sequence, LOGO employs a reference-free preference optimization strategy and adopts a position synthesis method to construct the training data.
Unleashing Reasoning Capability of LLMs via Scalable Question Synthesis from Scratch
The availability of high-quality data is one of the most important factors in improving the reasoning capability of LLMs.
Beware of Calibration Data for Pruning Large Language Models
As large language models (LLMs) are widely applied across various fields, model compression has become increasingly crucial for reducing costs and improving inference efficiency.
Revealing and Mitigating the Local Pattern Shortcuts of Mamba
Large language models (LLMs) have advanced significantly due to the attention mechanism, but their quadratic complexity and linear memory demands limit their performance on long-context tasks.
L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding?
Long-context models (LCMs) have made remarkable strides in recent years, offering users great convenience for handling tasks that involve long context, such as document summarization.
MemLong: Memory-Augmented Retrieval for Long Text Modeling
This work introduces MemLong: Memory-Augmented Retrieval for Long Text Generation, a method designed to enhance the capabilities of long-context language modeling by utilizing an external retriever for historical information retrieval.
ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM
Only a few research explored the conflicts between the inherent knowledge of LLMs and the retrieved contextual knowledge.
OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure
It searches the optimal tree structure that maximizes the mathematical expectation of the acceptance length in each decoding step.
Timo: Towards Better Temporal Reasoning for Language Models
Therefore, we propose a crucial question: Can we build a universal framework to handle a variety of temporal reasoning tasks?
A Survey on Human Preference Learning for Large Language Models
In this survey, we review the progress in exploring human preference learning for LLMs from a preference-centered perspective, covering the sources and formats of preference feedback, the modeling and usage of preference signals, as well as the evaluation of the aligned LLMs.
Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?
Temporal reasoning is fundamental for large language models (LLMs) to comprehend the world.
Demonstration Augmentation for Zero-shot In-context Learning
Large Language Models (LLMs) have demonstrated an impressive capability known as In-context Learning (ICL), which enables them to acquire knowledge from textual demonstrations without the need for parameter updates.
Fennec: Fine-grained Language Model Evaluation and Correction Extended through Branching and Bridging
The rapid advancement of large language models has given rise to a plethora of applications across a myriad of real-world tasks, mainly centered on aligning with human intent.
Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization
To address the challenges of low-rank compression in LLMs, we conduct empirical research on the low-rank characteristics of large models.
OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruning
Large Language Models (LLMs) have played an important role in many fields due to their powerful capabilities. However, their massive number of parameters leads to high deployment requirements and incurs significant inference costs, which impedes their practical applications.
Rethinking Negative Instances for Generative Named Entity Recognition
In the Named Entity Recognition (NER) task, recent advancements have seen the remarkable improvement of LLMs in a broad range of entity domains via instruction tuning, by adopting entity-centric schema.
StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis
To leverage LLMs for visual synthesis, traditional methods convert raster image information into discrete grid tokens through specialized visual modules, while disrupting the model's ability to capture the true semantic representation of visual scenes.
Resolving Crash Bugs via Large Language Models: An Empirical Study
Evaluating IntDiagSolver on multiple LLMs reveals consistent enhancement in the accuracy of crash bug resolution, including ChatGPT, Claude, and CodeLlama.
KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model
We achieve a biomedical multilingual corpus by incorporating three granularity knowledge alignments (entity, fact, and passage levels) into monolingual corpora.
Beyond Hard Samples: Robust and Effective Grammatical Error Correction with Cycle Self-Augmenting
By leveraging the augmenting data from the GEC models themselves in the post-training process and introducing regularization data for cycle training, our proposed method can effectively improve the model robustness of well-trained GEC models with only a few more training epochs as an extra cost.
G-SPEED: General SParse Efficient Editing MoDel
Large Language Models~(LLMs) have demonstrated incredible capabilities in understanding, generating, and manipulating languages.
OpenBA: An Open-sourced 15B Bilingual Asymmetric seq2seq Model Pre-trained from Scratch
This report provides the main details to pre-train an analogous model, including pre-training data processing, Bilingual Flan data collection, the empirical observations that inspire our model architecture design, training objectives of different stages, and other enhancement techniques.
LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models
Graphic layout generation, a growing research field, plays a significant role in user engagement and information perception.
Harnessing the Power of David against Goliath: Exploring Instruction Data Generation without Using Closed-Source Models
Instruction tuning is instrumental in enabling Large Language Models~(LLMs) to follow user instructions to complete various open-domain tasks.
GameEval: Evaluating LLMs on Conversational Games
In this paper, we propose GameEval, a novel approach to evaluating LLMs through goal-driven conversational games, overcoming the limitations of previous methods.
CMD: a framework for Context-aware Model self-Detoxification
In view of this, we introduce a Context-aware Model self-Detoxification~(CMD) framework that pays attention to both the context and the detoxification process, i. e., first detoxifying the context and then making the language model generate along the safe context.
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Diffusion models have gained significant attention in the realm of image generation due to their exceptional performance.
Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!
Diffusion models have been successfully adapted to text generation tasks by mapping the discrete text into the continuous space.
Test-Time Adaptation with Perturbation Consistency Learning
Accordingly, in this paper, we propose perturbation consistency learning (PCL), a simple test-time adaptation method to promote the model to make stable predictions for samples with distribution shifts.
Gated Mechanism Enhanced Multi-Task Learning for Dialog Routing
Although most existing methods can fulfil this requirement, they can only model single-source dialog data and cannot effectively capture the underlying knowledge of relations among data and subtasks.
RenewNAT: Renewing Potential Translation for Non-Autoregressive Transformer
Non-autoregressive neural machine translation (NAT) models are proposed to accelerate the inference process while maintaining relatively high performance.
AMOM: Adaptive Masking over Masking for Conditional Masked Language Model
Experiments on \textbf{3} different tasks (neural machine translation, summarization, and code generation) with \textbf{15} datasets in total confirm that our proposed simple method achieves significant performance improvement over the strong CMLM model.
Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Study
To answer this question, we evaluate test-time adaptation (TTA) to improve a model after deployment.
Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change
Existing methods mainly perform continual training to mitigate such a misalignment.
SelfMix: Robust Learning Against Textual Label Noise with Self-Mixup Training
The conventional success of textual classification relies on annotated data, and the new paradigm of pre-trained language models (PLMs) still requires a few labeled data for downstream tasks.
Chinese grammatical error correction based on knowledge distillation
In view of the poor robustness of existing Chinese grammatical error correction models on attack test sets and large model parameters, this paper uses the method of knowledge distillation to compress model parameters and improve the anti-attack ability of the model.
A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond
While NAR generation can significantly accelerate inference speed for machine translation, the speedup comes at the cost of sacrificed translation accuracy compared to its counterpart, autoregressive (AR) generation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+12
Image-text Retrieval: A Survey on Recent Research and Development
On top of this, the efficiency-focused study on the ITR system is introduced as the third perspective.
CT4Rec: Simple yet Effective Consistency Training for Sequential Recommendation
State-of-the-art sequential recommendation models proposed very recently combine contrastive learning techniques for obtaining high-quality user representations.
Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning
For this task, the adoption of pre-trained language models (such as BERT) has led to remarkable progress in a number of benchmarks.
DM-CT: Consistency Training with Data and Model Perturbation
The common approach of consistency training is performed on the data-level, which typically utilizes the data augmentation strategy (or adversarial training) to make the predictions from the augmented input and the original input to be consistent, so that the model is more robust and attains better generalization ability.
Are BERT Families Zero-Shot Learners? A Study on Their Potential and Limitations
Starting from the resurgence of deep learning, language models (LMs) have never been so popular.
R-Drop: Regularized Dropout for Neural Networks
Dropout is a powerful and widely used technique to regularize the training of deep neural networks.
Ranked #4 on
Machine Translation
on WMT2014 English-French
Learning to Organize a Bag of Words into Sentences with Neural Networks: An Empirical Study
Sequential information, a. k. a., orders, is assumed to be essential for processing a sequence with recurrent neural network or convolutional neural network based encoders.
Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots
To fill the gap between these up-to-date methods and the real-world applications, we incorporate user-specific dialogue history into the response selection and propose a personalized hybrid matching network (PHMN).
How does Truth Evolve into Fake News? An Empirical Study of Fake News Evolution
Automatically identifying fake news from the Internet is a challenging problem in deception detection tasks.
Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model
Experimental results show that our proposed method achieves significant performance improvements over the state-of-the-art pretrained cross-lingual language model in the CLCD setting.
Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training
To improve the robustness of self-training, in this paper we present class-aware feature self-distillation (CFd) to learn discriminative features from PrLMs, in which PrLM features are self-distilled into a feature adaptation module and the features from the same class are more tightly clustered.
Solution Path Algorithm for Twin Multi-class Support Vector Machine
Next, Lagrangian multipliers are proved to be 1 as the regularization parameter approaches infinity, thus, a simple yet effective initialization algorithm is devised.
Cross-Lingual Low-Resource Set-to-Description Retrieval for Global E-Commerce
We manually collect a new and high-quality paired dataset, where each pair contains an unordered product attribute set in the source language and an informative product description in the target language.
Modeling Personalization in Continuous Space for Response Generation via Augmented Wasserstein Autoencoders
In this work, we improve the WAE for response generation.
Stick to the Facts: Learning towards a Fidelity-oriented E-Commerce Product Description Generation
Different from other text generation tasks, in product description generation, it is of vital importance to generate faithful descriptions that stick to the product attribute information.
Are Training Samples Correlated? Learning to Generate Dialogue Responses with Multiple References
Due to its potential applications, open-domain dialogue generation has become popular and achieved remarkable progress in recent years, but sometimes suffers from generic responses.
Generating Classical Chinese Poems via Conditional Variational Autoencoder and Adversarial Training
It is a challenging task to automatically compose poems with not only fluent expressions but also aesthetic wording.
