Search Results for author:
Found 32 papers, 19 papers with code
Leveraging Word-Formation Knowledge for Chinese Word Sense Disambiguation
In this paper, we propose to leverage word-formation knowledge to enhance Chinese WSD.
基于词信息嵌入的汉语构词结构识别研究(Chinese Word-Formation Prediction based on Representations of Word-Related Features)
“作为一种意合型语言, 汉语中的构词结构刻画了构词成分之间的组合关系, 是认知、理解词义的关键。在中文信息处理领域, 此前的构词结构识别工作大多沿用句法层面的粗粒度标签, 且主要基于上下文等词间信息建模, 忽略了语素义、词义等词内信息对构词结构识别的作用。本文采用语言学视域下的构词结构标签体系, 构建汉语构词结构及相关信息数据集, 提出了一种基于Bi-LSTM和Self-attention的模型, 以此来探究词内、词间等多方面信息对构词结构识别的潜在影响和能达到的性能。实验取得了良好的预测效果, 准确率77. 87%, F1值78. 36%;同时, 对比测试揭示, 词内的语素义信息对构词结构识别具有显著的贡献, 而词间的上下文信息贡献较弱且带有较强的不稳定性。该预测方法与数据集, 将为中文信息处理的多种任务, 如语素和词结构分析、词义识别与生成、语言文字研究与词典编纂等提供新的观点和方案。”
Large Language Models Are Unconscious of Unreasonability in Math Problems
Large language models (LLMs) demonstrate substantial capabilities in solving math problems.
PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
Although LoRA fine-tuning is effective, there is still a performance gap compared to full fine-tuning, since its weight update is limited to low-rank matrices.
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Subsequently, we scale up DeepSeekMoE to 16B parameters and show that it achieves comparable performance with LLaMA2 7B, with only about 40% of computations.
Language Models Understand Numbers, at Least Partially
Large language models (LLMs) have exhibited impressive competence in various tasks, but their opaque internal mechanisms hinder their use in mathematical problems.
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
The rapid development of open-source large language models (LLMs) has been truly remarkable.
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
In this paper, we present an innovative process-oriented math process reward model called \textbf{Math-Shepherd}, which assigns a reward score to each step of math problem solutions.
Ranked #13 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning
To assess the quality of weights in the absence of additional validation data, we design a masked self-prediction (MSP) score that exhibits a strong correlation with the final ICL performance.
Denoising Bottleneck with Mutual Information Maximization for Video Multimodal Fusion
However, unlike other image-text multimodal tasks, video has longer multimodal sequences with more redundancy and noise in both visual and audio modalities.
Bi-Drop: Enhancing Fine-tuning Generalization via Synchronous sub-net Estimation and Optimization
Also, Bi-Drop needs only one mini-batch to estimate the sub-net so it achieves higher utility of training data.
Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning
In-context learning (ICL) emerges as a promising capability of large language models (LLMs) by providing them with demonstration examples to perform diverse tasks.
A Survey on In-context Learning
With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few examples.
Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers
We comprehensively compare the behaviors of in-context learning and explicit finetuning on real tasks to provide empirical evidence that supports our understanding.
Calibrating Factual Knowledge in Pretrained Language Models
However, we find that facts stored in the PLMs are not always correct.
Neural Knowledge Bank for Pretrained Transformers
The ability of pretrained Transformers to remember factual knowledge is essential but still limited for existing models.
Robust Fine-tuning via Perturbation and Interpolation from In-batch Instances
For each instance in a batch, we involve other instances in the same batch to interact with it.
On the Representation Collapse of Sparse Mixture of Experts
We also present a comprehensive analysis on the representation and routing behaviors of our models.
StableMoE: Stable Routing Strategy for Mixture of Experts
We point out that existing learning-to-route MoE methods suffer from the routing fluctuation issue, i. e., the target expert of the same input may change along with training, but only one expert will be activated for the input during inference.
Mixture of Experts for Biomedical Question Answering
In this paper, in order to alleviate the parameter competition problem, we propose a Mixture-of-Expert (MoE) based question answering method called MoEBQA that decouples the computation for different types of questions by sparse routing.
Hierarchical Curriculum Learning for AMR Parsing
Abstract Meaning Representation (AMR) parsing aims to translate sentences to semantic representation with a hierarchical structure, and is recently empowered by pretrained sequence-to-sequence models.
Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classification
However, we find they suffer from trigger biases that signify the statistical homogeneity between some trigger words and target event types, which we summarize as trigger overlapping and trigger separability.
Explicit Interaction Network for Aspect Sentiment Triplet Extraction
Table encoder extracts sentiment at token-pair level, so that the compositional feature between targets and opinions can be easily captured.
Decompose, Fuse and Generate: A Formation-Informed Method for Chinese Definition Generation
In this paper, we tackle the task of Definition Generation (DG) in Chinese, which aims at automatically generating a definition for a word.
Knowledge Neurons in Pretrained Transformers
In this paper, we present preliminary studies on how factual knowledge is stored in pretrained Transformers by introducing the concept of knowledge neurons.
Incorporating Connections Beyond Knowledge Embeddings: A Plug-and-Play Module to Enhance Commonsense Reasoning in Machine Reading Comprehension
Conventional Machine Reading Comprehension (MRC) has been well-addressed by pattern matching, but the ability of commonsense reasoning remains a gap between humans and machines.
Coarse-to-Fine Entity Representations for Document-level Relation Extraction
In classification, we combine the entity representations from both two levels into more comprehensive representations for relation extraction.
Ranked #34 on
Relation Extraction
on DocRED
Inductively Representing Out-of-Knowledge-Graph Entities by Optimal Estimation Under Translational Assumptions
Conventional Knowledge Graph Completion (KGC) assumes that all test entities appear during training.
Learning to Control the Fine-grained Sentiment for Story Ending Generation
Therefore, we propose a generic and novel framework which consists of a sentiment analyzer and a sentimental generator, respectively addressing the two challenges.
LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts
We introduce the task of automatic live commenting.
Sememe Prediction: Learning Semantic Knowledge from Unstructured Textual Wiki Descriptions
In the experiments, we take a real-world sememe knowledge base HowNet and the corresponding descriptions of the words in Baidu Wiki for training and evaluation.
Live Video Comment Generation Based on Surrounding Frames and Live Comments
In this paper, we propose the task of live comment generation.
