Search Results for author:
Found 9 papers, 5 papers with code
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
To help the open-source community have a better understanding of Mixture-of-Experts (MoE) based large language models (LLMs), we train and release OpenMoE, a series of fully open-sourced and reproducible decoder-only MoE LLMs, ranging from 650M to 34B parameters and trained on up to over 1T tokens.
A Survey on Semantic Processing Techniques
We connect the surveyed tasks with downstream applications because this may inspire future scholars to fuse these low-level semantic processing tasks with high-level natural language processing tasks.
Finding the Pillars of Strength for Multi-Head Attention
Specifically, the heads of MHA were originally designed to attend to information from different representation subspaces, whereas prior studies found that some attention heads likely learn similar features and can be pruned without harming performance.
Logical Reasoning over Natural Language as Knowledge Representation: A Survey
This paper provides a comprehensive overview on a new paradigm of logical reasoning, which uses natural language as knowledge representation and pretrained language models as reasoners, including philosophical definition and categorization of logical reasoning, advantages of the new paradigm, benchmarks and methods, challenges of the new paradigm, possible future directions, and relation to related NLP fields.
Adaptive Knowledge Distillation between Text and Speech Pre-trained Models
Learning on a massive amount of speech corpus leads to the recent success of many self-supervised speech models.
A Class-Aware Representation Refinement Framework for Graph Classification
In this paper, we propose a Class-Aware Representation rEfinement (CARE) framework for the task of graph classification.
Fusing task-oriented and open-domain dialogues in conversational agents
It features inter-mode contextual dependency, i. e., the dialogue turns from the two modes depend on each other.
Ranked #1 on
Dialogue Generation
on FusedChat
Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey
To the best of our knowledge, this survey is the most comprehensive and up-to-date one at present for deep learning based dialogue systems, extensively covering the popular techniques.
An Embarrassingly Simple Model for Dialogue Relation Extraction
Dialogue relation extraction (RE) is to predict the relation type of two entities mentioned in a dialogue.
Ranked #9 on
Dialog Relation Extraction
on DialogRE
