Search Results for author:
Found 67 papers, 37 papers with code
An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
We present Expert-Token-Routing, a unified generalist framework that facilitates seamless integration of multiple expert LLMs.
Empowering Large Language Model Agents through Action Learning
Large Language Model (LLM) Agents have recently garnered increasing interest yet they are limited in their ability to learn from trial and error, a key element of intelligent behavior.
Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation
In this work, we leverage the intrinsic segmentation of language sequences and design a new positional encoding method called Bilevel Positional Encoding (BiPE).
InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks
In this paper, we introduce InfiAgent-DABench, the first benchmark specifically designed to evaluate LLM-based agents on data analysis tasks.
Transformer Multivariate Forecasting: Less is More?
From the model perspective, one of the PCA-enhanced models: PCA+Crossformer, reduces mean square errors (MSE) by 33. 3% and decreases runtime by 49. 2% on average.
Trustworthy AI: Deciding What to Decide
We aim to use this framework to conduct the TAI experiments by quantitive and qualitative research methods to satisfy TAI properties for the decision-making context.
CloudBrain-NMR: An Intelligent Cloud Computing Platform for NMR Spectroscopy Processing, Reconstruction and Analysis
Particularly, the emerging deep learning tools is hard to be widely used in NMR due to the sophisticated setup of computation.
Extrapolating Large Language Models to Non-English by Aligning Languages
We start from targeting individual languages by performing cross-lingual instruction-tuning (CoIT) on LLaMA, i. e. tuning it with translation task data and cross-lingual general task data to obtain cross-lingual models (x-LLaMAs), and formulate underlying scaling laws to investigate the advantages of using scalable translation data.
INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation
We propose an effective training framework INK to directly smooth the representation space via adjusting representations of kNN neighbors with a small number of new parameters.
M$^3$IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning
To tackle this challenge and promote research in the vision-language field, we introduce the Multi-Modal, Multilingual Instruction Tuning (M$^3$IT) dataset, designed to optimize VLM alignment with human instructions.
ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories
Recently, Large Language Models (LLMs) have been serving as general-purpose interfaces, posing a significant demand for comprehensive visual knowledge.
Can Language Models Understand Physical Concepts?
Language models~(LMs) gradually become general-purpose interfaces in the interactive and embodied world, where the understanding of physical concepts is an essential prerequisite.
Extrapolating Multilingual Understanding Models as Multilingual Generators
Considering that encoder-based models have the advantage of efficient generation and self-correction abilities, this paper explores methods to empower multilingual understanding models the generation abilities to get a unified model.
Can We Edit Factual Knowledge by In-Context Learning?
Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge.
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis
Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT).
A Challenging Benchmark for Low-Resource Learning
Experiments on a wide range of models show that neural networks, even pre-trained language models, have sharp performance drops on our benchmark, demonstrating the effectiveness on evaluating the weaknesses of neural networks.
OpenICL: An Open-Source Framework for In-context Learning
However, the implementation of ICL is sophisticated due to the diverse retrieval and inference methods involved, as well as the varying pre-processing requirements for different models, datasets, and tasks.
Analyzing And Improving Neural Speaker Embeddings for ASR
By further adding neural speaker embeddings, we gain additional ~3% relative WER improvement on Hub5'00.
A Survey on In-context Learning
With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few examples.
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation
To address the needs of learning representations for all languages in a unified space, we propose a novel efficient training recipe, upon which we build an effective detachable model, Lego-MT.
Go-tuning: Improving Zero-shot Learning Abilities of Smaller Language Models
With increasing scale, large language models demonstrate both quantitative improvement and new qualitative capabilities, especially as zero-shot learners, like GPT-3.
BigText-QA: Question Answering over a Large-Scale Hybrid Knowledge Graph
Answering complex questions over textual resources remains a challenge, particularly when dealing with nuanced relationships between multiple entities expressed within natural-language sentences.
Enhancing and Adversarial: Improve ASR with Speaker Labels
Detailed analysis and experimental verification are conducted to show the optimal positions in the ASR neural network (NN) to apply speaker enhancing and adversarial training.
Calibrating Factual Knowledge in Pretrained Language Models
However, we find that facts stored in the PLMs are not always correct.
Improving the Training Recipe for a Robust Conformer-based Hybrid Model
In this work, we investigate various methods for speaker adaptive training (SAT) based on feature-space approaches for a conformer-based acoustic model (AM) on the Switchboard 300h dataset.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Contextual Representation Learning beyond Masked Language Modeling
How do masked language models (MLMs) such as BERT learn contextual representations?
KNAS: Green Neural Architecture Search
Many existing neural architecture search (NAS) solutions rely on downstream training for architecture evaluation, which takes enormous computations.
A Survey on Green Deep Learning
In recent years, larger and deeper models are springing up and continuously pushing state-of-the-art (SOTA) results across various fields like natural language processing (NLP) and computer vision (CV).
Conformer-based Hybrid ASR System for Switchboard Dataset
The recently proposed conformer architecture has been successfully used for end-to-end automatic speech recognition (ASR) architectures achieving state-of-the-art performance on different datasets.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Non-Autoregressive Models are Better Multilingual Translators
Multilingual machine translation aims to develop a single model for multiple language directions.
MTG: A Benchmark Suite for Multilingual Text Generation
We introduce MTG, a new benchmark suite for training and evaluating multilingual text generation.
Duplex Sequence-to-Sequence Learning for Reversible Machine Translation
Thus REDER enables reversible machine translation by simply flipping the input and output ends.
Information-theoretic Vocabularization via Optimal Transport
In this paper, we find an exciting relation between an information-theoretic feature and the performance of NLP tasks such as machine translation with a given vocabulary.
A Gradient-based Kernel Approach for Efficient Network Architecture Search
Inspired by our new finding, we explore a simple yet effective network architecture search (NAS) approach that leverages gradient correlation and gradient values to find well-performing architectures.
Vocabulary Learning via Optimal Transport for Neural Machine Translation
The choice of token vocabulary affects the performance of machine translation.
Graph-based Multi-hop Reasoning for Long Text Generation
The reasoning module is responsible for searching skeleton paths from a knowledge graph to imitate the imagination process in the human writing for semantic transfer.
MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
In this work, we explore parallel multi-scale representation learning on sequence data, striving to capture both long-range and short-range language structures.
Ranked #8 on
Machine Translation
on WMT2014 English-French
Understanding and Improving Layer Normalization
Unlike them, we find that the derivatives of the mean and variance are more important than forward normalization by re-centering and re-scaling backward gradients.
Ranked #5 on
Machine Translation
on IWSLT2015 English-Vietnamese
LexicalAT: Lexical-Based Adversarial Reinforcement Training for Robust Sentiment Classification
The generator learns to generate examples to attack the classifier while the classifier learns to defend these attacks.
Specificity-Driven Cascading Approach for Unsupervised Sentiment Modification
The task of unsupervised sentiment modification aims to reverse the sentiment polarity of the input text while preserving its semantic content without any parallel data.
Asking Clarification Questions in Knowledge-Based Question Answering
We provide representative baselines for these tasks and further introduce a coarse-to-fine model for clarification question generation.
VisDrone-DET2019: The Vision Meets Drone Object Detection in Image Challenge Results
Results of 33 object detection algorithms are presented.
Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering
In this work, we propose to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence.
Ranked #13 on
Common Sense Reasoning
on CommonsenseQA
Reasoning Over Semantic-Level Graph for Fact Checking
We evaluate our system on FEVER, a benchmark dataset for fact checking, and find that rich structural information is helpful and both our graph-based mechanisms improve the accuracy.
Ranked #2 on
Fact Verification
on FEVER
Coherent Comments Generation for Chinese Articles with a Graph-to-Sequence Model
In this paper, we propose to generate comments with a graph-to-sequence model that models the input news as a topic interaction graph.
PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation
Through this method, we generate synthetic data using a large amount of unlabeled data in the target domain and then obtain a word segmentation model for the target domain.
Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
In this paper, we propose to generate comments with a graph-to-sequence model that models the input news as a topic interaction graph.
Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy
In order to avoid such sophisticated alternate optimization, we propose to learn unsupervised word mapping by directly maximizing the mean discrepancy between the distribution of transferred embedding and target embedding.
Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation
Existing text generation methods tend to produce repeated and {''}boring{''} expressions.
Knowledge-Aware Conversational Semantic Parsing Over Web Tables
Results show that our knowledge-aware model outperforms the state-of-the-art approaches.
Evaluating Semantic Rationality of a Sentence: A Sememe-Word-Matching Neural Network based on HowNet
To evaluate the effectiveness of the proposed model, we build a large-scale rationality evaluation dataset.
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
Different from conventional text generation tasks, the mapping between inputs and responses in conversations is more complicated, which highly demands the understanding of utterance-level semantic dependency, a relation between the whole meanings of inputs and outputs.
Ranked #1 on
Text Generation
on DailyDialog
Review-Driven Multi-Label Music Style Classification by Exploiting Style Correlations
This task requires the system to identify multiple styles of music based on its reviews on websites.
Learning Sentiment Memories for Sentiment Modification without Parallel Data
The task of sentiment modification requires reversing the sentiment of the input and preserving the sentiment-independent content.
A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation
Compared to the state-of-the-art models, our skeleton-based model can generate significantly more coherent text according to human evaluation and automatic evaluation.
Primal Meaning Recommendation via On-line Encyclopedia
We define the primal meaning of an expression to be a frequently used sense of that expression from which its other frequent senses can be deduced.
Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach
We evaluate our approach on two review datasets, Yelp and Amazon.
Ranked #6 on
Unsupervised Text Style Transfer
on Yelp
DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
Existing text generation methods tend to produce repeated and "boring" expressions.
A Discourse-Level Named Entity Recognition and Relation Extraction Dataset for Chinese Literature Text
To build a high quality dataset, we propose two tagging methods to solve the problem of data inconsistency, including a heuristic tagging method and a machine auxiliary tagging method.
Training Simplification and Model Simplification for Deep Learning: A Minimal Effort Back Propagation Method
Based on the sparsified gradients, we further simplify the model by eliminating the rows or columns that are seldom updated, which will reduce the computational cost both in the training and decoding, and potentially accelerate decoding in real-world applications.
Deep Stacking Networks for Low-Resource Chinese Word Segmentation with Transfer Learning
In this paper, we propose a deep stacking framework to improve the performance on word segmentation tasks with insufficient data by integrating datasets from diverse domains.
Shallow Discourse Parsing with Maximum Entropy Model
The head-based representation of the PDTB is adopted in the arguments identifier, which turns the problem of indentifying the arguments of discourse connective into finding the head and end of the arguments.
Improving Social Media Text Summarization by Learning Sentence Weight Distribution
Recently, encoder-decoder models are widely used in social media text summarization.
Minimal Effort Back Propagation for Convolutional Neural Networks
As traditional neural network consumes a significant amount of computing resources during back propagation, \citet{Sun2017mePropSB} propose a simple yet effective technique to alleviate this problem.
Improving Semantic Relevance for Sequence-to-Sequence Learning of Chinese Social Media Text Summarization
In this work, our goal is to improve semantic relevance between source texts and summaries for Chinese social media summarization.
Transfer Deep Learning for Low-Resource Chinese Word Segmentation with a Novel Neural Network
First, we train a teacher model on high-resource corpora and then use the learned knowledge to initialize a student model.
