Search Results for author:
Found 79 papers, 40 papers with code
Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
We introduce Seed1. 5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks.
Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning
Specifically, we align the steps of Chain-of-Thought (CoT) reasoning with loop iterations and apply intermediate supervision during the training of Looped Transformers.
Teaching Language Models to Critique via Reinforcement Learning
Teaching large language models (LLMs) to critique and refine their outputs is crucial for building systems that can iteratively improve, yet it is fundamentally limited by the ability to provide accurate judgments and actionable suggestions.
Efficient Supernet Training with Orthogonal Softmax for Scalable ASR Model Compression
ASR systems are deployed across diverse environments, each with specific hardware constraints.
A Unified Hyperparameter Optimization Pipeline for Transformer-Based Time Series Forecasting Models
Transformer-based models for time series forecasting (TSF) have attracted significant attention in recent years due to their effectiveness and versatility.
The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Model
Based on our observations, we propose the Babel Tower Hypothesis, which describes the entire process of LLMs acquiring new language capabilities.
FullStack Bench: Evaluating LLMs as Full Stack Coders
As the capabilities of code large language models (LLMs) continue to expand, their applications across diverse code intelligence domains are rapidly increasing.
Why Does the Effective Context Length of LLMs Fall Short?
Advancements in distributed training and efficient attention mechanisms have significantly expanded the context window sizes of large language models (LLMs).
FAN: Fourier Analysis Networks
Despite the remarkable successes of general-purpose neural networks, such as MLPs and Transformers, we find that they exhibit notable shortcomings in modeling and reasoning about periodic phenomena, achieving only marginal performance within the training domain and failing to generalize effectively to out-of-domain (OOD) scenarios.
Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting
We review the previous research works from a data-centric AI perspective and we intend to lay the foundation work for the future development of transformer-based architecture and data-centric AI.
Dynamic Encoder Size Based on Data-Driven Layer-wise Pruning for Speech Recognition
Varying-size models are often required to deploy ASR systems under different hardware and/or application constraints such as memory and latency.
Let the Code LLM Edit Itself When You Edit the Code
Simply encoding the edited subsequence and integrating it to the original KV cache meets the temporal confusion problem, leading to significantly worse performance.
An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
We present Expert-Token-Routing, a unified generalist framework that facilitates seamless integration of multiple expert LLMs.
Empowering Large Language Model Agents through Action Learning
Large Language Model (LLM) Agents have recently garnered increasing interest yet they are limited in their ability to learn from trial and error, a key element of intelligent behavior.
Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation
In this work, we leverage the intrinsic segmentation of language sequences and design a new positional encoding method called Bilevel Positional Encoding (BiPE).
InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks
In this paper, we introduce InfiAgent-DABench, the first benchmark specifically designed to evaluate LLM-based agents on data analysis tasks.
Transformer Multivariate Forecasting: Less is More?
From the model perspective, one of the PCA-enhanced models: PCA+Crossformer, reduces mean square errors (MSE) by 33. 3% and decreases runtime by 49. 2% on average.
Trustworthy AI: Deciding What to Decide
We aim to use this framework to conduct the TAI experiments by quantitive and qualitative research methods to satisfy TAI properties for the decision-making context.
CloudBrain-NMR: An Intelligent Cloud Computing Platform for NMR Spectroscopy Processing, Reconstruction and Analysis
Particularly, the emerging deep learning tools is hard to be widely used in NMR due to the sophisticated setup of computation.
Extrapolating Large Language Models to Non-English by Aligning Languages
We start from targeting individual languages by performing cross-lingual instruction-tuning (CoIT) on LLaMA, i. e. tuning it with translation task data and cross-lingual general task data to obtain cross-lingual models (x-LLaMAs), and formulate underlying scaling laws to investigate the advantages of using scalable translation data.
INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation
We propose an effective training framework INK to directly smooth the representation space via adjusting representations of kNN neighbors with a small number of new parameters.
M$^3$IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning
To tackle this challenge and promote research in the vision-language field, we introduce the Multi-Modal, Multilingual Instruction Tuning (M$^3$IT) dataset, designed to optimize VLM alignment with human instructions.
ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories
Recently, Large Language Models (LLMs) have been serving as general-purpose interfaces, posing a significant demand for comprehensive visual knowledge.
Can Language Models Understand Physical Concepts?
Language models~(LMs) gradually become general-purpose interfaces in the interactive and embodied world, where the understanding of physical concepts is an essential prerequisite.
Can We Edit Factual Knowledge by In-Context Learning?
Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge.
Extrapolating Multilingual Understanding Models as Multilingual Generators
Considering that encoder-based models have the advantage of efficient generation and self-correction abilities, this paper explores methods to empower multilingual understanding models the generation abilities to get a unified model.
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis
Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT).
A Challenging Benchmark for Low-Resource Learning
Experiments on a wide range of models show that neural networks, even pre-trained language models, have sharp performance drops on our benchmark, demonstrating the effectiveness on evaluating the weaknesses of neural networks.
OpenICL: An Open-Source Framework for In-context Learning
However, the implementation of ICL is sophisticated due to the diverse retrieval and inference methods involved, as well as the varying pre-processing requirements for different models, datasets, and tasks.
Analyzing And Improving Neural Speaker Embeddings for ASR
By further adding neural speaker embeddings, we gain additional ~3% relative WER improvement on Hub5'00.
A Survey on In-context Learning
With the increasing capabilities of large language models (LLMs), in-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP), where LLMs make predictions based on contexts augmented with a few examples.
Go-tuning: Improving Zero-shot Learning Abilities of Smaller Language Models
With increasing scale, large language models demonstrate both quantitative improvement and new qualitative capabilities, especially as zero-shot learners, like GPT-3.
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation
To address the needs of learning representations for all languages in a unified space, we propose a novel efficient training recipe, upon which we build an effective detachable model, Lego-MT.
BigText-QA: Question Answering over a Large-Scale Hybrid Knowledge Graph
Answering complex questions over textual resources remains a challenge, particularly when dealing with nuanced relationships between multiple entities expressed within natural-language sentences.
Enhancing and Adversarial: Improve ASR with Speaker Labels
Detailed analysis and experimental verification are conducted to show the optimal positions in the ASR neural network (NN) to apply speaker enhancing and adversarial training.
Calibrating Factual Knowledge in Pretrained Language Models
However, we find that facts stored in the PLMs are not always correct.
Improving the Training Recipe for a Robust Conformer-based Hybrid Model
In this work, we investigate various methods for speaker adaptive training (SAT) based on feature-space approaches for a conformer-based acoustic model (AM) on the Switchboard 300h dataset.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Contextual Representation Learning beyond Masked Language Modeling
How do masked language models (MLMs) such as BERT learn contextual representations?
KNAS: Green Neural Architecture Search
Many existing neural architecture search (NAS) solutions rely on downstream training for architecture evaluation, which takes enormous computations.
A Survey on Green Deep Learning
In recent years, larger and deeper models are springing up and continuously pushing state-of-the-art (SOTA) results across various fields like natural language processing (NLP) and computer vision (CV).
Conformer-based Hybrid ASR System for Switchboard Dataset
The recently proposed conformer architecture has been successfully used for end-to-end automatic speech recognition (ASR) architectures achieving state-of-the-art performance on different datasets.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Non-Autoregressive Models are Better Multilingual Translators
Multilingual machine translation aims to develop a single model for multiple language directions.
MTG: A Benchmark Suite for Multilingual Text Generation
We introduce MTG, a new benchmark suite for training and evaluating multilingual text generation.
Duplex Sequence-to-Sequence Learning for Reversible Machine Translation
Thus REDER enables reversible machine translation by simply flipping the input and output ends.
Information-theoretic Vocabularization via Optimal Transport
In this paper, we find an exciting relation between an information-theoretic feature and the performance of NLP tasks such as machine translation with a given vocabulary.
A Gradient-based Kernel Approach for Efficient Network Architecture Search
Inspired by our new finding, we explore a simple yet effective network architecture search (NAS) approach that leverages gradient correlation and gradient values to find well-performing architectures.
Vocabulary Learning via Optimal Transport for Neural Machine Translation
The choice of token vocabulary affects the performance of machine translation.
Graph-based Multi-hop Reasoning for Long Text Generation
The reasoning module is responsible for searching skeleton paths from a knowledge graph to imitate the imagination process in the human writing for semantic transfer.
MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
In this work, we explore parallel multi-scale representation learning on sequence data, striving to capture both long-range and short-range language structures.
Ranked #8 on
Machine Translation
on WMT2014 English-French
Understanding and Improving Layer Normalization
Unlike them, we find that the derivatives of the mean and variance are more important than forward normalization by re-centering and re-scaling backward gradients.
Ranked #5 on
Machine Translation
on IWSLT2015 English-Vietnamese
Specificity-Driven Cascading Approach for Unsupervised Sentiment Modification
The task of unsupervised sentiment modification aims to reverse the sentiment polarity of the input text while preserving its semantic content without any parallel data.
LexicalAT: Lexical-Based Adversarial Reinforcement Training for Robust Sentiment Classification
The generator learns to generate examples to attack the classifier while the classifier learns to defend these attacks.
Asking Clarification Questions in Knowledge-Based Question Answering
We provide representative baselines for these tasks and further introduce a coarse-to-fine model for clarification question generation.
VisDrone-DET2019: The Vision Meets Drone Object Detection in Image Challenge Results
Results of 33 object detection algorithms are presented.
Reasoning Over Semantic-Level Graph for Fact Checking
We evaluate our system on FEVER, a benchmark dataset for fact checking, and find that rich structural information is helpful and both our graph-based mechanisms improve the accuracy.
Ranked #2 on
Fact Verification
on FEVER
Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering
In this work, we propose to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence.
Ranked #14 on
Common Sense Reasoning
on CommonsenseQA
PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation
Through this method, we generate synthetic data using a large amount of unlabeled data in the target domain and then obtain a word segmentation model for the target domain.
Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
In this paper, we propose to generate comments with a graph-to-sequence model that models the input news as a topic interaction graph.
Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy
In order to avoid such sophisticated alternate optimization, we propose to learn unsupervised word mapping by directly maximizing the mean discrepancy between the distribution of transferred embedding and target embedding.
Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation
Existing text generation methods tend to produce repeated and {''}boring{''} expressions.
Knowledge-Aware Conversational Semantic Parsing Over Web Tables
Results show that our knowledge-aware model outperforms the state-of-the-art approaches.
Evaluating Semantic Rationality of a Sentence: A Sememe-Word-Matching Neural Network based on HowNet
To evaluate the effectiveness of the proposed model, we build a large-scale rationality evaluation dataset.
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
Different from conventional text generation tasks, the mapping between inputs and responses in conversations is more complicated, which highly demands the understanding of utterance-level semantic dependency, a relation between the whole meanings of inputs and outputs.
Ranked #2 on
Text Generation
on DailyDialog
Review-Driven Multi-Label Music Style Classification by Exploiting Style Correlations
This task requires the system to identify multiple styles of music based on its reviews on websites.
Learning Sentiment Memories for Sentiment Modification without Parallel Data
The task of sentiment modification requires reversing the sentiment of the input and preserving the sentiment-independent content.
A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation
Compared to the state-of-the-art models, our skeleton-based model can generate significantly more coherent text according to human evaluation and automatic evaluation.
Primal Meaning Recommendation via On-line Encyclopedia
We define the primal meaning of an expression to be a frequently used sense of that expression from which its other frequent senses can be deduced.
Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach
We evaluate our approach on two review datasets, Yelp and Amazon.
Ranked #6 on
Unsupervised Text Style Transfer
on Yelp
DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text
Existing text generation methods tend to produce repeated and "boring" expressions.
A Discourse-Level Named Entity Recognition and Relation Extraction Dataset for Chinese Literature Text
To build a high quality dataset, we propose two tagging methods to solve the problem of data inconsistency, including a heuristic tagging method and a machine auxiliary tagging method.
Training Simplification and Model Simplification for Deep Learning: A Minimal Effort Back Propagation Method
Based on the sparsified gradients, we further simplify the model by eliminating the rows or columns that are seldom updated, which will reduce the computational cost both in the training and decoding, and potentially accelerate decoding in real-world applications.
Deep Stacking Networks for Low-Resource Chinese Word Segmentation with Transfer Learning
In this paper, we propose a deep stacking framework to improve the performance on word segmentation tasks with insufficient data by integrating datasets from diverse domains.
Shallow Discourse Parsing with Maximum Entropy Model
The head-based representation of the PDTB is adopted in the arguments identifier, which turns the problem of indentifying the arguments of discourse connective into finding the head and end of the arguments.
Minimal Effort Back Propagation for Convolutional Neural Networks
As traditional neural network consumes a significant amount of computing resources during back propagation, \citet{Sun2017mePropSB} propose a simple yet effective technique to alleviate this problem.
Improving Semantic Relevance for Sequence-to-Sequence Learning of Chinese Social Media Text Summarization
In this work, our goal is to improve semantic relevance between source texts and summaries for Chinese social media summarization.
Transfer Deep Learning for Low-Resource Chinese Word Segmentation with a Novel Neural Network
First, we train a teacher model on high-resource corpora and then use the learned knowledge to initialize a student model.
