Search Results for author:
Found 29 papers, 11 papers with code
BIT-Xiaomi’s System for AutoSimTrans 2022
This system paper describes the BIT-Xiaomi simultaneous translation system for Autosimtrans 2022 simultaneous translation challenge.
Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study
Large language models (LLMs) have shown continuously improving multilingual capabilities, and even small-scale open-source models have demonstrated rapid performance enhancement.
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models
To address this, we propose Weight-Decomposed Tensor Adaptation (DoTA), which leverages the Matrix Product Operator (MPO) decomposition of pre-trained weights for effective initialization in fine-tuning LLMs.
KG-Retriever: Efficient Knowledge Indexing for Retrieval-Augmented Large Language Models
Large language models with retrieval-augmented generation encounter a pivotal challenge in intricate retrieval tasks, e. g., multi-hop question answering, which requires the model to navigate across multiple documents and generate comprehensive responses based on fragmented information.
HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
Furthermore, we conduct a detailed analysis of rotary position encoding (RoPE, a prevalent relative positional encoding in LLMs), and found that the U-shape attention is caused by some learned components, which are also the key factor limiting RoPE's expressiveness and extrapolation. Inspired by these insights, we propose High-frequency rotary Position Encoding (HoPE).
PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning
Low-rank adaptation (LoRA) and its variants have recently gained much interest due to their ability to avoid excessive inference costs.
ToolPlanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedback
Recently, tool-augmented LLMs have gained increasing attention.
MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding
These fine-tuned VLMs may still ignore the relationships between UI pages, neglect the roles of elements in page transitions and lack inter-UI understanding.
Mixture of Diverse Size Experts
The Sparsely-Activated Mixture-of-Experts (MoE) has gained increasing popularity for scaling up large language models (LLMs) without exploding computational costs.
LLaVA-SG: Leveraging Scene Graphs as Visual Semantic Expression in Vision-Language Models
Recent advances in large vision-language models (VLMs) typically employ vision encoders based on the Vision Transformer (ViT) architecture.
Pruning Large Language Models to Intra-module Low-rank Architecture with Transitional Activations
However, as the coarse-grained structured pruning poses large damage to the highly interconnected model, achieving a high compression ratio for scaled-up LLMs remains a challenge.
Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents
With the remarkable advancements of large language models (LLMs), LLM-based agents have become a research hotspot in human-computer interaction.
SUBLLM: A Novel Efficient Architecture with Token Sequence Subsampling for LLM
In inference, it boosts speeds by up to 37% and reduces memory by 1GB per GPU.
ToolRerank: Adaptive and Hierarchy-Aware Reranking for Tool Retrieval
Specifically, our proposed ToolRerank includes Adaptive Truncation, which truncates the retrieval results related to seen and unseen tools at different positions, and Hierarchy-Aware Reranking, which makes retrieval results more concentrated for single-tool queries and more diverse for multi-tool queries.
A Comprehensive Evaluation of Quantization Strategies for Large Language Models
Our experimental results indicate that LLMs with 4-bit quantization can retain performance comparable to their non-quantized counterparts, and perplexity can serve as a proxy metric for quantized LLMs on most benchmarks.
Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security
Next, we discuss several key challenges to achieve intelligent, efficient and secure Personal LLM Agents, followed by a comprehensive survey of representative solutions to address these challenges.
CBSiMT: Mitigating Hallucination in Simultaneous Machine Translation with Weighted Prefix-to-Prefix Training
Simultaneous machine translation (SiMT) is a challenging task that requires starting translation before the full source sentence is available.
DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy
Recent advances in large language models (LLMs) have revolutionized the landscape of reasoning tasks.
CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?
We present the Chinese Elementary School Math Word Problems (CMATH) dataset, comprising 1. 7k elementary school-level math word problems with detailed annotations, source from actual Chinese workbooks and exams.
UniMC: A Unified Framework for Long-Term Memory Conversation via Relevance Representation Learning
In this paper, we propose a Unified framework for Long-term Memory Conversations (UniMC), which increases the connection between different stages by learning relevance representation.
Exploring Better Text Image Translation with Multimodal Codebook
Text image translation (TIT) aims to translate the source texts embedded in the image to target translations, which has a wide range of applications and thus has important research value.
Rethinking the Reasonability of the Test Set for Simultaneous Machine Translation
Simultaneous machine translation (SimulMT) models start translation before the end of the source sentence, making the translation monotonically aligned with the source sentence.
BERT-ERC: Fine-tuning BERT is Enough for Emotion Recognition in Conversation
Accordingly, we propose a novel paradigm, i. e., exploring contextual information and dialogue structure information in the fine-tuning step, and adapting the PLM to the ERC task in terms of input text, classification structure, and training strategy.
Improve Bilingual TTS Using Dynamic Language and Phonology Embedding
We introduce phonology embedding to capture the English differences between different phonology.
PAMA-TTS: Progression-Aware Monotonic Attention for Stable Seq2Seq TTS With Accurate Phoneme Duration Control
Sequence expansion between encoder and decoder is a critical challenge in sequence-to-sequence TTS.
HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis
To tackle the difficulty of singing modeling caused by high sampling rate (wider frequency band and longer waveform), we introduce multi-scale adversarial training in both the acoustic model and vocoder to improve singing modeling.
DeepSinger: Singing Voice Synthesis with Data Mined From the Web
DeepSinger has several advantages over previous SVS systems: 1) to the best of our knowledge, it is the first SVS system that directly mines training data from music websites, 2) the lyrics-to-singing alignment model further avoids any human efforts for alignment labeling and greatly reduces labeling cost, 3) the singing model based on a feed-forward Transformer is simple and efficient, by removing the complicated acoustic feature modeling in parametric synthesis and leveraging a reference encoder to capture the timbre of a singer from noisy singing data, and 4) it can synthesize singing voices in multiple languages and multiple singers.
Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizer
This paper presents a high quality singing synthesizer that is able to model a voice with limited available recordings.
XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System
This paper presents XiaoiceSing, a high-quality singing voice synthesis system which employs an integrated network for spectrum, F0 and duration modeling.
