Search Results for author:
Found 17 papers, 3 papers with code
HW-TSC’s Participation in the WMT 2021 Efficiency Shared Task
This paper presents the submission of Huawei Translation Services Center (HW-TSC) to WMT 2021 Efficiency Shared Task.
Large Language Models Orchestrating Structured Reasoning Achieve Kaggle Grandmaster Level
We introduce Agent K v1. 0, an end-to-end autonomous data science agent designed to automate, optimise, and generalise across diverse data science tasks.
FastAttention: Extend FlashAttention2 to NPUs and Low-resource GPUs
In this work, we propose FastAttention which pioneers the adaptation of FlashAttention series for NPUs and low-resource GPUs to boost LLM inference efficiency.
FlatQuant: Flatness Matters for LLM Quantization
In this paper, we propose FlatQuant (Fast and Learnable Affine Transformation), a new post-training quantization approach to enhance flatness of weights and activations.
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions
GPT-4o, an omni-modal model that enables vocal conversations with diverse emotions and tones, marks a milestone for omni-modal foundation models.
Compound Expression Recognition via Multi Model Ensemble for the ABAW7 Challenge
Compound Expression Recognition (CER) is vital for effective interpersonal interactions.
Facial Affect Recognition based on Multi Architecture Encoder and Feature Fusion for the ABAW7 Challenge
In this paper, we present our approach to addressing the challenges of the 7th ABAW competition.
IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
Such outliers are found to allocate most of the attention scores on initial tokens of input, termed as pivot tokens, which are crucial to the performance of quantized LLMs.
Machine Learning Insides OptVerse AI Solver: Design Principles and Applications
To this end, we present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI Solver, which aims to mitigate the scarcity of real-world mathematical programming instances, and to surpass the capabilities of traditional optimization techniques.
PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation
We then demonstrate that the proposed approach is significantly effective for enhancing the model nonlinearity through carefully designed ablations; thus, we present a new efficient model architecture for establishing modern, namely, PanGu-$\pi$.
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing
In this work, we develop a system that trained a trillion-parameter language model on a cluster of Ascend 910 AI processors and MindSpore framework, and present the language model with 1. 085T parameters named PanGu-{\Sigma}.
ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations
Moreover, we show that contrastive learning fails to learn domain-invariant features, which limits its transferability.
When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration Method
In this paper, to handle the problem and address the limitations of prior works, we propose a representation calibration method RCAL.
Deep Stock Trading: A Hierarchical Reinforcement Learning Framework for Portfolio Optimization and Order Execution
Portfolio management via reinforcement learning is at the forefront of fintech research, which explores how to optimally reallocate a fund into different financial assets over the long term by trial-and-error.
Correction of Faulty Background Knowledge based on Condition Aware and Revise Transformer for Question Answering
CAR-Transformer (1) revises each condition value based on the whole conversation and original conditions values, and (2) it encodes the revised conditions and utilizes the conditions embedding to select an answer.
Sub-Architecture Ensemble Pruning in Neural Architecture Search
Neural architecture search (NAS) is gaining more and more attention in recent years due to its flexibility and remarkable capability to reduce the burden of neural network design.
