no code implementations • LT4HALA (LREC) 2022 • Hailin Zhang, Ziyu Yang, Yingwen Fu, Ruoyao Ding
In addition, we perform a series of training strategies based on the provided ancient Chinese pre-trained model to enhance the model performance.
Chinese Word Segmentation
Cultural Vocal Bursts Intensity Prediction
+5
no code implementations • 13 Aug 2024 • Haiyue Jing, Wenchi Cheng, Zan Li, Hailin Zhang
Numerical simulations are presented to validate and evaluate that our developed concentric UCAs based low-order RowComms can significantly increase the capacity as compared with that of singular UCA based RowComms.
no code implementations • 13 Aug 2024 • Wenchi Cheng, Haiyue Jing, Wei zhang, Zan Li, Hailin Zhang
To maintain the orthogonality among different OAM modes at the receiver, the strict alignment between transmit and receive antennas is highly demanded.
no code implementations • 13 Aug 2024 • Wenchi Cheng, Hailin Zhang, Liping Liang, Haiyue Jing, Zan Li
Traditional massive-MIMO based mmWave communications did not concern the potential spectrum-efficiency-gain (SE-gain) offered by orthogonal states of OAM.
no code implementations • 13 Aug 2024 • Haiyue Jing, Wenchi Cheng, Xiang-Gen Xia, Hailin Zhang
The plane wave based wireless communications have becoming more and more matured, along with the well utilization of the traditional resources such as time and frequency.
no code implementations • 10 Aug 2024 • Wenchi Cheng, Haiyue Jing, Wei zhang, Keyi Zhang, Hailin Zhang
We perform the two-dimension OAM modulation (TOM) and demodulation (TOD) schemes with the orthogonal OAM mode number exceeding the array-element number, which is beyond the traditional concept of multiple antennas based wireless communications.
no code implementations • 2 Aug 2024 • Fan Qin, Yifei Liu, Chao Gu, Linfeng Zeng, Wenchi Cheng, Hailin Zhang, Steven Gao
To accomplish the reconfiguration of both unidirectional and bidirectional radiation states in the +z, -z, and +/-z directions, a polarization switchable multi-feed array (MFA) is placed at the focal plane between the TA and FTA, radiating x-polarization, y-polarization, and 45-degree oblique polarization waves, respectively.
no code implementations • 24 Jul 2024 • Zhuohui Yao, Wenchi Cheng, Wei zhang, Hailin Zhang
Numerical results show that the new heterogeneous Fisher-Snedecor $\mathcal{F}$ composite fading channel adapted resource allocation schemes can achieve higher capacity and energy efficiency than those of traditional channel model adapted resource allocation schemes, thus providing better communications service for post-disaster areas.
no code implementations • 24 Jul 2024 • Jianyu Wang, Wenchi Cheng, Wei zhang, Hailin Zhang
In ZIMS-VFD, the transceiver inserts a zero-interval for each symbol in the transmit signal and provides self-interference (SI)-free intervals for itself.
no code implementations • 18 Jul 2024 • Liping Liang, Wenchi Cheng, Wei zhang, Hailin Zhang
In this paper, a hybrid orthogonal division multiplexing (HODM) scheme by using OAM multiplexing and orthogonal frequency division multiplexing (OFDM) in conjunction is proposed to achieve high-capacity wireless communications in sparse multipath environments, where the scatterers are sparse.
no code implementations • 18 Jul 2024 • Liping Liang, Wenchi Cheng, Wei zhang, Hailin Zhang
In particular, we propose the mode hopping (MH) scheme for antijamming within the narrow frequency band.
no code implementations • 18 Jul 2024 • Liping Liang, Wenchi Cheng, Wei zhang, Hailin Zhang
In this paper, we propose an MH with OAM-based index modulation scheme, where several OAM-modes are activated for hopping, to achieve high SE at a given bit error rate in radio vortex wireless communications.
no code implementations • 17 Jul 2024 • Liping Liang, Wenchi Cheng, Wei zhang, Hailin Zhang
To achieve efficient anti-jamming and increase SE of wireless communications with slight computational complexity cost, in this paper we propose an index-modulation embedded mode-hopping (IM-MH) scheme, which simultaneously activates several OAM-modes for hopping along with additional index information and signal information transmission.
no code implementations • 16 Jul 2024 • Minmin Wang, Liping Liang, Wenchi Cheng, Wei zhang, Ruirui Chen, Hailin Zhang
As a kind of wavefront with helical phase, orbital angular momentum (OAM) shows the great potential to enhance the security results of wireless communications due to its unique orthogonality and central hollow electromagnetic wave structure.
no code implementations • 16 Jul 2024 • Pinxue Zhao, Hailin Zhang, Fangcheng Fu, Xiaonan Nie, Qibin Liu, Fang Yang, Yuanbo Peng, Dian Jiao, Shuaipeng Li, Jinbao Xue, Yangyu Tao, Bin Cui
By leveraging fine-grained activation memory management, MEMO facilitates efficient training of 7B LLM with 1 million sequence length on just 8 A800 GPUs, achieving an MFU of 52. 30%.
no code implementations • 1 Jul 2024 • Hailin Zhang, Xiaodong Ji, Yilin Chen, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, WeiPeng Chen, Bin Cui
During the prefilling phase, we apply PQ to tokens' keys for each LLM layer and head.
no code implementations • 8 Jun 2024 • Runyu Lyu, Wenchi Cheng, Bazhong Shen, Zhiyuan Ren, Hailin Zhang
In this article, we introduce the OAM-based SWIPT as well as discuss some basic advantages and challenges for it.
no code implementations • 23 May 2024 • Shuaipeng Li, Penghao Zhao, Hailin Zhang, Xingwu Sun, Hao Wu, Dian Jiao, Weiyan Wang, Chengjun Liu, Zheng Fang, Jinbao Xue, Yangyu Tao, Bin Cui, Di Wang
First, we raise the scaling law between batch sizes and optimal learning rates in the sign of gradient case, in which we prove that the optimal learning rate first rises and then falls as the batch size increases.
3 code implementations • 29 Feb 2024 • Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Jie Jiang, Bin Cui
We first classify RAG foundations according to how the retriever augments the generator, distilling the fundamental abstractions of the augmentation methodologies for various retrievers and generators.
1 code implementation • 6 Dec 2023 • Hailin Zhang, Zirui Liu, Boxuan Chen, Yikai Zhao, Tong Zhao, Tong Yang, Bin Cui
Guided by our design philosophy, we further propose a multi-level hash embedding framework to optimize the embedding tables of non-hot features.
1 code implementation • 27 Nov 2023 • Hailin Zhang, Penghao Zhao, Xupeng Miao, Yingxia Shao, Zirui Liu, Tong Yang, Bin Cui
Learnable embedding vector is one of the most important applications in machine learning, and is widely used in various database-related domains.
1 code implementation • NeurIPS 2023 • Hailin Zhang, Yujing Wang, Qi Chen, Ruiheng Chang, Ting Zhang, Ziming Miao, Yingyan Hou, Yang Ding, Xupeng Miao, Haonan Wang, Bochen Pang, Yuefeng Zhan, Hao Sun, Weiwei Deng, Qi Zhang, Fan Yang, Xing Xie, Mao Yang, Bin Cui
We empirically show that our model achieves better performance on the commonly used academic benchmarks MSMARCO Passage and Natural Questions, with comparable serving latency to dense retrieval solutions.
1 code implementation • 11 Jun 2023 • Hailin Zhang, Defang Chen, Can Wang
Multi-Teacher knowledge distillation provides students with additional supervision from multiple pre-trained teachers with diverse information sources.
2 code implementations • 25 Nov 2022 • Xupeng Miao, Yujie Wang, Youhe Jiang, Chunan Shi, Xiaonan Nie, Hailin Zhang, Bin Cui
Transformer models have achieved state-of-the-art performance on various domains of applications and gradually becomes the foundations of the advanced large deep learning (DL) models.
1 code implementation • CVPR 2022 • Defang Chen, Jian-Ping Mei, Hailin Zhang, Can Wang, Yan Feng, Chun Chen
Knowledge distillation aims to compress a powerful yet cumbersome teacher model into a lightweight student model without much sacrifice of performance.
Ranked #4 on
Knowledge Distillation
on CIFAR-100
1 code implementation • 30 Dec 2021 • Hailin Zhang, Defang Chen, Can Wang
Knowledge distillation is initially introduced to utilize additional supervision from a single teacher model for the student model training.
3 code implementations • 14 Dec 2021 • Xupeng Miao, Hailin Zhang, Yining Shi, Xiaonan Nie, Zhi Yang, Yangyu Tao, Bin Cui
Embedding models have been an effective learning paradigm for high-dimensional data.
no code implementations • 22 Aug 2019 • Man Qi, Niv DeMalach, Tao Sun, Hailin Zhang
Thus, we developed an extension of resource competition theory to investigate partial and total preemption (in the latter, the preemptor is unaffected by species with lower preemption rank).