Search Results for author:
Found 99 papers, 16 papers with code
Learning Joint Source-Channel Encoding in IRS-assisted Multi-User Semantic Communications
We aim to maximize the users' semantic throughput by jointly optimizing the users' scheduling, the IRS's passive beamforming, and the semantic encoding strategies.
Generative AI Enabled Robust Data Augmentation for Wireless Sensing in ISAC Networks
Building on this, the scheme further utilizes another diffusion model to enhance the sample quality, thereby facilitating the data augmentation in scenarios where the original sensing data is insufficient and unevenly distributed.
Meta-Computing Enhanced Federated Learning in IIoT: Satisfaction-Aware Incentive Scheme via DRL-Based Stackelberg Game
This approach ensures balanced rewards and enhances the applicability of the incentive scheme for IIoT.
Aerial Reliable Collaborative Communications for Terrestrial Mobile Users via Evolutionary Multi-Objective Deep Reinforcement Learning
Specifically, this algorithm introduces an evolutionary learning approach to obtain the approximate Pareto set for the formulated MOP.
Deep Reinforcement Learning
Multi-Objective Reinforcement Learning
Detecting Neurocognitive Disorders through Analyses of Topic Evolution and Cross-modal Consistency in Visual-Stimulated Narratives
DTM-based approach validated the effectiveness of dynamic topic consistency as a macrostructural metric (F1=0. 61, AUC=0. 78).
Not All Errors Are Equal: Investigation of Speech Recognition Errors in Alzheimer's Disease Detection
Automatic Speech Recognition (ASR) plays an important role in speech-based automatic detection of Alzheimer's disease (AD).
Generative AI for Data Augmentation in Wireless Networks: Analysis, Applications, and Case Study
This article systemically explores the potential and effectiveness of GenAI-driven data augmentation in wireless networks.
Improving Grapheme-to-Phoneme Conversion through In-Context Knowledge Retrieval with Large Language Models
Using GPT-4 in the ICKR system can increase of 3. 5% absolute (3. 8% relative) on the Librig2p dataset.
Diffusion-based Auction Mechanism for Efficient Resource Management in 6G-enabled Vehicular Metaverses
In 6G-enable Vehicular Metaverses, vehicles are represented by Vehicle Twins (VTs), which serve as digital replicas of physical vehicles to support real-time vehicular applications such as large Artificial Intelligence (AI) model-based Augmented Reality (AR) navigation, called VT tasks.
MetaTrading: An Immersion-Aware Model Trading Framework for Vehicular Metaverse Services
To comprehensively evaluate the contribution of locally trained learning models provided by MUs to AR services, we design a new immersion metric that captures service immersion by considering the freshness and accuracy of learning models, as well as the amount and potential value of raw data used for training.
Generative Diffusion-based Contract Design for Efficient AI Twins Migration in Vehicular Embodied AI Networks
Embodied twins are digital models of these embodied agents, with various embodied AI twins for intelligent applications in cyberspace.
Toward Mixture-of-Experts Enabled Trustworthy Semantic Communication for 6G Networks
This system comprises a gating network and multiple experts, each specializing in different security challenges.
Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trends
We review the current state of LM agents, the key technologies enabling LM agent collaboration, and the security and privacy challenges they face during cooperative operations.
Towards Within-Class Variation in Alzheimer's Disease Detection from Spontaneous Speech
Unlike conventional classification tasks, we identify within-class variation as a critical challenge in AD detection: individuals with AD exhibit a spectrum of cognitive impairments.
Disentangling Speakers in Multi-Talker Speech Recognition with Speaker-Aware CTC
SACTC is a tailored CTC variant for multi-talker scenarios, it explicitly models speaker disentanglement by constraining the encoder to represent different speakers' tokens at specific time frames.
Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASR
Self-supervised learning (SSL) based discrete speech representations are highly compact and domain adaptable.
Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructions
Recent advancements in large language models (LLMs) have revolutionized various domains, bringing significant progress and new opportunities.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Sustainable Diffusion-based Incentive Mechanism for Generative AI-driven Digital Twins in Industrial Cyber-Physical Systems
In this paper, we first develop a GenAI-driven DT architecture in ICPSs.
Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition System
In this study, we propose a pioneering approach to empower Whisper, which is a speech foundation model, to tackle joint multi-talker and target-talker speech recognition tasks.
Hybrid-Generative Diffusion Models for Attack-Oriented Twin Migration in Vehicular Metaverses
The vehicular metaverse is envisioned as a blended immersive domain that promises to bring revolutionary changes to the automotive industry.
Purple-teaming LLMs with Adversarial Defender Training
In PAD, we automatically collect conversational data that cover the vulnerabilities of an LLM around specific safety risks in a self-play manner, where the attacker aims to elicit unsafe responses and the defender generates safe responses to these attacks.
Hybrid RAG-empowered Multi-modal LLM for Secure Data Management in Internet of Medical Things: A Diffusion-based Contract Approach
Moreover, it enhances the output quality of MLLMs through hybrid RAG, which employs multi-modal metrics to filter various unimodal RAG results and incorporates these retrieval results as additional inputs to MLLMs.
QoE Maximization for Multiple-UAV-Assisted Multi-Access Edge Computing: An Online Joint Optimization Approach
Then, we formulate a joint task offloading, resource allocation, and UAV trajectory planning optimization problem (JTRTOP) to maximize the QoE of UDs while considering the energy consumption constraints of UAVs.
Efficient Prompting for LLM-based Generative Internet of Things
Large language models (LLMs) have demonstrated remarkable capacities on various tasks, and integrating the capacities of LLMs into the Internet of Things (IoT) applications has drawn much research attention recently.
Ranked #6 on
Table-based Fact Verification
on TabFact
Multi-attribute Auction-based Resource Allocation for Twins Migration in Vehicular Metaverses: A GPT-based DRL Approach
In this mechanism, we propose a two-stage matching for vehicular users and Metaverse service providers in multi-attribute resource markets.
Diffusion-based Reinforcement Learning for Dynamic UAV-assisted Vehicle Twins Migration in Vehicular Metaverses
In this paper, we propose a dynamic Unmanned Aerial Vehicle (UAV)-assisted VT migration framework in air-ground integrated networks, where UAVs act as aerial edge servers to assist ground RSUs during VT task offloading.
Optimizing 6G Integrated Sensing and Communications (ISAC) via Expert Networks
Integrated Sensing and Communications (ISAC) is one of the core technologies of 6G, which combines sensing and communication functions into a single system.
Generative AI for Deep Reinforcement Learning: Framework, Analysis, and Use Cases
We first introduce several classic GAI and DRL algorithms and demonstrate the applications of GAI-enhanced DRL algorithms.
Generative AI for Low-Carbon Artificial Intelligence of Things with Large Language Models
In this article, we explore the potential of GAI for carbon emissions reduction and propose a novel GAI-enabled solution for low-carbon AIoT.
Integration of Mixture of Experts and Multimodal Generative AI in Internet of Vehicles: A Survey
Generative AI (GAI) can enhance the cognitive, reasoning, and planning capabilities of intelligent modules in the Internet of Vehicles (IoV) by synthesizing augmented datasets, completing sensor data, and making sequential decisions.
Generative Artificial Intelligence Assisted Wireless Sensing: Human Flow Detection in Practical Communication Environments
Finally, through clustering, G-HFD determines the number of subflows and the number of targets in each subflow, i. e., the subflow size.
Generative AI Agents with Large Language Model for Satellite Networks via a Mixture of Experts Transmission
Furthermore, the superiority of the proposed MoE-ppo approach over other benchmarks is confirmed in solving the formulated problem.
Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning
Specifically, it is found that DCB enables terminals that cannot reach the uplink achievable threshold to achieve efficient direct uplink transmission, which thus reveals that DCB is an effective solution for enabling direct ground-space communications.
Blockchain-based Pseudonym Management for Vehicle Twin Migrations in Vehicular Edge Metaverse
As highly computerized avatars of Vehicular Metaverse Users (VMUs), the Vehicle Twins (VTs) deployed in edge servers can provide valuable metaverse services to improve driving safety and on-board satisfaction for their VMUs throughout journeys.
On-demand Quantization for Green Federated Generative Diffusion in Mobile Edge Networks
Federated learning is a promising technique for effectively training GAI models in mobile edge networks due to its data distribution.
Generative AI for Unmanned Vehicle Swarms: Challenges, Applications and Opportunities
For that, this paper aims to provide a comprehensive survey on applications, challenges, and opportunities of GAI in unmanned vehicle swarms.
Compressing Deep Reinforcement Learning Networks with a Dynamic Structured Pruning Method for Autonomous Driving
Our method consists of two steps, i. e. training DRL models with a group sparse regularizer and removing unimportant neurons with a dynamic pruning threshold.
Generative AI-Driven Human Digital Twin in IoT-Healthcare: A Comprehensive Survey
The Internet of things (IoT) can significantly enhance the quality of human life, specifically in healthcare, attracting extensive attentions to IoT-healthcare services.
Tiny Multi-Agent DRL for Twins Migration in UAV Metaverses: A Multi-Leader Multi-Follower Stackelberg Game Approach
To address the challenges, we propose a tiny machine learning-based Stackelberg game framework based on pruning techniques for efficient UT migration in UAV metaverses.
When Large Language Model Agents Meet 6G Networks: Perception, Grounding, and Alignment
AI agents based on multimodal large language models (LLMs) are expected to revolutionize human-computer interaction and offer more personalized assistant services across various domains like healthcare, education, manufacturing, and entertainment.
Cross-Speaker Encoding Network for Multi-Talker Speech Recognition
To the best of our knowledge, this work represents an early effort to integrate SIMO and SISO for multi-talker speech recognition.
Acceleration Estimation of Signal Propagation Path Length Changes for Wireless Sensing
Furthermore, in multi-target scenarios, the fall detection achieves an average true positive rate of 89. 56% and a false positive rate of 11. 78%, demonstrating its importance in enhancing indoor wireless sensing capabilities.
Generative AI-driven Semantic Communication Networks: Architecture, Technologies and Applications
Furthermore, GAI algorithms also play an important role in the management of SemCom networks.
Resource-efficient Generative Mobile Edge Networks in 6G Era: Fundamentals, Framework and Case Study
By integrating Generative Artificial Intelligence (GAI) with mobile edge networks, generative mobile edge networks possess immense potential to enhance the intelligence and efficiency of wireless communication networks.
From Generative AI to Generative Internet of Things: Fundamentals, Framework, and Outlooks
Generative Artificial Intelligence (GAI) possesses the capabilities of generating realistic data and facilitating advanced decision-making.
Generative AI for Integrated Sensing and Communication: Insights from the Physical Layer Perspective
In this article, we investigate applications of GAI in the physical layer and analyze its support for integrated sensing and communications (ISAC) systems.
Knowledge Base Aware Semantic Communication in Vehicular Networks
Semantic communication (SemCom) has recently been considered a promising solution for the inevitable crisis of scarce communication resources.
Generative AI-aided Joint Training-free Secure Semantic Communications via Multi-modal Prompts
The system jointly optimizes the diffusion step, jamming, and transmitting power with the aid of the generative diffusion models, enabling successful and secure transmission of the source messages.
QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learning
This paper proposes a novel semi-supervised TTS framework, QS-TTS, to improve TTS quality with lower supervised data requirements via Vector-Quantized Self-Supervised Speech Representation Learning (VQ-S3RL) utilizing more unlabeled speech audio.
Artificial Intelligence for Web 3.0: A Comprehensive Survey
Then, we thoroughly analyze the current state of AI technology applications in the four layers of Web 3. 0 and offer some insights into its potential future development direction.
Vision-based Semantic Communications for Metaverse Services: A Contest Theoretic Approach
The framework provides a novel solution to resource allocation for avatar association in VR environments, ensuring a smooth and immersive experience for all users.
Semantic Communications for Artificial Intelligence Generated Content (AIGC) Toward Effective Content Creation
Artificial Intelligence Generated Content (AIGC) Services have significant potential in digital content creation.
Service Reservation and Pricing for Green Metaverses: A Stackelberg Game Approach
A single-leader multi-follower Stackelberg game is formulated between the MSP and users while each user optimizes an offloading probability to minimize the weighted sum of time, energy consumption and monetary cost.
Blockchain-empowered Federated Learning for Healthcare Metaverses: User-centric Incentive Mechanism with Optimal Data Freshness
Given the revolutionary role of metaverses, healthcare metaverses are emerging as a transformative force, creating intelligent healthcare systems that offer immersive and personalized services.
A Revolution of Personalized Healthcare: Enabling Human Digital Twin with Mobile AIGC
To promote the development of this new breed of paradigm, in this article, we propose a system architecture of mobile AIGC-driven HDT and highlight the corresponding design requirements and challenges.
Federated Learning-Empowered AI-Generated Content in Wireless Networks
Artificial intelligence generated content (AIGC) has emerged as a promising technology to improve the efficiency, quality, diversity and flexibility of the content creation process by adopting a variety of generative AI models.
Multi-Agent Deep Reinforcement Learning for Dynamic Avatar Migration in AIoT-enabled Vehicular Metaverses with Trajectory Prediction
Specifically, we propose a model to predict the future trajectories of intelligent vehicles based on their historical data, indicating the future workloads of RSUs. Based on the expected workloads of RSUs, we formulate the avatar task migration problem as a long-term mixed integer programming problem.
Towards Effective and Compact Contextual Representation for Conformer Transducer Speech Recognition Systems
Current ASR systems are mainly trained and evaluated at the utterance level.
Adversarial Attacks and Defenses for Semantic Communication in Vehicular Metaverses
In this article, we propose a hierarchical SemCom-enabled vehicular metaverses framework consisting of the global metaverse, local metaverses, SemCom module, and resource pool.
Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator
Extending on this, we incorporate a diarization branch into the Sidecar, allowing for unified modeling of both ASR and diarization with a negligible overhead of only 768 parameters.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
The defender's perspective on automatic speaker verification: An overview
Automatic speaker verification (ASV) plays a critical role in security-sensitive environments.
DADFNet: Dual Attention and Dual Frequency-Guided Dehazing Network for Video-Empowered Intelligent Transportation
To eliminate the influences of adverse weather conditions, we propose a dual attention and dual frequency-guided dehazing network (termed DADFNet) for real-time visibility enhancement.
Deep Generative Model and Its Applications in Efficient Wireless Network Management: A Tutorial and Case Study
With the phenomenal success of diffusion models and ChatGPT, deep generation models (DGMs) have been experiencing explosive growth from 2022.
Guiding AI-Generated Digital Content with Wireless Perception
Experiments results verify the effectiveness of the WP-AIGC framework, highlighting its potential as a novel approach for guiding AI models in the accurate generation of digital content.
AI-Generated Incentive Mechanism and Full-Duplex Semantic Communications for Information Sharing
Specifically, a user can transmit the generated content and semantic information extracted from their view image to nearby users, who can then use this information to obtain the spatial matching of computation results under their view images.
A Sidecar Separator Can Convert a Single-Talker Speech Recognition System to a Multi-Talker One
Although automatic speech recognition (ASR) can perform well in common non-overlapping environments, sustaining performance in multi-talker overlapping speech recognition remains challenging.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses
In the vehicular mixed reality (MR) Metaverse, the distance between physical and virtual entities can be overcome by fusing the physical and virtual environments with multi-dimensional communications in autonomous driving systems.
Generative AI-empowered Effective Physical-Virtual Synchronization in the Vehicular Metaverse
Furthermore, we propose a multi-task enhanced auction-based mechanism to match and price AVs and MARs for RSUs to provision real-time and effective services.
Enabling AI-Generated Content (AIGC) Services in Wireless Edge Networks
To achieve efficient AaaS and maximize the quality of generated content in wireless edge networks, we propose a deep reinforcement learning-enabled algorithm for optimal ASP selection.
AnycostFL: Efficient On-Demand Federated Learning over Heterogeneous Edge Devices
We propose a cost-adjustable FL framework, named AnycostFL, that enables diverse edge devices to efficiently perform local updates under a wide range of efficiency constraints.
When Quantum Information Technologies Meet Blockchain in Web 3.0
Therefore, in this paper, we introduce a quantum blockchain-driven Web 3. 0 framework that provides information-theoretic security for decentralized data transferring and payment transactions.
Performance Analysis of Free-Space Information Sharing in Full-Duplex Semantic Communications
In this paper, we propose a free-space information sharing mechanism based on full-duplex device-to-device (D2D) semantic communications.
Semantic Communications for Wireless Sensing: RIS-aided Encoding and Self-supervised Decoding
To select the task-related signal spectrums for achieving efficient encoding, a semantic hash sampling method is introduced.
Wireless Sensing Data Collection and Processing for Metaverse Avatar Construction
We observe that the collected sensing data, i. e., channel state information (CSI), suffers from a phase shift problem, which negatively affects the extraction of user information such as behavior and heartbeat and further deteriorates the avatar construction.
Semantic-Aware Sensing Information Transmission for Metaverse: A Contest Theoretic Approach
Inspired by emerging semantic communication, in this paper, we propose a semantic transmission framework for transmitting sensing information from the physical world to Metaverse.
Personalized Saliency in Task-Oriented Semantic Communications: Image Transmission and Performance Analysis
Semantic communication, as a promising technology, has emerged to break through the Shannon limit, which is envisioned as the key enabler and fundamental paradigm for future 6G networks and applications, e. g., smart healthcare.
Attention-aware Resource Allocation and QoE Analysis for Metaverse xURLLC Services
Although conventional ultra-reliable and low-latency communications (URLLC) can satisfy objective KPIs, it is difficult to provide a personalized immersive experience that is a distinctive feature of the Metaverse.
Exploring Attention-Aware Network Resource Allocation for Customized Metaverse Services
With the help of UOAL, we propose an attention-aware network resource allocation algorithm that has two steps, i. e., attention prediction and QoE maximization.
Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion
However, previous works show that state-of-the-art ASV models are seriously vulnerable to voice spoofing attacks, and the recently proposed high-performance spoofing countermeasure (CM) models only focus solely on the standalone anti-spoofing tasks, and ignore the subsequent speaker verification process.
Efficient Federated Learning with Spike Neural Networks for Traffic Sign Recognition
However, for machine learning-based traffic sign recognition on the Internet of Vehicles (IoV), a large amount of traffic sign data from distributed vehicles is needed to be gathered in a centralized server for model training, which brings serious privacy leakage risk because of traffic sign data containing lots of location privacy information.
Spoofing-Aware Speaker Verification by Multi-Level Fusion
In the second-level fusion, the CM score and ASV scores directly from ASV systems will be concatenated into a prediction block for the final decision.
Reconfigurable Intelligent Surface-Aided Joint Radar and Covert Communications: Fundamentals, Optimization, and Challenges
Thus, we first apply covert communication into JRC and propose a joint radar and covert communication (JRCC) system to achieve high spectrum utilization and secure data transmission simultaneously.
TalkTive: A Conversational Agent Using Backchannels to Engage Older Adults in Neurocognitive Disorders Screening
This is used in the development of TalkTive, a CA which can predict both timing and form of backchanneling during cognitive assessments.
The CUHK-TENCENT speaker diarization system for the ICASSP 2022 multi-channel multi-party meeting transcription challenge
This paper describes our speaker diarization system submitted to the Multi-channel Multi-party Meeting Transcription (M2MeT) challenge, where Mandarin meeting data were recorded in multi-channel format for diarization and automatic speech recognition (ASR) tasks.
Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT
In this paper, we propose FedGS, which is a hierarchical cloud-edge-end FL framework for 5G empowered industries, to improve industrial FL performance on non-i. i. d.
Robust Semi-supervised Federated Learning for Images Automatic Recognition in Internet of Drones
This emerging technology relies on sharing ground truth labeled data between Unmanned Aerial Vehicle (UAV) swarms to train a high-quality automatic image recognition model.
Towards Communication-efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Things
Second, we propose an asynchronous local differential privacy mechanism, which improves communication efficiency and mitigates gradient leakage attacks by adding well-designed noise to the gradients of edge nodes.
Squeezing value of cross-domain labels: a decoupled scoring approach for speaker verification
Domain mismatch often occurs in real applications and causes serious performance reduction on speaker verification systems.
Scalable and Communication-efficient Decentralized Federated Edge Learning with Multi-blockchain Framework
This framework can achieve scalable and flexible decentralized FEL by individually manage local model updates or model sharing records for performance isolation.
Cryptography and Security
Deep Anomaly Detection for Time-series Data in Industrial IoT: A Communication-Efficient On-device Federated Learning Approach
Third, to adapt the proposed framework to the timeliness of industrial anomaly detection, we propose a gradient compression mechanism based on Top-\textit{k} selection to improve communication efficiency.
Federated Learning for 6G Communications: Challenges, Methods, and Future Directions
As the 5G communication networks are being widely deployed worldwide, both industry and academia have started to move beyond 5G and explore 6G communications.
Domain-Invariant Speaker Vector Projection by Model-Agnostic Meta-Learning
Domain generalization remains a critical problem for speaker recognition, even with the state-of-the-art architectures based on deep neural nets.
Audio and Speech Processing
A Secure Federated Learning Framework for 5G Networks
In this article, we propose a blockchain-based secure FL framework to create smart contracts and prevent malicious or unreliable participants from involving in FL.
Towards Federated Learning in UAV-Enabled Internet of Vehicles: A Multi-Dimensional Contract-Matching Approach
Coupled with the rise of Deep Learning, the wealth of data and enhanced computation capabilities of Internet of Vehicles (IoV) components enable effective Artificial Intelligence (AI) based models to be built.
Signal Processing Networking and Internet Architecture
Privacy-preserving Traffic Flow Prediction: A Federated Learning Approach
Through extensive case studies on a real-world dataset, it is shown that FedGRU's prediction accuracy is 90. 96% higher than the advanced deep learning models, which confirm that FedGRU can achieve accurate and timely traffic prediction without compromising the privacy and security of raw data.
CN-CELEB: a challenging Chinese speaker recognition dataset
These datasets tend to deliver over optimistic performance and do not meet the request of research on speaker recognition in unconstrained conditions.
Reliable Federated Learning for Mobile Networks
Based on this metric, a reliable worker selection scheme is proposed for federated learning tasks.
Cryptography and Security
Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach
To strengthen data privacy and security, federated learning as an emerging machine learning technique is proposed to enable large-scale nodes, e. g., mobile devices, to distributedly train and globally share models without revealing their local data.
Blockchain for Secure and Efficient Data Sharing in Vehicular Edge Computing and Networks
Due to limited resources with vehicles, vehicular edge computing and networks (VECONs) i. e., the integration of mobile edge computing and vehicular networks, can provide powerful computing and massive storage resources.
