Search Results for author:
Found 112 papers, 45 papers with code
Privacy Meets Explainability: Managing Confidential Data and Transparency Policies in LLM-Empowered Science
As Large Language Models (LLMs) become integral to scientific workflows, concerns over the confidentiality and ethical handling of confidential data have emerged.
Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data
These challenges are particularly pronounced in the domain of few-shot learning, where the ability to learn from limited labeled data is crucial.
Federated Learning with Differential Privacy: An Utility-Enhanced Approach
In order to successfully avoid data leakage, adopting differential privacy (DP) in the local optimization process or in the local update aggregation process has emerged as two feasible ways for achieving sample-level or user-level privacy guarantees respectively, in federated learning models.
Multi-Objective Optimization for Privacy-Utility Balance in Differentially Private Federated Learning
In this work, we propose an adaptive clipping mechanism that dynamically adjusts the clipping norm using a multi-objective optimization framework.
Do Fairness Interventions Come at the Cost of Privacy: Evaluations for Binary Classifiers
While in-processing fairness approaches show promise in mitigating biased predictions, their potential impact on privacy leakage remains under-explored.
From Data to Action: Charting A Data-Driven Path to Combat Antimicrobial Resistance
Antimicrobial-resistant (AMR) microbes are a growing challenge in healthcare, rendering modern medicines ineffective.
VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation
We present a general strategy to aligning visual generation models -- both image and video generation -- with human preference.
From Principles to Practice: A Deep Dive into AI Ethics and Regulations
In the rapidly evolving domain of Artificial Intelligence (AI), the complex interaction between innovation and regulation has become an emerging focus of our society.
Face De-identification: State-of-the-art Methods and Comparative Studies
The widespread use of image acquisition technologies, along with advances in facial recognition, has raised serious privacy concerns.
DreamPolish: Domain Score Distillation With Progressive Geometry Generation
We introduce DreamPolish, a text-to-3D generation model that excels in producing refined geometry and high-quality textures.
MulCPred: Learning Multi-modal Concepts for Explainable Pedestrian Action Prediction
These limitations are tackled accordingly through the following approaches: 1) a linear aggregator to integrate the activation results of the concepts into predictions, which associates concepts of different modalities and provides ante-hoc explanations of the relevance between the concepts and the predictions; 2) a channel-wise recalibration module that attends to local spatiotemporal regions, which enables the concepts with locality; 3) a feature regularization loss that encourages the concepts to learn diverse patterns.
CogVLM2: Visual Language Models for Image and Video Understanding
Beginning with VisualGLM and CogVLM, we are continuously exploring VLMs in pursuit of enhanced vision-language fusion, efficient higher-resolution architecture, and broader modalities and applications.
Ranked #10 on
Visual Question Answering
on MM-Vet
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
We present CogVideoX, a large-scale text-to-video generation model based on diffusion transformer, which can generate 10-second continuous videos aligned with text prompt, with a frame rate of 16 fps and resolution of 768 * 1360 pixels.
VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents
Large Multimodal Models (LMMs) have ushered in a new era in artificial intelligence, merging capabilities in both language and vision to form highly capable Visual Foundation Agents.
AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models
Evaluating the alignment capabilities of large Vision-Language Models (VLMs) is essential for determining their effectiveness as helpful assistants.
LVBench: An Extreme Long Video Understanding Benchmark
To address this gap, we introduce LVBench, a benchmark specifically designed for long video understanding.
Memorization in deep learning: A survey
Deep Learning (DL) powered by Deep Neural Networks (DNNs) has revolutionized various domains, yet understanding the intricacies of DNN decision-making and learning processes remains a significant challenge.
Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimization
Driven by this issue, we propose a novel sparse FedAdam algorithm called FedAdam-SSM, wherein distributed devices sparsify the updates of local model parameters and moment estimates and subsequently upload the sparse representations to the centralized server.
Trustworthy DNN Partition for Blockchain-enabled Digital Twin in Wireless IIoT Networks
To promote the efficiency and trustworthiness of DT for wireless IIoT networks, we propose a blockchain-enabled DT (B-DT) framework that employs deep neural network (DNN) partitioning technique and reputation-based consensus mechanism, wherein the DTs maintained at the gateway side execute DNN inference tasks using the data collected from their associated IIoT devices.
EmInspector: Combating Backdoor Attacks in Federated Self-Supervised Learning Through Embedding Inspection
To tackle this issue, we dive into the fundamental mechanism of backdoor attacks on FSSL, proposing the Embedding Inspector (EmInspector) that detects malicious clients by inspecting the embedding space of local models.
Robust Model Aggregation for Heterogeneous Federated Learning: Analysis and Optimizations
To evaluate the learning performance of T-SFL, we provide an upper bound on the global loss function.
Inf-DiT: Upsampling Any-Resolution Image with Memory-Efficient Diffusion Transformer
However, due to a quadratic increase in memory during generating ultra-high-resolution images (e. g. 4096*4096), the resolution of generated images is often limited to 1024*1024.
Privacy at a Price: Exploring its Dual Impact on AI Fairness
The worldwide adoption of machine learning (ML) and deep learning models, particularly in critical sectors, such as healthcare and finance, presents substantial challenges in maintaining individual privacy and fairness.
The Frontier of Data Erasure: Machine Unlearning for Large Language Models
Large Language Models (LLMs) are foundational to AI advancements, facilitating applications like predictive text generation.
CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion
Recent advancements in text-to-image generative systems have been largely driven by diffusion models.
Does Negative Sampling Matter? A Review with Insights into its Theory and Applications
Is there a general framework that can incorporate all existing negative sampling methods?
Privacy for Fairness: Information Obfuscation for Fair Representation Learning with Local Differential Privacy
Based on the proposed method, we further develop a variational representation encoding approach that simultaneously achieves fairness and LDP.
Private Knowledge Sharing in Distributed Learning: A Survey
In this light, it is crucial to utilize information in learning processes that are either distributed or owned by different entities.
CogCoM: Train Large Vision-Language Models Diving into Details through Chain of Manipulations
Drawing inspiration from human cognition in solving visual problems (e. g., marking, zoom in), this paper introduces Chain of Manipulations, a mechanism that enables VLMs to solve problems step-by-step with evidence.
CogAgent: A Visual Language Model for GUI Agents
People are spending an enormous amount of time on digital devices through graphical user interfaces (GUIs), e. g., computer or smartphone screens.
Ranked #4 on
on
PnPNet: Pull-and-Push Networks for Volumetric Segmentation with Boundary Confusion
Precise boundary segmentation of volumetric images is a critical task for image-guided diagnosis and computer-assisted intervention, especially for boundary confusion in clinical practice.
Refine, Discriminate and Align: Stealing Encoders via Sample-Wise Prototypes and Multi-Relational Extraction
This paper introduces RDA, a pioneering approach designed to address two primary deficiencies prevalent in previous endeavors aiming at stealing pre-trained encoders: (1) suboptimal performances attributed to biased optimization objectives, and (2) elevated query costs stemming from the end-to-end paradigm that necessitates querying the target encoder every epoch.
DRUformer: Enhancing the driving scene Important object detection with driving relationship self-understanding
The DRUformer is a transformer-based multi-modal important object detection model that takes into account the relationships between all the participants in the driving scenario.
When Fairness Meets Privacy: Exploring Privacy Threats in Fair Binary Classifiers via Membership Inference Attacks
It leverages the difference in the predictions from both the original and fairness-enhanced models and exploits the observed prediction gaps as attack clues.
CogVLM: Visual Expert for Pretrained Language Models
We introduce CogVLM, a powerful open-source visual language foundation model.
Ranked #4 on
Visual Question Answering (VQA)
on InfiMM-Eval
Federated Meta-Learning for Few-Shot Fault Diagnosis with Representation Encoding
However, the domain discrepancy and data scarcity problems among clients deteriorate the performance of the global FL model.
Semantic Difference Guidance for the Uncertain Boundary Segmentation of CT Left Atrial Appendage
In this paper, we have modeled relative relations between the LA and LAA via deep segmentation networks for the first time, and introduce a new LA & LAA CT dataset.
Federated Learning in Intelligent Transportation Systems: Recent Applications and Open Problems
Intelligent transportation systems (ITSs) have been fueled by the rapid development of communication technologies, sensor technologies, and the Internet of Things (IoT).
Sparse Federated Training of Object Detection in the Internet of Vehicles
However, the current object detection methods are mostly based on centralized deep training, that is, the sensitive data obtained by edge devices need to be uploaded to the server, which raises privacy concerns.
GPT Can Solve Mathematical Problems Without a Calculator
Previous studies have typically assumed that large language models are unable to accurately perform arithmetic operations, particularly multiplication of >8 digits, and operations involving decimals and fractions, without the use of calculator tools.
Personalized Federated Deep Reinforcement Learning-based Trajectory Optimization for Multi-UAV Assisted Edge Computing
This work proposes a novel solution to address these challenges, namely personalized federated deep reinforcement learning (PF-DRL), for multi-UAV trajectory optimization.
Relay Diffusion: Unifying diffusion process across resolutions for image synthesis
Diffusion models achieved great success in image synthesis, but still face challenges in high-resolution generation.
Ranked #1 on
Image Generation
on CelebA-HQ 256x256
(using extra training data)
Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity
With the rapid proliferation of smart mobile devices, federated learning (FL) has been widely considered for application in wireless networks for distributed model training.
R-Cut: Enhancing Explainability in Vision Transformers with Relationship Weighted Out and Cut
Our method aims to improve trust in classification results and empower users to gain a deeper understanding of the model for downstream tasks by providing visualizations of class-specific maps.
Mobility-Aware Joint User Scheduling and Resource Allocation for Low Latency Federated Learning
As an efficient distributed machine learning approach, Federated learning (FL) can obtain a shared model by iterative local model training at the user side and global model aggregating at the central server side, thereby protecting privacy of users.
G$^2$uardFL: Safeguarding Federated Learning Against Backdoor Attacks through Attributed Client Graph Clustering
Through empirical evaluation, comparing G$^2$uardFL with cutting-edge defenses, such as FLAME (USENIX Security 2022) [28] and DeepSight (NDSS 2022) [36], against various backdoor attacks including 3DFed (SP 2023) [20], our results demonstrate its significant effectiveness in mitigating backdoor attacks while having a negligible impact on the aggregated model's performance on benign samples (i. e., the primary task performance).
BatchSampler: Sampling Mini-Batches for Contrastive Learning in Vision, Language, and Graphs
To make each mini-batch have fewer false negatives, we design the proximity graph of randomly-selected instances.
Learn to Unlearn: A Survey on Machine Unlearning
This inspired recent research on removing the influence of specific data samples from a trained ML model.
ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
We present a comprehensive solution to learn and improve text-to-image models from human preference feedback.
Ranked #5 on
Preference Mapping
on Pick-a-Pic v1 test
(using extra training data)
Gradient Sparsification for Efficient Wireless Federated Learning with Differential Privacy
Specifically, we first design a random sparsification algorithm to retain a fraction of the gradient elements in each client's local training, thereby mitigating the performance degradation induced by DP and and reducing the number of transmission parameters over wireless channels.
Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning
The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.
Efficient and Low Overhead Website Fingerprinting Attacks and Defenses based on TCP/IP Traffic
To achieve a configurable trade-off between the defense and the network overhead, we further improve the list-based defense by a traffic splitting mechanism, which can combat the mentioned attacks as well as save a considerable amount of network overhead.
Differentially Private Vertical Federated Learning
Thus, in this paper, we aim to explore how to protect the privacy of individual organisation data in a differential privacy (DP) setting.
Decentralized Federated Reinforcement Learning for User-Centric Dynamic TFDD Control
The explosive growth of dynamic and heterogeneous data traffic brings great challenges for 5G and beyond mobile networks.
Privacy-preserving Deep Learning based Record Linkage
The global model is then used by a linkage unit to distinguish unlabelled record pairs as matches and non-matches.
Parameter-Efficient Tuning Makes a Good Classification Head
In this paper, we find that parameter-efficient tuning makes a good classification head, with which we can simply replace the randomly initialized heads for a stable performance gain.
How Does a Deep Learning Model Architecture Impact Its Privacy? A Comprehensive Study of Privacy Attacks on CNNs and Transformers
As a booming research area in the past decade, deep learning technologies have been driven by big data collected and processed on an unprecedented scale.
GLM-130B: An Open Bilingual Pre-trained Model
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters.
Ranked #1 on
Language Modelling
on CLUE (OCNLI_50K)
Robust Information Bottleneck for Task-Oriented Communication with Digital Modulation
Task-oriented communications, mostly using learning-based joint source-channel coding (JSCC), aim to design a communication-efficient edge inference system by transmitting task-relevant information to the receiver.
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers
Large-scale pretrained transformers have created milestones in text (GPT-3) and text-to-image (DALL-E and CogView) generation.
Ranked #22 on
Video Generation
on UCF-101
Rethinking the Setting of Semi-supervised Learning on Graphs
graph benchmark (IGB) consisting of 4 datasets.
CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers
The development of the transformer-based text-to-image models are impeded by its slow generation and complexity for high-resolution images.
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
Vertical Federated Learning: Challenges, Methodologies and Experiments
As a special architecture in FL, vertical FL (VFL) is capable of constructing a hyper ML model by embracing sub-models from different clients.
Negative-ResNet: Noisy Ambulatory Electrocardiogram Signal Classification Scheme
With recently successful applications of deep learning in computer vision and general signal processing, deep learning has shown many unique advantages in medical signal processing.
A Wearable ECG Monitor for Deep Learning Based Real-Time Cardiovascular Disease Detection
Cardiovascular disease has become one of the most significant threats endangering human life and health.
Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements.
Adaptive Diffusion in Graph Neural Networks
Notably, message passing based GNNs, e. g., graph convolutional networks, leverage the immediate neighbors of each node during the aggregation process, and recently, graph diffusion convolution (GDC) is proposed to expand the propagation neighborhood by leveraging generalized graph diffusion.
Federated Learning for Smart Healthcare: A Survey
Recent advances in communication technologies and Internet-of-Medical-Things have transformed smart healthcare enabled by artificial intelligence (AI).
Federated Learning for COVID-19 Detection with Generative Adversarial Networks in Edge Cloud Computing
COVID-19 has spread rapidly across the globe and become a deadly pandemic.
Cooperative Task Offloading and Block Mining in Blockchain-based Edge Computing with Multi-agent Deep Reinforcement Learning
The convergence of mobile edge computing (MEC) and blockchain is transforming the current computing services in mobile networks, by offering task offloading solutions with security enhancement empowered by blockchain mining.
BEdgeHealth: A Decentralized Architecture for Edge-based IoMT Networks Using Blockchain
The healthcare industry has witnessed significant transformations in e-health services by using mobile edge computing (MEC) and blockchain to facilitate healthcare operations.
FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding
The few-shot natural language understanding (NLU) task has attracted much recent attention.
6G Internet of Things: A Comprehensive Survey
The sixth generation (6G) wireless communication networks are envisioned to revolutionize customer services and applications via the Internet of Things (IoT) towards a future of fully intelligent and autonomous systems.
Low-Latency Federated Learning over Wireless Channels with Differential Privacy
Then, we convert the MAMAB to a max-min bipartite matching problem at each communication round, by estimating rewards with the upper confidence bound (UCB) approach.
Federated Learning for Industrial Internet of Things in Future Industries
The Industrial Internet of Things (IIoT) offers promising opportunities to transform the operation of industrial systems and becomes a key enabler for future industries.
M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers
Conditional image synthesis aims to create an image according to some multi-modal guidance in the forms of textual descriptions, reference images, and image blocks to preserve, as well as their combinations.
CogView: Mastering Text-to-Image Generation via Transformers
Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding.
Ranked #53 on
Text-to-Image Generation
on COCO (Common Objects in Context)
(using extra training data)
UFC-BERT: Unifying Multi-Modal Controls for Conditional Image Synthesis
Conditional image synthesis aims to create an image according to some multi-modal guidance in the forms of textual descriptions, reference images, and image blocks to preserve, as well as their combinations.
Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design
Owing to the low communication costs and privacy-promoting capabilities, Federated Learning (FL) has become a promising tool for training effective machine learning models among distributed clients.
Federated Learning for Internet of Things: A Comprehensive Survey
The Internet of Things (IoT) is penetrating many facets of our daily life with the proliferation of intelligent services and applications empowered by artificial intelligence (AI).
Realistic Differentially-Private Transmission Power Flow Data Release
For the modeling, design and planning of future energy transmission networks, it is vital for stakeholders to access faithful and useful power flow data, while provably maintaining the privacy of business confidentiality of service providers.
Controllable Generation from Pre-trained Language Models via Inverse Prompting
To tackle this challenge, we propose an innovative method, inverse prompting, to better control text generation.
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
On a wide range of tasks across NLU, conditional and unconditional generation, GLM outperforms BERT, T5, and GPT given the same model sizes and data, and achieves the best performance from a single pretrained model with 1. 25x parameters of BERT Large , demonstrating its generalizability to different downstream tasks.
Ranked #4 on
Language Modelling
on WikiText-103
(using extra training data)
GPT Understands, Too
Prompting a pretrained language model with natural language patterns has been proved effective for natural language understanding (NLU).
DP-Image: Differential Privacy for Image Data in Feature Space
The excessive use of images in social networks, government databases, and industrial applications has posed great privacy risks and raised serious concerns from the public.
IdentityDP: Differential Private Identification Protection for Face Images
We propose IdentityDP, a face anonymization framework that combines a data-driven deep neural network with a differential privacy (DP) mechanism.
M6: A Chinese Multimodal Pretrainer
In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1. 9TB images and 292GB texts that cover a wide range of domains.
Covert Model Poisoning Against Federated Learning: Algorithm Design and Optimization
An attacker in FL may control a number of participant clients, and purposely craft the uploaded model parameters to manipulate system outputs, namely, model poisoning (MP).
Blockchain Assisted Decentralized Federated Learning (BLADE-FL): Performance Analysis and Resource Allocation
Focusing on this problem, we explore the impact of lazy clients on the learning performance of BLADE-FL, and characterize the relationship among the optimal K, the learning parameters, and the proportion of lazy clients.
Generalizing Graph Convolutional Networks
Graph convolutional networks (GCNs) have emerged as a powerful framework for mining and learning with graphs.
F-manifold color algebras
In this paper, we introduce the notion of F-manifold color algebras and study their properties which extend some results for F-manifold algebras.
Rings and Algebras
Blockchain Assisted Decentralized Federated Learning (BLADE-FL) with Lazy Clients
The proposed BLADE-FL has a good performance in terms of privacy preservation, tamper resistance, and effective cooperation of learning.
CogLTX: Applying BERT to Long Texts
BERTs are incapable of processing long texts due to its quadratically increasing memory and time consumption.
When Machine Learning Meets Privacy: A Survey and Outlook
The newly emerged machine learning (e. g. deep learning) methods have become a strong driving force to revolutionize a wide range of industries, such as smart healthcare, financial technology, and surveillance systems.
When Federated Learning Meets Blockchain: A New Distributed Learning Paradigm
Motivated by the explosive computing capabilities at end user equipments, as well as the growing privacy concerns over sharing sensitive raw data, a new machine learning paradigm, named federated learning (FL) has emerged.
Networking and Internet Architecture
RDP-GAN: A Rényi-Differential Privacy based Generative Adversarial Network
Generative adversarial network (GAN) has attracted increasing attention recently owing to its impressive ability to generate realistic samples with high privacy protection.
GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
Graph representation learning has emerged as a powerful technique for addressing real-world problems.
Understanding Negative Sampling in Graph Representation Learning
To the best of our knowledge, we are the first to derive the theory and quantify that the negative sampling distribution should be positively but sub-linearly correlated to their positive sampling distribution.
User-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization
According to our analysis, the UDP framework can realize $(\epsilon_{i}, \delta_{i})$-LDP for the $i$-th MT with adjustable privacy protection levels by varying the variances of the artificial noise processes.
Federated Learning with Differential Privacy: Algorithms and Performance Analysis
Specifically, the theoretical bound reveals the following three key properties: 1) There is a tradeoff between the convergence performance and privacy protection levels, i. e., a better convergence performance leads to a lower protection level; 2) Given a fixed privacy protection level, increasing the number $N$ of overall clients participating in FL can improve the convergence performance; 3) There is an optimal number of maximum aggregation times (communication rounds) in terms of convergence performance for a given protection level.
On Safeguarding Privacy and Security in the Framework of Federated Learning
Motivated by the advancing computational capacity of wireless end-user equipment (UE), as well as the increasing concerns about sharing private data, a new machine learning (ML) paradigm has emerged, namely federated learning (FL).
Networking and Internet Architecture
Secure Computation Offloading in Blockchain based IoT Networks with Deep Reinforcement Learning
How to implement offloading to alleviate computation burdens at MDs while guaranteeing high security in mobile edge cloud is a challenging problem.
Privacy-Preserved Task Offloading in Mobile Blockchain with Deep Reinforcement Learning
Blockchain technology with its secure, transparent and decentralized nature has been recently employed in many mobile applications.
Towards Knowledge-Based Recommender Dialog System
In this paper, we propose a novel end-to-end framework called KBRD, which stands for Knowledge-Based Recommender Dialog System.
Ranked #5 on
Text Generation
on ReDial
ProNE: Fast and Scalable Network Representation Learning
Recent advances in network embedding have revolutionized the field of graph and network mining.
Cognitive Knowledge Graph Reasoning for One-shot Relational Learning
Inferring new facts from existing knowledge graphs (KG) with explainable reasoning processes is a significant problem and has received much attention recently.
Cognitive Graph for Multi-Hop Reading Comprehension at Scale
We propose a new CogQA framework for multi-hop question answering in web-scale documents.
Ranked #50 on
Question Answering
on HotpotQA
Privacy Preserving Location Data Publishing: A Machine Learning Approach
By introducing a new formulation of the problem, we are able to apply machine learning algorithms for clustering the trajectories and propose to use $k$-means algorithm for this purpose.
Semi-supervised Learning on Graphs with Generative Adversarial Nets
We first provide insights on working principles of adversarial learning over graphs and then present GraphSGAN, a novel approach to semi-supervised learning on graphs.
Spectral Network Embedding: A Fast and Scalable Method via Sparsity
In this paper, we propose a $10\times \sim 100\times$ faster network embedding method, called Progle, by elegantly utilizing the sparsity property of online networks and spectral analysis.
Privacy Preservation in Location-Based Services: A Novel Metric and Attack Model
In this paper, we incorporate a new type of side information based on consecutive location changes of users and propose a new metric called transition-entropy to investigate the location privacy preservation, followed by two algorithms to improve the transition-entropy for a given dummy generation algorithm.
