Search Results for author:
Found 231 papers, 81 papers with code
p-Norm Flow Diffusion for Local Graph Clustering
Local graph clustering and the closely related seed set expansion problem are primitives on graphs that are central to a wide range of analytic and learning tasks such as local clustering, community detection, nodes ranking and feature inference.
TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series
Satellite image time series (SITS) provide continuous observations of the Earth's surface, making them essential for applications such as environmental management and disaster assessment.
Matching Distance and Geometric Distribution Aided Learning Multiview Point Cloud Registration
This paper concentrates on pose graph construction and motion synchronization within multiview registration.
DGSolver: Diffusion Generalist Solver with Universal Posterior Sampling for Image Restoration
To address these challenges, we introduce \textbf{DGSolver}, a diffusion generalist solver with universal posterior sampling.
Understanding the Repeat Curse in Large Language Models from a Feature Perspective
Large language models (LLMs) have made remarkable progress in various domains, yet they often suffer from repetitive text generation, a phenomenon we refer to as the "Repeat Curse".
Understanding Aha Moments: from External Observations to Internal Mechanisms
We demonstrate that the "aha moment" is externally manifested in a more frequent use of anthropomorphic tones for self-reflection and an adaptive adjustment of uncertainty based on problem difficulty.
TransMamba: Flexibly Switching between Transformer and Mamba
Transformers are the cornerstone of modern large language models, but their quadratic computational complexity limits efficiency in long-sequence processing.
XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?
On top of the XLRS-Bench, 16 sub-tasks are defined to evaluate MLLMs' 10 kinds of perceptual capabilities and 6 kinds of reasoning capabilities, with a primary emphasis on advanced cognitive processes that facilitate real-world decision-making and the capture of spatiotemporal changes.
Improved Rates of Differentially Private Nonconvex-Strongly-Concave Minimax Optimization
In this paper, we study the problem of (finite sum) minimax optimization in the Differential Privacy (DP) model.
ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory
Alternatively, zeroth-order (ZO) techniques can compute gradients using just forward operations, eliminating the need to store activations.
RoMA: Scaling up Mamba-based Foundation Models for Remote Sensing
Recent advances in self-supervised learning for Vision Transformers (ViTs) have fueled breakthroughs in remote sensing (RS) foundation models.
C^2 ATTACK: Towards Representation Backdoor on CLIP via Concept Confusion
Backdoor attacks pose a significant threat to deep learning models, enabling adversaries to embed hidden triggers that manipulate the behavior of the model during inference.
Nearly Optimal Differentially Private ReLU Regression
We first show that when $\varepsilon = \tilde{O}(\sqrt{\frac{1}{N}})$ and there is some public data, it is possible to achieve an upper bound of $\Tilde{O}(\frac{d^2}{N^2 \varepsilon^2})$ for the excess population risk in $(\epsilon, \delta)$-DP, where $d$ is the dimension and $N$ is the number of data samples.
Online Bidding under RoS Constraints without Knowing the Value
This establishes the first optimal regret and constraint violation bounds for bidding in the online setting with unknown impression values.
UQABench: Evaluating User Embedding for Prompting LLMs in Personalized Question Answering
To address this concern, we propose \name, a benchmark designed to evaluate the effectiveness of user embeddings in prompting LLMs for personalization.
Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluation
We present a comprehensive evaluation framework for assessing Large Language Models' (LLMs) capabilities in suicide prevention, focusing on two critical aspects: the Identification of Implicit Suicidal ideation (IIS) and the Provision of Appropriate Supportive responses (PAS).
Towards User-level Private Reinforcement Learning with Human Feedback
We first show that the classical random response algorithm, which achieves an acceptable performance in item-level privacy, leads to suboptimal utility when in the user-level settings.
Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements
We introduce Fraud-R1, a benchmark designed to evaluate LLMs' ability to defend against internet fraud and phishing in dynamic, real-world scenarios.
Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model Reasoning
Transformer-based language models have achieved notable success, yet their internal reasoning mechanisms remain largely opaque due to complex non-linear interactions and high-dimensional operations.
EAP-GP: Mitigating Saturation Effect in Gradient-based Automated Circuit Identification
In this paper, we revisit existing gradient-based circuit identification methods and find that their performance is either affected by the zero-gradient problem or saturation effects, where edge attribution scores become insensitive to input changes, resulting in noisy and unreliable attribution evaluations for circuit components.
"Short-length" Adversarial Training Helps LLMs Defend "Long-length" Jailbreak Attacks: Theoretical and Empirical Evidence
This paper focuses on adversarial suffix jailbreak attacks and unveils that to defend against a jailbreak attack with an adversarial suffix of length $\Theta(M)$, it is enough to align LLMs on prompts with adversarial suffixes of length $\Theta(\sqrt{M})$.
Evaluating Data Influence in Meta Learning
As one of the most fundamental models, meta learning aims to effectively address few-shot learning challenges.
Autonomy-of-Experts Models
We argue that the separation between the router's decision-making and the experts' execution is a critical yet overlooked issue, leading to suboptimal expert selection and ineffective learning.
Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation
This system includes two foundation components: a large-scale shape generation model -- Hunyuan3D-DiT, and a large-scale texture synthesis model -- Hunyuan3D-Paint.
Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption
We introduce a multi-dimensional framework to elucidate common learning-based IVIF methods, from visual enhancement strategies to data compatibility and task adaptability.
Plug-and-Play DISep: Separating Dense Instances for Scene-to-Pixel Weakly-Supervised Change Detection in High-Resolution Remote Sensing Images
Existing Weakly-Supervised Change Detection (WSCD) methods often encounter the problem of "instance lumping" under scene-level supervision, particularly in scenarios with a dense distribution of changed instances (i. e., changed objects).
Advancing the Understanding of Fine-Grained 3D Forest Structures using Digital Cousins and Simulation-to-Reality: Methods and Datasets
The Boreal3D dataset, and more broadly, the synthetic data augmentation framework, is poised to become a critical resource for advancing research in large-scale 3D forest scene understanding and structural parameter estimation.
Scaling Laws for Floating Point Quantization Training
Low-precision training is considered an effective strategy for reducing both training and downstream inference costs.
Understanding the Dark Side of LLMs' Intrinsic Self-Correction
Intrinsic self-correction was proposed to improve LLMs' responses via feedback prompts solely based on their inherent capability.
CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information
This results in the loss of critical multimodal information in EEG.
FD2-Net: Frequency-Driven Feature Decomposition Network for Infrared-Visible Object Detection
Infrared-visible object detection (IVOD) seeks to harness the complementary information in infrared and visible images, thereby enhancing the performance of detectors in complex environments.
SAMCL: Empowering SAM to Continually Learn from Dynamic Domains
SAMCL decouples stability and plasticity during CS by two components: $\textit{AugModule}$ and $\textit{Module Selector}$.
HunyuanVideo: A Systematic Framework For Large Video Generative Models
In this report, we introduce HunyuanVideo, an innovative open-source video foundation model that demonstrates performance in video generation comparable to, or even surpassing, that of leading closed-source models.
HumanRig: Learning Automatic Rigging for Humanoid Character in a Large Scale Dataset
With the rapid evolution of 3D generation algorithms, the cost of producing 3D humanoid character models has plummeted, yet the field is impeded by the lack of a comprehensive dataset for automatic rigging, which is a pivotal step in character animation.
Data Lineage Inference: Uncovering Privacy Vulnerabilities of Dataset Pruning
Our findings reveal, for the first time, that even if data in the redundant set is solely used before model training, its pruning-phase membership status can still be detected through attacks.
CompetitorFormer: Competitor Transformer for 3D Instance Segmentation
Transformer-based methods have become the dominant approach for 3D instance segmentation.
Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning
Manipulating the interaction trajectories between the intelligent agent and the environment can control the agent's training and behavior, exposing the potential vulnerabilities of reinforcement learning (RL).
Dissecting Representation Misalignment in Contrastive Learning via Influence Function
ECIF considers both positive and negative samples and provides a closed-form approximation of contrastive learning models, eliminating the need for retraining.
More Expressive Attention with Negative Weights
We propose a novel attention mechanism, named Cog Attention, that enables attention weights to be negative for enhanced expressiveness, which stems from two key factors: (1) Cog Attention enhances parameter flexibility.
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens.
Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation
To address this issue, we propose a two-stage approach named Hunyuan3D 1. 0 including a lite version and a standard version, that both support text- and image-conditioned generation.
Show Me What and Where has Changed? Question Answering and Grounding for Remote Sensing Change Detection
Furthermore, we present VisTA, a simple yet effective baseline method that unifies the tasks of question answering and grounding by delivering both visual and textual answers.
CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentation
The field of Remote Sensing Domain Generalization (RSDG) has emerged as a critical and valuable research frontier, focusing on developing models that generalize effectively across diverse scenarios.
Towards Multi-dimensional Explanation Alignment for Medical Classification
The lack of interpretability in the field of medical image analysis has significant ethical and legal implications.
TAAD: Time-varying adversarial anomaly detection in dynamic graphs
The time-varying discriminator cooperates with the feature extractor to conduct adversarial training, which decreases the distributional differences in the feature representations of nodes between historical and newly emerged moments to learn transferable features.
Private Language Models via Truncated Laplacian Mechanism
To prevent privacy leakage, researchers have investigated word-level perturbations, relying on the formal guarantees of differential privacy (DP) in the embedding space.
Dissecting Fine-Tuning Unlearning in Large Language Models
Our findings reveal that these methods alter the model's knowledge retrieval process, providing further evidence that they do not genuinely erase the problematic knowledge embedded in the model parameters.
Faithful Interpretation for Graph Neural Networks
In this paper, we propose a solution to this problem by introducing a novel notion called Faithful Graph Attention-based Interpretation (FGAI).
Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
In this paper, leveraging tools in mechanistic interpretability, we first identify that in multi-hop tasks, LLMs tend to retrieve knowledge with implicit subject information from deeper MLP layers, unlike single-hop tasks, which rely on shallow layers.
What makes your model a low-empathy or warmth person: Exploring the Origins of Personality in LLMs
Large language models (LLMs) have demonstrated remarkable capabilities in generating human-like text and exhibiting personality traits similar to those in humans.
Understanding Reasoning in Chain-of-Thought from the Hopfieldian View
Moreover, we propose the Representation-of-Thought (RoT) framework, which leverages the robustness of low-dimensional representation spaces to enhance the robustness of the reasoning process in CoTs.
Selective Transformer for Hyperspectral Image Classification
The SFormer is designed to dynamically select receptive fields for capturing both spatial and spectral contextual information, while mitigating the impact of redundant data by prioritizing the most relevant features.
Multilateral Cascading Network for Semantic Segmentation of Large-Scale Outdoor Point Clouds
Semantic segmentation of large-scale outdoor point clouds is of significant importance in environment perception and scene understanding.
Component-based Sketching for Deep ReLU Nets
To address this issue, we develop a novel sketching scheme based on deep net components for various tasks.
MQA-KEAL: Multi-hop Question Answering under Knowledge Editing for Arabic Language
Later for each sub-problem, it iteratively queries the external memory and/or target LLM in order to generate the final response.
Multiple Rotation Averaging with Constrained Reweighting Deep Matrix Factorization
Our method synthesizes the merit of both optimization-based and learning-based methods.
Efficiently Expanding Receptive Fields: Local Split Attention and Parallel Aggregation for Enhanced Large-scale Point Cloud Semantic Segmentation
In light of the aforementioned designs, we put forth a novel framework, designated as LSNet, for large-scale point cloud semantic segmentation.
HMoE: Heterogeneous Mixture of Experts for Language Modeling
To address the imbalance in expert activation, we propose a novel training objective that encourages the frequent activation of smaller experts, enhancing computational efficiency and parameter utilization.
Incremental Structure Discovery of Classification via Sequential Monte Carlo
To alleviate the requirement of prior knowledge used in GPs and learn new features from data that arrive successively, this paper presents a novel method to automatically discover models of classification on complex data with little prior knowledge.
Pre-trained Encoder Inference: Revealing Upstream Encoders In Downstream Machine Learning Services
Experiments show that the PEI attack succeeds in revealing the hidden encoder in most cases and seldom makes mistakes even when the hidden encoder is not in the candidate set.
XTraffic: A Dataset Where Traffic Meets Incidents with Explainability and More
Our data can revolutionalize traditional traffic-related tasks towards higher interpretability and practice: instead of traditional prediction or classification tasks, we conduct: (1) post-incident traffic forecasting to quantify the impact of different incidents on traffic indexes; (2) incident classification using traffic indexes to determine the incidents types for precautions measures; (3) global causal analysis among the traffic indexes, meta-attributes, and incidents to give high-level guidance of the interrelations of various factors; (4) local causal analysis within road nodes to examine how different incidents affect the road segments' relations.
Deep Reinforcement Learning for Sequential Combinatorial Auctions
Revenue-optimal auction design is a challenging problem with significant theoretical and practical implications.
Releasing Malevolence from Benevolence: The Menace of Benign Data on Machine Unlearning
Our evaluation demonstrates that unlearning this benign data, comprising no more than 1% of the total training data, can reduce model accuracy by up to 50%.
Semi-supervised Concept Bottleneck Models
Concept Bottleneck Models (CBMs) have garnered increasing attention due to their ability to provide concept-based explanations for black-box deep learning models while achieving high final prediction accuracy using human-like concepts.
Beyond Statistical Estimation: Differentially Private Individual Computation via Shuffling
PIC enables personalized outputs while preserving privacy, and enjoys privacy amplification through shuffling.
A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoning
Chain-of-Thought (CoT) holds a significant place in augmenting the reasoning performance for large language models (LLMs).
HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model
Accurate hyperspectral image (HSI) interpretation is critical for providing valuable insights into various earth observation-related applications such as urban planning, precision agriculture, and environmental monitoring.
Scaling Efficient Masked Image Modeling on Large Remote Sensing Dataset
To address these issues, we present a new pre-training pipeline for RS models, featuring the creation of a large-scale RS dataset and an efficient MIM approach.
Improving Generalization of Neural Vehicle Routing Problem Solvers Through the Lens of Model Architecture
The DS decoder explicitly models VRPs of multiple training distribution patterns through multiple auxiliary light decoders, expanding the model representation space to encompass a broader range of distributional scenarios.
How to Strategize Human Content Creation in the Era of GenAI?
In time-sensitive content domains (e. g., news or pop music creation) where contents' value diminishes over time, we show that there is no polynomial time algorithm for finding the human's optimal (dynamic) strategy, unless the randomized exponential time hypothesis is false.
Neural Combinatorial Optimization Algorithms for Solving Vehicle Routing Problems: A Comprehensive Survey with Perspectives
Although several surveys on Neural Combinatorial Optimization (NCO) solvers specifically designed to solve Vehicle Routing Problems (VRPs) have been conducted.
Text Guided Image Editing with Automatic Concept Locating and Forgetting
With the advancement of image-to-image diffusion models guided by text, significant progress has been made in image editing.
A Real-Time Voice Activity Detection Based On Lightweight Neural
Voice activity detection (VAD) is the task of detecting speech in an audio stream, which is challenging due to numerous unseen noises and low signal-to-noise ratios in real environments.
Understanding Forgetting in Continual Learning with Linear Regression
In this paper, we provide a general theoretical analysis of forgetting in the linear regression model via Stochastic Gradient Descent (SGD) applicable to both underparameterized and overparameterized regimes.
Editable Concept Bottleneck Models
Concept Bottleneck Models (CBMs) have garnered much attention for their ability to elucidate the prediction process through a humanunderstandable concept layer.
Leveraging Logical Rules in Knowledge Editing: A Cherry on the Top
To address these issues, in this paper, we propose a novel framework named RULE-KE, i. e., RULE based Knowledge Editing, which is a cherry on the top for augmenting the performance of all existing MQA methods under KE.
Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling
First, we raise the scaling law between batch sizes and optimal learning rates in the sign of gradient case, in which we prove that the optimal learning rate first rises and then falls as the batch size increases.
Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality
We present a demonstration of a multimodal fine-grained training assistant for LEGO brick assembly in a pilot XR environment.
LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation
Experimental results in classification and dense prediction tasks show that LeMeViT has a significant $1. 7 \times$ speedup, fewer parameters, and competitive performance compared to the baseline models, and achieves a better trade-off between efficiency and performance.
Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding
For fine-grained language understanding, we train a Multimodal Large Language Model to refine the captions of the images.
Closing the Gap: Achieving Global Convergence (Last Iterate) of Actor-Critic under Markovian Sampling with Neural Network Parametrization
The current state-of-the-art theoretical analysis of Actor-Critic (AC) algorithms significantly lags in addressing the practical aspects of AC implementations.
Auctions with LLM Summaries
We study an auction setting in which bidders bid for placement of their content within a summary generated by a large language model (LLM), e. g., an ad auction in which the display is a summary paragraph of multiple ads.
Multi-hop Question Answering under Temporal Knowledge Editing
Multi-hop question answering (MQA) under knowledge editing (KE) has garnered significant attention in the era of large language models.
Dialectical Alignment: Resolving the Tension of 3H and Security Threats of LLMs
With the rise of large language models (LLMs), ensuring they embody the principles of being helpful, honest, and harmless (3H), known as Human Alignment, becomes crucial.
Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression
We also propose GSM8K-aug, i. e., an extended version of the existing GSM8K benchmark for task-agnostic prompts in order to provide a comprehensive evaluation platform.
MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining
However, transferring the pretrained models to downstream tasks may encounter task discrepancy due to their formulation of pretraining as image classification or object discrimination tasks.
Aerial Scene Classification
Building change detection for remote sensing images
+13
Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Approximate Unlearning Completeness
Our analysis, conducted across multiple unlearning benchmarks, reveals that these algorithms inconsistently fulfill their unlearning commitments due to two main issues: 1) unlearning new data can significantly affect the unlearning utility of previously requested data, and 2) approximate algorithms fail to ensure equitable unlearning utility across different groups.
Revisiting Differentially Private Hyper-parameter Tuning
Recent works propose a generic private selection solution for the tuning process, yet a fundamental question persists: is this privacy bound tight?
Privacy-Preserving Low-Rank Adaptation against Membership Inference Attacks for Latent Diffusion Models
To mitigate this issue, we further propose a Stable Membership-Privacy-preserving LoRA (SMP-LoRA) that adapts the LDM by minimizing the ratio of the adaptation loss to the MI gain.
MONAL: Model Autophagy Analysis for Modeling Human-AI Interactions
The increasing significance of large models and their multi-modal variants in societal information processing has ignited debates on social safety and ethics.
MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning
Adapting large language models (LLMs) to new domains/tasks and enabling them to be efficient lifelong learners is a pivotal challenge.
NIV-SSD: Neighbor IoU-Voting Single-Stage Object Detector From Point Cloud
NIV strategy can serve as a bridge between classification and regression branches by calculating two types of statistical data from the regression output to correct the classification confidence.
Communication Efficient and Provable Federated Unlearning
We start by giving a rigorous definition of \textit{exact} federated unlearning, which guarantees that the unlearned model is statistically indistinguishable from the one trained without the deleted data.
Antonym vs Synonym Distinction using InterlaCed Encoder NETworks (ICE-NET)
In this paper, we propose InterlaCed Encoder NETworks (i. e., ICE-NET) for antonym vs synonym distinction, that aim to capture and model the relation-specific properties of the antonyms and synonyms pairs in order to perform the classification task in a performance-enhanced manner.
Weighted Spectral Filters for Kernel Interpolation on Spheres: Estimates of Prediction Accuracy for Noisy Data
In this paper, we introduce a weighted spectral filter approach to reduce the condition number of the kernel matrix and then stabilize kernel interpolation.
Unleashing Channel Potential: Space-Frequency Selection Convolution for SAR Object Detection
In this paper we propose an efficient convolution module for SAR object detection called SFS-Conv which increases feature diversity within each convolutional layer through a shunt-perceive-select strategy.
A comprehensive framework for occluded human pose estimation
The challenges posed by occlusion can be attributed to the following factors: 1) Data: The collection and annotation of occluded human pose samples are relatively challenging.
Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
We introduce Truth Forest, a method that enhances truthfulness in LLMs by uncovering hidden truth representations using multi-dimensional orthogonal probes.
Refining Latent Homophilic Structures over Heterophilic Graphs for Robust Graph Convolution Networks
This finding motivates us to present a novel method that aims to harden GCNs by automatically learning Latent Homophilic Structures over heterophilic graphs.
Ranked #5 on
Node Classification
on Actor
Anchoring Path for Inductive Relation Prediction in Knowledge Graphs
To address this challenge, we propose Anchoring Path Sentence Transformer (APST) by introducing Anchoring Paths (APs) to alleviate the reliance of CPs.
Distilling Autoregressive Models to Obtain High-Performance Non-Autoregressive Solvers for Vehicle Routing Problems with Faster Inference Speed
To the best of our knowledge, this study is first-of-its-kind to obtain NAR VRP solvers from AR ones through knowledge distillation.
SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation
In this paper, we address the following question: "Only sparse human keypoint locations are detected for human pose estimation, is it really necessary to describe the whole image in a dense, high-resolution manner?"
Image Content Generation with Causal Reasoning
Considering causal reasoning in visual content generation is significant.
Improving Interpretation Faithfulness for Vision Transformers
However, ViTs suffer from issues with explanation faithfulness, as their focal points are fragile to adversarial attacks and can be easily changed with even slight perturbations on the input image.
Fair Text-to-Image Diffusion via Fair Mapping
In this paper, we address the limitations of existing text-to-image diffusion models in generating demographically fair results when given human-related descriptions.
Preserving Node-level Privacy in Graph Neural Networks
In this study, we propose a solution that specifically addresses the issue of node-level privacy.
Distributed Uncertainty Quantification of Kernel Interpolation on Spheres
For radial basis function (RBF) kernel interpolation of scattered data, Schaback in 1995 proved that the attainable approximation error and the condition number of the underlying interpolation matrix cannot be made small simultaneously.
GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddings
Bilingual Lexical Induction (BLI) is a core challenge in NLP, it relies on the relative isomorphism of individual embedding spaces.
GRI: Graph-based Relative Isomorphism of Word Embedding Spaces
Automated construction of bilingual dictionaries using monolingual embedding spaces is a core challenge in machine translation.
Worst-Case Analysis is Maximum-A-Posteriori Estimation
The worst-case resource usage of a program can provide useful information for many software-engineering tasks, such as performance optimization and algorithmic-complexity-vulnerability discovery.
Differentially Private Non-convex Learning for Multi-layer Neural Networks
This paper focuses on the problem of Differentially Private Stochastic Optimization for (multi-layer) fully connected neural networks with a single output node.
Improved Analysis of Sparse Linear Regression in Local Differential Privacy Model
To address these issues, we first consider the problem in the $\epsilon$ non-interactive LDP model and provide a lower bound of $\Omega(\frac{\sqrt{dk\log d}}{\sqrt{n}\epsilon})$ on the $\ell_2$-norm estimation error for sub-Gaussian data, where $n$ is the sample size and $d$ is the dimension of the space.
Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approach
Adversarial training (AT) is a canonical method for enhancing the robustness of deep neural networks (DNNs).
Fake News Detectors are Biased against Texts Generated by Large Language Models
The spread of fake news has emerged as a critical challenge, undermining trust and posing threats to society.
Adaptive Distributed Kernel Ridge Regression: A Feasible Distributed Learning Scheme for Data Silos
Data silos, mainly caused by privacy and interoperability, significantly constrain collaborations among different organizations with similar data for the same purpose.
Improving Misaligned Multi-modality Image Fusion with One-stage Progressive Dense Registration
Both stages directly estimate the respective target deformation fields.
Cost-effective On-device Continual Learning over Memory Hierarchy with Miro
Continual learning (CL) trains NN models incrementally from a continuous stream of tasks.
Reinforcement Learning-based Non-Autoregressive Solver for Traveling Salesman Problems
The Traveling Salesman Problem (TSP) is a well-known combinatorial optimization problem with broad real-world applications.
On the Global Convergence of Natural Actor-Critic with Two-layer Neural Network Parametrization
To achieve that, we propose a Natural Actor-Critic algorithm with 2-Layer critic parametrization (NAC2L).
Differentially Private Episodic Reinforcement Learning with Heavy-tailed Rewards
Under each framework, we consider both joint differential privacy (JDP) and local differential privacy (LDP) models.
Generalization Guarantees of Gradient Descent for Multi-Layer Neural Networks
This sheds light on sufficient or necessary conditions for under-parameterized and over-parameterized NNs trained by GD to attain the desired risk rate of $O(1/\sqrt{n})$.
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text
One is called DetectLLM-LRR, which is fast and efficient, and the other is called DetectLLM-NPR, which is more accurate, but slower due to the need for perturbations.
An Interactively Reinforced Paradigm for Joint Infrared-Visible Image Fusion and Saliency Object Detection
Their common characteristic of seeking complementary cues from different source images motivates us to explore the collaborative relationship between Fusion and Salient object detection tasks on infrared and visible images via an Interactively Reinforced multi-task paradigm for the first time, termed IRFS.
SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model
In this study, we leverage SAM and existing RS object detection datasets to develop an efficient pipeline for generating a large-scale RS segmentation dataset, dubbed SAMRS.
Empowering Agrifood System with Artificial Intelligence: A Survey of the Progress, Challenges and Opportunities
With the world population rapidly increasing, transforming our agrifood systems to be more productive, efficient, safe, and sustainable is crucial to mitigate potential food shortages.
HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search
Recent neural architecture search (NAS) based approaches have made great progress in hyperspectral image (HSI) classification tasks.
DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification
Hyperspectral image (HSI) classification is challenging due to spatial variability caused by complex imaging conditions.
Practical Differentially Private and Byzantine-resilient Federated Learning
In contrast, we leverage the random noise to construct an aggregation that effectively rejects many existing Byzantine attacks.
Inductive Graph Unlearning
To extend machine unlearning to graph data, \textit{GraphEraser} has been proposed.
Differentially Private Stochastic Convex Optimization in (Non)-Euclidean Space Revisited
In this paper, we revisit the problem of Differentially Private Stochastic Convex Optimization (DP-SCO) in Euclidean and general $\ell_p^d$ spaces.
DIME-Net: Neural Network-Based Dynamic Intrinsic Parameter Rectification for Cameras with Optical Image Stabilization System
Here we propose a novel neural network-based approach that estimates $\mathrm{K}$ matrix in real-time so that pose estimation or scene reconstruction can be run at camera native resolution for the highest accuracy on mobile devices.
Sketching with Spherical Designs for Noisy Data Fitting on Spheres
These interesting findings show that the proposed sketching strategy is capable of fitting massive and noisy data on spheres.
Kernel-Based Distributed Q-Learning: A Scalable Reinforcement Learning Approach for Dynamic Treatment Regimes
The novelties of our research are as follows: 1) From a methodological perspective, we present a novel and scalable approach for generating DTRs by combining distributed learning with Q-learning.
Personalized and privacy-preserving federated heterogeneous medical image analysis with PPPML-HMI
However, there are few open-source frameworks for federated heterogeneous medical image analysis with personalization and privacy protection simultaneously without the demand to modify the existing model structures or to share any private data.
Quantum Computing Provides Exponential Regret Improvement in Episodic Reinforcement Learning
This improvement is a key to the significant regret improvement in quantum reinforcement learning.
Robust Budget Pacing with a Single Sample
Given the inherent non-stationarity in an advertiser's value and also competing advertisers' values over time, a commonly used approach is to learn a target expenditure plan that specifies a target spend as a function of time, and then run a controller that tracks this plan.
Quantum Heavy-tailed Bandits
In this paper, we study multi-armed bandits (MAB) and stochastic linear bandits (SLB) with heavy-tailed rewards and quantum reward oracle.
Differentially Private Natural Language Models: Recent Advances and Future Directions
In this paper, we provide the first systematic review of recent advances in DP deep learning models in NLP.
USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text Retrieval
(2) It limits the scale of negative sample pairs by employing the mini-batch based end-to-end training mechanism.
Loss-Controlling Calibration for Predictive Models
By comparison, the predictors built by the proposed loss-controlling approach are not limited to set predictors, and the loss function can be any measurable function without the monotone assumption.
Conformal Loss-Controlling Prediction
Conformal prediction is a learning framework controlling prediction coverage of prediction sets, which can be built on any learning algorithm for point prediction.
Multi-Aspect Explainable Inductive Relation Prediction by Sentence Transformer
Recent studies on knowledge graphs (KGs) show that path-based methods empowered by pre-trained language models perform well in the provision of inductive and explainable relation predictions.
Broad Learning System with Takagi-Sugeno Fuzzy Subsystem for Tobacco Origin Identification based on Near Infrared Spectroscopy
In this paper, a novel broad learning system with Takagi-Sugeno (TS) fuzzy subsystem is proposed for rapid identification of tobacco origin.
Quantizing Heavy-tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recovery
Our major standpoint is that (near) minimax rates of estimation error are achievable merely from the quantized data produced by the proposed scheme.
Intelligent Feature Extraction, Data Fusion and Detection of Concrete Bridge Cracks: Current Development and Challenges
As a common appearance defect of concrete bridges, cracks are important indices for bridge structure health assessment.
SEAT: Stable and Explainable Attention
Results show that SEAT is more stable against different perturbations and randomness while also keeps the explainability of attention, which indicates it is a more faithful explanation.
Semantic-aware Texture-Structure Feature Collaboration for Underwater Image Enhancement
To address the above limitations, we develop an efficient and compact enhancement network in collaboration with a high-level semantic-aware pretrained model, aiming to exploit its hierarchical feature representation as an auxiliary for the low-level underwater image enhancement.
1st ICLR International Workshop on Privacy, Accountability, Interpretability, Robustness, Reasoning on Structured Data (PAIR^2Struct)
Recent years have seen advances on principles and guidance relating to accountable and ethical use of artificial intelligence (AI) spring up around the globe.
On Stability and Generalization of Bilevel Optimization Problem
In this paper, we conduct a thorough analysis on the generalization of first-order (gradient-based) methods for the bilevel optimization problem.
On PAC Learning Halfspaces in Non-interactive Local Privacy Model with Public Unlabeled Data
In this paper, we study the problem of PAC learning halfspaces in the non-interactive local differential privacy model (NLDP).
Truthful Generalized Linear Models
In the first part of the paper, we consider the case where the covariates are sub-Gaussian and the responses are heavy-tailed where they only have the finite fourth moments.
Online Bidding Algorithms for Return-on-Spend Constrained Advertisers
We contribute a simple online algorithm that achieves near-optimal regret in expectation while always respecting the specified RoS constraint when the input sequence of queries are i. i. d.
KL-divergence Based Deep Learning for Discrete Time Model
Neural Network (Deep Learning) is a modern model in Artificial Intelligence and it has been exploited in Survival Analysis.
Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model
Large-scale vision foundation models have made significant progress in visual tasks on natural images, with vision transformers being the primary choice due to their good scalability and representation ability.
Ranked #1 on
Aerial Scene Classification
on AID (50% as trainset)
Deep neural network heatmaps capture Alzheimer's disease patterns reported in a large meta-analysis of neuroimaging studies
Deep neural networks currently provide the most advanced and accurate machine learning models to distinguish between structural MRI scans of subjects with Alzheimer's disease and healthy controls.
Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration
Moreover, to better fuse the registered infrared images and visible images, we present a feature Interaction Fusion Module (IFM) to adaptively select more meaningful features for fusion in the Dual-path Interaction Fusion Network (DIFN).
An Empirical Study of Remote Sensing Pretraining
To this end, we train different networks from scratch with the help of the largest RS scene recognition dataset up to now -- MillionAID, to obtain a series of RS pretrained backbones, including both convolutional neural networks (CNN) and vision transformers such as Swin and ViTAE, which have shown promising performance on computer vision tasks.
Ranked #1 on
Aerial Scene Classification
on UCM (80% as trainset)
Aerial Scene Classification
Building change detection for remote sensing images
+5
High Dimensional Statistical Estimation under Uniformly Dithered One-bit Quantization
In heavy-tailed regime, while the rates of our estimators become essentially slower, these results are either the first ones in an 1-bit quantized and heavy-tailed setting, or already improve on existing comparable results from some respect.
Differentially Private $\ell_1$-norm Linear Regression with Heavy-tailed Data
Firstly, we study the case where the $\ell_2$ norm of data has bounded second order moment.
VDPC: Variational Density Peak Clustering Algorithm
Furthermore, we divide all data points into different levels according to their local density and propose a unified clustering framework by combining the advantages of both DPC and DBSCAN.
Fed2: Feature-Aligned Federated Learning
Federated learning learns from scattered data by fusing collaborative models from local nodes.
A Survey of Large-Scale Deep Learning Serving System Optimization: Challenges and Opportunities
With both scaling trends, new problems and challenges emerge in DL inference serving systems, which gradually trends towards Large-scale Deep learning Serving systems (LDS).
Single volume lung biomechanics from chest computed tomography using a mode preserving generative adversarial network
Once trained, the framework can be used as a registration-free method for predicting local tissue expansion.
Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning
In particular, in mobile and IoT devices, real-time data can be stored not just in high-speed RAMs but in internal storage devices as well, which offer significantly larger capacity than the RAMs.
Incorporating Surprisingly Popular Algorithm and Euclidean Distance-based Adaptive Topology into PSO
While many Particle Swarm Optimization (PSO) algorithms only use fitness to assess the performance of particles, in this work, we adopt Surprisingly Popular Algorithm (SPA) as a complementary metric in addition to fitness.
PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction
In this paper, we propose a Pre-trained masked Language model with Misspelled knowledgE (PLOME) for CSC, which jointly learns how to understand language and correct spelling errors.
Concept-Based Label Embedding via Dynamic Routing for Hierarchical Text Classification
In this paper, we propose a novel concept-based label embedding method that can explicitly represent the concept and model the sharing mechanism among classes for the hierarchical text classification.
Faster Rates of Private Stochastic Convex Optimization
Specifically, we first show that under some mild assumptions on the loss functions, there is an algorithm whose output could achieve an upper bound of $\tilde{O}((\frac{1}{\sqrt{n}}+\frac{\sqrt{d\log \frac{1}{\delta}}}{n\epsilon})^\frac{\theta}{\theta-1})$ for $(\epsilon, \delta)$-DP when $\theta\geq 2$, here $n$ is the sample size and $d$ is the dimension of the space.
High Dimensional Differentially Private Stochastic Optimization with Heavy-tailed Data
To better understand the challenges arising from irregular data distribution, in this paper we provide the first study on the problem of DP-SCO with heavy-tailed data in the high dimensional space.
Spectral-Spatial Global Graph Reasoning for Hyperspectral Image Classification
To tackle these problems, in this paper, different from previous approaches, we perform the superpixel generation on intermediate features during network training to adaptively produce homogeneous regions, obtain graph structures, and further generate spatial descriptors, which are served as graph nodes.
UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Prediction
Keyphrase Prediction (KP) task aims at predicting several keyphrases that can summarize the main idea of the given document.
Optimal Rates of (Locally) Differentially Private Heavy-tailed Multi-Armed Bandits
Finally, we establish the lower bound to show that the instance-dependent regret of our improved algorithm is optimal.
$\ell_2$-norm Flow Diffusion in Near-Linear Time
Diffusion is a fundamental graph procedure and has been a basic building block in a wide range of theoretical and empirical applications such as graph partitioning and semi-supervised learning on graphs.
GSA-Forecaster: Forecasting Graph-Based Time-Dependent Data with Graph Sequence Attention
This task is challenging as models need not only to capture spatial dependency and temporal dependency within the data, but also to leverage useful auxiliary information for accurate predictions.
3DMNDT:3D multi-view registration method based on the normal distributions transform
Under the NDT framework, this paper proposes a novel multi-view registration method, named 3D multi-view registration based on the normal distributions transform (3DMNDT), which integrates the K-means clustering and Lie algebra solver to achieve multi-view registration.
Minimum Cost Flows, MDPs, and $\ell_1$-Regression in Nearly Linear Time for Dense Instances
In the special case of the minimum cost flow problem on $n$-vertex $m$-edge graphs with integer polynomially-bounded costs and capacities we obtain a randomized method which solves the problem in $\tilde{O}(m+n^{1. 5})$ time.
Data Structures and Algorithms Optimization and Control
CARE: Commonsense-Aware Emotional Response Generation with Latent Concepts
Experimental results on two large-scale datasets support our hypothesis and show that our model can produce more accurate and commonsense-aware emotional responses and achieve better human ratings than state-of-the-art models that only specialize in one aspect.
Third ArchEdge Workshop: Exploring the Design Space of Efficient Deep Neural Networks
This paper gives an overview of our ongoing work on the design space exploration of efficient deep neural networks (DNNs).
Empirical Risk Minimization in the Non-interactive Local Model of Differential Privacy
In our second attempt, we show that for any $1$-Lipschitz generalized linear convex loss function, there is an $(\epsilon, \delta)$-LDP algorithm whose sample complexity for achieving error $\alpha$ is only linear in the dimensionality $p$.
Deep Learning Analysis and Age Prediction from Shoeprints
We collected 100, 000 shoeprints of subjects ranging from 7 to 80 years old and used the data to develop a deep learning end-to-end model ShoeNet to analyze age-related patterns and predict age.
Differentially Private (Gradient) Expectation Maximization Algorithm with Statistical Guarantees
To address this issue, we propose in this paper the first DP version of (Gradient) EM algorithm with statistical guarantees.
On Differentially Private Stochastic Convex Optimization with Heavy-tailed Data
For this case, we propose a method based on the sample-and-aggregate framework, which has an excess population risk of $\tilde{O}(\frac{d^3}{n\epsilon^4})$ (after omitting other factors), where $n$ is the sample size and $d$ is the dimensionality of the data.
Estimating Stochastic Linear Combination of Non-linear Regressions Efficiently and Scalably
To the best of our knowledge, this is the first work that studies and provides theoretical guarantees for the stochastic linear combination of non-linear regressions model.
Robust High Dimensional Expectation Maximization Algorithm via Trimmed Hard Thresholding
In this paper, we study the problem of estimating latent variable models with arbitrarily corrupted samples in high dimensional space ({\em i. e.,} $d\gg n$) where the underlying parameter is assumed to be sparse.
Learning Robust Algorithms for Online Allocation Problems Using Adversarial Training
In this paper, we focus on the AdWords problem, which is a classical online budgeted matching problem of both theoretical and practical significance.
ECG Beats Fast Classification Base on Sparse Dictionaries
In VQ method, a set of dictionaries corresponding to segments of ECG beats is trained, and VQ codes are used to represent each heartbeat.
Heterogeneous Federated Learning
Specifically, we design a feature-oriented regulation method ({$\Psi$-Net}) to ensure explicit feature information allocation in different neural network structures.
AntiDote: Attention-based Dynamic Optimization for Neural Network Runtime Efficiency
Based on the neural network attention mechanism, we propose a comprehensive dynamic optimization framework including (1) testing-phase channel and column feature map pruning, as well as (2) training-phase optimization by targeted dropout.
Raising Expectations: Automating Expected Cost Analysis with Types
The effectiveness of the technique is evaluated by analyzing the sample complexity of discrete distributions and with a novel average-case estimation for deterministic programs that combines expected cost analysis with statistical methods.
Programming Languages
$p$-Norm Flow Diffusion for Local Graph Clustering
Local graph clustering and the closely related seed set expansion problem are primitives on graphs that are central to a wide range of analytic and learning tasks such as local clustering, community detection, nodes ranking and feature inference.
Towards Assessment of Randomized Smoothing Mechanisms for Certifying Adversarial Robustness
Based on our framework, we assess the Gaussian and Exponential mechanisms by comparing the magnitude of additive noise required by these mechanisms and the lower bounds (criteria).
Distributed Kernel Ridge Regression with Communications
This paper focuses on generalization performance analysis for distributed algorithms in the framework of learning theory.
Robust Feature-Based Point Registration Using Directional Mixture Model
This paper presents a robust probabilistic point registration method for estimating the rigid transformation (i. e. rotation matrix and translation vector) between two pointcloud dataset.
Unsupervised Domain Adaptation for Object Detection via Cross-Domain Semi-Supervised Learning
In this work, we show that such adversarial-based methods can only reduce the domain style gap, but cannot address the domain content distribution gap that is shown to be important for object detectors.
Facility Location Problem in Differential Privacy Model Revisited
On the negative side, we show that the approximation ratio of any $\epsilon$-DP algorithm is lower bounded by $\Omega(\frac{1}{\sqrt{\epsilon}})$, even for instances on HST metrics with uniform facility cost, under the super-set output setting.
Estimating Smooth GLM in Non-interactive Local Differential Privacy Model with Public Unlabeled Data
In the second part of the paper, we extend our idea to the problem of estimating non-linear regressions and show similar results as in GLMs for both multivariate Gaussian and sub-Gaussian cases.
INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps
3) The driving behavior is highly interactive and complex with adversarial and cooperative motions of various traffic participants.
Faster width-dependent algorithm for mixed packing and covering LPs
As a special case of our result, we report a $1+\eps$ approximation algorithm for the densest subgraph problem which runs in time $O(md/ \eps)$, where $m$ is the number of edges in the graph and $d$ is the maximum graph degree.
A Unified framework for randomized smoothing based certified defenses
We answer the above two questions by first demonstrating that Gaussian mechanism and Exponential mechanism are the (near) optimal options to certify the $\ell_2$ and $\ell_\infty$-normed robustness.
YaoGAN: Learning Worst-case Competitive Algorithms from Self-generated Inputs
To answer this question, we draw insights from classic results in game theory, analysis of algorithms, and online learning to introduce a novel framework.
