Search Results for author:
Found 34 papers, 16 papers with code
Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representations
Domain Generalization (DG) aims to enhance model robustness in unseen or distributionally shifted target domains through training exclusively on source domains.
LoRA Users Beware: A Few Spurious Tokens Can Manipulate Your Finetuned Model
Second, for light SSTI, the reliance on spurious tokens is proportional to the LoRA rank.
IRBridge: Solving Image Restoration Bridge with Pre-trained Generative Diffusion Models
The main challenge is that standard generative models are typically designed for a diffusion process that starts from pure noise, while restoration tasks begin with a low-quality image, resulting in a mismatch in the state distributions between the two processes.
RDI: An adversarial robustness evaluation metric for deep neural networks based on model statistical features
The average computation time of RDI is only 1/30 of the evaluation method based on the PGD attack.
Enhancing Multi-modal Models with Heterogeneous MoE Adapters for Fine-tuning
Multi-modal models excel in cross-modal tasks but are computationally expensive due to their billions of parameters.
Towards Transformer-Based Aligned Generation with Self-Coherence Guidance
We introduce a novel, training-free approach for enhancing alignment in Transformer-based Text-Guided Diffusion Models (TGDMs).
Query-Based and Unnoticeable Graph Injection Attack from Neighborhood Perspective
Various attack methods have been proposed to explore the vulnerabilities of GNNs, ranging from Graph Modification Attacks (GMA) to the more practical and flexible Graph Injection Attacks (GIA).
Compositional Concept-Based Neuron-Level Interpretability for Deep Reinforcement Learning
In this work, we propose a novel concept-based interpretability method that provides fine-grained explanations of DRL models at the neuron level.
Semantic Residual for Multimodal Unified Discrete Representation
Recent research in the domain of multimodal unified representations predominantly employs codebook as representation forms, utilizing Vector Quantization(VQ) for quantization, yet there has been insufficient exploration of other quantization representation forms.
Evaluating and Enhancing LLMs for Multi-turn Text-to-SQL with Multiple Question Types
Recent advancements in large language models (LLMs) have significantly advanced text-to-SQL systems.
Ranked #1 on
MMSQL performance
on MMSQL
How Much Data is Enough? Optimization of Data Collection for Artifact Detection in EEG Recordings
Our work addresses the need for effective data utilization in biological data collection, offering a systematic and dynamic quantitative approach.
Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Step
SiDA utilizes the encoder from the generator's score network as a discriminator, allowing it to distinguish between real images and those generated by SiD.
Ranked #1 on
Image Generation
on FFHQ 64x64
ALLoRA: Adaptive Learning Rate Mitigates LoRA Fatal Flaws
We identify three core limitations to LoRA for finetuning--a setting that employs limited amount of data and training steps.
A load forecasting approach for integrated energy systems based on aggregation hybrid modal decomposition and combined model
To improve the granularity of load decomposition within integrated energy system (IES), a novel multiple load forecasting approach is proposed.
ACE: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling
Generative retrieval, which has demonstrated effectiveness in text-to-text retrieval, utilizes a sequence-to-sequence model to directly generate candidate identifiers based on natural language queries.
ADBA:Approximation Decision Boundary Approach for Black-Box Adversarial Attacks
Many machine learning models are susceptible to adversarial attacks, with decision-based black-box attacks representing the most critical threat in real-world applications.
Long and Short Guidance in Score identity Distillation for One-Step Text-to-Image Generation
Specifically, its data-free distillation of Stable Diffusion 1. 5 achieves a record low FID of 8. 15 on the COCO-2014 validation set, with a CLIP score of 0. 304 at an LSG scale of 1. 5, and an FID of 9. 56 with a CLIP score of 0. 313 at an LSG scale of 2.
ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control
In this paper, we present ControlSpeech, a text-to-speech (TTS) system capable of fully cloning the speaker's voice and enabling arbitrary control and adjustment of speaking style.
Less is More: on the Over-Globalizing Problem in Graph Transformers
Then we propose a novel Bi-Level Global Graph Transformer with Collaborative Training (CoBFormer), including the inter-cluster and intra-cluster Transformers, to prevent the over-globalizing problem while keeping the ability to extract valuable information from distant nodes.
GCC: Generative Calibration Clustering
Deep clustering as an important branch of unsupervised representation learning focuses on embedding semantically similar samples into the identical feature space.
Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation
This achievement not only redefines the benchmarks for efficiency and effectiveness in diffusion distillation but also in the broader field of diffusion-based generation.
Ranked #2 on
Image Generation
on AFHQ-v2 64x64
Enhancing Multimodal Unified Representations for Cross Modal Generalization
To enhance the interpretability of multimodal unified representations, many studies have focused on discrete unified representations.
Language-Codec: Bridging Discrete Codec Representations and Speech Language Models
Furthermore, we also validate the efficiency of the Language-Codec on downstream speech language models.
Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts
In the field of large language models (LLMs), aligning models with the diverse preferences of users is a critical challenge.
Rethinking Evaluation Metrics of Open-Vocabulary Segmentaion
We benchmarked the proposed evaluation metrics on 12 open-vocabulary methods of three segmentation tasks.
CVPR 2023 Text Guided Video Editing Competition
In this paper we present a retrospective on the competition and describe the winning method.
Prompt Backdoors in Visual Prompt Learning
Specifically, the VPPTaaS provider optimizes a visual prompt given downstream data, and downstream users can use this prompt together with the large pre-trained model for prediction.
Composite Backdoor Attacks Against Large Language Models
Such a Composite Backdoor Attack (CBA) is shown to be stealthier than implanting the same multiple trigger keys in only a single component.
Parallel Computing Based Solution for Reliability-Constrained Distribution Network Planning
The main goal of distribution network (DN) expansion planning is essentially to achieve minimal investment constrained with specified reliability requirements.
On the Privacy Risks of Cell-Based NAS Architectures
Existing studies on neural architecture search (NAS) mainly focus on efficiently and effectively searching for network architectures with better performance.
Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for Trajectory Prediction
First, with the continuation of time, the prediction error at each time step increases significantly, causing the final displacement error to be impossible to ignore.
Data Poisoning Attacks to Deep Learning Based Recommender Systems
Specifically, we formulate our attack as an optimization problem, such that the injected ratings would maximize the number of normal users to whom the target items are recommended.
Assimilating Soil Moisture Retrieved from Sentinel-1 and Sentinel-2 Data into WOFOST Model to Improve Winter Wheat Yield Estimation
The aim of this study is to improve the accuracy for winter wheat yield estimation by assimilating time series soil moisture images, which are retrieved by a water cloud model using SAR and optical data as input, into the crop model.
A GPU-accelerated package for simulation of flow in nanoporous source rocks with many-body dissipative particle dynamics
Lastly, we demonstrate, through a flow simulation in realistic shale pores, that the CPU counterpart requires 840 Power9 cores to rival the performance delivered by our package with four V100 GPUs on ORNL's Summit architecture.
Computational Physics
