Search Results for author:
Found 49 papers, 26 papers with code
Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection
WikiDYK contains 12, 290 facts and 77, 180 questions, which is also seamlessly extensible with future updates from Wikipedia editors.
Enhancing Dance-to-Music Generation via Negative Conditioning Latent Diffusion Model
Conditional diffusion models have gained increasing attention since their impressive results for cross-modal synthesis, where the strong alignment between conditioning input and generated output can be achieved by training a time-conditioned U-Net augmented with cross-attention mechanism.
ProDiF: Protecting Domain-Invariant Features to Secure Pre-Trained Models Against Extraction
Pre-trained models are valuable intellectual property, capturing both domain-specific and domain-invariant features within their weight spaces.
Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-tuning
While current alignment strategies primarily rely on supervised safety fine-tuning with curated datasets, we identify a fundamental limitation we call the "safety mirage" where supervised fine-tuning inadvertently reinforces spurious correlations between superficial textual patterns and safety responses, rather than fostering deep, intrinsic mitigation of harm.
Attention Reveals More Than Tokens: Training-Free Long-Context Reasoning with Attention-guided Retrieval
Interestingly, we observe that the internal attention weights from the generated CoT tokens can effectively ground implicit facts, even when these facts are not explicitly recalled.
Towards Vector Optimization on Low-Dimensional Vector Symbolic Architecture
Vector Symbolic Architecture (VSA) is emerging in machine learning due to its efficiency, but they are hindered by issues of hyperdimensionality and accuracy.
The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1
The rapid development of large reasoning models, such as OpenAI-o3 and DeepSeek-R1, has led to significant improvements in complex reasoning over non-reasoning large language models~(LLMs).
A First-order Generative Bilevel Optimization Framework for Diffusion Models
Diffusion models, which iteratively denoise data samples to synthesize high-quality outputs, have achieved empirical success across domains.
Investigating the Shortcomings of LLMs in Step-by-Step Legal Reasoning
We then develop an LLM-based automated evaluation framework to identify reasoning errors and evaluate the performance of LLMs.
Forget Vectors at Play: Universal Input Perturbations Driving Machine Unlearning in Image Classification
Machine unlearning (MU), which seeks to erase the influence of specific unwanted data from already-trained models, is becoming increasingly vital in model editing, particularly to comply with evolving data regulations like the ``right to be forgotten''.
Diverse Score Distillation
Score distillation of 2D diffusion models has proven to be a powerful mechanism to guide 3D optimization, for example enabling text-based 3D generation or single-view reconstruction.
Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization
Our method introduces a novel optimization framework that leverages both the continuous prompt embedding and the injected noise trajectory in the sampling process to generate safe images.
QUOTA: Quantifying Objects with Text-to-Image Models for Any Domain
We tackle the problem of quantifying the number of objects by a generative text-to-image model.
UOE: Unlearning One Expert Is Enough For Mixture-of-experts LLMS
As MoE LLMs are celebrated for their exceptional performance and highly efficient inference processes, we ask: How can unlearning be performed effectively and efficiently on MoE LLMs?
Prompt Diffusion Robustifies Any-Modality Prompt Learning
This paper introduces prompt diffusion, which uses a diffusion model to gradually refine the prompts to obtain a customized prompt for each sample.
UniMuMo: Unified Text, Music and Motion Generation
We introduce a music-motion parallel generation scheme that unifies all music and motion generation tasks into a single transformer decoder architecture with a single training task of music-motion joint generation.
Boosting Online 3D Multi-Object Tracking through Camera-Radar Cross Check
In the domain of autonomous driving, the integration of multi-modal perception techniques based on data from diverse sensors has demonstrated substantial progress.
Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry
Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations.
Open-world Multi-label Text Classification with Extremely Weak Supervision
We study open-world multi-label text classification under extremely weak supervision (XWS), where the user only provides a brief description for classification objectives without any labels or ground-truth label space.
SegVG: Transferring Object Bounding Box to Segmentation for Visual Grounding
Specifically, we propose the Multi-layer Multi-task Encoder-Decoder as the target grounding stage, where we learn a regression query and multiple segmentation queries to ground the target by regression and segmentation of the box in each decoding layer, respectively.
Pruning One More Token is Enough: Leveraging Latency-Workload Non-Linearities for Vision Transformers on the Edge
For similar latency (within 5. 2% or 7ms) across devices we achieve 78. 6%-84. 5% ImageNet1K accuracy, while the state-of-the-art, Token Merging, achieves 45. 8%-85. 4%.
Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Difference
To achieve both goals, a mainstream class of LLM unlearning methods introduces an optimization framework with a combination of two objectives - maximizing the prediction loss on the forget documents while minimizing that on the retain documents, which suffers from two challenges, degenerated output and catastrophic forgetting.
Towards Hierarchical Multi-Agent Workflows for Zero-Shot Prompt Optimization
Specifically, we include a hierarchy of LLMs, first constructing a prompt with precise instructions and accurate wording in a hierarchical manner, and then using this prompt to generate the final answer to the user query.
Efficient Multitask Dense Predictor via Binarization
Moreover, we introduce a knowledge distillation mechanism to correct the direction of information flow in backward propagation.
Advancing the Robustness of Large Language Models through Self-Denoised Smoothing
Although large language models (LLMs) have achieved significant success, their vulnerability to adversarial perturbations, including recent jailbreak attacks, has raised considerable concerns.
MULTIFLOW: Shifting Towards Task-Agnostic Vision-Language Pruning
In this challenging setting, the transferable representations already encoded in the pretrained model are a key aspect to preserve.
MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection
Monocular 3D object detection (Mono3D) holds noteworthy promise for autonomous driving applications owing to the cost-effectiveness and rich visual context of monocular camera sensors.
A Survey on Large Language Model-Based Game Agents
The development of game agents holds a critical role in advancing towards Artificial General Intelligence.
Training-Free Semantic Segmentation via LLM-Supervision
Additionally, we propose an assembly that merges the segmentation maps from the various subclass descriptors to ensure a more comprehensive representation of the different aspects in the test images.
Urban Scene Diffusion through Semantic Occupancy Map
Generating unbounded 3D scenes is crucial for large-scale scene understanding and simulation.
ThermoHands: A Benchmark for 3D Hand Pose Estimation from Egocentric Thermal Images
Designing egocentric 3D hand pose estimation systems that can perform reliably in complex, real-world scenarios is crucial for downstream applications.
Self-Adapting Large Visual-Language Models to Edge Devices across Visual Modalities
Recent advancements in Vision-Language (VL) models have sparked interest in their deployment on edge devices, yet challenges in handling diverse visual modalities, manual annotation, and computational constraints remain.
UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Models
The technological advancements in diffusion models (DMs) have demonstrated unprecedented capabilities in text-to-image generation and are widely used in diverse applications.
Answer is All You Need: Instruction-following Text Embedding via Answering the Question
This work aims to build a text embedder that can capture characteristics of texts specified by user instructions.
Enhancing Post-training Quantization Calibration through Contrastive Learning
We aim to calibrate the quantized activations by maximizing the mutual information between the pre- and post-quantized activations.
From Trojan Horses to Castle Walls: Unveiling Bilateral Data Poisoning Effects in Diffusion Models
This is because the art necessitates modifications to the diffusion training and sampling procedures.
CenterRadarNet: Joint 3D Object Detection and Tracking Framework using 4D FMCW Radar
CenterRadarNet achieves the state-of-the-art result on the K-Radar 3D object detection benchmark.
Causal-DFQ: Causality Guided Data-free Network Quantization
Inspired by the causal understanding, we propose the Causality-guided Data-free Network Quantization method, Causal-DFQ, to eliminate the reliance on data via approaching an equilibrium of causality-driven intervened distributions.
Fast and Resource-Efficient Object Tracking on Edge Devices: A Measurement Study
Object tracking is an important functionality of edge video analytic systems and services.
$A^2$Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models
We study the task of zero-shot vision-and-language navigation (ZS-VLN), a practical yet challenging problem in which an agent learns to navigate following a path described by language instructions without requiring any path-instruction annotation data.
Riemannian Multinomial Logistics Regression for SPD Neural Networks
Besides, our framework offers a novel intrinsic explanation for the most popular LogEig classifier in existing SPD networks.
Model Sparsity Can Simplify Machine Unlearning
We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner, closing the approximation gap, while continuing to be efficient.
Optical Flow Estimation in 360$^\circ$ Videos: Dataset, Model and Application
Moreover, we present a novel Siamese representation Learning framework for Omnidirectional Flow (SLOF) estimation, which is trained in a contrastive manner via a hybrid loss that combines siamese contrastive and optical flow losses.
Adaptive Deep Neural Network Inference Optimization with EENet
Instead of having every sample go through all DNN layers during prediction, EENet learns an early exit scheduler, which can intelligently terminate the inference earlier for certain predictions, which the model has high confidence of early exit.
Learning Omnidirectional Flow in 360-degree Video via Siamese Representation
Optical flow estimation in omnidirectional videos faces two significant issues: the lack of benchmark datasets and the challenge of adapting perspective video-based methods to accommodate the omnidirectional nature.
Cross-View Exocentric to Egocentric Video Synthesis
Particularly, we propose a novel Bi-directional Spatial Temporal Attention Fusion Generative Adversarial Network (STA-GAN) to learn both spatial and temporal information to generate egocentric video sequences from the exocentric view.
A Metamodel and Framework for Artificial General Intelligence From Theory to Practice
This paper introduces a new metamodel-based knowledge representation that significantly improves autonomous learning and adaptation.
Exocentric to Egocentric Image Generation via Parallel Generative Adversarial Network
In this paper, we investigate exocentric (third-person) view to egocentric (first-person) view image generation.
Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation
In this work, we propose a novel Cycle In Cycle Generative Adversarial Network (C$^2$GAN) for the task of keypoint-guided image generation.
