Search Results for author:
Found 53 papers, 19 papers with code
L2COcc: Lightweight Camera-Centric Semantic Scene Completion via Distillation of LiDAR Model
Semantic Scene Completion (SSC) constitutes a pivotal element in autonomous driving perception systems, tasked with inferring the 3D semantic occupancy of a scene from sensory data.
Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descent
By explicitly disentangling the optimization of visual understanding from task-specific alignment, MDGD preserves pre-trained visual knowledge while enabling efficient task adaptation.
Latent Swap Joint Diffusion for 2D Long-Form Latent Generation
To address this issue, we propose Self-Loop Latent Swap, a frame-level bidirectional swap applied to the overlapping region of adjacent views.
The Silent Prompt: Initial Noise as Implicit Guidance for Goal-Driven Image Generation
Text-to-image synthesis (T2I) has advanced remarkably with the emergence of large-scale diffusion models.
SOWing Information: Cultivating Contextual Coherence with MLLMs in Image Generation
Originating from the diffusion phenomenon in physics, which describes the random movement and collisions of particles, diffusion generative models simulate a random walk in the data space along the denoising trajectory.
Regularized Multi-LLMs Collaboration for Enhanced Score-based Causal Discovery
As the significance of understanding the cause-and-effect relationships among variables increases in the development of modern systems and algorithms, learning causality from observational data has become a preferred and efficient approach over conducting randomized control trials.
OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Models
To overcome the reasoning heterogeneity and grounding problems, we leverage on-policy KG exploration and RL to model a KG policy that generates token-level likelihood distributions for LLM-generated chain-of-thought reasoning paths, simulating KG reasoning preference.
CL-HOI: Cross-Level Human-Object Interaction Distillation from Vision Large Language Models
Human-object interaction (HOI) detection has seen advancements with Vision Language Models (VLMs), but these methods often depend on extensive manual annotations.
Space-time 2D Gaussian Splatting for Accurate Surface Reconstruction under Complex Dynamic Scenes
To tackle the dynamic contents and the occlusions in complex scenes, we present a space-time 2D Gaussian Splatting approach.
Incorporating Spatial Cues in Modular Speaker Diarization for Multi-channel Multi-party Meetings
This paper proposes a three-stage modular system to enhance single-channel neural speaker diarization systems and recognition performance by utilizing spatial cues from multi-channel speech to provide more accurate initialization for each stage of neural speaker diarization (NSD) decoding: (1) Overlap detection and continuous speech separation (CSS) on multi-channel speech are used to obtain cleaner single speaker speech segments for clustering, followed by the first NSD decoding pass.
ASFT: Aligned Supervised Fine-Tuning through Absolute Likelihood
Additionally, we compare ASFT to DPO and its latest variants, such as the single-step approach ORPO, using the latest instruction-tuned model Llama3, which has been fine-tuned on UltraFeedback and HH-RLHF.
Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs
Large Language Models (LLMs) have demonstrated remarkable efficiency in tackling various tasks based on human instructions, but studies reveal that they often struggle with tasks requiring reasoning, such as math or physics.
Causality-Aware Transformer Networks for Robotic Navigation
We address these constraints by initially exploring the unique differences between Navigation tasks and other sequential data tasks through the lens of Causality, presenting a causal framework to elucidate the inadequacies of conventional sequential methods for Navigation.
Independence Constrained Disentangled Representation Learning from Epistemological Perspective
Disentangled Representation Learning aims to improve the explainability of deep learning methods by training a data encoder that identifies semantically meaningful latent variables in the data generation process.
The USTC-NERCSLIP Systems for the CHiME-8 NOTSOFAR-1 Challenge
This technical report outlines our submission system for the CHiME-8 NOTSOFAR-1 Challenge.
AdaOcc: Adaptive-Resolution Occupancy Prediction
We conducted comprehensive experiments on the nuScenes dataset, demonstrating significant improvements over existing methods.
ARVO: Atlas of Reproducible Vulnerabilities for Open Source Software
High-quality datasets of real-world vulnerabilities are enormously valuable for downstream research in software security, but existing datasets are typically small, require extensive manual effort to update, and are missing crucial features that such research needs.
MultiFuser: Multimodal Fusion Transformer for Enhanced Driver Action Recognition
Driver action recognition, aiming to accurately identify drivers' behaviours, is crucial for enhancing driver-vehicle interactions and ensuring driving safety.
CM2-Net: Continual Cross-Modal Mapping Network for Driver Action Recognition
Therefore, previous works have suggested independently learning each non-RGB modality by fine-tuning a model pre-trained on RGB videos, but these methods are less effective in extracting informative features when faced with newly-incoming modalities due to large domain gaps.
MALT: Multi-scale Action Learning Transformer for Online Action Detection
In this way, output features transition from coarse to fine as the branches deepen.
QA-MDT: Quality-aware Masked Diffusion Transformer for Enhanced Music Generation
In recent years, diffusion-based text-to-music (TTM) generation has gained prominence, offering an innovative approach to synthesizing musical content from textual descriptions.
Ranked #1 on
Music Generation
on Song Describer Dataset
Multi-agent Traffic Prediction via Denoised Endpoint Distribution
The exploration of high-speed movement by robots or road traffic agents is crucial for autonomous driving and navigation.
Behind the Veil: Enhanced Indoor 3D Scene Reconstruction with Occluded Surfaces Completion
The 3D Inpainter with abstract representation at coarse levels is trained offline using various scenes to complete occluded surfaces.
TCLC-GS: Tightly Coupled LiDAR-Camera Gaussian Splatting for Autonomous Driving
Most 3D Gaussian Splatting (3D-GS) based methods for urban scenes initialize 3D Gaussians directly with 3D LiDAR points, which not only underutilizes LiDAR data capabilities but also overlooks the potential advantages of fusing LiDAR with camera data.
SUP-NeRF: A Streamlined Unification of Pose Estimation and NeRF for Monocular 3D Object Reconstruction
While gradient-based optimization in a NeRF framework updates the initial pose, this paper highlights that scale-depth ambiguity in monocular object reconstruction causes failures when the initial pose deviates moderately from the true pose.
Multitask frame-level learning for few-shot sound event detection
This paper focuses on few-shot Sound Event Detection (SED), which aims to automatically recognize and classify sound events with limited samples.
StainFuser: Controlling Diffusion for Faster Neural Style Transfer in Multi-Gigapixel Histology Images
Stain normalization algorithms aim to transform the color and intensity characteristics of a source multi-gigapixel histology image to match those of a target image, mitigating inconsistencies in the appearance of stains used to highlight cellular components in the images.
A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition
In this paper, we investigate this contrasting phenomenon from the perspective of modality bias and reveal that an excessive modality bias on the audio caused by dropout is the underlying reason.
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
To measure these risks of malicious use, government institutions and major AI labs are developing evaluations for hazardous capabilities in LLMs.
Multi-modality action recognition based on dual feature shift in vehicle cabin monitoring
Driver Action Recognition (DAR) is crucial in vehicle cabin monitoring systems.
DETER: Detecting Edited Regions for Deterring Generative Manipulations
While face swapping and attribute editing are performed on similar face regions such as eyes and nose, the inpainting operation can be performed on random image regions, removing the spurious correlations of previous datasets.
An Automated Pipeline for Tumour-Infiltrating Lymphocyte Scoring in Breast Cancer
Tumour-infiltrating lymphocytes (TILs) are considered as a valuable prognostic markers in both triple-negative and human epidermal growth factor receptor 2 (HER2) positive breast cancer.
Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding with Sequence-to-Sequence Architecture
We propose a novel neural speaker diarization system using memory-aware multi-speaker embedding with sequence-to-sequence architecture (NSD-MS2S), which integrates the strengths of memory-aware multi-speaker embedding (MA-MSE) and sequence-to-sequence (Seq2Seq) architecture, leading to improvement in both efficiency and performance.
The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction
This pioneering effort aims to set the first benchmark for the AVTSE task, offering fresh insights into enhancing the ac-curacy of back-end speech recognition systems through AVTSE in challenging and real acoustic environments.
The USTC-NERCSLIP Systems for the CHiME-7 DASR Challenge
This technical report details our submission system to the CHiME-7 DASR Challenge, which focuses on speaker diarization and speech recognition under complex multi-speaker scenarios.
Concavity-Induced Distance for Unoriented Point Cloud Decomposition
We propose Concavity-induced Distance (CID) as a novel way to measure the dissimilarity between a pair of points in an unoriented point cloud.
Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generation
In this work, we investigate the diffusion (physics) in diffusion (machine learning) properties and propose our Cyclic One-Way Diffusion (COW) method to control the direction of diffusion phenomenon given a pre-trained frozen diffusion model for versatile customization application scenarios, where the low-level pixel information from the conditioning needs to be preserved.
P$^2$SDF for Neural Indoor Scene Reconstruction
As the unsupervised plane segments are usually noisy and inaccurate, we propose to assign different weights to the sampled points on the plane in plane estimation as well as the regularization loss.
An Aggregation of Aggregation Methods in Computational Pathology
Image analysis and machine learning algorithms operating on multi-gigapixel whole-slide images (WSIs) often process a large number of tiles (sub-images) and require aggregating predictions from the tiles in order to predict WSI-level labels.
PlaneDepth: Self-supervised Depth Estimation via Orthogonal Planes
PlaneDepth estimates the depth distribution using a Laplacian Mixture Model based on orthogonal planes for an input image.
Self-Supervised Place Recognition by Refining Temporal and Featural Pseudo Labels from Panoramic Data
Inspired by noisy label learning, we propose a novel self-supervised framework named TF-VPR that uses temporal neighborhoods and learnable feature neighborhoods to discover unknown spatial neighborhoods.
Breaking Correlation Shift via Conditional Invariant Regularizer
The correlation shift is caused by the spurious attributes that correlate to the class label, as the correlation between them may vary in training and test data.
TIAger: Tumor-Infiltrating Lymphocyte Scoring in Breast Cancer for the TiGER Challenge
The Tumor InfiltratinG lymphocytes in breast cancER (TiGER) challenge, aims to assess the prognostic significance of computer-generated TILs scores for predicting survival as part of a Cox proportional hazards model.
Rank the triplets: A ranking-based multiple instance learning framework for detecting HPV infection in head and neck cancers using routine H&E images
Unique gene expression profiles were also identified with respect to HPV infection status, and is in line with existing findings.
Out-of-distribution Generalization with Causal Invariant Transformations
In this work, we obviate these assumptions and tackle the OOD problem without explicitly recovering the causal feature.
ECRECer: Enzyme Commission Number Recommendation and Benchmarking based on Multiagent Dual-core Learning
Take UniPort protein "A0A0U5GJ41" as an example (1. 14.-.-), ECRECer annotated it with "1. 14. 11. 38", which supported by further protein structure analysis based on AlphaFold2.
Active Learning-Based Optimization of Scientific Experimental Design
Active learning (AL) is a machine learning algorithm that can achieve greater accuracy with fewer labeled training instances, for having the ability to ask oracles to label the most valuable unlabeled data chosen iteratively and heuristically by query strategies.
Improving OOD Generalization with Causal Invariant Transformations
In this work, we obviate these assumptions and tackle the OOD problem without explicitly recovering the causal feature.
Constructing Flow Graphs from Procedural Cybersecurity Texts
We must read the whole text to identify the relevant information or identify the instruction flows to complete a task, which is prone to failures.
Deep Weakly Supervised Positioning
PoseNet can map a photo to the position where it is taken, which is appealing in robotics.
Characterization of Excess Risk for Locally Strongly Convex Population Risk
Our bounds underscore that with locally strongly convex population risk, the models trained by any proper iterative algorithm can generalize well, even for non-convex problems, and $d$ is large.
SPARE3D: A Dataset for SPAtial REasoning on Three-View Line Drawings
Our experiments show that although convolutional networks have achieved superhuman performance in many visual learning tasks, their spatial reasoning performance on SPARE3D tasks is either lower than average human performance or even close to random guesses.
Real-time Soft Body 3D Proprioception via Deep Vision-based Sensing
Soft bodies made from flexible and deformable materials are popular in many robotics applications, but their proprioceptive sensing has been a long-standing challenge.
Robotics
