Search Results for author:
Found 684 papers, 308 papers with code
Momentum Batch Normalization for Deep Learning with Small Batch Size
To make a deeper understanding of BN, in this work we prove that BN actually introduces a certain level of noise into the sample mean and variance during the training process, while the noise level depends only on the batch size.
A Decoupled Learning Scheme for Real-world Burst Denoising from Raw Images
On one hand, most of the models are trained on video sequences with synthetic noise.
LST-Net: Learning a Convolutional Neural Network with a Learnable Sparse Transform
The 2D convolutional (Conv2d) layer is the fundamental element to a deep convolutional neural network (CNN).
Mining Word Boundaries from Speech-Text Parallel Data for Cross-domain Chinese Word Segmentation
Inspired by early research on exploring naturally annotated data for Chinese Word Segmentation (CWS), and also by recent research on integration of speech and text processing, this work for the first time proposes to explicitly mine word boundaries from speech-text parallel data.
Manta: Enhancing Mamba for Few-Shot Action Recognition of Long Sub-Sequence
The Matryoshka Mamba and the hybrid contrastive learning paradigm operate in parallel branches within Manta, enhancing Mamba for FSAR of long sub-sequence.
Evaluating and Aligning CodeLLMs on Human Preference
Code large language models (codeLLMs) have made significant strides in code generation.
Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach
By introducing two adjustable guidance scales on the two LoRA modules to control the strengths of pixel-wise fidelity and semantic-level details during inference, PiSASR can offer flexible SR results according to user preference without re-training.
Single-Cell Omics Arena: A Benchmark Study for Large Language Models on Cell Type Annotation Using Single-Cell Data
To thoroughly evaluate the capability of modern instruction-tuned LLMs in automating the cell type identification process, we introduce SOAR, a comprehensive benchmarking study of LLMs for cell type annotation tasks in single-cell genomics.
HandOS: 3D Hand Reconstruction in One Stage
In this work, we propose HandOS, an end-to-end framework for 3D hand reconstruction.
Learning on Less: Constraining Pre-trained Model Learning for Generalizable Diffusion-Generated Image Detection
Based on these facts, we propose a simple yet effective training method called Learning on Less (LoL).
Don't Let Your Robot be Harmful: Responsible Robotic Manipulation
Additionally, we create the SafeBox synthetic dataset, which includes one hundred responsible robotic manipulation tasks with different safety risk scenarios and instructions, effectively reducing the risks associated with real-world experiments.
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding
From the data perspective, we build a fully automated data engine and construct the Rexverse-2M dataset which possesses multiple granularities to support the joint training of perception and understanding.
TAPTRv3: Spatial and Temporal Context Foster Robust Tracking of Any Point in Long Video
In this paper, we present TAPTRv3, which is built upon TAPTRv2 to improve its point tracking robustness in long videos.
Scaling Speech-Text Pre-training with Synthetic Interleaved Data
Starting from a pre-trained language model and scaling our pre-training to 1 trillion tokens (with 600B synthetic interleaved speech-text data), we achieve state-of-the-art performance in speech language modeling and spoken question answering, improving performance on spoken questions tasks from the previous SOTA of 13% (Moshi) to 31%.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Neural Network-based High-index Saddle Dynamics Method for Searching Saddle Points and Solution Landscape
The high-index saddle dynamics (HiSD) method is a powerful approach for computing saddle points and solution landscape.
OSMamba: Omnidirectional Spectral Mamba with Dual-Domain Prior Generator for Exposure Correction
Exposure correction is a fundamental problem in computer vision and image processing.
DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding
DINO-X employs the same Transformer-based encoder-decoder architecture as Grounding DINO 1. 5 to pursue an object-level representation for open-world object understanding.
Advancing Sustainability via Recommender Systems: A Survey
Human behavioral patterns and consumption paradigms have emerged as pivotal determinants in environmental degradation and climate change, with quotidian decisions pertaining to transportation, energy utilization, and resource consumption collectively precipitating substantial ecological impacts.
A Learned Proximal Alternating Minimization Algorithm and Its Induced Network for a Class of Two-block Nonconvex and Nonsmooth Optimization
For smoothed nonconvex problems we modify the proximal alternating linearized minimization (PALM) scheme by incorporating the residual learning architecture, which has proven to be highly effective in deep network training, and employing the block coordinate decent (BCD) iterates as a safeguard for the convergence of the algorithm.
Minder: Faulty Machine Detection for Large-scale Distributed Model Training
To address the drawbacks of the time-consuming and labor-intensive manual scrutiny, we propose Minder, an automatic faulty machine detector for distributed training tasks.
Optical Flow Representation Alignment Mamba Diffusion Model for Medical Video Generation
Medical video generation models are expected to have a profound impact on the healthcare industry, including but not limited to medical education and training, surgical planning, and simulation.
Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learning
Meta-learning offers a promising avenue for few-shot learning (FSL), enabling models to glean a generalizable feature embedding through episodic training on synthetic FSL tasks in a source domain.
Automating Quantum Software Maintenance: Flakiness Detection and Root Cause Analysis
We aim to create an automated framework to detect flaky tests in quantum software and an extended dataset of quantum flaky tests, overcoming the limitations of manual methods.
AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language Models
To overcome this issue, we introduce \textit{adaptive negative proxies}, which are dynamically generated during testing by exploring actual OOD images, to align more closely with the underlying OOD label space and enhance the efficacy of negative proxy guidance.
MotionCLR: Motion Generation and Training-free Editing via Understanding Attention Mechanisms
Previous motion diffusion models lack explicit modeling of the word-level text-motion correspondence and good explainability, hence restricting their fine-grained editing ability.
FreCaS: Efficient Higher-Resolution Image Generation via Frequency-aware Cascaded Sampling
While image generation with diffusion models has achieved a great success, generating images of higher resolution than the training size remains a challenging task due to the high computational cost.
Efficient Antibody Structure Refinement Using Energy-Guided SE(3) Flow Matching
In this paper, we develop a novel antibody structure refinement method termed FlowAB based on energy-guided flow matching.
Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion
Selective state space models (SSMs), such as Mamba, highly excel at capturing long-range dependencies in 1D sequential data, while their applications to 2D vision tasks still face challenges.
Toward Generalizing Visual Brain Decoding to Unseen Subjects
Prior works typically focus on decoding brain activity of individuals based on the observation that different subjects exhibit different brain activities, while it remains unclear whether brain decoding can be generalized to unseen subjects.
UniG: Modelling Unitary 3D Gaussians for View-consistent 3D Reconstruction
In this work, we present UniG, a view-consistent 3D reconstruction and novel view synthesis model that generates a high-fidelity representation of 3D Gaussians from sparse images.
Self-Supervised Scene Flow Estimation with Point-Voxel Fusion and Surface Representation
Scene flow estimation aims to generate the 3D motion field of points between two consecutive frames of point clouds, which has wide applications in various fields.
LKASeg:Remote-Sensing Image Semantic Segmentation with Large Kernel Attention and Full-Scale Skip Connections
In this paper, we propose a remote-sensing image semantic segmentation network named LKASeg, which combines Large Kernel Attention(LSKA) and Full-Scale Skip Connections(FSC).
Visual-O1: Understanding Ambiguous Instructions via Multi-modal Multi-turn Chain-of-thoughts Reasoning
As large-scale models evolve, language instructions are increasingly utilized in multi-modal tasks.
Ruler: A Model-Agnostic Method to Control Generated Length for Large Language Models
Specifically, Ruler equips LLMs with the ability to generate responses of a specified length based on length constraints within the instructions.
Self-supervised Monocular Depth Estimation with Large Kernel Attention
Self-supervised monocular depth estimation has emerged as a promising approach since it does not rely on labeled training data.
LKA-ReID:Vehicle Re-Identification with Large Kernel Attention
Using attention mechanism to capture global and local features is crucial to solve the challenge of high similarity between classes in vehicle Re-ID tasks.
DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D Diffusion
The core of this framework lies in Skeleton-guided Score Distillation and Hybrid 3D Gaussian Avatar representation.
Path-adaptive Spatio-Temporal State Space Model for Event-based Recognition with Arbitrary Duration
Our key insight is to learn the spatiotemporal relationships from the encoded event features via the state space model (SSM) -- whose linear complexity makes it ideal for modeling high temporal resolution events with longer sequences.
Qwen2.5-Coder Technical Report
In this report, we introduce the Qwen2. 5-Coder series, a significant upgrade from its predecessor, CodeQwen1. 5.
ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation
Transparent object depth perception poses a challenge in everyday life and logistics, primarily due to the inability of standard 3D sensors to accurately capture depth on transparent or reflective surfaces.
TMFNet: Two-Stream Multi-Channels Fusion Networks for Color Image Operation Chain Detection
To solve these issues, in this article, we propose a novel two-stream multi-channels fusion networks for color image operation chain detection in which the spatial artifact stream and the noise residual stream are explored in a complementary manner.
CustomContrast: A Multilevel Contrastive Perspective For Subject-Driven Text-to-Image Customization
This misconstruction leads to both overfitting or underfitting of irrelevant and intrinsic attributes of the subject, i. e., these attributes are over-represented or under-represented simultaneously, causing a trade-off between similarity and controllability.
A Survey of Multimodal Composite Editing and Retrieval
To facilitate a deeper understanding of this promising direction, this survey explores multimodal composite editing and retrieval in depth, covering image-text composite editing, image-text composite retrieval, and other multimodal composite retrieval.
Fed-MUnet: Multi-modal Federated Unet for Brain Tumor Segmentation
To address the above challenges, we propose a novel multi-modal FL framework for brain tumor segmentation (Fed-MUnet) that is suitable for FL training.
LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation
We introduce LLaVA-MoD, a novel framework designed to enable the efficient training of small-scale Multimodal Language Models (s-MLLM) by distilling knowledge from large-scale MLLM (l-MLLM).
Guided and Fused: Efficient Frozen CLIP-ViT with Feature Guidance and Multi-Stage Feature Fusion for Generalizable Deepfake Detection
Recent methods leveraging the frozen pre-trained CLIP-ViT model have made great progress in deepfake detection.
Factorized-Dreamer: Training A High-Quality Video Generator with Limited and Low-Quality Data
Text-to-video (T2V) generation has gained significant attention due to its wide applications to video generation, editing, enhancement and translation, \etc.
VRCopilot: Authoring 3D Layouts with Generative AI Models in VR
Through a series of user studies, we evaluated the potential and challenges in manual, scaffolded, and automatic creation in immersive authoring.
SkillMimic: Learning Reusable Basketball Skills from Demonstrations
SkillMimic employs a unified configuration to learn diverse skills from human-ball motion datasets, with skill diversity and generalization improving as the dataset grows.
SSL: A Self-similarity Loss for Improving Generative Image Super-resolution
Based on the fact that natural images exhibit high self-similarities, i. e., a local patch can have many similar patches to it in the whole image, in this work we propose a simple yet effective self-similarity loss (SSL) to improve the performance of generative Real-ISR models, enhancing the hallucination of structural and textural details while reducing the unpleasant visual artifacts.
Dual-path Collaborative Generation Network for Emotional Video Captioning
Emotional Video Captioning is an emerging task that aims to describe factual content with the intrinsic emotions expressed in videos.
DNA-SE: Towards Deep Neural-Nets Assisted Semiparametric Estimation
Semiparametric statistics play a pivotal role in a wide range of domains, including but not limited to missing data, causal inference, and transfer learning, to name a few.
Boosting Gaze Object Prediction via Pixel-level Supervision from Vision Foundation Model
Gaze object prediction (GOP) aims to predict the category and location of the object that a human is looking at.
Gradient Harmonization in Unsupervised Domain Adaptation
In this paper, we delve into this issue and introduce two effective solutions known as Gradient Harmonization, including GH and GH++, to mitigate the conflict between domain alignment and classification tasks.
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #2 on
Multi-task Language Understanding
on MMLU
GaussianSR: High Fidelity 2D Gaussian Splatting for Arbitrary-Scale Image Super-Resolution
Although these approaches have shown promising results, their performance is constrained by the limited representation ability of discrete latent codes in the encoded features.
Occlusion-Aware 3D Motion Interpretation for Abnormal Behavior Detection
Estimating abnormal posture based on 3D pose is vital in human pose analysis, yet it presents challenges, especially when reconstructing 3D human poses from monocular datasets with occlusions.
SOAP: Enhancing Spatio-Temporal Relation and Motion Information Capturing for Few-Shot Action Recognition
High frame-rate (HFR) videos of action recognition improve fine-grained expression while reducing the spatio-temporal relation and motion information density.
TAPTRv2: Attention-based Position Update Improves Tracking Any Point
In this paper, we present TAPTRv2, a Transformer-based approach built upon TAPTR for solving the Tracking Any Point (TAP) task.
LIDIA: Precise Liver Tumor Diagnosis on Multi-Phase Contrast-Enhanced CT via Iterative Fusion and Asymmetric Contrastive Learning
The early detection and precise diagnosis of liver tumors are tasks of critical clinical value, yet they pose significant challenges due to the high heterogeneity and variability of liver tumors.
Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillation
The scarcity of large-scale 3D-text paired data poses a great challenge on open vocabulary 3D scene understanding, and hence it is popular to leverage internet-scale 2D data and transfer their open vocabulary capabilities to 3D models through knowledge distillation.
General Geometry-aware Weakly Supervised 3D Object Detection
However, the annotation of large-scale 3D datasets requires significant human effort.
Close the Sim2real Gap via Physically-based Structured Light Synthetic Data Simulation
Despite the substantial progress in deep learning, its adoption in industrial robotics projects remains limited, primarily due to challenges in data acquisition and labeling.
Dilated convolution neural operator for multiscale partial differential equations
This paper introduces a data-driven operator learning method for multiscale partial differential equations, with a particular emphasis on preserving high-frequency information.
Dense Multimodal Alignment for Open-Vocabulary 3D Scene Understanding
Recent vision-language pre-training models have exhibited remarkable generalization ability in zero-shot recognition tasks.
LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models
The LAPT framework operates autonomously, requiring only ID class names as input and eliminating the need for manual intervention.
Urban Waterlogging Detection: A Challenging Benchmark and Large-Small Model Co-Adapter
Urban waterlogging poses a major risk to public safety and infrastructure.
A Text-to-Game Engine for UGC-Based Role-Playing Games
The shift from professionally generated content (PGC) to user-generated content (UGC) has revolutionized various media formats, from text to video.
MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesis
Auto-regressive models have made significant progress in the realm of language generation, yet they do not perform on par with diffusion models in the domain of image synthesis.
AI-based Automatic Segmentation of Prostate on Multi-modality Images: A Review
However, prostate segmentation is challenging due to imperfections in the images and the prostate's complex tissue structure.
EMBANet: A Flexible Efffcient Multi-branch Attention Network
The proposed MBC module brings new degrees of freedom (DoF) for the design of attention networks by allowing the type of transformation operators and the number of branches to be flexibly adjusted.
SymPoint Revolutionized: Boosting Panoptic Symbol Spotting with Layer Feature Enhancement
In particular, we first propose a Layer Feature-Enhanced module (LFE) to encode the graphical layer information into the primitive feature, which significantly boosts the performance.
TokenPacker: Efficient Visual Projector for Multimodal LLM
However, the visual tokens are redundant and can be considerably increased when dealing with high-resolution images, impairing the efficiency of MLLMs significantly.
Ranked #160 on
Visual Question Answering
on MM-Vet
ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillation
Current state-of-the-arts such as Variational Score Distillation finetune the pretrained diffusion model to minimize the noise prediction error so as to align the distributions, which are however unstable to train and will impair the model's comprehension capability to numerous text prompts.
YuLan: An Open-source Large Language Model
This paper presents the development of YuLan, a series of open-source LLMs with $12$ billion parameters.
Hierarchical Context Pruning: Optimizing Real-World Code Completion with Repository-Level Pretrained Code LLMs
We applied the HCP strategy in experiments with six Repo-Code LLMs, and the results demonstrate that our proposed method can significantly enhance completion accuracy while substantially reducing the length of input.
Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
Long-context modeling capabilities have garnered widespread attention, leading to the emergence of Large Language Models (LLMs) with ultra-context windows.
SyncNoise: Geometrically Consistent Noise Prediction for Text-based 3D Scene Editing
Meanwhile, the 2D diffusion models also exhibit substantial potentials for 3D editing tasks.
CausalMMM: Learning Causal Structure for Marketing Mix Modeling
We argue that causal MMM needs dynamically discover specific causal structures for different shops and the predictions should comply with the prior known marketing response patterns.
Soft Masked Mamba Diffusion Model for CT to MRI Conversion
Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) are the predominant modalities utilized in the field of medical imaging.
Toward Exploring the Code Understanding Capabilities of Pre-trained Code Generation Models
While these models acquire vast amounts of code knowledge, they perform poorly on code understanding tasks, such as code search and clone detection, as they are specifically trained for generation.
Integrating sensing and communications: Simultaneously transmitting and reflecting digital coding metasurfaces
Wireless networks are undergoing a transformative shift, driven by the crucial factors of cost effectiveness and sustainability.
Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection
Inspired by the recent advances of state space models (SSMs), we present a Voxel SSM, termed as Voxel Mamba, which employs a group-free strategy to serialize the whole space of voxels into a single sequence.
Interference Analysis for Coexistence of UAVs and Civil Aircrafts Based on Automatic Dependent Surveillance-Broadcast
However, due to the limited resource of channel capacity, UAVs equipped with ADS-B results in the interference between UAVs and civil aircrafts (CAs), which further impacts the accuracy of received information at GSs.
One-Step Effective Diffusion Network for Real-World Image Super-Resolution
Most of the existing methods start from random noise to reconstruct the high-quality (HQ) image under the guidance of the given low-quality (LQ) image.
Open-World Human-Object Interaction Detection via Multi-modal Prompts
In this paper, we develop \textbf{MP-HOI}, a powerful Multi-modal Prompt-based HOI detector designed to leverage both textual descriptions for open-set generalization and visual exemplars for handling high ambiguity in descriptions, realizing HOI detection in the open world.
Autoregressive Pretraining with Mamba in Vision
The vision community has started to build with the recently developed state space model, Mamba, as the new backbone for a range of tasks.
Methodology and Real-World Applications of Dynamic Uncertain Causality Graph for Clinical Diagnosis with Explainability and Invariance
After verifications, the 46 DUCG models were applied in the real-world in China.
MotionLLM: Understanding Human Behaviors from Human Motions and Videos
This study delves into the realm of multi-modality (i. e., video and motion modalities) human behavior understanding by leveraging the powerful capabilities of Large Language Models (LLMs).
Network Interdiction Goes Neural
Network interdiction problems are combinatorial optimization problems involving two players: one aims to solve an optimization problem on a network, while the other seeks to modify the network to thwart the first player's objectives.
Spiking Neural Network Phase Encoding for Cognitive Computing
Additionally, the paper discusses the encoding of impulse delays and the phase differences between adjacent frequency components.
DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception
The development of large language models (LLMs) has significantly advanced the emergence of large multimodal models (LMMs).
LIRE: listwise reward enhancement for preference alignment
Recently, tremendous strides have been made to align the generation of Large Language Models (LLMs) with human values to mitigate toxic or unhelpful content.
Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detection
Empirical results demonstrate the effectiveness of Grounding DINO 1. 5, with the Grounding DINO 1. 5 Pro model attaining a 54. 3 AP on the COCO detection benchmark and a 55. 7 AP on the LVIS-minival zero-shot transfer benchmark, setting new records for open-set object detection.
Ranked #1 on
Zero-Shot Object Detection
on MSCOCO
(using extra training data)
NTIRE 2024 Restore Any Image Model (RAIM) in the Wild Challenge
In this paper, we review the NTIRE 2024 challenge on Restore Any Image Model (RAIM) in the Wild.
DoLLM: How Large Language Models Understanding Network Flow Data to Detect Carpet Bombing DDoS
We evaluate DoLLM with public datasets CIC-DDoS2019 and real NetFlow trace from Top-3 countrywide ISP.
PCLMix: Weakly Supervised Medical Image Segmentation via Pixel-Level Contrastive Learning and Dynamic Mix Augmentation
In weakly supervised medical image segmentation, the absence of structural priors and the discreteness of class feature distribution present a challenge, i. e., how to accurately propagate supervision signals from local to global regions without excessively spreading them to other irrelevant regions?
MasterWeaver: Taming Editability and Face Identity for Personalized Text-to-Image Generation
In this work, we present MasterWeaver, a test-time tuning-free method designed to generate personalized images with both faithful identity fidelity and flexible editability.
