Search Results for author:
Found 148 papers, 60 papers with code
SASE: A Searching Architecture for Squeeze and Excitation Operations
Neural Architecture Search (NAS), meanwhile, is able to automate the design of neural networks and spares the numerous experiments required for an optimal architecture.
DSDE: Using Proportion Estimation to Improve Model Selection for Out-of-Distribution Detection
In this paper, we emphasize the significance of the proportion of models in the library that identify the test sample as an OoD sample.
Enhancing Zeroth-order Fine-tuning for Language Models with Low-rank Structures
However, traditional first-order (FO) fine-tuning algorithms incur substantial memory overhead due to the need to store activation values for back-propagation during gradient computation, particularly in long-context fine-tuning tasks.
SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference
In vision-language models (VLMs), visual tokens usually consume a significant amount of computational overhead, despite their sparser information density compared to text tokens.
Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMs
Vision-and-Language Navigation (VLN) tasks require an agent to follow textual instructions to navigate through 3D environments.
Generic Diagonalizability, Structural Functional Observability and Output Controllability
This paper investigates the structural functional observability (SFO) and structural output controllability (SOC) of a class of systems with generically diagonalizable state matrices and explores the associated minimal sensor and actuator placement problems.
A Novel Perspective for Multi-modal Multi-label Skin Lesion Classification
The efficacy of deep learning-based Computer-Aided Diagnosis (CAD) methods for skin diseases relies on analyzing multiple data modalities (i. e., clinical+dermoscopic images, and patient metadata) and addressing the challenges of multi-label classification.
Human-AI Collaborative Multi-modal Multi-rater Learning for Endometriosis Diagnosis
HAICOMM is the first method that explores three important aspects of this problem: 1) multi-rater learning to extract a cleaner label from the multiple "noisy" labels available per training sample; 2) multi-modal learning to leverage the presence of T1/T2 MRI images for training and testing; and 3) human-AI collaboration to build a system that leverages the predictions from clinicians and the AI model to provide more accurate classification than standalone clinicians and AI models.
La-SoftMoE CLIP for Unified Physical-Digital Face Attack Detection
Some studies attempt to combine the sparse data from both types of attacks into a single dataset and try to find a common feature space, which is often impractical due to the space is difficult to be found or even non-existent.
A Unified Framework for Iris Anti-Spoofing: Introducing IrisGeneral Dataset and Masked-MoE Method
Specifically, we utilize the Mixture of Experts (MoE) to fit complex data distributions using multiple sub-neural networks.
Minimal Sensor Placement for Generic State and Unknown Input Observability
Based on these conditions, we prove that determining the minimum number of dedicated sensors to achieve generic state and input observability is NP-hard, which contrasts sharply with the polynomial-time complexity of the corresponding problem with known inputs.
RadioDiff: An Effective Generative Diffusion Model for Sampling-Free Dynamic Radio Map Construction
Thus, to enhance RM construction performance, in this paper, the sampling-free RM construction is modeled as a conditional generative problem, where a denoised diffusion-based method, named RadioDiff, is proposed to achieve high-quality RM construction.
Focus, Distinguish, and Prompt: Unleashing CLIP for Efficient and Flexible Scene Text Retrieval
Scene text retrieval aims to find all images containing the query text from an image gallery.
PlacidDreamer: Advancing Harmony in Text-to-3D Generation
Previous methods utilize end-to-end 3D generation models to initialize 3D Gaussians, multi-view diffusion models to enforce multi-view consistency, and text-to-image diffusion models to refine details with score distillation algorithms.
4Dynamic: Text-to-4D Generation with Hybrid Priors
The static 3D generation is achieved under the guidance of the input text and the first frame of the reference video, while in the dynamic generation stage, we introduce a customized SDS loss to ensure multi-view consistency, a video-based SDS loss to improve temporal consistency, and most importantly, direct priors from the reference video to ensure the quality of geometry and texture.
Latent Linear Quadratic Regulator for Robotic Control Tasks
Model predictive control (MPC) has played a more crucial role in various robotic control tasks, but its high computational requirements are concerning, especially for nonlinear dynamical models.
DSCENet: Dynamic Screening and Clinical-Enhanced Multimodal Fusion for MPNs Subtype Classification
(1) A dynamic screening module is proposed to flexibly adapt the feature learning of local patches, reducing the interference of irrelevant features and enhancing their diagnostic representativeness.
LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control
Instead of following mainstream diffusion-based methods, we explore and extend the potential of the implicit-keypoint-based framework, which effectively balances computational efficiency and controllability.
ClawMachine: Fetching Visual Tokens as An Entity for Referring and Grounding
Existing methods, including proxy encoding and geometry encoding, incorporate additional syntax to encode the object's location, bringing extra burdens in training MLLMs to communicate between language and vision.
Artemis: Towards Referential Understanding in Complex Videos
Videos carry rich visual information including object description, action, interaction, etc., but the existing multimodal large language models (MLLMs) fell short in referential understanding scenarios such as video-based referring.
Benchmarking and Improving Detail Image Caption
We also design a more reliable caption evaluation metric called CAPTURE (CAPtion evaluation by exTracting and coUpling coRE information).
SG-Adapter: Enhancing Text-to-Image Generation with Scene Graph Guidance
In this paper, we introduce the Scene Graph Adapter(SG-Adapter), leveraging the structured representation of scene graphs to rectify inaccuracies in the original text embeddings.
Unveiling the Tapestry of Consistency in Large Vision-Language Models
To this end, we provide a multi-modal benchmark ConBench, to intuitively analyze how LVLMs perform when the solution space of a prompt revolves around a knowledge point.
A rapid approach to urban traffic noise mapping with a generative adversarial network
With rapid urbanisation and the accompanying increase in traffic density, traffic noise has become a major concern in urban planning.
UDUC: An Uncertainty-driven Approach for Learning-based Robust Control
Learning-based techniques have become popular in both model predictive control (MPC) and reinforcement learning (RL).
Multimodal Emotion Recognition by Fusing Video Semantic in MOOC Learning Scenarios
The method proposed in this paper not only contributes to a deeper understanding of the impact of instructional videos on learners' emotional states but also provides a beneficial reference for future research on emotion recognition in MOOC learning scenarios.
ReALM: Reference Resolution As Language Modeling
Reference resolution is an important problem, one that is essential to understand and successfully handle context of different kinds.
Constrained Reinforcement Learning with Smoothed Log Barrier Function
However, for many real-world problems, it is often more convenient to formulate optimization problems in terms of rewards and constraints simultaneously.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we introduce the Gemini 1. 5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Ranked #1 on
Visual Question Answering
on MM-Vet
Unifying Generation and Compression: Ultra-low bitrate Image Coding Via Multi-stage Transformer
Recent progress in generative compression technology has significantly improved the perceptual quality of compressed data.
"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach
Modern deep neural networks (DNNs) are extremely powerful; however, this comes at the price of increased depth and having more parameters per layer, making their training and inference more computationally challenging.
Open Ad Hoc Teamwork with Cooperative Game Theory
Ad hoc teamwork poses a challenging problem, requiring the design of an agent to collaborate with teammates without prior coordination or joint training.
Instruction Backdoor Attacks Against Customized LLMs
The increasing demand for customized Large Language Models (LLMs) has led to the development of solutions like GPTs.
Intersectional Two-sided Fairness in Recommendation
Fairness of recommender systems (RS) has attracted increasing attention recently.
Understanding Time Series Anomaly State Detection through One-Class Classification
Therefore, in this article, we try to re-understand and define the time series anomaly detection problem through OCC, which we call 'time series anomaly state detection problem'.
Can Large Language Models Understand Context?
Understanding context is key to understanding human language, an ability which Large Language Models (LLMs) have been increasingly seen to demonstrate to an impressive extent.
ChatterBox: Multi-round Multimodal Referring and Grounding
In this study, we establish a baseline for a new task named multimodal multi-round referring and grounding (MRG), opening up a promising direction for instance-level multimodal dialogues.
Enhancing the Power of OOD Detection via Sample-Aware Model Selection
In this work we present a novel perspective on detecting out-of-distribution (OOD) samples and propose an algorithm for sample-aware model selection to enhance the effectiveness of OOD detection.
Cloud-Device Collaborative Learning for Multimodal Large Language Models
However, the deployment of these large-scale MLLMs on client devices is hindered by their extensive model parameters, leading to a notable decline in generalization capabilities when these models are compressed for device deployment.
DVIS++: Improved Decoupled Framework for Universal Video Segmentation
We present the \textbf{D}ecoupled \textbf{VI}deo \textbf{S}egmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video instance segmentation (VIS), video semantic segmentation (VSS), and video panoptic segmentation (VPS).
Ranked #1 on
Video Semantic Segmentation
on VSPW
FedSODA: Federated Cross-assessment and Dynamic Aggregation for Histopathology Segmentation
Federated learning (FL) for histopathology image segmentation involving multiple medical sites plays a crucial role in advancing the field of accurate disease diagnosis and treatment.
Stable Segment Anything Model
Thus, our solution, termed Stable-SAM, offers several advantages: 1) improved SAM's segmentation stability across a wide range of prompt qualities, while 2) retaining SAM's powerful promptable segmentation efficiency and generality, with 3) minimal learnable parameters (0. 08 M) and fast adaptation (by 1 training epoch).
FreeKD: Knowledge Distillation via Semantic Frequency Prompt
(2) During the distillation period, a pixel-wise frequency mask is generated via Frequency Prompt, to localize those pixel of interests (PoIs) in various frequency bands.
MARRS: Multimodal Reference Resolution System
Successfully handling context is essential for any dialog understanding task.
Collaboration and Transition: Distilling Item Transitions into Multi-Query Self-Attention for Sequential Recommendation
Second, we develop a transition-aware embedding distillation module that distills global item-to-item transition patterns into item embeddings, which enables the model to memorize and leverage transitional signals and serves as a calibrator for collaborative signals.
Generalized Cactus and Structural Controllability of Switched Linear Continuous-Time Systems
This paper explores the structural controllability of switched linear continuous-time systems.
1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation
Video instance segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving.
Label-free Deep Learning Driven Secure Access Selection in Space-Air-Ground Integrated Networks
Considering the inherent co-channel interference due to spectrum sharing among multi-tier access networks in SAGIN, it can be leveraged to assist the physical layer security among heterogeneous transmissions.
Effectively Heterogeneous Federated Learning: A Pairing and Split Learning Based Approach
As a promising paradigm federated Learning (FL) is widely used in privacy-preserving machine learning, which allows distributed devices to collaboratively train a model while avoiding data transmission among clients.
Functional Observability, Structural Functional Observability and Optimal Sensor Placement
Based on these results, the problems of selecting the minimal sensors from a prior set to achieve functional observability and SFO are shown to be NP-hard.
Measuring Item Global Residual Value for Fair Recommendation
In this paper, we call for a shift of attention from modeling user preferences to developing fair exposure mechanisms for items.
Dynamic Snake Convolution based on Topological Geometric Constraints for Tubular Structure Segmentation
In this work, we note the specificity of tubular structures and use this knowledge to guide our DSCNet to simultaneously enhance perception in three stages: feature extraction, feature fusion, and loss constraint.
Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation
Through theoretical analyses, we find that the conservatism of existing methods fails in pursuing users' long-term satisfaction.
Distilling Missing Modality Knowledge from Ultrasound for Endometriosis Diagnosis with Magnetic Resonance Images
Next, we distill the knowledge from the teacher TVUS POD obliteration detector to train the student MRI model by minimizing a regression loss that approximates the output of the student to the teacher using unpaired TVUS and MRI data.
Razor SNN: Efficient Spiking Neural Network with Temporal Embeddings
The event streams generated by dynamic vision sensors (DVS) are sparse and non-uniform in the spatial domain, while still dense and redundant in the temporal domain.
1st Place Solution for PVUW Challenge 2023: Video Panoptic Segmentation
In this report, we successfully validated the effectiveness of the decoupling strategy in video panoptic segmentation.
DVIS: Decoupled Video Instance Segmentation Framework
The efficacy of the decoupling strategy relies on two crucial elements: 1) attaining precise long-term alignment outcomes via frame-by-frame association during tracking, and 2) the effective utilization of temporal information predicated on the aforementioned accurate alignment outcomes during refinement.
Ranked #4 on
Video Panoptic Segmentation
on VIPSeg
Knowledge Diffusion for Distillation
To address this, we propose to denoise student features using a diffusion model trained by teacher features.
Distribution-Free Matrix Prediction Under Arbitrary Missing Pattern
This paper studies the open problem of conformalized entry prediction in a row/column-exchangeable matrix.
Avatar Knowledge Distillation: Self-ensemble Teacher Paradigm with Uncertainty
Knowledge distillation is an effective paradigm for boosting the performance of pocket-size model, especially when multiple teacher models are available, the student would break the upper limit again.
Observability Blocking for Functional Privacy of Linear Dynamic Networks
This paper addresses the problem of determining the minimum set of state variables in a network that need to be blocked from direct measurements in order to protect functional privacy with respect to {\emph{any}} output matrices.
Judicial Intelligent Assistant System: Extracting Events from Divorce Cases to Detect Disputes for the Judge
It is a difficult but necessary task to extract the key information for the cases from these textual materials and to clarify the dispute focus of related parties.
Time-Variance Aware Real-Time Speech Enhancement
Time-variant factors often occur in real-world full-duplex communication applications.
On the Reachability and Controllability of Temporal Continuous-Time Linear Networks: A Generic Analysis
This paper investigates the reachability and controllability of temporal continuous-time linear networks from a generic viewpoint, where only the zero-nonzero patterns of subsystem matrices are known.
Real-time speech enhancement with dynamic attention span
For real-time speech enhancement (SE) including noise suppression, dereverberation and acoustic echo cancellation, the time-variance of the audio signals becomes a severe challenge.
Divide and Conquer: Towards Better Embedding-based Retrieval for Recommender Systems From a Multi-task Perspective
Embedding-based retrieval (EBR) methods are widely used in modern recommender systems thanks to its simplicity and effectiveness.
Incorporating Recurrent Reinforcement Learning into Model Predictive Control for Adaptive Control in Autonomous Driving
Model Predictive Control (MPC) is attracting tremendous attention in the autonomous driving task as a powerful control technique.
Capacity Analysis of Holographic MIMO Channels with Practical Constraints
However, the channel might not satisfy isotropic scattering because of generalized angle distributions, and the antenna gain is limited by the array aperture in reality.
Mask the Correct Tokens: An Embarrassingly Simple Approach for Error Correction
Since the error rate of the incorrect sentence is usually low (e. g., 10\%), the correction model can only learn to correct on limited error tokens but trivially copy on most tokens (correct tokens), which harms the effective training of error correction.
DAMO-YOLO : A Report on Real-Time Object Detection Design
Based on these new techs, we build a suite of models at various scales to meet the needs of different scenarios.
Ranked #46 on
Real-Time Object Detection
on MS COCO
Disentangled Feature Learning for Real-Time Neural Speech Coding
Recently end-to-end neural audio/speech coding has shown its great potential to outperform traditional signal analysis based audio codecs.
Differentially Private Optimizers Can Learn Adversarially Robust Models
Machine learning models have shone in a variety of domains and attracted increasing attention from both the security and the privacy communities.
FV2ES: A Fully End2End Multimodal System for Fast Yet Effective Video Emotion Recognition Inference
In the latest social networks, more and more people prefer to express their emotions in videos through text, speech, and rich facial expressions.
Real-time Short Video Recommendation on Mobile Devices
However, as users continue to watch videos and feedback, the changing context leads the ranking of the server-side recommendation system inaccurate.
KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos
To facilitate model learning, we further collect rich features of users and items as well as users' behavior history.
Higher-order accurate two-sample network inference and network hashing
Two-sample hypothesis testing for network comparison presents many significant challenges, including: leveraging repeated network observations and known node registration, but without requiring them to operate; relaxing strong structural assumptions; achieving finite-sample higher-order accuracy; handling different network sizes and sparsity levels; fast computation and memory parsimony; controlling false discovery rate (FDR) in multiple testing; and theoretical understandings, particularly regarding finite-sample accuracy and minimax optimality.
Latent-Domain Predictive Neural Speech Coding
Neural audio/speech coding has recently demonstrated its capability to deliver high quality at much lower bitrates than traditional methods.
Cross-Scale Vector Quantization for Scalable Neural Speech Coding
In this paper, we introduce a cross-scale scalable vector quantization scheme (CSVQ), in which multi-scale features are encoded progressively with stepwise feature fusion and refinement.
Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization
To deal with unknown uncertainty sets, we further propose a novel adversarial approach to generate them based on the value function.
Masked Distillation with Receptive Tokens
To obtain a group of masks, the receptive tokens are learned via the regular task loss but with teacher fixed, and we also leverage a Dice loss to enrich the diversity of learned masks.
WSSS4LUAD: Grand Challenge on Weakly-supervised Tissue Semantic Segmentation for Lung Adenocarcinoma
According to the technical reports of the top-tier teams, CAM is still the most popular approach in WSSS.
Data Augmentation
Weakly supervised Semantic Segmentation
+1
CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System
The basic idea is to first learn a causal user model on historical data to capture the overexposure effect of items on user satisfaction.
On Polynomially Solvable Constrained Input Selections for Fixed and Switched Linear Structured Systems
In this paper, it is found that, if the input structure satisfies certain `regularizations', which are characterized by the proposed restricted total unimodulairty notion, those problems can be solvable in polynomial time via linear programming (LP) relaxations.
LBCF: A Large-Scale Budget-Constrained Causal Forest Algorithm
The proposed approach is currently serving over hundreds of millions of users on the platform and achieves one of the most tremendous improvements over these months.
In Defense of Kalman Filtering for Polyp Tracking from Colonoscopy Videos
We advocate that the field should instead focus on the development of simple and efficient detectors that an be combined with effective trackers to allow the implementation of real-time polyp detectors.
End-to-End Neural Speech Coding for Real-Time Communications
Deep-learning based methods have shown their advantages in audio coding over traditional ones but limited attention has been paid on real-time communications (RTC).
On real structured controllability/stabilizability/stability radius: Complexity and unified rank-relaxation based methods
This paper addresses the real structured controllability, stabilizability, and stability radii (RSCR, RSSZR, and RSSR, respectively) of linear systems, which involve determining the distance (in terms of matrix norms) between a (possibly large-scale) system and its nearest uncontrollable, unstabilizable, and unstable systems, respectively, with a prescribed affine structure.
Learning Multi-granularity User Intent Unit for Session-based Recommendation
Specifically, we creatively propose Multi-granularity Intent Heterogeneous Session Graph which captures the interactions between different granularity intent units and relieves the burden of long-dependency.
MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Image
In this work, we propose a framework for single-view hand mesh reconstruction, which can simultaneously achieve high reconstruction accuracy, fast inference speed, and temporal coherence.
Ranked #2 on
3D Hand Pose Estimation
on FreiHAND
Controllable Semantic Parsing via Retrieval Augmentation
In practical applications of semantic parsing, we often want to rapidly change the behavior of the parser, such as enabling it to handle queries in a new domain, or changing its predictions on certain targeted queries.
A free lunch from ViT:Adaptive Attention Multi-scale Fusion Transformer for Fine-grained Visual Recognition
Learning subtle representation about object parts plays a vital role in fine-grained visual recognition (FGVR) field.
Ranked #12 on
Fine-Grained Image Classification
on Stanford Dogs
Fine-Grained Image Classification
Fine-Grained Visual Recognition
Practical and Private Heterogeneous Federated Learning
Heterogeneous federated learning (HFL) enables clients with different computation/communication capabilities to collaboratively train their own customized models, in which the knowledge of models is shared via clients' predictions on a public dataset.
AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance
While previous pruning methods have mostly focused on identifying unimportant channels, channel pruning is considered as a special case of neural architecture search in recent years.
r-GAT: Relational Graph Attention Network for Multi-Relational Graphs
To tackle this issue, we propose r-GAT, a relational graph attention network to learn multi-channel entity representations.
PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition
In this paper, we propose a Parallel, Iterative and Mimicking Network (PIMNet) to balance accuracy and efficiency.
Total Unimodularity and Strongly Polynomial Solvability of Constrained Minimum Input Selections for Structural Controllability: an LP-based Method
Subject to a certain condition on the prescribed input configurations that contains the dedicated input one as a special case, we demonstrate that the constraint matrices of these ILPs are totally unimodular.
Distilling Neuron Spike with High Temperature in Reinforcement Learning Agents
This method uses distillation to effectively avoid the weakness of STBP, which can achieve SOTA performance in classification, and can obtain a smaller, faster convergence and lower power consumption SNN reinforcement learning model.
QA-Driven Zero-shot Slot Filling with Weak Supervision Pretraining
To address this problem, we introduce QA-driven slot filling (QASF), which extracts slot-filler spans from utterances with a span-based QA model.
Latent Space Model for Higher-order Networks and Generalized Tensor Decomposition
We formulate the relationship between the latent positions and the observed data via a generalized multilinear kernel as the link function.
Partial Strong Structural Controllability
This paper introduces a new controllability notion, termed partial strong structural controllability (PSSC), on a structured system whose entries of system matrices are either fixed zero or indeterminate, which naturally extends the conventional strong structural controllability (SSC) and bridges the gap between structural controllability and SSC.
SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning
This paper studies a theoretical framework for value factorisation with interpretability via Shapley value theory.
Recent Standard Development Activities on Video Coding for Machines
Specifically, we will first provide an overview of the MPEG VCM group including use cases, requirements, processing pipelines, plan for potential VCM standards, followed by the evaluation framework including machine-vision tasks, dataset, evaluation metrics, and anchor generation.
Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations
Sequence-to-sequence (seq2seq) models are prevalent in semantic parsing, but have been found to struggle at out-of-distribution compositional generalization.
Ranked #3 on
Semantic Parsing
on CFQ
Few-shot Intent Classification and Slot Filling with Retrieved Examples
Few-shot learning arises in important practical scenarios, such as when a natural language understanding system needs to learn new semantic labels for an emerging, resource-scarce domain.
ProDCoNN-server: a web server for protein sequence prediction and design from a three-dimensional structure
The prediction is very fast compared with other protein sequence prediction servers - it takes only a few minutes for a query protein on average.
PTSC: a New Definition for Structural Controllability under Numerical Perturbations
This paper proposes a novel notion for structural controllability under structured numerical perturbations, namely the perturbation-tolerant structural controllability (PTSC), on a single-input structured system whose entries can be classified into three categories: fixed zero entries, unknown generic entries whose values are fixed but unknown, and perturbed entries that can take arbitrary complex values.