Search Results for author:
Found 121 papers, 43 papers with code
Multimodal Emotion Recognition by Fusing Video Semantic in MOOC Learning Scenarios
The method proposed in this paper not only contributes to a deeper understanding of the impact of instructional videos on learners' emotional states but also provides a beneficial reference for future research on emotion recognition in MOOC learning scenarios.
ReALM: Reference Resolution As Language Modeling
Reference resolution is an important problem, one that is essential to understand and successfully handle context of different kinds.
Constrained Reinforcement Learning with Smoothed Log Barrier Function
However, for many real-world problems, it is often more convenient to formulate optimization problems in terms of rewards and constraints simultaneously.
Unifying Generation and Compression: Ultra-low bitrate Image Coding Via Multi-stage Transformer
Recent progress in generative compression technology has significantly improved the perceptual quality of compressed data.
"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach
Modern deep neural networks (DNNs) are extremely powerful; however, this comes at the price of increased depth and having more parameters per layer, making their training and inference more computationally challenging.
Open Ad Hoc Teamwork with Cooperative Game Theory
Open ad hoc teamwork further complicates this challenge by considering environments with a changing number of teammates, referred to as open teams.
Rapid Adoption, Hidden Risks: The Dual Impact of Large Language Model Customization
The increasing demand for customized Large Language Models (LLMs) has led to the development of solutions like GPTs.
Intersectional Two-sided Fairness in Recommendation
Fairness of recommender systems (RS) has attracted increasing attention recently.
Understanding Time Series Anomaly State Detection through One-Class Classification
Therefore, in this article, we try to re-understand and define the time series anomaly detection problem through OCC, which we call 'time series anomaly state detection problem'.
Can Large Language Models Understand Context?
Understanding context is key to understanding human language, an ability which Large Language Models (LLMs) have been increasingly seen to demonstrate to an impressive extent.
ChatterBox: Multi-round Multimodal Referring and Grounding
In this study, we establish a baseline for a new task named multimodal multi-round referring and grounding (MRG), opening up a promising direction for instance-level multimodal dialogues.
Cloud-Device Collaborative Learning for Multimodal Large Language Models
However, the deployment of these large-scale MLLMs on client devices is hindered by their extensive model parameters, leading to a notable decline in generalization capabilities when these models are compressed for device deployment.
FedSODA: Federated Cross-assessment and Dynamic Aggregation for Histopathology Segmentation
Federated learning (FL) for histopathology image segmentation involving multiple medical sites plays a crucial role in advancing the field of accurate disease diagnosis and treatment.
DVIS++: Improved Decoupled Framework for Universal Video Segmentation
We present the \textbf{D}ecoupled \textbf{VI}deo \textbf{S}egmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video instance segmentation (VIS), video semantic segmentation (VSS), and video panoptic segmentation (VPS).
Ranked #1 on
Video Semantic Segmentation
on VSPW
Stable Segment Anything Model
Thus, our solution, termed Stable-SAM, offers several advantages: 1) improved SAM's segmentation stability across a wide range of prompt qualities, while 2) retaining SAM's powerful promptable segmentation efficiency and generality, with 3) minimal learnable parameters (0. 08 M) and fast adaptation (by 1 training epoch).
FreeKD: Knowledge Distillation via Semantic Frequency Prompt
(2) During the distillation period, a pixel-wise frequency mask is generated via Frequency Prompt, to localize those pixel of interests (PoIs) in various frequency bands.
MARRS: Multimodal Reference Resolution System
Successfully handling context is essential for any dialog understanding task.
Collaboration and Transition: Distilling Item Transitions into Multi-Query Self-Attention for Sequential Recommendation
Second, we develop a transition-aware embedding distillation module that distills global item-to-item transition patterns into item embeddings, which enables the model to memorize and leverage transitional signals and serves as a calibrator for collaborative signals.
Structural Controllability of Switched Continuous and Discrete Time Linear Systems
This paper explores the structural controllability of switched continuous and discrete time linear systems.
Label-free Deep Learning Driven Secure Access Selection in Space-Air-Ground Integrated Networks
Considering the inherent co-channel interference due to spectrum sharing among multi-tier access networks in SAGIN, it can be leveraged to assist the physical layer security among heterogeneous transmissions.
1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation
Video instance segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving.
Effectively Heterogeneous Federated Learning: A Pairing and Split Learning Based Approach
As a promising paradigm federated Learning (FL) is widely used in privacy-preserving machine learning, which allows distributed devices to collaboratively train a model while avoiding data transmission among clients.
Functional Observability, Structural Functional Observability and Optimal Sensor Placement
Based on these results, the problems of selecting the minimal sensors from a prior set to achieve functional observability and SFO are shown to be NP-hard.
Measuring Item Global Residual Value for Fair Recommendation
In this paper, we call for a shift of attention from modeling user preferences to developing fair exposure mechanisms for items.
Dynamic Snake Convolution based on Topological Geometric Constraints for Tubular Structure Segmentation
In this work, we note the specificity of tubular structures and use this knowledge to guide our DSCNet to simultaneously enhance perception in three stages: feature extraction, feature fusion, and loss constraint.
Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation
Through theoretical analyses, we find that the conservatism of existing methods fails in pursuing users' long-term satisfaction.
Distilling Missing Modality Knowledge from Ultrasound for Endometriosis Diagnosis with Magnetic Resonance Images
Next, we distill the knowledge from the teacher TVUS POD obliteration detector to train the student MRI model by minimizing a regression loss that approximates the output of the student to the teacher using unpaired TVUS and MRI data.
Razor SNN: Efficient Spiking Neural Network with Temporal Embeddings
The event streams generated by dynamic vision sensors (DVS) are sparse and non-uniform in the spatial domain, while still dense and redundant in the temporal domain.
1st Place Solution for PVUW Challenge 2023: Video Panoptic Segmentation
In this report, we successfully validated the effectiveness of the decoupling strategy in video panoptic segmentation.
DVIS: Decoupled Video Instance Segmentation Framework
The efficacy of the decoupling strategy relies on two crucial elements: 1) attaining precise long-term alignment outcomes via frame-by-frame association during tracking, and 2) the effective utilization of temporal information predicated on the aforementioned accurate alignment outcomes during refinement.
Ranked #3 on
Video Panoptic Segmentation
on VIPSeg
Knowledge Diffusion for Distillation
To address this, we propose to denoise student features using a diffusion model trained by teacher features.
Distribution-Free Matrix Prediction Under Arbitrary Missing Pattern
This paper studies the open problem of conformalized entry prediction in a row/column-exchangeable matrix.
Avatar Knowledge Distillation: Self-ensemble Teacher Paradigm with Uncertainty
Knowledge distillation is an effective paradigm for boosting the performance of pocket-size model, especially when multiple teacher models are available, the student would break the upper limit again.
Observability Blocking for Functional Privacy of Linear Dynamic Networks
This paper addresses the problem of determining the minimum set of state variables in a network that need to be blocked from direct measurements in order to protect functional privacy with respect to {\emph{any}} output matrices.
Judicial Intelligent Assistant System: Extracting Events from Divorce Cases to Detect Disputes for the Judge
It is a difficult but necessary task to extract the key information for the cases from these textual materials and to clarify the dispute focus of related parties.
Time-Variance Aware Real-Time Speech Enhancement
Time-variant factors often occur in real-world full-duplex communication applications.
On the Reachability and Controllability of Temporal Continuous-Time Linear Networks: A Generic Analysis
This paper investigates the reachability and controllability of temporal continuous-time linear networks from a generic viewpoint, where only the zero-nonzero patterns of subsystem matrices are known.
Real-time speech enhancement with dynamic attention span
For real-time speech enhancement (SE) including noise suppression, dereverberation and acoustic echo cancellation, the time-variance of the audio signals becomes a severe challenge.
Divide and Conquer: Towards Better Embedding-based Retrieval for Recommender Systems From a Multi-task Perspective
Embedding-based retrieval (EBR) methods are widely used in modern recommender systems thanks to its simplicity and effectiveness.
Incorporating Recurrent Reinforcement Learning into Model Predictive Control for Adaptive Control in Autonomous Driving
Model Predictive Control (MPC) is attracting tremendous attention in the autonomous driving task as a powerful control technique.
Capacity Analysis of Holographic MIMO Channels with Practical Constraints
However, the channel might not satisfy isotropic scattering because of generalized angle distributions, and the antenna gain is limited by the array aperture in reality.
Mask the Correct Tokens: An Embarrassingly Simple Approach for Error Correction
Since the error rate of the incorrect sentence is usually low (e. g., 10\%), the correction model can only learn to correct on limited error tokens but trivially copy on most tokens (correct tokens), which harms the effective training of error correction.
DAMO-YOLO : A Report on Real-Time Object Detection Design
Based on these new techs, we build a suite of models at various scales to meet the needs of different scenarios.
Ranked #33 on
Real-Time Object Detection
on MS COCO
Disentangled Feature Learning for Real-Time Neural Speech Coding
Recently end-to-end neural audio/speech coding has shown its great potential to outperform traditional signal analysis based audio codecs.
Differentially Private Optimizers Can Learn Adversarially Robust Models
Machine learning models have shone in a variety of domains and attracted increasing attention from both the security and the privacy communities.
FV2ES: A Fully End2End Multimodal System for Fast Yet Effective Video Emotion Recognition Inference
In the latest social networks, more and more people prefer to express their emotions in videos through text, speech, and rich facial expressions.
Real-time Short Video Recommendation on Mobile Devices
However, as users continue to watch videos and feedback, the changing context leads the ranking of the server-side recommendation system inaccurate.
KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos
To facilitate model learning, we further collect rich features of users and items as well as users' behavior history.
Higher-order accurate two-sample network inference and network hashing
Two-sample hypothesis testing for network comparison presents many significant challenges, including: leveraging repeated network observations and known node registration, but without requiring them to operate; relaxing strong structural assumptions; achieving finite-sample higher-order accuracy; handling different network sizes and sparsity levels; fast computation and memory parsimony; controlling false discovery rate (FDR) in multiple testing; and theoretical understandings, particularly regarding finite-sample accuracy and minimax optimality.
Latent-Domain Predictive Neural Speech Coding
Neural audio/speech coding has recently demonstrated its capability to deliver high quality at much lower bitrates than traditional methods.
Cross-Scale Vector Quantization for Scalable Neural Speech Coding
In this paper, we introduce a cross-scale scalable vector quantization scheme (CSVQ), in which multi-scale features are encoded progressively with stepwise feature fusion and refinement.
Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization
To deal with unknown uncertainty sets, we further propose a novel adversarial approach to generate them based on the value function.
Masked Distillation with Receptive Tokens
To obtain a group of masks, the receptive tokens are learned via the regular task loss but with teacher fixed, and we also leverage a Dice loss to enrich the diversity of learned masks.
WSSS4LUAD: Grand Challenge on Weakly-supervised Tissue Semantic Segmentation for Lung Adenocarcinoma
According to the technical reports of the top-tier teams, CAM is still the most popular approach in WSSS.
Data Augmentation
Weakly supervised Semantic Segmentation
+1
CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System
The basic idea is to first learn a causal user model on historical data to capture the overexposure effect of items on user satisfaction.
On Polynomially Solvable Constrained Input Selections for Fixed and Switched Linear Structured Systems
In this paper, it is found that, if the input structure satisfies certain `regularizations', which are characterized by the proposed restricted total unimodulairty notion, those problems can be solvable in polynomial time via linear programming (LP) relaxations.
LBCF: A Large-Scale Budget-Constrained Causal Forest Algorithm
The proposed approach is currently serving over hundreds of millions of users on the platform and achieves one of the most tremendous improvements over these months.
In Defense of Kalman Filtering for Polyp Tracking from Colonoscopy Videos
We advocate that the field should instead focus on the development of simple and efficient detectors that an be combined with effective trackers to allow the implementation of real-time polyp detectors.
End-to-End Neural Speech Coding for Real-Time Communications
Deep-learning based methods have shown their advantages in audio coding over traditional ones but limited attention has been paid on real-time communications (RTC).
On real structured controllability/stabilizability/stability radius: Complexity and unified rank-relaxation based methods
This paper addresses the real structured controllability, stabilizability, and stability radii (RSCR, RSSZR, and RSSR, respectively) of linear systems, which involve determining the distance (in terms of matrix norms) between a (possibly large-scale) system and its nearest uncontrollable, unstabilizable, and unstable systems, respectively, with a prescribed affine structure.
Learning Multi-granularity User Intent Unit for Session-based Recommendation
Specifically, we creatively propose Multi-granularity Intent Heterogeneous Session Graph which captures the interactions between different granularity intent units and relieves the burden of long-dependency.
MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Image
In this work, we propose a framework for single-view hand mesh reconstruction, which can simultaneously achieve high reconstruction accuracy, fast inference speed, and temporal coherence.
Ranked #7 on
3D Hand Pose Estimation
on DexYCB
Controllable Semantic Parsing via Retrieval Augmentation
In practical applications of semantic parsing, we often want to rapidly change the behavior of the parser, such as enabling it to handle queries in a new domain, or changing its predictions on certain targeted queries.
A free lunch from ViT:Adaptive Attention Multi-scale Fusion Transformer for Fine-grained Visual Recognition
Learning subtle representation about object parts plays a vital role in fine-grained visual recognition (FGVR) field.
Ranked #10 on
Fine-Grained Image Classification
on Stanford Dogs
Fine-Grained Image Classification
Fine-Grained Visual Recognition
Practical and Private Heterogeneous Federated Learning
Heterogeneous federated learning (HFL) enables clients with different computation/communication capabilities to collaboratively train their own customized models, in which the knowledge of models is shared via clients' predictions on a public dataset.
AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance
While previous pruning methods have mostly focused on identifying unimportant channels, channel pruning is considered as a special case of neural architecture search in recent years.
r-GAT: Relational Graph Attention Network for Multi-Relational Graphs
To tackle this issue, we propose r-GAT, a relational graph attention network to learn multi-channel entity representations.
PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition
In this paper, we propose a Parallel, Iterative and Mimicking Network (PIMNet) to balance accuracy and efficiency.
Total Unimodularity and Strongly Polynomial Solvability of Constrained Minimum Input Selections for Structural Controllability: an LP-based Method
Subject to a certain condition on the prescribed input configurations that contains the dedicated input one as a special case, we demonstrate that the constraint matrices of these ILPs are totally unimodular.
Distilling Neuron Spike with High Temperature in Reinforcement Learning Agents
This method uses distillation to effectively avoid the weakness of STBP, which can achieve SOTA performance in classification, and can obtain a smaller, faster convergence and lower power consumption SNN reinforcement learning model.
QA-Driven Zero-shot Slot Filling with Weak Supervision Pretraining
To address this problem, we introduce QA-driven slot filling (QASF), which extracts slot-filler spans from utterances with a span-based QA model.
Latent Space Model for Higher-order Networks and Generalized Tensor Decomposition
We formulate the relationship between the latent positions and the observed data via a generalized multilinear kernel as the link function.
Partial Strong Structural Controllability
This paper introduces a new controllability notion, termed partial strong structural controllability (PSSC), on a structured system whose entries of system matrices are either fixed zero or indeterminate, which naturally extends the conventional strong structural controllability (SSC) and bridges the gap between structural controllability and SSC.
SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning
This paper studies a theoretical framework for value factorisation with interpretability via Shapley value theory.
Recent Standard Development Activities on Video Coding for Machines
Specifically, we will first provide an overview of the MPEG VCM group including use cases, requirements, processing pipelines, plan for potential VCM standards, followed by the evaluation framework including machine-vision tasks, dataset, evaluation metrics, and anchor generation.
Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations
Sequence-to-sequence (seq2seq) models are prevalent in semantic parsing, but have been found to struggle at out-of-distribution compositional generalization.
Ranked #3 on
Semantic Parsing
on CFQ
Few-shot Intent Classification and Slot Filling with Retrieved Examples
Few-shot learning arises in important practical scenarios, such as when a natural language understanding system needs to learn new semantic labels for an emerging, resource-scarce domain.
ProDCoNN-server: a web server for protein sequence prediction and design from a three-dimensional structure
The prediction is very fast compared with other protein sequence prediction servers - it takes only a few minutes for a query protein on average.
PTSC: a New Definition for Structural Controllability under Numerical Perturbations
This paper proposes a novel notion for structural controllability under structured numerical perturbations, namely the perturbation-tolerant structural controllability (PTSC), on a single-input structured system whose entries can be classified into three categories: fixed zero entries, unknown generic entries whose values are fixed but unknown, and perturbed entries that can take arbitrary complex values.
Interactive Speech and Noise Modeling for Speech Enhancement
In this paper, we propose a novel idea to model speech and noise simultaneously in a two-branch convolutional neural network, namely SN-Net.
Ranked #1 on
Speech Enhancement
on Deep Noise Suppression (DNS) Challenge
(SI-SDR-NB metric)
End-to-End Chinese Parsing Exploiting Lexicons
Chinese parsing has traditionally been solved by three pipeline systems including word-segmentation, part-of-speech tagging and dependency parsing modules.
Cross-Layer Distillation with Semantic Calibration
Knowledge distillation is a technique to enhance the generalization ability of a student model by exploiting outputs from a teacher model.
Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective
Based on our theoretical and empirical analysis, we establish a universal theoretical framework of GCN from an optimization perspective and derive a novel convolutional kernel named GCN+ which has lower parameter amount while relieving the over-smoothing inherently.
Improving Monocular Depth Estimation by Leveraging Structural Awareness and Complementary Datasets
Monocular depth estimation plays a crucial role in 3D recognition and understanding.
LIT Team's System Description for Japanese-Chinese Machine Translation Task in IWSLT 2020
This paper describes the LIT Team{'}s submission to the IWSLT2020 open domain translation task, focusing primarily on Japanese-to-Chinese translation direction.
Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System
We test HDNO on MultiWoz 2. 0 and MultiWoz 2. 1, the datasets on multi-domain dialogues, in comparison with word-level E2E model trained with RL, LaRL and HDSA, showing improvements on the performance evaluated by automatic evaluation metrics and human evaluation.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
Noise Robust Named Entity Understanding for Voice Assistants
Named Entity Recognition (NER) and Entity Linking (EL) play an essential role in voice assistant interaction, but are challenging due to the special difficulties associated with spoken user queries.
Mapping Natural Language Instructions to Mobile UI Action Sequences
We present a new problem: grounding natural language instructions to mobile user interface actions, and create three new datasets for it.
Edgeworth expansions for network moments
In this paper, we present the first higher-order accurate approximation to the sampling CDF of a studentized network moment by Edgeworth expansion.
FocalMix: Semi-Supervised Learning for 3D Medical Image Detection
Applying artificial intelligence techniques in medical imaging is one of the most promising areas in medicine.
Neural Inheritance Relation Guided One-Shot Layer Assignment Search
In this paper, for the first time, we systematically investigate the impact of different layer assignments to the network performance by building an architecture dataset of layer assignment on CIFAR-100.
Distilling Structured Knowledge into Embeddings for Explainable and Accurate Recommendation
Recently, the embedding-based recommendation models (e. g., matrix factorization and deep models) have been prevalent in both academia and industry due to their effectiveness and flexibility.
Large-scale representation learning from visually grounded untranscribed speech
Systems that can associate images with their spoken audio captions are an important step towards visually grounded language learning.
PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification
Most existing work on adversarial data generation focuses on English.
Shapley Q-value: A Local Reward Approach to Solve Global Reward Games
To deal with this problem, we i) introduce a cooperative-game theoretical framework called extended convex game (ECG) that is a superset of global reward game, and ii) propose a local reward approach called Shapley Q-value.
Tree Communication Models for Sentiment Analysis
Tree-LSTMs have been used for tree-based sentiment analysis over Stanford Sentiment Treebank, which allows the sentiment signals over hierarchical phrase structures to be calculated simultaneously.
PAWS: Paraphrase Adversaries from Word Scrambling
Existing paraphrase identification datasets lack sentence pairs that have high lexical overlap without being paraphrases.
All You Need is a Few Shifts: Designing Efficient Convolutional Neural Networks for Image Classification
In this family of architectures, the basic block is only composed by 1x1 convolutional layers with only a few shift operations applied to the intermediate feature maps.
Neural IR Meets Graph Embedding: A Ranking Model for Product Search
Recently, neural models for information retrieval are becoming increasingly popular.
A Layer Decomposition-Recomposition Framework for Neuron Pruning towards Accurate Lightweight Networks
A better alternative is to propagate the entire useful information to reconstruct the pruned layer instead of directly discarding the less important neurons.
Modeling Attention Flow on Graphs
Real-world scenarios demand reasoning about process, more than final outcome prediction, to discover latent causal chains and better understand complex systems.
A Fast, Compact, Accurate Model for Language Identification of Codemixed Text
We address fine-grained multilingual language identification: providing a language code for every token in a sentence, including codemixed text containing multiple languages.
Consistent polynomial-time unseeded graph matching for Lipschitz graphons
We propose a consistent polynomial-time method for the unseeded node matching problem for networks with smooth underlying structures.
Backprop-Q: Generalized Backpropagation for Stochastic Computation Graphs
Then, the SCG can be trained based on these surrogate costs using standard backpropagation.
A Localization Method Avoiding Flip Ambiguities for micro-UAVs with Bounded Distance Measurement Errors
This study proposes a localization method that can avoid the occurrence of flip ambiguities in bounded distance measurement errors and constrained flying motions; to demonstrate its efficacy, the method is implemented on bilateration and trilateration.
Signal Processing Other Computer Science
Blind Predicting Similar Quality Map for Image Quality Assessment
A key problem in blind image quality assessment (BIQA) is how to effectively model the properties of human visual system in a data-driven manner.
Aspect-augmented Adversarial Networks for Domain Adaptation
We introduce a neural method for transfer learning between two (source and target) classification tasks or aspects over the same domain.
Stack-propagation: Improved Representation Learning for Syntax
Traditional syntax models typically leverage part-of-speech (POS) information by constructing features from hand-tuned templates.
Estimating network edge probabilities by neighborhood smoothing
The estimation of probabilities of network edges from the observed adjacency matrix has important applications to predicting missing links and network denoising.
Community Detection in Networks with Node Features
Many methods have been proposed for community detection in networks, but most of them do not take into account additional information on the nodes that is often available in practice.
Detecting Overlapping Communities in Networks Using Spectral Methods
Community detection is a fundamental problem in network analysis which is made more challenging by overlaps between communities which often occur in practice.
