Search Results for author:
Found 75 papers, 34 papers with code
Learning Progressive Joint Propagation for Human Motion Prediction
Last, in order to incorporate a general motion space for high-quality prediction, we build a memory-based dictionary, which aims to preserve the global motion patterns in training data to guide the predictions.
FedMPQ: Secure and Communication-Efficient Federated Learning with Multi-codebook Product Quantization
In federated learning, particularly in cross-device scenarios, secure aggregation has recently gained popularity as it effectively defends against inference attacks by malicious aggregators.
RD2Bench: Toward Data-Centric Automatic R&D
The progress of humanity is driven by those successful discoveries accompanied by countless failed experiments.
Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On
Second, we propose a novel diffusion-based method that predicts a precise inpainting mask based on the person and reference garment images, further enhancing the reliability of the try-on results.
DA-PFL: Dynamic Affinity Aggregation for Personalized Federated Learning
In this paper, we propose a novel Dynamic Affinity-based Personalized Federated Learning model (DA-PFL) to alleviate the class imbalanced problem during federated learning.
FedHCDR: Federated Cross-Domain Recommendation with Hypergraph Signal Decoupling
Specifically, to address the data heterogeneity across domains, we introduce an approach called hypergraph signal decoupling (HSD) to decouple the user features into domain-exclusive and domain-shared features.
MemoNav: Working Memory Model for Visual Navigation
Subsequently, a graph attention module encodes the retained STM and the LTM to generate working memory (WM) which contains the scene features essential for efficient navigation.
Analysis of the Two-Step Heterogeneous Transfer Learning for Laryngeal Blood Vessel Classification: Issue and Improvement
Accurate classification of laryngeal vascular as benign or malignant is crucial for early detection of laryngeal cancer.
A Lightweight Inception Boosted U-Net Neural Network for Routability Prediction
As the modern CPU, GPU, and NPU chip design complexity and transistor counts keep increasing, and with the relentless shrinking of semiconductor technology nodes to nearly 1 nanometer, the placement and routing have gradually become the two most pivotal processes in modern very-large-scale-integrated (VLSI) circuit back-end design.
Do You Guys Want to Dance: Zero-Shot Compositional Human Dance Generation with Multiple Persons
Human dance generation (HDG) aims to synthesize realistic videos from images and sequences of driving poses.
ICD-LM: Configuring Vision-Language In-Context Demonstrations by Language Modeling
Moreover, during data construction, we use the LVLM intended for ICL implementation to validate the strength of each ICD sequence, resulting in a model-specific dataset and the ICD-LM trained by this dataset is also model-specific.
Building Variable-sized Models via Learngene Pool
To overcome these challenges, motivated by the recently proposed Learngene framework, we propose a novel method called Learngene Pool.
Transformer as Linear Expansion of Learngene
Under the situation where we need to produce models of varying depths adapting for different resource constraints, TLEG achieves comparable results while reducing around 19x parameters stored to initialize these models and around 5x pre-training costs, in contrast to the pre-training and fine-tuning approach.
How to Configure Good In-Context Sequence for Visual Question Answering
Inspired by the success of Large Language Models in dealing with new tasks via In-Context Learning (ICL) in NLP, researchers have also developed Large Vision-Language Models (LVLMs) with ICL capabilities.
Manipulating the Label Space for In-Context Classification
Specifically, using our approach on ImageNet, we increase accuracy from 74. 70\% in a 4-shot setting to 76. 21\% with just 2 shots.
Category-Wise Fine-Tuning for Image Multi-label Classification with Partial Labels
A single model submitted to the competition server for the official evaluation achieves mAUC 91. 82% on the test set, which is the highest single model score in the leaderboard and literature.
Ranked #1 on
Multi-Label Classification
on CheXpert
ChartLlama: A Multimodal LLM for Chart Understanding and Generation
Next, we introduce ChartLlama, a multi-modal large language model that we've trained using our created dataset.
Rethinking Residual Connection in Training Large-Scale Spiking Neural Networks
Spiking Neural Network (SNN) is known as the most famous brain-inspired model, but the non-differentiable spiking mechanism makes it hard to train large-scale SNNs.
Rethinking Evaluation Metrics of Open-Vocabulary Segmentaion
We benchmarked the proposed evaluation metrics on 12 open-vocabulary methods of three segmentation tasks.
Leveraging Large Language Model for Automatic Evolving of Industrial Data-Centric R&D Cycle
In the wake of relentless digital transformation, data-driven solutions are emerging as powerful tools to address multifarious industrial tasks such as forecasting, anomaly detection, planning, and even complex decision-making.
SeisT: A foundational deep learning model for earthquake monitoring tasks
The most significant improvements, in comparison to existing models, are observed in phase-P picking, phase-S picking, and magnitude estimation, with gains of 1. 7%, 9. 5%, and 8. 0%, respectively.
FedDCSR: Federated Cross-domain Sequential Recommendation via Disentangled Representation Learning
Nonetheless, the sequence feature heterogeneity across different domains significantly impacts the overall performance of FL.
Temporal Difference Learning for High-Dimensional PIDEs with Jumps
In this paper, we propose a deep learning framework for solving high-dimensional partial integro-differential equations (PIDEs) based on the temporal difference learning.
Genes in Intelligent Agents
Inspired by the biological intelligence, artificial intelligence (AI) has devoted to building the machine intelligence.
Exploring Diverse In-Context Configurations for Image Captioning
After discovering that Language Models (LMs) can be good in-context few-shot learners, numerous strategies have been proposed to optimize in-context sequence configurations.
Transforming Visual Scene Graphs to Image Captions
In TSG, we apply multi-head attention (MHA) to design the Graph Neural Network (GNN) for embedding scene graphs.
Learngene: Inheriting Condensed Knowledge from the Ancestry Model to Descendant Models
(i) Accumulating: the knowledge is accumulated during the continuous learning of an ancestry model.
SC-ML: Self-supervised Counterfactual Metric Learning for Debiased Visual Question Answering
Unfortunately, language bias is a common problem in VQA, which refers to the model generating answers only by associating with the questions while ignoring the visual content, resulting in biased results.
Spatial Attention and Syntax Rule Enhanced Tree Decoder for Offine Handwritten Mathematical Expression Recognition
In this paper, we propose a novel model called Spatial Attention and Syntax Rule Enhanced Tree Decoder (SS-TD), which is equipped with spatial attention mechanism to alleviate the prediction error of tree structure and use syntax masks (obtained from the transformation of syntax rules) to constrain the occurrence of ungrammatical mathematical expression.
Learning Trajectory-Word Alignments for Video-Language Tasks
To amend this, we propose a novel TW-BERT to learn Trajectory-Word alignment by a newly designed trajectory-to-word (T2W) attention for solving video-language tasks.
Adaptively Clustering Neighbor Elements for Image Captioning
We design a novel global-local Transformer named \textbf{Ada-ClustFormer} (\textbf{ACF}) to generate captions.
Spikeformer: A Novel Architecture for Training High-Performance Low-Latency Spiking Neural Network
Spiking neural networks (SNNs) have made great progress on both performance and efficiency over the last few years, but their unique working pattern makes it hard to train a high-performance low-latency SNN. Thus the development of SNNs still lags behind traditional artificial neural networks (ANNs). To compensate this gap, many extraordinary works have been proposed. Nevertheless, these works are mainly based on the same kind of network structure (i. e. CNN) and their performance is worse than their ANN counterparts, which limits the applications of SNNs. To this end, we propose a novel Transformer-based SNN, termed "Spikeformer", which outperforms its ANN counterpart on both static dataset and neuromorphic dataset and may be an alternative architecture to CNN for training high-performance SNNs. First, to deal with the problem of "data hungry" and the unstable training period exhibited in the vanilla model, we design the Convolutional Tokenizer (CT) module, which improves the accuracy of the original model on DVS-Gesture by more than 16%. Besides, in order to better incorporate the attention mechanism inside Transformer and the spatio-temporal information inherent to SNN, we adopt spatio-temporal attention (STA) instead of spatial-wise or temporal-wise attention. With our proposed method, we achieve competitive or state-of-the-art (SOTA) SNN performance on DVS-CIFAR10, DVS-Gesture, and ImageNet datasets with the least simulation time steps (i. e. low latency). Remarkably, our Spikeformer outperforms other SNNs on ImageNet by a large margin (i. e. more than 5%) and even outperforms its ANN counterpart by 3. 1% and 2. 2% on DVS-Gesture and ImageNet respectively, indicating that Spikeformer is a promising architecture for training large-scale SNNs and may be more suitable for SNNs compared to CNN. We believe that this work shall keep the development of SNNs in step with ANNs as much as possible. Code will be available.
Image Projective Transformation Rectification with Synthetic Data for Smartphone-captured Chest X-ray Photos Classification
To the best of our knowledge, it is the first work to predict the projective transformation matrix as the learning goal for photo rectification.
Ranked #1 on
Medical Image Classification
on CheXphoto
Learning to Collocate Visual-Linguistic Neural Modules for Image Captioning
This is because the language is only partially observable, for which we need to dynamically collocate the modules during the process of image captioning.
MemoNav: Selecting Informative Memories for Visual Navigation
To address this limitation, we present the MemoNav, a novel memory mechanism for image-goal navigation, which retains the agent's informative short-term memory and long-term memory to improve the navigation performance on a multi-goal task.
Automatically Discovering Novel Visual Categories with Self-supervised Prototype Learning
In the first stage, we obtain a robust feature extractor, which could serve for all images with base and novel categories.
Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning
Some methods recognize state and object with two trained classifiers, ignoring the impact of the interaction between object and state; the other methods try to learn the joint representation of the state-object compositions, leading to the domain gap between seen and unseen composition sets.
iExam: A Novel Online Exam Monitoring and Analysis System Based on Face Detection and Recognition
In this paper, we propose iExam, an intelligent online exam monitoring and analysis system that can not only use face detection to assist invigilators in real-time student identification, but also be able to detect common abnormal behaviors (including face disappearing, rotating faces, and replacing with a different person during the exams) via a face recognition-based post-exam video analysis.
Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptation
In this paper, we emphasize the adaptation process across sim2real domains and model it as a learning problem on the BatchNorm parameters of a simulation-trained model.
Weakly Aligned Feature Fusion for Multimodal Object Detection
In this article, we propose a general multimodal detector named aligned region CNN (AR-CNN) to tackle the position shift problem.
Not Just Selection, but Exploration: Online Class-Incremental Continual Learning via Dual View Consistency
Unfortunately, these methods only focus on selecting samples from the memory bank for replay and ignore the adequate exploration of semantic information in the single-pass data stream, leading to poor classification accuracy.
Show, Deconfound and Tell: Image Captioning With Causal Inference
In the encoding stage, the IOD is able to disentangle the region-based visual features by deconfounding the visual confounder.
Towards End-to-End Image Compression and Analysis with Transformers
Meanwhile, we propose a feature aggregation module to fuse the compressed features with the selected intermediate features of the Transformer, and feed the aggregated features to a deconvolutional neural network for image reconstruction.
Auto-Encoding Score Distribution Regression for Action Quality Assessment
Traditionally, AQA is treated as a regression problem to learn the underlying mappings between videos and action scores.
Ranked #1 on
Action Quality Assessment
on JIGSAWS
EMScore: Evaluating Video Captioning via Coarse-Grained and Fine-Grained Embedding Matching
The datasets will be released to facilitate the development of video captioning metrics.
Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for Trajectory Prediction
First, with the continuation of time, the prediction error at each time step increases significantly, causing the final displacement error to be impossible to ignore.
Can AI detect pain and express pain empathy? A review from emotion recognition and a human-centered AI perspective
Furthermore, we discuss challenges for responsible evaluation of cognitive methods and computational techniques and show approaches to future work to contribute to affective assistants capable of empathy.
Text-Driven Image Manipulation via Semantic-Aware Knowledge Transfer
And on this basis, a novel attribute transfer method, named semantic directional decomposition network (SDD-Net), is proposed to achieve semantic-level facial attribute transfer by latent semantic direction decomposition, improving the interpretability and editability of our method.
Open Set Domain Adaptation with Zero-shot Learning on Graph
Thus the classification ability of the source domain is transferred to the target domain and the model can distinguish the unknown classes with prior knowledge.
Auto-Parsing Network for Image Captioning and Visual Question Answering
We propose an Auto-Parsing Network (APN) to discover and exploit the input data's hidden tree structures for improving the effectiveness of the Transformer-based vision-language systems.
Towards Unbiased Visual Emotion Recognition via Causal Intervention
In this work, we scrutinize this problem from the perspective of causal inference, where such dataset characteristic is termed as a confounder which misleads the system to learn the spurious correlation.
Nearest Neighbor Matching for Deep Clustering
Specifically, for the local level, we match the nearest neighbors based on batch embedded features, as for the global one, we match neighbors from overall embedded features.
SelfSAGCN: Self-Supervised Semantic Alignment for Graph Convolution Network
Specifically, the Identity Aggregation is applied to extract semantic features from labeled nodes, the Semantic Alignment is utilized to align node features obtained from different aspects using the class central similarity.
Doubly Contrastive Deep Clustering
In this paper, we present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views to obtain more discriminative features and competitive results.
Causal Attention for Vision-Language Tasks
Specifically, CATT is implemented as a combination of 1) In-Sample Attention (IS-ATT) and 2) Cross-Sample Attention (CS-ATT), where the latter forcibly brings other samples into every IS-ATT, mimicking the causal intervention.
A Distributed Implementation of Steady-State Kalman Filter
This paper studies the distributed state estimation in sensor network, where $m$ sensors are deployed to infer the $n$-dimensional state of a linear time-invariant (LTI) Gaussian system.
Incremental Embedding Learning via Zero-Shot Translation
Different from traditional incremental classification networks, the semantic gap between the embedding spaces of two adjacent tasks is the main challenge for embedding networks under incremental learning setting.
Adversarial Learning for Robust Deep Clustering
Meanwhile, we devise an adversarial attack strategy to explore samples that easily fool the clustering layers but do not impact the performance of the deep embedding.
Cloud Cover and Aurora Contamination at Dome A in 2017 from KLCAM
Dome A in Antarctica has many characteristics that make it an excellent site for astronomical observations, from the optical to the terahertz.
Instrumentation and Methods for Astrophysics
Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioning
In this paper, we propose a novel visual encoder to explicitly distinguish viewpoint changes from semantic changes in the change captioning task.
Learning to Scan: A Deep Reinforcement Learning Approach for Personalized Scanning in CT Imaging
It is more desirable to design a personalized scanning strategy for each subject to obtain better reconstruction result.
Deconfounded Image Captioning: A Causal Retrospect
Dataset bias in vision-language tasks is becoming one of the main problems which hinders the progress of our community.
Classical limit for the varying-mass Schrödinger equation with random inhomogeneities
The varying-mass Schr\"odinger equation (VMSE) has been successfully applied to model electronic properties of semiconductor hetero-structures, for example, quantum dots and quantum wells.
Numerical Analysis Numerical Analysis
Automated Pavement Crack Segmentation Using U-Net-based Convolutional Neural Network
In this paper, we propose a deep learning technique based on a convolutional neural network to perform segmentation tasks on pavement crack images.
TBC-Net: A real-time detector for infrared small target detection using semantic constraint
Meanwhile, we propose a joint loss function and a training method.
mu-Forcing: Training Variational Recurrent Autoencoders for Text Generation
It has been previously observed that training Variational Recurrent Autoencoders (VRAE) for text generation suffers from serious uninformative latent variables problem.
Deep Spectral Clustering using Dual Autoencoder Network
In this paper, we propose a joint learning framework for discriminative embedding and spectral clustering.
Learning to Collocate Neural Modules for Image Captioning
To this end, we make the following technical contributions for CNM training: 1) compact module design --- one for function words and three for visual content words (eg, noun, adjective, and verb), 2) soft module fusion and multi-step module execution, robustifying the visual reasoning in partial observation, 3) a linguistic loss for module controller being faithful to part-of-speech collocations (eg, adjective is before noun).
Unpaired Image Captioning via Scene Graph Alignments
Most of current image captioning models heavily rely on paired image-caption datasets.
Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection
In this paper, we propose a novel Aligned Region CNN (AR-CNN) to handle the weakly aligned multispectral data in an end-to-end way.
Auto-Encoding Scene Graphs for Image Captioning
We propose Scene Graph Auto-Encoder (SGAE) that incorporates the language inductive bias into the encoder-decoder image captioning framework for more human-like captions.
Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
By "agnostic", we mean that the feature is less likely biased to the classes of paired objects.
Face Photo Sketch Synthesis via Larger Patch and Multiresolution Spline
In order to get a smoother sketch, we propose a new method to reduce such jagged parts and mottled points.
A Weighted Common Subgraph Matching Algorithm
We propose a weighted common subgraph (WCS) matching algorithm to find the most similar subgraphs in two labeled weighted graphs.
