Search Results for author:
Found 400 papers, 176 papers with code
Learning to See in the Dark
Imaging in low light is challenging due to low photon count and low SNR.
Ranked #4 on
Image Denoising
on SID x300
Seeing Motion in the Dark
By carefully designing a learning-based pipeline and introducing a new loss function to encourage temporal stability, we train a siamese network on static raw videos, for which ground truth is available, such that the network generalizes to videos of dynamic scenes at test time.
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities
The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework.
Semantic Image Inpainting with Deep Generative Models
In this paper, we propose a novel method for semantic image inpainting, which generates the missing content by conditioning on the available data.
Less is More: Removing Text-regions Improves CLIP Training Efficiency and Robustness
In this paper, we discuss two effective approaches to improve the efficiency and robustness of CLIP training: (1) augmenting the training dataset while maintaining the same number of optimization steps, and (2) filtering out samples that contain text regions in the image.
VEGA: Towards an End-to-End Configurable AutoML Pipeline
Automated Machine Learning (AutoML) is an important industrial solution for automatic discovery and deployment of the machine learning models.
Real-world Anomaly Detection in Surveillance Videos
To avoid annotating the anomalous segments or clips in training videos, which is very time consuming, we propose to learn anomaly through the deep multiple instance ranking framework by leveraging weakly labeled training videos, i. e. the training labels (anomalous or normal) are at video-level instead of clip-level.
Ranked #2 on
Abnormal Event Detection In Video
on UBI-Fights
Activity Recognition
Anomaly Detection In Surveillance Videos
+2
3D Human Pose Estimation with Spatial and Temporal Transformers
Transformer architectures have become the model of choice in natural language processing and are now being introduced into computer vision tasks such as image classification, object detection, and semantic segmentation.
Ranked #13 on
Monocular 3D Human Pose Estimation
on Human3.6M
Deep Learning-Based Human Pose Estimation: A Survey
Furthermore, 2D and 3D human pose estimation datasets and evaluation metrics are included.
PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation
However, in real scenarios, the performance of PoseFormer and its follow-ups is limited by two factors: (a) The length of the input joint sequence; (b) The quality of 2D joint detection.
Ranked #8 on
3D Human Pose Estimation
on MPI-INF-3DHP
TransBTS: Multimodal Brain Tumor Segmentation Using Transformer
To capture the local 3D context information, the encoder first utilizes 3D CNN to extract the volumetric spatial feature maps.
TransBTSV2: Towards Better and More Efficient Volumetric Segmentation of Medical Images
Different from TransBTS, the proposed TransBTSV2 is not limited to brain tumor segmentation (BTS) but focuses on general medical image segmentation, providing a stronger and more efficient 3D baseline for volumetric segmentation of medical images.
Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation
Few-shot semantic segmentation (FSS) has great potential for medical imaging applications.
RenderIH: A Large-scale Synthetic Dataset for 3D Interacting Hand Pose Estimation
The current interacting hand (IH) datasets are relatively simplistic in terms of background and texture, with hand joints being annotated by a machine annotator, which may result in inaccuracies, and the diversity of pose distribution is limited.
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks
Conventional task- and modality-specific artificial intelligence (AI) models are inflexible in real-world deployment and maintenance for biomedicine.
Ranked #1 on
Text Summarization
on MeQSum
GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generation
Recent breakthroughs in the field of language-guided image generation have yielded impressive achievements, enabling the creation of high-quality and diverse images based on user instructions. Although the synthesis performance is fascinating, one significant limitation of current image generation models is their insufficient ability to generate text coherently within images, particularly for complex glyph structures like Chinese characters.
Optical Character Recognition (OCR)
Text-to-Image Generation
Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning
In this paper, we propose Subject-Diffusion, a novel open-domain personalized image generation model that, in addition to not requiring test-time fine-tuning, also only requires a single reference image to support personalized generation of single- or multi-subject in any domain.
Diffusion Personalization Tuning Free
Text-to-Image Generation
AIM: Adapting Image Models for Efficient Video Action Recognition
Recent vision transformer based video models mostly follow the ``image pre-training then finetuning" paradigm and have achieved great success on multiple video benchmarks.
Ranked #2 on
Action Recognition
on Diving-48
(using extra training data)
Lightweight image super-resolution with enhanced CNN
Finally, the IRB uses coarse high-frequency features from the RB to learn more accurate SR features and construct a SR image.
Ranked #45 on
Image Super-Resolution
on Set14 - 4x upscaling
3D Dilated Multi-Fiber Network for Real-time Brain Tumor Segmentation in MRI
In this work, we aim to segment brain MRI volumes.
MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and Resolution
We propose the width-resolution mutual learning method (MutualNet) to train a network that is executable at dynamic resource constraints to achieve adaptive accuracy-efficiency trade-offs at runtime.
MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurations
MutualNet is a general training methodology that can be applied to various network structures (e. g., 2D networks: MobileNets, ResNet, 3D networks: SlowFast, X3D) and various tasks (e. g., image classification, object detection, segmentation, and action recognition), and is demonstrated to achieve consistent improvements on a variety of datasets.
ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback
To this end, we propose ControlNet++, a novel approach that improves controllable generation by explicitly optimizing pixel-level cycle consistency between generated images and conditional controls.
Multi-Scale Progressive Fusion Network for Single Image Deraining
In this work, we explore the multi-scale collaborative representation for rain streaks from the perspective of input image scales and hierarchical deep features in a unified framework, termed multi-scale progressive fusion network (MSPFN) for single image rain streak removal.
Ranked #6 on
Single Image Deraining
on Test2800
Enhanced 3D Human Pose Estimation from Videos by using Attention-Based Neural Network with Dilated Convolutions
In this work, we show a systematic design (from 2D to 3D) for how conventional networks and other forms of constraints can be incorporated into the attention framework for learning long-range dependencies for the task of pose estimation.
Ranked #6 on
3D Human Pose Estimation
on HumanEva-I
GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
Leveraging the rich linguistic knowledge and strong reasoning abilities of LLMs, our new paradigm can integrate the rich information in N-best candidates to generate a higher-quality translation result.
AlignDet: Aligning Pre-training and Fine-tuning in Object Detection
To this end, we propose AlignDet, a unified pre-training framework that can be adapted to various existing detectors to alleviate the discrepancies.
Learning Normal Dynamics in Videos with Meta Prototype Network
Frame reconstruction (current or future frame) based on Auto-Encoder (AE) is a popular method for video anomaly detection.
Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition
Technically, given the character segmentation maps predicted by a VR model, we construct a subgraph for each instance, where nodes represent the pixels in it and edges are added between nodes based on their spatial similarity.
Ranked #9 on
Scene Text Recognition
on ICDAR2015
(using extra training data)
Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information
In this paper, we propose an end-to-end speech emotion recognition system using multi-level acoustic information with a newly designed co-attention module.
TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization
It does not rely on polar transform and infers faster than CNN-based methods.
Ranked #3 on
Image-Based Localization
on VIGOR Cross Area
GradAug: A New Regularization Method for Deep Neural Networks
The key idea is utilizing randomly transformed training samples to regularize a set of sub-networks, which are originated by sampling the width of the original network, in the training process.
Density Map Guided Object Detection in Aerial Images
Specifically, we propose a Density-Map guided object detection Network (DMNet), which is inspired from the observation that the object density map of an image presents how objects distribute in terms of the pixel intensity of the map.
SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection
We observe that the prevailing set abstraction design for down-sampling points may maintain too much unimportant background information that can affect feature learning for detecting objects.
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
To this end, we propose to extract a language-space noise embedding from the N-best list to represent the noise conditions of source speech, which can promote the denoising process in GER.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Visual Explanation for Deep Metric Learning
This work explores the visual explanation for deep metric learning and its applications.
Causality-inspired Single-source Domain Generalization for Medical Image Segmentation
In this work, we investigate the single-source domain generalization problem: training a deep network that is robust to unseen domains, under the condition that training data is only available from one source domain, which is common in medical imaging applications.
Noise-aware Speech Enhancement using Diffusion Probabilistic Model
Specifically, we design a noise classification (NC) model to produce acoustic embedding as a noise conditioner for guiding the reverse denoising process.
Consistency-based Active Learning for Object Detection
Unlike most recent works that focused on applying active learning for image classification, we propose an effective Consistency-based Active Learning method for object Detection (CALD), which fully explores the consistency between original and augmented data.
The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions
Multi-agent behavior modeling aims to understand the interactions that occur between agents.
SoccerNet 2023 Challenges Results
More information on the tasks, challenges, and leaderboards are available on https://www. soccer-net. org.
SoccerNet 2022 Challenges Results
The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team.
Unsupervised Anomaly Detection in Medical Images Using Masked Diffusion Model
It can be challenging to identify brain MRI anomalies using supervised deep-learning techniques due to anatomical heterogeneity and the requirement for pixel-level labeling.
VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval
In this paper, we redefine this problem with a more realistic assumption that the query image can be arbitrary in the area of interest and the reference images are captured before the queries emerge.
Stock price prediction using Generative Adversarial Networks
In this paper, it proposes a stock prediction model using Generative Adversarial Network (GAN) with Gated Recurrent Units (GRU) used as a generator that inputs historical stock price and generates future stock price and Convolutional Neural Network (CNN) as a discriminator to discriminate between the real stock price and generated stock price.
LatentEditor: Text Driven Local Editing of 3D Scenes
Our approach achieves faster editing speeds and superior output quality compared to existing 3D editing models, bridging the gap between textual instructions and high-quality 3D scene editing in latent space.
Geometric Transformers for Protein Interface Contact Prediction
Computational methods for predicting the interface contacts between proteins come highly sought after for drug discovery as they can significantly advance the accuracy of alternative approaches, such as protein-protein docking, protein function analysis tools, and other computational methods for protein bioinformatics.
MSINet: Twins Contrastive Search of Multi-Scale Interaction for Object ReID
Neural Architecture Search (NAS) has been increasingly appealing to the society of object Re-Identification (ReID), for that task-specific architectures significantly improve the retrieval performance.
Ranked #8 on
Vehicle Re-Identification
on VehicleID Large
SuperPCA: A Superpixelwise PCA Approach for Unsupervised Feature Extraction of Hyperspectral Imagery
(1) Unlike the traditional PCA method based on a whole image, SuperPCA takes into account the diversity in different homogeneous regions, that is, different regions should have different projections.
Realistic Adversarial Data Augmentation for MR Image Segmentation
In this work, we propose an adversarial data augmentation method for training neural networks for medical image segmentation.
Enhancing MR Image Segmentation with Realistic Adversarial Data Augmentation
The success of neural networks on medical image segmentation tasks typically relies on large labeled datasets for model training.
MOFI: Learning Image Representations from Noisy Entity Annotated Images
Our approach involves employing a named entity recognition model to extract entities from the alt-text, and then using a CLIP model to select the correct entities as labels of the paired image.
Fairness in Graph Mining: A Survey
Recently, algorithmic fairness has been extensively studied in graph-based applications.
Towards Resolving the Challenge of Long-tail Distribution in UAV Images for Object Detection
To this end, we rethink long-tailed object detection in UAV images and propose the Dual Sampler and Head detection Network (DSHNet), which is the first work that aims to resolve long-tail distribution in UAV images.
Towards Geospatial Foundation Models via Continual Pretraining
Geospatial technologies are becoming increasingly essential in our world for a wide range of applications, including agriculture, urban planning, and disaster response.
Cross-View Image Matching for Geo-localization in Urban Environments
Next, for each building in the query image, we retrieve the $k$ nearest neighbors from the reference buildings using a Siamese network trained on both positive matching image pairs and negative pairs.
Cross-View Image-to-Image Translation
Image Classification
+2
Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning
To alleviate this issue, many FL algorithms focus on mitigating the effects of data heterogeneity across clients by introducing a variety of proximal terms, some incurring considerable compute and/or memory overheads, to restrain local updates with respect to the global model.
HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models
We make our results publicly accessible for reproducible pipelines with released pre-trained models, thus providing a new evaluation paradigm for ASR error correction with LLMs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
POSTER: A Pyramid Cross-Fusion Transformer Network for Facial Expression Recognition
In this paper, we propose a two-stream Pyramid crOss-fuSion TransformER network (POSTER), that aims to holistically solve all three issues.
Facial Expression Recognition
Facial Expression Recognition (FER)
POTTER: Pooling Attention Transformer for Efficient Human Mesh Recovery
In this paper, we propose a pure transformer architecture named POoling aTtention TransformER (POTTER) for the HMR task from single images.
Ranked #33 on
3D Human Pose Estimation
on 3DPW
CEFHRI: A Communication Efficient Federated Learning Framework for Recognizing Industrial Human-Robot Interaction
Machine learning plays a vital role in industrial HRI by enhancing the adaptability and autonomy of robots in complex environments.
DIPS-Plus: The Enhanced Database of Interacting Protein Structures for Interface Prediction
In this work, we expand on a dataset recently introduced for this task, the Database of Interacting Protein Structures (DIPS), to present DIPS-Plus, an enhanced, feature-rich dataset of 42, 112 complexes for geometric deep learning of protein interfaces.
MGANet: A Robust Model for Quality Enhancement of Compressed Video
In video compression, most of the existing deep learning approaches concentrate on the visual quality of a single frame, while ignoring the useful priors as well as the temporal information of adjacent frames.
RaSa: Relation and Sensitivity Aware Representation Learning for Text-based Person Search
RA offsets the overfitting risk by introducing a novel positive relation detection task (i. e., learning to distinguish strong and weak positive pairs).
Ranked #2 on
Text based Person Retrieval
on RSTPReid
MMM: Generative Masked Motion Model
MMM consists of two key components: (1) a motion tokenizer that transforms 3D human motion into a sequence of discrete tokens in latent space, and (2) a conditional masked motion transformer that learns to predict randomly masked motion tokens, conditioned on the pre-computed text tokens.
Ranked #5 on
Motion Synthesis
on KIT Motion-Language
A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition
The goal of building a benchmark (suite of datasets) is to provide a unified protocol for fair evaluation and thus facilitate the evolution of a specific area.
R2Former: Unified Retrieval and Reranking Transformer for Place Recognition
Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database.
Cooperative Training and Latent Space Data Augmentation for Robust Medical Image Segmentation
In this paper, we present a cooperative framework for training image segmentation models and a latent space augmentation method for generating hard examples.
Interactive Feature Fusion for End-to-End Noise-Robust Speech Recognition
Speech enhancement (SE) aims to suppress the additive noise from a noisy speech signal to improve the speech's perceptual quality and intelligibility.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Dual-Path Style Learning for End-to-End Noise-Robust Speech Recognition
Then, we propose style learning to map the fused feature close to clean feature, in order to learn latent speech information from the latter, i. e., clean "speech style".
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
MaxStyle: Adversarial Style Composition for Robust Medical Image Segmentation
We propose a novel data augmentation framework called MaxStyle, which maximizes the effectiveness of style augmentation for model OOD performance.
Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces
This small set of principal vectors is provided to the server so that the server can directly identify distribution similarities among the clients to form clusters.
Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models
Our approach produces a more semantically accurate synthesis by constraining the attention regions of each token in the prompt to the image.
Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation
To alleviate this problem, we propose a novel network to unify speech enhancement and separation with gradient modulation to improve noise-robustness.
Multi-view Self-supervised Disentanglement for General Image Denoising
It is understandable as the model is designed to learn paired mapping (e. g. from a noisy image to its clean version).
Ranked #1 on
Denoising
on CBSD68 sigm75
Skeleton-in-Context: Unified Skeleton Sequence Modeling with In-Context Learning
Under this setting, the model can perceive tasks from prompts and accomplish them without any extra task-specific head predictions or model fine-tuning.
Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learning
How to collect informative trajectories of which the corresponding context reflects the specification of tasks?
What About Inputing Policy in Value Function: Policy Representation and Policy-extended Value Function Approximator
We study Policy-extended Value Function Approximator (PeVFA) in Reinforcement Learning (RL), which extends conventional value function approximator (VFA) to take as input not only the state (and action) but also an explicit policy representation.
Annotating Columns with Pre-trained Language Models
Inferring meta information about tables, such as column headers or relationships between columns, is an active research topic in data management as we find many tables are missing some of this information.
Ranked #1 on
Column Type Annotation
on VizNet-Sato-MultiColumn
A Lightweight Graph Transformer Network for Human Mesh Reconstruction from 2D Human Pose
We propose a pose analysis module that uses graph transformers to exploit structured and implicit joint correlations, and a mesh regression module that combines the extracted pose feature with the mesh template to reconstruct the final human mesh.
Ranked #60 on
3D Human Pose Estimation
on 3DPW
Knowledge Is Flat: A Seq2Seq Generative Framework for Various Knowledge Graph Completion
To remedy the KG structure information loss from the "flat" text, we further improve the input representations of entities and relations, and the inference algorithm in KG-S2S.
OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition
Due to the resource-intensive nature of training vision-language models on expansive video data, a majority of studies have centered on adapting pre-trained image-language models to the video domain.
Ranked #2 on
Zero-Shot Action Recognition
on Kinetics
GAMa: Cross-view Video Geo-localization
There are no existing datasets for this problem, therefore we propose GAMa dataset, a large-scale dataset with ground videos and corresponding aerial images.
Memory Attention Networks for Skeleton-based Action Recognition
Specifically, the TARM is deployed in a residual learning module that employs a novel attention learning network to recalibrate the temporal attention of frames in a skeleton sequence.
Ranked #89 on
Skeleton Based Action Recognition
on NTU RGB+D
Degrade is Upgrade: Learning Degradation for Low-light Image Enhancement
Different from existing methods tending to accomplish the relighting task directly by ignoring the fidelity and naturalness recovery, we investigate the intrinsic degradation and relight the low-light image while refining the details and color in two steps.
Balanced Knowledge Distillation for Long-tailed Learning
Existing methods usually modify the classification loss to increase the learning focus on tail classes, which unexpectedly sacrifice the performance on head classes.
Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Prompting
However, the fine-tuned PLMs often overwhelmingly focus on the textual information and overlook structural knowledge.
Adaptive FSS: A Novel Few-Shot Segmentation Framework via Prototype Enhancement
In this paper, we propose a novel framework based on the adapter mechanism, namely Adaptive FSS, which can efficiently adapt the existing FSS model to the novel classes.
Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks
Though successful, federated learning presents new challenges for machine learning, especially when the issue of data heterogeneity, also known as Non-IID data, arises.
Improving Image Captioning with Conditional Generative Adversarial Nets
In this paper, we propose a novel conditional-generative-adversarial-nets-based image captioning framework as an extension of traditional reinforcement-learning (RL)-based encoder-decoder architecture.
Multi-Task Learning for Left Atrial Segmentation on GE-MRI
Segmentation of the left atrium (LA) is crucial for assessing its anatomy in both pre-operative atrial fibrillation (AF) ablation planning and post-operative follow-up studies.
Efficient Deep Learning of Non-local Features for Hyperspectral Image Classification
Therefore, to incorporate the long-range contextual information, a deep fully convolutional network (FCN) with an efficient non-local module, named ENL-FCN, is proposed for HSI classification.
Attention guided global enhancement and local refinement network for semantic segmentation
First, commonly used upsampling methods in the decoder such as interpolation and deconvolution suffer from a local receptive field, unable to encode global contexts.
TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation
This paper focuses on an under-explored yet important problem: Federated Class-Continual Learning (FCCL), where new classes are dynamically added in federated learning.
UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learning
Multimodal learning aims to imitate human beings to acquire complementary information from multiple modalities for various downstream tasks.
Pseudo-label Alignment for Semi-supervised Instance Segmentation
Through extensive experiments conducted on the COCO and Cityscapes datasets, we demonstrate that PAIS is a promising framework for semi-supervised instance segmentation, particularly in cases where labeled data is severely limited.
Hyperspectral Image Classification in the Presence of Noisy Labels
The key idea of RLPA is to exploit knowledge (e. g., the superpixel based spectral-spatial constraints) from the observed hyperspectral images and apply it to the process of label propagation.
BDANet: Multiscale Convolutional Neural Network with Cross-directional Attention for Building Damage Assessment from Satellite Images
With a pair of pre- and post-disaster satellite images, building damage assessment aims at predicting the extent of damage to buildings.
Ranked #2 on
2D Semantic Segmentation
on xBD
Biomechanics-informed Neural Networks for Myocardial Motion Tracking in MRI
The learnt VAE regulariser then can be coupled with any deep learning based registration network to regularise the solution space to be biomechanically plausible.
DENSE: Data-Free One-Shot Federated Learning
One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round.
SPAct: Self-supervised Privacy Preservation for Action Recognition
Existing approaches for mitigating privacy leakage in action recognition require privacy labels along with the action labels from the video dataset.
Ranked #1 on
Action Classification
on UCF101
Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sports
Reasoning over sports videos for question answering is an important task with numerous applications, such as player training and information retrieval.
GaitMixer: Skeleton-based Gait Representation Learning via Wide-spectrum Multi-axial Mixer
Most existing gait recognition methods are appearance-based, which rely on the silhouettes extracted from the video data of human walking activities.
Ranked #6 on
Multiview Gait Recognition
on CASIA-B
ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
Some recent efforts have explored introducing ReLU or its variants as the substitutive activation function to help LLMs achieve activation sparsity and inference acceleration, but few can simultaneously obtain high sparsity and comparable model performance.
MLP: Motion Label Prior for Temporal Sentence Localization in Untrimmed 3D Human Motions
In this paper, we address the unexplored question of temporal sentence localization in human motions (TSLM), aiming to locate a target moment from a 3D human motion that semantically corresponds to a text query.
IDEAL: Query-Efficient Data-Free Learning from Black-box Models
Knowledge Distillation (KD) is a typical method for training a lightweight student model with the help of a well-trained teacher model.
Task-Adaptive Few-shot Node Classification
Then we transfer such knowledge to the classes with limited labeled nodes via our proposed task-adaptive modules.
Synthetic optical coherence tomography angiographs for detailed retinal vessel segmentation without human annotations
To address this issue, recent work has employed transfer learning, where a segmentation network is trained on synthetic OCTA images and is then applied to real data.
M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization
The proposed method, named Medical vision-language pre-training with Frozen language models and Latent spAce Geometry optimization (M-FLAG), leverages a frozen language model for training stability and efficiency and introduces a novel orthogonality loss to harmonize the latent space geometry.
Multimodal Transformer for Nursing Activity Recognition
Our method achieves state-of-the-art performance of 81. 8% accuracy on the benchmark dataset available for nurse activity recognition from the Nurse Care Activity Recognition Challenge.
Context Label Learning: Improving Background Class Representations in Semantic Segmentation
The results demonstrate that CoLab can guide the segmentation model to map the logits of background samples away from the decision boundary, resulting in significantly improved segmentation accuracy.
Edit Everything: A Text-Guided Generative System for Images Editing
We introduce a new generative system called Edit Everything, which can take image and text inputs and produce image outputs.
A Neural State-Space Model Approach to Efficient Speech Separation
In this work, we introduce S4M, a new efficient speech separation framework based on neural state-space models (SSM).
Deep Fisher Discriminant Learning for Mobile Hand Gesture Recognition
Gesture recognition is a challenging problem in the field of biometrics.
Ranked #1 on
Hand Gesture Recognition
on MGB
Cross-Modal Global Interaction and Local Alignment for Audio-Visual Speech Recognition
However, most existing AVSR approaches simply fuse the audio and visual features by concatenation, without explicit interactions to capture the deep correlations between them, which results in sub-optimal multimodal representations for downstream speech recognition task.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+3
First Place Solution to the CVPR'2023 AQTC Challenge: A Function-Interaction Centric Approach with Spatiotemporal Visual-Language Alignment
Affordance-Centric Question-driven Task Completion (AQTC) has been proposed to acquire knowledge from videos to furnish users with comprehensive and systematic instructions.
LAMP: Leveraging Language Prompts for Multi-person Pose Estimation
Human-centric visual understanding is an important desideratum for effective human-robot interaction.
Federated Few-shot Learning
In this way, the server can exploit the computational power of all clients and train the model on a larger set of data samples among all clients.
An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models
We derive several implications for system and middleware builders: (1) when adding a defense component against adversarial attacks, it is important to deploy multiple defense methods in tandem to achieve a good coverage of various attacks, (2) a blackbox attack is much less effective compared with a white-box attack, implying that it is important to keep model details (e. g., model architecture, hyperparameters) confidential via model obfuscation, and (3) driving models with a complex architecture are preferred if computing resources permit as they are more resilient to adversarial attacks than simple models.
RoPGen: Towards Robust Code Authorship Attribution via Automatic Coding Style Transformation
Recent studies show that current source code authorship attribution methods can be compromised by attackers exploiting adversarial examples and coding style manipulation.
Part Aware Contrastive Learning for Self-Supervised Action Recognition
This paper proposes an attention-based contrastive learning framework for skeleton representation learning, called SkeAttnCLR, which integrates local similarity and global features for skeleton-based action representations.
MIR-GAN: Refining Frame-Level Modality-Invariant Representations with Adversarial Network for Audio-Visual Speech Recognition
In this paper, we aim to learn the shared representations across modalities to bridge their gap.
PEA-Diffusion: Parameter-Efficient Adapter with Knowledge Distillation in non-English Text-to-Image Generation
In this paper, we are inspired to propose a simple plug-and-play language transfer method based on knowledge distillation.
Cross-lingual Text-to-Image Generation
Knowledge Distillation
+1
Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learning
Offline multi-agent reinforcement learning is challenging due to the coupling effect of both distribution shift issue common in offline setting and the high dimension issue common in multi-agent setting, making the action out-of-distribution (OOD) and value overestimation phenomenon excessively severe.
Robust Image Segmentation Using Contour-Guided Color Palettes
The contour-guided color palette (CCP) is proposed for robust image segmentation.
Hearing Lips in Noise: Universal Viseme-Phoneme Mapping and Transfer for Robust Audio-Visual Speech Recognition
In this work, we investigate the noise-invariant visual modality to strengthen robustness of AVSR, which can adapt to any testing noises while without dependence on noisy training data, a. k. a., unsupervised noise adaptation.
GeoSegNet: Point Cloud Semantic Segmentation via Geometric Encoder-Decoder Modeling
Semantic segmentation of point clouds, aiming to assign each point a semantic category, is critical to 3D scene understanding. Despite of significant advances in recent years, most of existing methods still suffer from either the object-level misclassification or the boundary-level ambiguity.
Estimating Model Performance under Domain Shifts with Class-Specific Confidence Scores
If we could estimate the performance that a pre-trained model would achieve on data from a specific deployment setting, for example a certain clinic, we could judge whether the model could safely be deployed or if its performance degrades unacceptably on the specific data.
Gradient Remedy for Multi-Task Learning in End-to-End Noise-Robust Speech Recognition
In this paper, we propose a simple yet effective approach called gradient remedy (GR) to solve interference between task gradients in noise-robust speech recognition, from perspectives of both angle and magnitude.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
FreMIM: Fourier Transform Meets Masked Image Modeling for Medical Image Segmentation
The research community has witnessed the powerful potential of self-supervised Masked Image Modeling (MIM), which enables the models capable of learning visual representation from unlabeled data.
A Global Benchmark of Algorithms for Segmenting Late Gadolinium-Enhanced Cardiac Magnetic Resonance Imaging
Segmentation of cardiac images, particularly late gadolinium-enhanced magnetic resonance imaging (LGE-MRI) widely used for visualizing diseased cardiac structures, is a crucial first step for clinical diagnosis and treatment.
Magic ELF: Image Deraining Meets Association Learning and Transformer
Convolutional neural network (CNN) and Transformer have achieved great success in multimedia applications.
GaitSADA: Self-Aligned Domain Adaptation for mmWave Gait Recognition
mmWave radar-based gait recognition is a novel user identification method that captures human gait biometrics from mmWave radar return signals.
Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting
In order to address this issue, we propose GAS, a \shorten approach that can successfully adapt existing robust AGRs to non-IID settings.
Generative Myocardial Motion Tracking via Latent Space Exploration with Biomechanics-informed Prior
In contrast to most existing approaches which impose explicit generic regularization such as smoothness, in this work we propose a novel method that can implicitly learn an application-specific biomechanics-informed prior and embed it into a neural network-parameterized transformation model.
FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learning
Personalized Federated Learning (PFL) represents a promising solution for decentralized learning in heterogeneous data environments.
Regress Before Construct: Regress Autoencoder for Point Cloud Self-supervised Learning
The proposed method decouples functions between the decoder and the encoder by introducing a mask regressor, which predicts the masked patch representation from the visible patch representation encoded by the encoder and the decoder reconstructs the target from the predicted masked patch representation.
Ranked #3 on
Few-Shot 3D Point Cloud Classification
on ModelNet40 10-way (20-shot)
(using extra training data)
Few-Shot 3D Point Cloud Classification
Representation Learning
+1
FAITH: Few-Shot Graph Classification with Hierarchical Task Graphs
Specifically, these works propose to accumulate meta-knowledge across diverse meta-training tasks, and then generalize such meta-knowledge to the target task with a disjoint label set.
ConPET: Continual Parameter-Efficient Tuning for Large Language Models
First, Static ConPET can adapt former continual learning methods originally designed for relatively smaller models to LLMs through PET and a dynamic replay strategy, which largely reduces the tuning costs and alleviates the over-fitting and forgetting issue.
Adversarial Attacks on Fairness of Graph Neural Networks
Fairness-aware graph neural networks (GNNs) have gained a surge of attention as they can reduce the bias of predictions on any demographic group (e. g., female) in graph-based applications.
A Coarse-to-Fine Indoor Layout Estimation (CFILE) Method
The task of estimating the spatial layout of cluttered indoor scenes from a single RGB image is addressed in this work.
Improving the generalizability of convolutional neural network-based segmentation on CMR images
We demonstrate that a neural network trained on a single-site single-scanner dataset from the UK Biobank can be successfully applied to segmenting cardiac MR images across different sites and different scanners without substantial loss of accuracy.
Revisiting Training-free NAS Metrics: An Efficient Training-based Method
In this paper, we revisit these training-free metrics and find that: (1) the number of parameters (\#Param), which is the most straightforward training-free metric, is overlooked in previous works but is surprisingly effective, (2) recent training-free metrics largely rely on the \#Param information to rank networks.
PGFed: Personalize Each Client's Global Objective for Federated Learning
Based on our observation, in this work, we propose Personalized Global Federated Learning (PGFed), a novel personalized FL framework that enables each client to personalize its own global objective by explicitly and adaptively aggregating the empirical risks of itself and other clients.
Plan To Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learning
In Model-based Reinforcement Learning (MBRL), model learning is critical since an inaccurate model can bias policy learning via generating misleading samples.
SAVE: Spectral-Shift-Aware Adaptation of Image Diffusion Models for Text-driven Video Editing
Text-to-Image (T2I) diffusion models have achieved remarkable success in synthesizing high-quality images conditioned on text prompts.
Source-free Domain Adaptive Human Pose Estimation
To this end, we propose a new task, named source-free domain adaptive HPE, which aims to address the challenges of cross-domain learning of HPE without access to source data during the adaptation process.
Robust and Scalable Model Editing for Large Language Models
Large language models (LLMs) can make predictions using parametric knowledge--knowledge encoded in the model weights--or contextual knowledge--knowledge presented in the context.
Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos
A video is first divided into equal length clips and for each clip a set of tube proposals are generated next based on 3D Convolutional Network (ConvNet) features.
Ranked #1 on
Action Detection
on UCF Sports
Category Guided Attention Network for Brain Tumor Segmentation in MRI
In this model, we design a Supervised Attention Module (SAM) based on the attention mechanism, which can capture more accurate and stable long-range dependency in feature maps without introducing much computational cost.
DIAGNOSIS: Detecting Unauthorized Data Usages in Text-to-image Diffusion Models
To address this issue, we propose a method for detecting such unauthorized data usage by planting the injected memorization into the text-to-image diffusion models trained on the protected dataset.
A Dual-Augmentor Framework for Domain Generalization in 3D Human Pose Estimation
Furthermore, the pose estimator's optimization is not exposed to domain shifts, limiting its overall generalization ability.
Domain Generalization
Weakly-supervised 3D Human Pose Estimation
Refined Semantic Enhancement towards Frequency Diffusion for Video Captioning
In this paper, we introduce a novel Refined Semantic enhancement method towards Frequency Diffusion (RSFD), a captioning model that constantly perceives the linguistic representation of the infrequent tokens.
Reconciling Object-Level and Global-Level Objectives for Long-Tail Detection
Specifically, complementary to the object-level classification loss for model discrimination, we design a generalized average precision (GAP) loss to explicitly optimize the global-level score ranking across different objects.
Progressive Bilateral-Context Driven Model for Post-Processing Person Re-Identification
Most existing person re-identification methods compute pairwise similarity by extracting robust visual features and learning the discriminative metric.
Embedding Gradient-based Optimization in Image Registration Networks
Deep learning (DL) image registration methods amortize the costly pair-wise iterative optimization by training deep neural networks to predict the optimal transformation in one fast forward-pass.
FeatER: An Efficient Network for Human Reconstruction via Feature Map-Based TransformER
Recently, vision transformers have shown great success in a set of human reconstruction tasks such as 2D human pose estimation (2D HPE), 3D human pose estimation (3D HPE), and human mesh reconstruction (HMR) tasks.
Ranked #29 on
3D Human Pose Estimation
on 3DPW
Graph Few-shot Learning with Task-specific Structures
Therefore, to adaptively learn node representations across meta-tasks, we propose a novel framework that learns a task-specific structure for each meta-task.
Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"
In the first step, we train different sub-predictors on different types of available low-fidelity information to extract beneficial knowledge as low-fidelity experts.
Combating Unknown Bias with Effective Bias-Conflicting Scoring and Gradient Alignment
For challenge B, we point out that the gradient contribution statistics can be a reliable indicator to inspect whether the optimization is dominated by bias-aligned samples.
CalFAT: Calibrated Federated Adversarial Training with Label Skewness
In this paper, we study the problem of FAT under label skewness, and reveal one root cause of the training instability and natural accuracy degradation issues: skewed labels lead to non-identical class probabilities and heterogeneous local models.
TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition
We observe that these representations complement each other depending on the nature of the action.
GCNext: Towards the Unity of Graph Convolutions for Human Motion Prediction
The past few years has witnessed the dominance of Graph Convolutional Networks (GCNs) over human motion prediction. Various styles of graph convolutions have been proposed, with each one meticulously designed and incorporated into a carefully-crafted network architecture.
Towards Improved Proxy-based Deep Metric Learning via Data-Augmented Domain Adaptation
Our experiments on benchmarks, including the popular CUB-200-2011, CARS196, Stanford Online Products, and In-Shop Clothes Retrieval, show that our learning algorithm significantly improves the existing proxy losses and achieves superior results compared to the existing methods.
CodaMal: Contrastive Domain Adaptation for Malaria Detection in Low-Cost Microscopes
Annotating images from LCM significantly increases the burden on medical experts compared to annotating images from high-cost microscopes (HCM).
Foresee then Evaluate: Decomposing Value Estimation with Latent Future Prediction
Value function is the central notion of Reinforcement Learning (RL).
Nonnegative Tensor Completion via Integer Optimization
Unlike matrix completion, tensor completion does not have an algorithm that is known to achieve the information-theoretic sample complexity rate.
Sampo: Unsupervised Knowledge Base Construction for Opinions and Implications
Since it can be expensive to obtain training data to learn to extract implications for each new domain of reviews, we propose an unsupervised KBC system, Sampo, Specifically, Sampo is tailored to build KBs for domains where many reviews on the same domain are available.
Accelerated Nonnegative Tensor Completion via Integer Programming
A recent approach, based on integer programming, resolves this tension for nonnegative tensor completion.
Few-shot Node Classification with Extremely Weak Supervision
Recent few-shot node classification methods typically learn from classes with abundant labeled nodes (i. e., meta-training classes) and then generalize to classes with limited labeled nodes (i. e., meta-test classes).
Learning Semantic Proxies from Visual Prompts for Parameter-Efficient Fine-Tuning in Deep Metric Learning
As a result of the success of recent pre-trained models trained from larger-scale datasets, it is challenging to adapt the model to the DML tasks in the local data domain while retaining the previously gained knowledge.
Ranked #2 on
Image Retrieval
on iNaturalist
Improved post-hoc probability calibration for out-of-domain MRI segmentation
In image segmentation, well-calibrated probabilities allow radiologists to identify regions where model-predicted segmentations are unreliable.
Getting the Most from Eye-Tracking: User-Interaction Based Reading Region Estimation Dataset and Models
A single digital newsletter usually contains many messages (regions).
Semi-Dynamic Load Balancing: Efficient Distributed Learning in Non-Dedicated Environments
Efficient model training requires eliminating such stragglers, yet for modern ML workloads, existing load balancing strategies are inefficient and even infeasible.
Key Person Aided Re-identification in Partially Ordered Pedestrian Set
Ideally person re-identification seeks for perfect feature representation and metric model that re-identify all various pedestrians well in non-overlapping views at different locations with different camera configurations, which is very challenging.
One-Two-One Networks for Compression Artifacts Reduction in Remote Sensing
Compression artifacts reduction (CAR) is a challenging problem in the field of remote sensing.
Gabor Convolutional Networks
Steerable properties dominate the design of traditional filters, e. g., Gabor filters, and endow features the capability of dealing with spatial transformations.
Robust 3D Action Recognition through Sampling Local Appearances and Global Distributions
Then, motion and shape cues are jointly used to generate robust and distinctive spatial-temporal interest points (STIPs): motion-based STIPs and shape-based STIPs.
An End-to-end 3D Convolutional Neural Network for Action Detection and Segmentation in Videos
A video is first divided into equal length clips and next for each clip a set of tube proposals are generated based on 3D CNN features.
Latent Constrained Correlation Filter
In this paper, we introduce an intermediate step -- solution sampling -- after the data sampling step to form a subspace, in which an optimal solution can be estimated.
Block building programming for symbolic regression
Symbolic regression that aims to detect underlying data-driven models has become increasingly important for industrial data analysis.
Multi-modal Aggregation for Video Classification
In this paper, we present a solution to Large-Scale Video Classification Challenge (LSVC2017) [1] that ranked the 1st place.
Manifold Constrained Low-Rank Decomposition
Low-rank decomposition (LRD) is a state-of-the-art method for visual data reconstruction and modelling.
Fast Modeling Methods for Complex System with Separable Features
In this paper, we analyze different types of separability of some real-world engineering equations and establish a mathematical model of generalized separable system (GS system).
A divide and conquer method for symbolic regression
This feature motivated us to develop a new method, divide and conquer (D&C), for symbolic regression, in which the target function is divided into a number of sub-functions and the sub-functions are then determined by any of a GP algorithm.
Latent Constrained Correlation Filters for Object Localization
There is a neglected fact in the traditional machine learning methods that the data sampling can actually lead to the solution sampling.
Elite Bases Regression: A Real-time Algorithm for Symbolic Regression
In this paper, a new non-evolutionary real-time algorithm for symbolic regression, Elite Bases Regression (EBR), is proposed.
Deep Spatio-temporal Manifold Network for Action Recognition
Visual data such as videos are often sampled from complex manifold.
Measuring and Predicting Tag Importance for Image Retrieval
Textual data such as tags, sentence descriptions are combined with visual cues to reduce the semantic gap for image retrieval applications in today's Multimodal Image Retrieval (MIR) systems.
Output Constraint Transfer for Kernelized Correlation Filter in Tracking
Kernelized Correlation Filter (KCF) is one of the state-of-the-art object trackers.
GAL: A Global-Attributes Assisted Labeling System for Outdoor Scenes
The proposed Global-attributes Assisted Labeling (GAL) system exploits both local features and global attributes.
Fast Iteratively Reweighted Least Squares Algorithms for Analysis-Based Sparsity Reconstruction
The convergence rate of the proposed algorithm is almost the same as that of the traditional IRLS algorithms, that is, exponentially fast.
SIRF: Simultaneous Image Registration and Fusion in A Unified Framework
In this paper, we propose a novel method for image fusion with a high-resolution panchromatic image and a low-resolution multispectral image at the same geographical location.
Forest Sparsity for Multi-channel Compressive Sensing
In this paper, we investigate a new compressive sensing model for multi-channel sparse data where each channel can be represented as a hierarchical tree and different channels are highly correlated.
Rain Removal By Image Quasi-Sparsity Priors
We present a novel rain removal method in this paper, which consists of two steps, i. e., detection of rain streaks and reconstruction of the rain-removed image.
Compressive Sensing MRI with Wavelet Tree Sparsity
On the other side, some algorithms have been proposed for tree sparsity regularization, but few of them has validated the benefit of tree structure in CS-MRI.
Dense Semantic and Topological Correspondence of 3D Faces without Landmarks
The dense correspondence goal is revisited in two perspectives: semantic and topological correspondence.
Preconditioning for Accelerated Iteratively Reweighted Least Squares in Structured Sparsity Reconstruction
In this paper, we propose a novel algorithm for structured sparsity reconstruction.
Image Fusion with Local Spectral Consistency and Dynamic Gradient Sparsity
In this paper, we propose a novel method for image fusion from a high resolution panchromatic image and a low resolution multispectral image at the same geographical location.
