Search Results for author:
Found 33 papers, 14 papers with code
Negative to Positive Co-learning with Aggressive Modality Dropout
This paper aims to document an effective way to improve multimodal co-learning by using aggressive modality dropout.
A2VIS: Amodal-Aware Approach to Video Instance Segmentation
Through extensive experiments and ablation studies, we show that A2VIS excels in both MOT and VIS tasks in identifying and tracking object instances with a keen understanding of their full shape.
Amodal Instance Segmentation with Diffusion Shape Prior Estimation
Drawing inspiration from this, we propose AISDiff with a Diffusion Shape Prior Estimation (DiffSP) module.
HENASY: Learning to Assemble Scene-Entities for Egocentric Video-Language Model
Current video-language models (VLMs) rely extensively on instance-level alignment between video and language modalities, which presents two major limitations: (1) visual reasoning disobeys the natural perception that humans do in first-person perspective, leading to a lack of reasoning interpretation; and (2) learning is limited in capturing inherent fine-grained relationships between two modalities.
S3Former: Self-supervised High-resolution Transformer for Solar PV Profiling
To meet this need, we introduce S3Former, designed to segment solar panels from aerial imagery and provide size and location information critical for analyzing the impact of such installations on the grid.
CarcassFormer: An End-to-end Transformer-based Framework for Simultaneous Localization, Segmentation and Classification of Poultry Carcass Defect
This study proposes an effective approach for automating the assessment of carcass quality without requiring skilled labor or inspector involvement.
ShapeFormer: Shape Prior Visible-to-Amodal Transformer-based Amodal Instance Segmentation
Consequently, this compromised quality of visible features during the subsequent visible-to-amodal transition.
Dyadic Interaction Modeling for Social Behavior Generation
Hence, an effective model for generating listener nonverbal behaviors requires understanding the dyadic context and interaction.
3FM: Multi-modal Meta-learning for Federated Tasks
We present a novel approach in the domain of federated learning (FL), particularly focusing on addressing the challenges posed by modality heterogeneity, variability in modality availability across clients, and the prevalent issue of missing data.
SolarFormer: Multi-scale Transformer for Solar PV Profiling
Accurate mapping of PV installations is crucial for understanding their adoption and informing energy policy.
Privacy-preserving Representation Learning for Speech Understanding
In this paper, we present a novel framework to anonymize utterance-level speech embeddings generated by pre-trained encoders and show its effectiveness for a range of speech classification tasks.
Open-Fusion: Real-time Open-Vocabulary 3D Mapping and Queryable Scene Representation
Open-Fusion harnesses the power of a pre-trained vision-language foundation model (VLFM) for open-set semantic comprehension and employs the Truncated Signed Distance Function (TSDF) for swift 3D scene reconstruction.
Personalized Adaptation with Pre-trained Speech Encoders for Continuous Emotion Recognition
There are individual differences in expressive behaviors driven by cultural norms and personality.
LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysis
Facial expression analysis is an important tool for human-computer interaction.
Representation Learning for Audio Privacy Preservation using Source Separation and Robust Adversarial Learning
Privacy preservation has long been a concern in smart acoustic monitoring systems, where speech can be passively recorded along with a target signal in the system's operating environment.
AerialFormer: Multi-resolution Transformer for Aerial Image Segmentation
Aerial Image Segmentation is a top-down perspective semantic segmentation and has several challenging characteristics such as strong imbalance in the foreground-background distribution, complex background, intra-class heterogeneity, inter-class homogeneity, and tiny objects.
Ranked #1 on
Semantic Segmentation
on ISPRS Potsdam
Adversarial Representation Learning for Robust Privacy Preservation in Audio
In this study, we propose a novel adversarial training method for learning representations of audio recordings that effectively prevents the detection of speech activity from the latent features of the recordings.
Multi-modal Facial Action Unit Detection with Large Pre-trained Models for the 5th Competition on Affective Behavior Analysis in-the-wild
Facial action unit detection has emerged as an important task within facial expression analysis, aimed at detecting specific pre-defined, objective facial expressions, such as lip tightening and cheek raising.
An Inception-Residual-Based Architecture with Multi-Objective Loss for Detecting Respiratory Anomalies
Notably, we achieved the Top-1 performance in Task 2-1 and Task 2-2 with the highest Score of 74. 5% and 53. 9%, respectively.
Meta Learning for Few-Shot Medical Text Classification
We investigate the use of meta-learning and robustness techniques on a broad corpus of benchmark text and medical data.
AISFormer: Amodal Instance Segmentation with Transformer
AISFormer explicitly models the complex coherence between occluder, visible, amodal, and invisible masks within an object's regions of interest by treating them as learnable queries.
3DConvCaps: 3DUnet with Convolutional Capsule Encoder for Medical Image Segmentation
Capsule network is a recent new architecture that has achieved better robustness in part-whole representation learning by replacing pooling layers with dynamic routing and convolutional strides, which has shown potential results on popular tasks such as digit classification and object segmentation.
Scaling Cross-Domain Content-Based Image Retrieval for E-commerce Snap and Search Application
In this industry talk at ECIR 2022, we illustrate how we approach the main challenges from large scale cross-domain content-based image retrieval using a cascade method and a combination of our visual search and classification capabilities.
A Speech Representation Anonymization Framework via Selective Noise Perturbation
Privacy and security are major concerns when communicating speech signals to cloud services such as automatic speech recognition (ASR) and speech emotion recognition (SER).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
CapsNet for Medical Image Segmentation
We then provide recent developments of CapsNet for the task of medical image segmentation.
A Pre-trained Audio-Visual Transformer for Emotion Recognition
In this paper, we introduce a pretrained audio-visual Transformer trained on more than 500k utterances from nearly 4000 celebrities from the VoxCeleb2 dataset for human behavior understanding.
SS-3DCapsNet: Self-supervised 3D Capsule Networks for Medical Segmentation on Less Labeled Data
Capsule network is a recent new deep network architecture that has been applied successfully for medical image segmentation tasks.
Deep Federated Learning for Autonomous Driving
We design a new Federated Autonomous Driving network (FADNet) that can improve the model stability, ensure convergence, and handle imbalanced data distribution problems while is being trained with federated learning methods.
Modeling Dynamics of Facial Behavior for Mental Health Assessment
Facial action unit (FAU) intensities are popular descriptors for the analysis of facial behavior.
Multiple Meta-model Quantifying for Medical Visual Question Answering
However, most of the existing medical VQA methods rely on external data for transfer learning, while the meta-data within the dataset is not fully utilized.
Ranked #5 on
Medical Visual Question Answering
on PathVQA
Towards A Friendly Online Community: An Unsupervised Style Transfer Framework for Profanity Redaction
Offensive and abusive language is a pressing problem on social media platforms.
Robust Deep Learning Framework For Predicting Respiratory Anomalies and Diseases
This paper presents a robust deep learning framework developed to detect respiratory diseases from recordings of respiratory sounds.
Are you really looking at me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze from Conventional Video
Despite a revolution in the pervasiveness of video cameras in our daily lives, one of the most meaningful forms of nonverbal affective communication, interpersonal eye gaze, i. e. eye gaze relative to a conversation partner, is not available from common video.
