Search Results for author:
Found 22 papers, 9 papers with code
Partial Federated Learning
Federated Learning (FL) is a popular algorithm to train machine learning models on user data constrained to edge devices (for example, mobile phones) due to privacy concerns.
Can Text-to-image Model Assist Multi-modal Learning for Visual Recognition with Visual Modality Missing?
Specifically, we propose a simple but effective multi-modal learning framework GTI-MM to enhance the data efficiency and model robustness against missing visual modality by imputing the missing data with generative transformers.
Audio-visual child-adult speaker classification in dyadic interactions
Building on the foundation of an audio-only child-adult speaker classification pipeline, we propose incorporating visual cues through active speaker detection and visual processing models.
FedAIoT: A Federated Learning Benchmark for Artificial Intelligence of Things
However, most existing FL works are not conducted on datasets collected from authentic IoT devices that capture unique modalities and inherent challenges of IoT data.
Scaling Representation Learning from Ubiquitous ECG with State-Space Models
We train this model in a self-supervised manner with 275, 000 10s ECG recordings collected in the wild and evaluate it on a range of downstream tasks.
MM-AU:Towards Multimodal Understanding of Advertisement Videos
Advertisement videos (ads) play an integral part in the domain of Internet e-commerce as they amplify the reach of particular products to a broad audience or can serve as a medium to raise awareness about specific issues through concise narrative structures.
Robust Self Supervised Speech Embeddings for Child-Adult Classification in Interactions involving Children with Autism
We address the problem of detecting who spoke when in child-inclusive spoken interactions i. e., automatic child-adult speaker classification.
Learning Behavioral Representations of Routines From Large-scale Unlabeled Wearable Time-series Data Streams using Hawkes Point Process
In this work, we propose a novel wearable time-series mining framework, Hawkes point process On Time series clusters for ROutine Discovery (HOT-ROD), for uncovering behavioral routines from completely unlabeled wearable recordings.
FedMultimodal: A Benchmark For Multimodal Federated Learning
In order to facilitate the research in multimodal FL, we introduce FedMultimodal, the first FL benchmark for multimodal learning covering five representative multimodal applications from ten commonly used datasets with a total of eight unique modalities.
GPT-FL: Generative Pre-trained Model-Assisted Federated Learning
Through comprehensive ablation analysis, we discover that the downstream model generated by synthetic data plays a crucial role in controlling the direction of gradient diversity during FL training, which enhances convergence speed and contributes to the notable accuracy boost observed with GPT-FL.
Understanding Spoken Language Development of Children with ASD Using Pre-trained Speech Embeddings
This paper proposes applications of speech processing technologies in support of automated assessment of children's spoken language development by classification between child and adult speech and between speech and nonverbal vocalization in NLS, with respective F1 macro scores of 82. 6% and 67. 8%, underscoring the potential for accurate and scalable tools for ASD research and clinical use.
Learning Sequence Descriptor based on Spatio-Temporal Attention for Visual Place Recognition
Visual Place Recognition (VPR) aims to retrieve frames from a geotagged database that are located at the same place as the query frame.
A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness
Speech-centric machine learning systems have revolutionized many leading domains ranging from transportation and healthcare to education and defense, profoundly changing how people live, work, and interact with each other.
Multimodal Estimation of Change Points of Physiological Arousal in Drivers
Detecting unsafe driving states, such as stress, drowsiness, and fatigue, is an important component of ensuring driving safety and an essential prerequisite for automatic intervention systems in vehicles.
Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling
In this work, we propose a semi-supervised federated learning framework, Semi-FedSER, that utilizes both labeled and unlabeled data samples to address the challenge of limited labeled data samples in FL.
Patch-NetVLAD+: Learned patch descriptor and weighted matching strategy for place recognition
Visual Place Recognition (VPR) in areas with similar scenes such as urban or indoor scenarios is a major challenge.
Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings
To assess the information leakage of SER systems trained using FL, we propose an attribute inference attack framework that infers sensitive attribute information of the clients from shared gradients or model parameters, corresponding to the FedSGD and the FedAvg training algorithms, respectively.
Unsupervised Joint Learning of Depth, Optical Flow, Ego-motion from Video
Compared with independent training, joint training can make full use of the geometric relationship between geometric elements and provide dynamic and static information of the scene.
Screenplay Quality Assessment: Can We Predict Who Gets Nominated?
Toward that goal, in this work, we present a method to evaluate the quality of a screenplay based on linguistic cues.
TILES-2018, a longitudinal physiologic and behavioral data set of hospital workers
We designed the study to investigate the use of off-the-shelf wearable and environmental sensors to understand individual-specific constructs such as job performance, interpersonal interaction, and well-being of hospital workers over time in their natural day-to-day job settings.
Occlusion Aware Unsupervised Learning of Optical Flow From Video
In this paper, we proposed an unsupervised learning method for estimating the optical flow between video frames, especially to solve the occlusion problem.
Vision-based Semantic Mapping and Localization for Autonomous Indoor Parking
In this paper, we proposed a novel and practical solution for the real-time indoor localization of autonomous driving in parking lots.
