Search Results for author:
Found 21 papers, 9 papers with code
SpeechGuard: Exploring the Adversarial Robustness of Multimodal Large Language Models
Despite safety guardrails, experiments on jailbreaking demonstrate the vulnerability of SLMs to adversarial perturbations and transfer attacks, with average attack success rates of 90% and 10% respectively when evaluated on a dataset of carefully designed harmful questions spanning 12 different toxic categories.
SpeechVerse: A Large-scale Generalizable Audio Language Model
The models are instruction finetuned using continuous latent representations extracted from the speech foundation model to achieve optimal zero-shot performance on a diverse range of speech processing tasks using natural language instructions.
Mask The Bias: Improving Domain-Adaptive Generalization of CTC-based ASR with Internal Language Model Estimation
Internal language model estimation (ILME) has been proposed to mitigate this bias for autoregressive models such as attention-based encoder-decoder and RNN-T.
NeuroMapper: In-browser Visualizer for Neural Network Training
We present our ongoing work NeuroMapper, an in-browser visualization tool that helps machine learning (ML) developers interpret the evolution of a model during training, providing a new way to monitor the training process and visually discover reasons for suboptimal training.
Hear No Evil: Towards Adversarial Robustness of Automatic Speech Recognition via Multi-Task Learning
In this work, we investigate the impact of performing such multi-task learning on the adversarial robustness of ASR models in the speech domain.
SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning
Person tracking using computer vision techniques has wide ranging applications such as autonomous driving, home security and sports analytics.
Concept Evolution in Deep Learning Training: A Unified Interpretation Framework and Discoveries
We present ConceptEvo, a unified interpretation framework for deep neural networks (DNNs) that reveals the inception and evolution of learned concepts during training.
DetectorDetective: Investigating the Effects of Adversarial Examples on Object Detectors
With deep learning based systems performing exceedingly well in many vision-related tasks, a major concern with their widespread deployment especially in safety-critical applications is their susceptibility to adversarial attacks.
NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks
Through a large-scale human evaluation, we demonstrate that our technique discovers neuron groups that represent coherent, human-meaningful concepts.
EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models
The advent of larger machine learning (ML) models have improved state-of-the-art (SOTA) performance in various modeling tasks, ranging from computer vision to natural language.
Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learning
Training deep neural networks for automatic speech recognition (ASR) requires large amounts of transcribed speech.
SkeletonVis: Interactive Visualization for Understanding Adversarial Attacks on Human Action Recognition Models
Skeleton-based human action recognition technologies are increasingly used in video based applications, such as home robotics, healthcare on aging population, and surveillance.
Bluff: Interactively Deciphering Adversarial Attacks on Deep Neural Networks
Deep neural networks (DNNs) are now commonly used in many domains.
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization
Deep learning's great success motivates many practitioners and students to learn about this exciting technology.
Massif: Interactive Interpretation of Adversarial Attacks on Deep Learning
Deep neural networks (DNNs) are increasingly powering high-stakes applications such as autonomous cars and healthcare; however, DNNs are often treated as "black boxes" in such applications.
CNN 101: Interactive Visual Learning for Convolutional Neural Networks
The success of deep learning solving previously-thought hard problems has inspired many non-experts to learn and understand this exciting technology.
GOGGLES: Automatic Image Labeling with Affinity Coding
We build the GOGGLES system that implements affinity coding for labeling image datasets by designing a novel set of reusable affinity functions for images, and propose a novel hierarchical generative model for class inference using a small development set.
The Efficacy of SHIELD under Different Threat Models
To evaluate the robustness of the defense against an adaptive attacker, we consider the targeted-attack success rate of the Projected Gradient Descent (PGD) attack, which is a strong gradient-based adversarial attack proposed in adversarial machine learning research.
ADAGIO: Interactive Experimentation with Adversarial Attack and Defense for Audio
Adversarial machine learning research has recently demonstrated the feasibility to confuse automatic speech recognition (ASR) models by introducing acoustically imperceptible perturbations to audio samples.
Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compression
The rapidly growing body of research in adversarial machine learning has demonstrated that deep neural networks (DNNs) are highly vulnerable to adversarially generated images.
Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression
Deep neural networks (DNNs) have achieved great success in solving a variety of machine learning (ML) problems, especially in the domain of image recognition.
