Search Results for author: Ahmed Tewfik

Found 22 papers, 3 papers with code

Streaming Anchor Loss: Augmenting Supervision with Temporal Significance

no code implementations9 Oct 2023 Utkarsh Oggy Sarawgi, John Berkowitz, Vineet Garg, Arnav Kundu, Minsik Cho, Sai Srujana Buddi, Saurabh Adya, Ahmed Tewfik

Streaming neural network models for fast frame-wise responses to various speech and sensory signals are widely adopted on resource-constrained platforms.

Leveraging Large Language Models for Exploiting ASR Uncertainty

no code implementations9 Sep 2023 Pranay Dighe, Yi Su, Shangshang Zheng, Yunshu Liu, Vineet Garg, Xiaochuan Niu, Ahmed Tewfik

While large language models excel in a variety of natural language processing (NLP) tasks, to perform well on spoken language understanding (SLU) tasks, they must either rely on off-the-shelf automatic speech recognition (ASR) systems for transcription, or be equipped with an in-built speech modality.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +7

Audio-to-Intent Using Acoustic-Textual Subword Representations from End-to-End ASR

no code implementations21 Oct 2022 Pranay Dighe, Prateeth Nayak, Oggi Rudovic, Erik Marchi, Xiaochuan Niu, Ahmed Tewfik

Accurate prediction of the user intent to interact with a voice assistant (VA) on a device (e. g. on the phone) is critical for achieving naturalistic, engaging, and privacy-centric interactions with the VA. To this end, we present a novel approach to predict the user's intent (the user speaking to the device or not) directly from acoustic and textual information encoded at subword tokens which are obtained via an end-to-end ASR model.

intent-classification Intent Classification

Radiomics-Guided Global-Local Transformer for Weakly Supervised Pathology Localization in Chest X-Rays

1 code implementation10 Jul 2022 Yan Han, Gregory Holste, Ying Ding, Ahmed Tewfik, Yifan Peng, Zhangyang Wang

Using the learned self-attention of its image branch, RGT extracts a bounding box for which to compute radiomic features, which are further processed by the radiomics branch; learned image and radiomic features are then fused and mutually interact via cross-attention layers.

End-to-end system for object detection from sub-sampled radar data

no code implementations8 Mar 2022 Madhumitha Sakthi, Ahmed Tewfik, Marius Arvinte, Haris Vikalo

We show robust detection based on radar data reconstructed using 20% of samples under extreme weather conditions such as snow or fog, and on low-illuminated nights.

object-detection Object Detection

CheXT: Knowledge-Guided Cross-Attention Transformer for Abnormality Classification and Localization in Chest X-rays

no code implementations29 Sep 2021 Yan Han, Ying Ding, Ahmed Tewfik, Yifan Peng, Zhangyang Wang

During training, the image branch leverages its learned attention to estimate pathology localization, which is then utilized to extract radiomic features from images in the radiomics branch.

Knowledge-Augmented Contrastive Learning for Abnormality Classification and Localization in Chest X-rays with Radiomics using a Feedback Loop

no code implementations11 Apr 2021 Yan Han, Chongyan Chen, Ahmed Tewfik, Benjamin Glicksberg, Ying Ding, Yifan Peng, Zhangyang Wang

The key knob of our framework is a unique positive sampling approach tailored for the medical images, by seamlessly integrating radiomic features as a knowledge augmentation.

Contrastive Learning

Automotive Radar Data Acquisition using Object Detection

1 code implementation5 Oct 2020 Madhumitha Sakthi, Ahmed Tewfik

In this paper, we introduce an algorithm to utilize object detection results from the image to adaptively sample and acquire radar data using Compressed Sensing (CS).

Autonomous Driving Object +2

Adaptive Automotive Radar data Acquisition

no code implementations28 Sep 2020 Madhumitha Sakthi, Ahmed Tewfik

We use previous radar frame information to mitigate the potential information loss of an object missed by the image or the object detection network.

Autonomous Driving Object +2

Constrained Variational Autoencoder for improving EEG based Speech Recognition Systems

no code implementations1 Jun 2020 Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik

In this paper we introduce a recurrent neural network (RNN) based variational autoencoder (VAE) model with a new constrained loss function that can generate more meaningful electroencephalography (EEG) features from raw EEG features to improve the performance of EEG based speech recognition systems.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Improving EEG based continuous speech recognition using GAN

no code implementations29 May 2020 Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik

In this paper we demonstrate that it is possible to generate more meaningful electroencephalography (EEG) features from raw EEG features using generative adversarial networks (GAN) to improve the performance of EEG based continuous speech recognition systems.

EEG speech-recognition +1

Predicting Different Acoustic Features from EEG and towards direct synthesis of Audio Waveform from EEG

no code implementations29 May 2020 Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik

In [1, 2] authors provided preliminary results for synthesizing speech from electroencephalography (EEG) features where they first predict acoustic features from EEG features and then the speech is reconstructed from the predicted acoustic features using griffin lim reconstruction algorithm.

EEG

Understanding effect of speech perception in EEG based speech recognition systems

no code implementations29 May 2020 Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik

The electroencephalography (EEG) signals recorded in parallel with speech are used to perform isolated and continuous speech recognition.

EEG speech-recognition +1

Predicting Video features from EEG and Vice versa

no code implementations16 May 2020 Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik

In this paper we explore predicting facial or lip video features from electroencephalography (EEG) features and predicting EEG features from recorded facial or lip video frames using deep learning models.

EEG

Advancing Speech Synthesis using EEG

no code implementations9 Apr 2020 Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik

In this paper we introduce attention-regression model to demonstrate predicting acoustic features from electroencephalography (EEG) features recorded in parallel with spoken sentences.

EEG regression +1

Speaker Identification using EEG

no code implementations7 Mar 2020 Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik

In this paper we explore speaker identification using electroencephalography (EEG) signals.

EEG Speaker Identification

Detecting Patch Adversarial Attacks with Image Residuals

1 code implementation28 Feb 2020 Marius Arvinte, Ahmed Tewfik, Sriram Vishwanath

We introduce an adversarial sample detection algorithm based on image residuals, specifically designed to guard against patch-based attacks.

Denoising

Continuous Silent Speech Recognition using EEG

no code implementations6 Feb 2020 Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik

Our results demonstrate the feasibility of using EEG signals for performing continuous silent speech recognition.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Cannot find the paper you are looking for? You can Submit a new open access paper.