Search Results for author:
Found 65 papers, 31 papers with code
Effective and Efficient Mixed Precision Quantization of Speech Foundation Models
This paper presents a novel mixed-precision quantization approach for speech foundation models that tightly integrates mixed-precision learning and quantized model parameter estimation into one single model compression stage.
3D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformer
Current 3D Large Multimodal Models (3D LMMs) have shown tremendous potential in 3D-vision-based dialogue and reasoning.
Structured Speaker-Deficiency Adaptation of Foundation Models for Dysarthric and Elderly Speech Recognition
Experiments on the UASpeech dysarthric and DementiaBank Pitt elderly speech corpora suggest structured speaker-deficiency adaptation of HuBERT and Wav2vec2-conformer models consistently outperforms baseline SFMs using either: a) no adapters; b) global adapters shared among all speakers; or c) single attribute adapters modelling speaker or deficiency labels alone by statistically significant WER reductions up to 3. 01% and 1. 50% absolute (10. 86% and 6. 94% relative) on the two tasks respectively.
RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion
For the latter, we initially incorporate a radar-guided depth head to refine the transformation from image view to BEV.
Hierarchical Context Alignment with Disentangled Geometric and Temporal Modeling for Semantic Occupancy Prediction
Camera-based 3D Semantic Occupancy Prediction (SOP) is crucial for understanding complex 3D scenes from limited 2D image observations.
3D Semantic Occupancy Prediction
LIDAR Semantic Segmentation
+2
Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASR
Self-supervised learning (SSL) based discrete speech representations are highly compact and domain adaptable.
RayFormer: Improving Query-Based Multi-Camera 3D Object Detection via Ray-Centric Strategies
To extract unique object-level features that cater to distinct queries, we design a ray sampling method that suitably organizes the distribution of feature sampling points on both images and bird's eye view.
Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation
The application of data-intensive automatic speech recognition (ASR) technologies to dysarthric and elderly adult speech is confronted by their mismatch against healthy and nonaged voices, data scarcity and large speaker-level variability.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Described Spatial-Temporal Video Detection
However, in the video domain, the existing setting, i. e., spatial-temporal video grounding (STVG), is formulated to only detect one pre-existing object in each frame, ignoring the fact that language descriptions can involve none or multiple entities within a video.
Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognition
Experiments are conducted on four tasks: the English UASpeech and TORGO dysarthric speech corpora; and the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets.
Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion
To address this problem, we present HTCL, a novel Hierarchical Temporal Context Learning paradigm for improving camera-based semantic scene completion.
Ranked #11 on
3D Semantic Scene Completion
on SemanticKITTI
One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model
We propose a novel one-pass multiple ASR systems joint compression and quantization approach using an all-in-one neural model.
Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Mask
This paper proposes a novel non-autoregressive (NAR) block-based Attention Mask Decoder (AMD) that flexibly balances performance-efficiency trade-offs for Conformer ASR systems.
End-to-End Rate-Distortion Optimized 3D Gaussian Representation
3D Gaussian Splatting (3DGS) has become an emerging technique with remarkable potential in 3D representation and image rendering.
HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation
We present a hybrid-view-based knowledge distillation framework, termed HVDistill, to guide the feature learning of a point cloud neural network with a pre-trained image network in an unsupervised man- ner.
Agent3D-Zero: An Agent for Zero-shot 3D Understanding
The ability to understand and reason the 3D real world is a crucial milestone towards artificial general intelligence.
PoIFusion: Multi-Modal 3D Object Detection via Fusion at Points of Interest
In particular, our PoIFusion follows the paradigm of query-based object detection, formulating object queries as dynamic 3D boxes and generating a set of PoIs based on each query box.
DeepEraser: Deep Iterative Context Mining for Generic Text Eraser
In this work, we present DeepEraser, an effective deep network for generic text removal.
Cycle-Consistency Learning for Captioning and Grounding
The proposed framework (1) allows the semi-weakly supervised training of visual grounding; (2) improves the performance of fully supervised visual grounding; (3) yields a general captioning model that can describe arbitrary image regions.
Towards Automatic Data Augmentation for Disordered Speech Recognition
Automatic recognition of disordered speech remains a highly challenging task to date due to data scarcity.
FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugin
We apply the FlashOCC to diverse occupancy prediction baselines on the challenging Occ3D-nuScenes benchmarks and conduct extensive experiments to validate the effectiveness.
I$^2$MD: 3D Action Representation Learning with Inter- and Intra-modal Mutual Distillation
Different from existing distillation solutions that transfer the knowledge of a pre-trained and fixed teacher to the student, in CMD, the knowledge is continuously updated and bidirectionally distilled between modalities during pre-training.
Invariant Training 2D-3D Joint Hard Samples for Few-Shot Point Cloud Recognition
We tackle the data scarcity challenge in few-shot point cloud recognition of 3D objects by using a joint prediction from a conventional 3D model and a well-trained 2D model.
SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning
To make the best of such rectification cues, we introduce SimFIR, a simple framework for fisheye image rectification based on self-supervised representation learning.
Masked Motion Predictors are Strong 3D Action Representation Learners
To be specific, the proposed MAMP takes as input the masked spatio-temporal skeleton sequence and predicts the corresponding temporal motion of the masked human joints.
Few-Shot Skeleton-Based Action Recognition
motion prediction
+2
Cyclic-Bootstrap Labeling for Weakly Supervised Object Detection
These inaccurate high-scoring region proposals will mislead the training of subsequent refinement modules and thus hamper the detection performance.
Audio-visual End-to-end Multi-channel Speech Separation, Dereverberation and Recognition
Accurate recognition of cocktail party speech containing overlapping speakers, noise and reverberation remains a highly challenging task to date.
Hyper-parameter Adaptation of Conformer ASR Systems for Elderly and Dysarthric Speech Recognition
Automatic recognition of disordered and elderly speech remains highly challenging tasks to date due to data scarcity.
Factorised Speaker-environment Adaptive Training of Conformer Speech Recognition Systems
Rich sources of variability in natural speech present significant challenges to current data intensive speech recognition technologies.
Towards Effective and Compact Contextual Representation for Conformer Transducer Speech Recognition Systems
Current ASR systems are mainly trained and evaluated at the utterance level.
Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection
LiDAR and Radar are two complementary sensing approaches in that LiDAR specializes in capturing an object's 3D shape while Radar provides longer detection ranges as well as velocity hints.
Use of Speech Impairment Severity for Dysarthric Speech Recognition
A key challenge in dysarthric speech recognition is the speaker-level diversity attributed to both speaker-identity associated factors such as gender, and speech impairment severity.
Deep Unrestricted Document Image Rectification
To our best knowledge, this is the first learning-based method for the rectification of unrestricted document images.
Ranked #1 on
Local Distortion
on DocUNet
Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition
Experiments conducted on the UASpeech dysarthric and DementiaBank Pitt elderly speech corpora suggest TDNN and Conformer ASR systems integrated domain adapted wav2vec2. 0 models consistently outperform the standalone wav2vec2. 0 models by statistically significant WER reductions of 8. 22% and 3. 43% absolute (26. 71% and 15. 88% relative) on the two tasks respectively.
Confidence Score Based Speaker Adaptation of Conformer Speech Recognition Systems
Practical application of unsupervised model-based speaker adaptation techniques to data intensive end-to-end ASR systems is hindered by the scarcity of speaker-level data and performance sensitivity to transcription errors.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Recurrent Generic Contour-based Instance Segmentation with Progressive Learning
It maintains a single estimate of the contour that is progressively deformed toward the object boundary.
Ranked #1 on
Semantic Contour Prediction
on Sbd val
OA-BEV: Bringing Object Awareness to Bird's-Eye-View Representation for Multi-Camera 3D Object Detection
To this end, we propose OA-BEV, a network that can be plugged into the BEV-based 3D object detection framework to bring out the objects by incorporating object-aware pseudo-3D features and depth features.
3DPPE: 3D Point Positional Encoding for Transformer-based Multi-Camera 3D Object Detection
Although 3D measurements are not available at the inference time of monocular 3D object detection, 3DPPE uses predicted depth to approximate the real point positions.
3DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformers
Although 3D measurements are not available at the inference time of monocular 3D object detection, 3DPPE uses predicted depth to approximate the real point positions.
Adversarial Data Augmentation Using VAE-GAN for Disordered Speech Recognition
After LHUC speaker adaptation, the best system using VAE-GAN based augmentation produced an overall WER of 27. 78% on the UASpeech test set of 16 dysarthric speakers, and the lowest published WER of 57. 31% on the subset of speakers with "Very Low" intelligibility.
Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care and to delay further progression.
Geometric Representation Learning for Document Image Rectification
In document image rectification, there exist rich geometric constraints between the distorted image and the ground truth one.
CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation
In this work, we formulate the cross-modal interaction as a bidirectional knowledge distillation problem.
3D Action Recognition
Few-Shot Skeleton-Based Action Recognition
+3
Confidence Score Based Conformer Speaker Adaptation for Speech Recognition
A key challenge for automatic speech recognition (ASR) systems is to model the speaker level variability.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Two-pass Decoding and Cross-adaptation Based System Combination of End-to-end Conformer and Hybrid TDNN ASR Systems
Fundamental modelling differences between hybrid and end-to-end (E2E) automatic speech recognition (ASR) systems create large diversity and complementarity among them.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detection
Early diagnosis of Alzheimer's disease (AD) is crucial in facilitating preventive care to delay further progression.
Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition
Articulatory features are inherently invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition (ASR) systems designed for normal speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer
For another, we devise Language Conditioned Vision Transformer that removes external fusion modules and reuses the uni-modal ViT for vision-language fusion at the intermediate layers.
Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition
Despite the rapid progress of automatic speech recognition (ASR) technologies targeting normal speech, accurate recognition of dysarthric and elderly speech remains highly challenging tasks to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Audio-visual multi-channel speech separation, dereverberation and recognition
Despite the rapid advance of automatic speech recognition (ASR) technologies, accurate recognition of cocktail party speech characterised by the interference from overlapping speakers, background noise and room reverberation remains a highly challenging task to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Neural Architecture Search For LF-MMI Trained Time Delay Neural Networks
State-of-the-art automatic speech recognition (ASR) system development is data and computation intensive.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusion
However, this approach often suffers from the mismatch between the resolution of point clouds and RGB images, leading to sub-optimal performance.
DocScanner: Robust Document Image Rectification with Progressive Learning
The iterative refinements make DocScanner converge to a robust and superior rectification performance, while the lightweight recurrent architecture ensures the running efficiency.
DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction
Specifically, DocTr consists of a geometric unwarping transformer and an illumination correction transformer.
From Multi-View to Hollow-3D: Hallucinated Hollow-3D R-CNN for 3D Object Detection
To this end, in this work, we regard point clouds as hollow-3D data and propose a new architecture, namely Hallucinated Hollow-3D R-CNN ($\text{H}^2$3D R-CNN), to address the problem of 3D object detection.
Neighbor-Vote: Improving Monocular 3D Object Detection through Neighbor Distance Voting
As cameras are increasingly deployed in new application domains such as autonomous driving, performing 3D object detection on monocular images becomes an important task for visual scene understanding.
Weakly Supervised Temporal Adjacent Network for Language Grounding
To this end, we introduce a novel weakly supervised temporal adjacent network (WSTAN) for temporal language grounding.
Multi-Modal 3D Object Detection in Autonomous Driving: a Survey
In this survey, we first introduce the background of popular sensors used for self-driving, their data properties, and the corresponding object detection algorithms.
TransVG: End-to-End Visual Grounding with Transformers
In this paper, we present a neat yet effective transformer-based framework for visual grounding, namely TransVG, to address the task of grounding a language query to the corresponding region onto an image.
Ranked #18 on
Referring Expression Comprehension
on RefCOCO
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection
3D object detection is receiving increasing attention from both industry and academia thanks to its wide applications in various fields.
Ranked #2 on
3D Object Detection
on KITTI Cars Easy val
Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection
In this paper, we take a slightly different viewpoint -- we find that precise positioning of raw points is not essential for high performance 3D object detection and that the coarse voxel granularity can also offer sufficient detection accuracy.
Ranked #4 on
3D Object Detection
on KITTI Cars Moderate val
Masked Contrastive Representation Learning for Reinforcement Learning
During inference, the CNN encoder and the policy network are used to take actions, and the Transformer module is discarded.
Single Shot Video Object Detector
Single shot detectors that are potentially faster and simpler than two-stage detectors tend to be more applicable to object detection in videos.
Adaptive Offline Quintuplet Loss for Image-Text Matching
For each image or text anchor in a training mini-batch, the model is trained to distinguish between a positive and the most confusing negative of the anchor mined from the mini-batch (i. e. online hard negative).
Relation Distillation Networks for Video Object Detection
In this paper, we introduce a new design to capture the interactions across the objects in spatio-temporal context.
