Search Results for author:
Found 47 papers, 12 papers with code
Ming-Omni: A Unified Multimodal Model for Perception and Generation
We propose Ming-Omni, a unified multimodal model capable of processing images, text, audio, and video, while demonstrating strong proficiency in both speech and image generation.
Multimodal Transformers are Hierarchical Modal-wise Heterogeneous Graphs
In this work, from the perspective of efficiency optimization, we propose and prove that MulTs are hierarchical modal-wise heterogeneous graphs (HMHGs), and we introduce the graph-structured representation pattern of MulTs.
Water Quality Data Imputation via A Fast Latent Factorization of Tensors with PID-based Optimizer
Water quality data can supply a substantial decision support for water resources utilization and pollution prevention.
Generative Zero-Shot Composed Image Retrieval
Composed Image Retrieval (CIR) is a vision-language task utilizing queries comprising images and textual descriptions to achieve precise image retrieval.
IV-tuning: Parameter-Efficient Transfer Learning for Infrared-Visible Tasks
By fine-tuning approximately 3% of the backbone parameters, IV-tuning outperforms full fine-tuning across various baselines in infrared-visible semantic segmentation and object detection, as well as previous state-of-the-art methods.
Confidence Diagram of Nonparametric Ranking for Uncertainty Assessment in Large Language Models Evaluation
We consider the inference for the ranking of large language models (LLMs).
OmniCreator: Self-Supervised Unified Generation with Universal Editing
On the other hand, when presented with a text prompt only, OmniCreator becomes generative, producing high-quality video as a result of the semantic correspondence learned.
SEAL: Semantic Attention Learning for Long Video Representation
Long video understanding presents challenges due to the inherent high computational complexity and redundant temporal information.
Structured Dialogue System for Mental Health: An LLM Chatbot Leveraging the PM+ Guidelines
The Structured Dialogue System, referred to as SuDoSys, is an innovative Large Language Model (LLM)-based chatbot designed to provide psychological counseling.
An End-To-End Stuttering Detection Method Based On Conformer And BILSTM
In the SLT 2024 Stuttering Speech Challenge based on the AS-70 dataset [1], our model improved the mean F1 score by 24. 8% compared to the baseline method and achieved first place.
A Tale of Two Cities: Pessimism and Opportunism in Offline Dynamic Pricing
We address this challenge by framing the problem to a partial identification framework.
Investigation of unsupervised and supervised hyperspectral anomaly detection
We later utilized a supervised classifier to determine the weights of a voting ensemble, creating a hybrid of heterogeneous unsupervised HS-AD algorithms with a supervised classifier in a model stacking, which improved detection accuracy.
Representation learning with CGAN for casual inference
Conditional Generative Adversarial Nets (CGAN) is often used to improve conditional image generation performance.
Perceiver-Prompt: Flexible Speaker Adaptation in Whisper for Chinese Disordered Speech Recognition
Disordered speech recognition profound implications for improving the quality of life for individuals afflicted with, for example, dysarthria.
Automatic Assessment of Dysarthria Using Audio-visual Vowel Graph Attention Network
Currently, research on the automatic assessment of dysarthria primarily focuses on two approaches: one that utilizes expert features combined with machine learning, and the other that employs data-driven deep learning methods to extract representations.
Private Optimal Inventory Policy Learning for Feature-based Newsvendor with Unknown Demand
By leveraging the structure of the newsvendor problem, we attain a faster excess population risk bound compared to that obtained from an indiscriminate application of existing results for general nonsmooth convex loss.
FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs
However, owing to the nature of their training process, these models have the potential to 1) propagate or amplify societal biases in the training data and 2) learn to rely on spurious features.
Action Reimagined: Text-to-Pose Video Editing for Dynamic Human Actions
While existing video editing tasks are limited to changes in attributes, backgrounds, and styles, our method aims to predict open-ended human action changes in video.
An Audio-textual Diffusion Model For Converting Speech Signals Into Ultrasound Tongue Imaging Data
In this model, the inherent acoustic characteristics of individuals related to the tongue motion details are encoded by using wav2vec 2. 0, while the ASR transcriptions related to the universality of tongue motions are encoded by using BERT.
Off-policy Evaluation in Doubly Inhomogeneous Environments
This work aims to study off-policy evaluation (OPE) under scenarios where two key reinforcement learning (RL) assumptions -- temporal stationarity and individual homogeneity are both violated.
Deep learning radiomics for assessment of gastroesophageal varices in people with compensated advanced chronic liver disease
Validation analysis revealed that the AUCs of DLRP were 0. 91 for GEV (95% CI 0. 90 to 0. 93, p < 0. 05) and 0. 88 for HRV (95% CI 0. 86 to 0. 89, p < 0. 01), which were significantly and robustly better than canonical risk indicators, including the value of LSM and SSM.
Context-aware Domain Adaptation for Time Series Anomaly Detection
We formulate context sampling into the Markov decision process and exploit deep reinforcement learning to optimize the time series domain adaptation process via context sampling and design a tailored reward function to generate domain-invariant features that better align two domains for anomaly detection.
ProTeGe: Untrimmed Pretraining for Video Temporal Grounding by Video Temporal Grounding
We present ProTeGe as the first method to perform VTG-based untrimmed pretraining to bridge the gap between trimmed pretrained backbones and downstream VTG tasks.
Quantile Off-Policy Evaluation via Deep Conditional Generative Learning
Off-Policy evaluation (OPE) is concerned with evaluating a new target policy using offline data generated by a potentially different behavior policy.
Offline Reinforcement Learning for Human-Guided Human-Machine Interaction with Private Information
To tackle the distributional mismatch, we leverage the idea of pessimism and use our OPE method to develop an off-policy learning algorithm for finding a desirable policy pair for both Alice and Bob.
Denoising Self-attentive Sequential Recommendation
Transformer-based sequential recommenders are very powerful for capturing both short-term and long-term sequential item dependencies.
Do learned representations respect causal relationships?
Second, we apply NCINet to identify the causal relations between image representations of different pairs of attributes with known and unknown causal relations between the labels.
A Robust Statistical Analysis of the Role of Hydropower on the System Electricity Price and Price Volatility
Hydroelectric power (hydropower) is unique in that it can function as both a conventional source of electricity and as backup storage (pumped hydroelectric storage) for providing energy in times of high demand on the grid.
Investigation of Deep Neural Network Acoustic Modelling Approaches for Low Resource Accented Mandarin Speech Recognition
Meanwhile, approaches of multi-accent modelling including multi-style training, multi-accent decision tree state tying, DNN tandem and multi-level adaptive network (MLAN) tandem hidden Markov model (HMM) modelling are combined and compared in this paper.
Variational Auto-Encoder Based Variability Encoding for Dysarthric Speech Recognition
Dysarthric speech recognition is a challenging task due to acoustic variability and limited amount of available data.
Forecast-based Multi-aspect Framework for Multivariate Time-series Anomaly Detection
Today's cyber-world is vastly multivariate.
Analysis of animal-related electric outages using species distribution models and community science data
These flexible, species-specific estimates can allow future animal-indicators of grid reliability to be investigated in more diverse regions and ecological communities, providing a better understanding of the variation that exists in animal-outage relationship.
Role of Variable Renewable Energy Penetration on Electricity Price and its Volatility Across Independent System Operators in the United States
After correcting for temporal effects, we found an increase in VRE penetration is associated with decrease in system electricity price in all ISOs studied.
Enhanced Memory Network: The novel network structure for Symbolic Music Generation
Symbolic melodies generation is one of the essential tasks for automatic music generation.
Adversarial Representation Learning With Closed-Form Solvers
Adversarial representation learning aims to learn data representations for a target task while removing unwanted sensitive information at the same time.
Unsupervised Cross-Lingual Speech Emotion Recognition Using Pseudo Multilabel
However, cross-lingual SER remains a challenge in real-world applications due to a great difference between the source and target domain distributions.
FDN: Finite Difference Network with Hierarchical Convolutional Features for Text-independent Speaker Verification
For example, RawNet and RawNet2 extracted speaker's feature embeddings from waveforms automatically for recognizing their voice, which can vastly reduce the front-end computation and obtain state-of-the-art performance.
A Multi-level Acoustic Feature Extraction Framework for Transformer Based End-to-End Speech Recognition
The shallow stream is used to acquire traditional shallow features that is beneficial for the classification of phones or words while the deep stream is used to obtain utterance-level speaker-invariant deep features for improving the feature diversity.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Event2Graph: Event-driven Bipartite Graph for Multivariate Time-series Anomaly Detection
Modeling inter-dependencies between time-series is the key to achieve high performance in anomaly detection for multivariate time-series data.
Generalization bounds via distillation
This paper theoretically investigates the following empirical phenomenon: given a high-complexity network with poor generalization bounds, one can distill it into a network with nearly identical predictions but low complexity and vastly smaller generalization bounds.
Critical Risk Indicators (CRIs) for the electric power grid: A survey and discussion of interconnected effects
We also provide relevant critical risk indicators (CRIs) across diverse domains that may influence electric power grid risks, including climate, ecology, hydrology, finance, space weather, and agriculture.
Applications
Bayesian Learning for Deep Neural Network Adaptation
A key task for speech recognition systems is to reduce the mismatch between training and evaluation data that is often attributable to speaker differences.
Data Augmentation for End-to-end Code-switching Speech Recognition
Training a code-switching end-to-end automatic speech recognition (ASR) model normally requires a large amount of data, while code-switching data is often limited.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Resampling-based Confidence Intervals for Model-free Robust Inference on Optimal Treatment Regimes
We first study a smoothed robust estimator that directly targets the parameter corresponding to the Bayes decision rule for optimal treatment regimes estimation.
A Survey of Tuning Parameter Selection for High-dimensional Regression
Penalized (or regularized) regression, as represented by Lasso and its variants, has become a standard technique for analyzing high-dimensional data when the number of variables substantially exceeds the sample size.
PM-GANs: Discriminative Representation Learning for Action Recognition Using Partial-modalities
As a result, using partial data channels to build a full representation of multi-modalities is clearly desired.
A novel learning-based frame pooling method for Event Detection
Detecting complex events in a large video collection crawled from video websites is a challenging task.
