Search Results for author:
Found 78 papers, 32 papers with code
Parallax-Tolerant Image Stitching with Epipolar Displacement Field
Large parallax image stitching is a challenging task.
PointOBB: Learning Oriented Object Detection via Single Point Supervision
Single point-supervised object detection is gaining attention due to its cost-effectiveness.
Melody-conditioned lyrics generation via fine-tuning language model and its evaluation with ChatGPT
We leverage character-level language models for syllable-level lyrics generation from symbolic melody.
LiveChat: Video Comment Generation from Audio-Visual Multimodal Contexts
Live commenting on video, a popular feature of live streaming platforms, enables viewers to engage with the content and share their comments, reactions, opinions, or questions with the streamer or other viewers while watching the video or live stream.
Music- and Lyrics-driven Dance Synthesis
To complement it, we introduce JustLMD, a new multimodal dataset of 3D dance motion with music and lyrics.
Towards Integrated Traffic Control with Operating Decentralized Autonomous Organization
Furthermore, an operation algorithm is proposed regarding the issue of structural rigidity in DAO.
Modify Training Directions in Function Space to Reduce Generalization Error
We propose theoretical analyses of a modified natural gradient descent method in the neural network function space based on the eigendecompositions of neural tangent kernel and Fisher information matrix.
ExposureDiffusion: Learning to Expose for Low-light Image Enhancement
Different from a vanilla diffusion model that has to perform Gaussian denoising, with the injected physics-based exposure model, our restoration process can directly start from a noisy image instead of pure noise.
Ranked #1 on
Image Denoising
on Image Denoising on SID x300
Beyond Learned Metadata-based Raw Image Reconstruction
Besides, we propose a novel design of the context model, which can better predict the order masks of encoding/decoding based on both the sRGB image and the masks of already processed features.
Controllable Lyrics-to-Melody Generation
Lyrics-to-melody generation is an interesting and challenging topic in AI music research field.
MIPI 2023 Challenge on RGB+ToF Depth Completion: Methods and Results
To evaluate the performance of different depth completion methods, we organized an RGB+sparse ToF depth completion competition.
Robust Andrew's sine estimate adaptive filtering
Two Andrew's sine estimator (ASE)-based robust adaptive filtering algorithms are proposed in this brief.
Frame-Level Multi-Label Playing Technique Detection Using Multi-Scale Network and Self-Attention Mechanism
Instrument playing technique (IPT) is a key element of musical presentation.
Instrument Playing Technique Detection
Multi-Label Classification
Emotionally Enhanced Talking Face Generation
To mitigate this, we build a talking face generation framework conditioned on a categorical emotion to generate videos with appropriate expressions, making them more realistic and convincing.
Ranked #1 on
Talking Face Generation
on CREMA-D
Augmented smartphone bilirubinometer enabled by a mobile app that turns smartphone into multispectral imager
This study highlights the potential of SpeCamX to improve the prediction of bio-chromophores, and its ability to transform an ordinary smartphone into a powerful medical tool without the need for additional investments or expertise.
Backdoor Attacks Against Deep Image Compression via Adaptive Frequency Trigger
Extensive experiments show that with our trained trigger injection models and simple modification of encoder parameters (of the compression model), the proposed attack can successfully inject several backdoors with corresponding triggers in a single image compression model.
Raw Image Reconstruction with Learned Compact Metadata
While raw images exhibit advantages over sRGB images (e. g., linearity and fine-grained quantization level), they are not widely used by common users due to the large storage requirements.
Deep Attention-Based Alignment Network for Melody Generation from Incomplete Lyrics
We propose a deep attention-based alignment network, which aims to automatically predict lyrics and melody with given incomplete lyrics as input in a way similar to the music creation of humans.
Emotional Talking Faces: Making Videos More Expressive and Realistic
Lip synchronization and talking face generation have gained a specific interest from the research community with the advent and need of digital communication in different fields.
Phase-Shifting Coder: Predicting Accurate Orientation in Oriented Object Detection
With the vigorous development of computer vision, oriented object detection has gradually been featured.
SoccerNet 2022 Challenges Results
The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team.
Playing Technique Detection by Fusing Note Onset Information in Guzheng Performance
Because each Guzheng playing technique is applied to a note, a dedicated onset detector is trained to divide an audio into several notes and its predictions are fused with frame-wise IPT predictions.
Widely Used and Fast De Novo Drug Design by a Protein Sequence-Based Reinforcement Learning Model
In the generative model, one of the reward components, a binding affinity predictor, is based on 1D protein sequence and molecular SMILES.
Study of General Robust Subband Adaptive Filtering
In this paper, we propose a general robust subband adaptive filtering (GR-SAF) scheme against impulsive noise by minimizing the mean square deviation under the random-walk model with individual weight uncertainty.
Interpretable Melody Generation from Lyrics with Discrete-Valued Adversarial Training
Generating melody from lyrics is an interesting yet challenging task in the area of artificial intelligence and music.
Design and Analysis of Robust Resilient Diffusion over Multi-Task Networks Against Byzantine Attacks
This paper studies distributed diffusion adaptation over clustered multi-task networks in the presence of impulsive interferences and Byzantine attacks.
Chemical transformer compression for accelerating both training and inference of molecular modeling
Transformer models have been developed in molecular science with excellent performance in applications including quantitative structure-activity relationship (QSAR) and virtual screening (VS).
Sparsity-Aware Robust Normalized Subband Adaptive Filtering algorithms based on Alternating Optimization
This paper proposes a unified sparsity-aware robust normalized subband adaptive filtering (SA-RNSAF) algorithm for identification of sparse systems under impulsive noise.
HarmoF0: Logarithmic Scale Dilated Convolution For Pitch Estimation
Sounds, especially music, contain various harmonic components scattered in the frequency dimension.
Lightweight Bimodal Network for Single-Image Super-Resolution via Symmetric CNN and Recursive Transformer
Single-image super-resolution (SISR) has achieved significant breakthroughs with the development of deep learning.
Towards Robust Rain Removal Against Adversarial Attacks: A Comprehensive Benchmark Analysis and Beyond
Finally, we examine various types of adversarial attacks that are specific to deraining problems and their effects on both human and machine vision tasks, including 1) rain region attacks, adding perturbations only in the rain regions to make the perturbations in the attacked rain images less visible; 2) object-sensitive attacks, adding perturbations only in regions near the given objects.
Conjugate Gradient Adaptive Learning with Tukey's Biweight M-Estimate
We propose a novel M-estimate conjugate gradient (CG) algorithm, termed Tukey's biweight M-estimate CG (TbMCG), for system identification in impulsive noise environments.
Recent Advances and Challenges in Deep Audio-Visual Correlation Learning
Audio-visual correlation learning aims to capture essential correspondences and understand natural phenomena between audio and video.
DEEPCHORUS: A Hybrid Model of Multi-scale Convolution and Self-attention for Chorus Detection
Chorus detection is a challenging problem in musical signal processing as the chorus often repeats more than once in popular songs, usually with rich instruments and complex rhythm forms.
Feature Distillation Interaction Weighting Network for Lightweight Image Super-Resolution
Convolutional neural networks based single-image super-resolution (SISR) has made great progress in recent years.
Variational Autoencoder with CCA for Audio-Visual Cross-Modal Retrieval
On the one hand, audio encoder and visual encoder separately encode audio data and visual data into two different latent spaces.
Active noise control techniques for nonlinear systems
Most of the literature focuses on the development of the linear active noise control (ANC) techniques.
A survey on active noise control techniques -- Part I: Linear systems
Active noise control (ANC) is an effective way for reducing the noise level in electroacoustic or electromechanical systems.
CRNNTL: convolutional recurrent neural network and transfer learning for QSAR modelling
Our strategy takes advantages of both convolutional and recurrent neural networks for feature extraction, as well as the data augmentation method.
FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation
Specifically, FBSNet employs a symmetrical encoder-decoder structure with two branches, semantic information branch and spatial detail branch.
Study of Proximal Normalized Subband Adaptive Algorithm for Acoustic Echo Cancellation
In addition, an adaptive approach for the choice of the thresholding parameter in the proximal step is also proposed based on the minimization of the mean square deviation.
Interpretable Visual Understanding with Cognitive Attention Network
While image understanding on recognition-level has achieved remarkable advancements, reliable visual scene understanding requires comprehensive image understanding on recognition-level but also cognition-level, which calls for exploiting the multi-source information as well as learning different levels of understanding and extensive commonsense knowledge.
Multi-TimeLine Summarization (MTLS): Improving Timeline Summarization by Generating Multiple Summaries
In this paper, we address a novel task, Multiple TimeLine Summarization (MTLS), which extends the flexibility and versatility of Time-Line Summarization (TLS).
Enhancing Social Relation Inference with Concise Interaction Graph and Discriminative Scene Representation
There has been a recent surge of research interest in attacking the problem of social relation inference based on images.
Adversarial Learning with Mask Reconstruction for Text-Guided Image Inpainting
In this paper, we propose an adversarial learning framework with mask reconstruction (ALMR) for image inpainting with textual guidance, which consists of a two-stage generator and dual discriminators.
Lattice partition recovery with dyadic CART
We study piece-wise constant signals corrupted by additive Gaussian noise over a $d$-dimensional lattice.
Few-Data Guided Learning Upon End-to-End Point Cloud Network for 3D Face Recognition
Without fine-tuning on the test set, the Rank-1 Recognition Rate (RR1) is achieved as follows: 98. 85% on FRGC v2. 0 dataset and 99. 33% on Bosphorus dataset, which proves the effectiveness and the potentiality of our method.
Leaning Compact and Representative Features for Cross-Modality Person Re-Identification
This paper pays close attention to the cross-modality visible-infrared person re-identification (VI Re-ID) task, which aims to match pedestrian samples between visible and infrared modes.
Cross-Modality Person Re-identification
Knowledge Distillation
+1
Hierarchical Deep CNN Feature Set-Based Representation Learning for Robust Cross-Resolution Face Recognition
(i) To learn more robust and discriminative features, we desire to adaptively fuse the contextual features from different layers.
Lightweight Image Super-Resolution with Multi-scale Feature Interaction Network
Recently, the single image super-resolution (SISR) approaches with deep and complex convolutional neural network structures have achieved promising performance.
MSCFNet: A Lightweight Network With Multi-Scale Context Fusion for Real-Time Semantic Segmentation
In recent years, how to strike a good trade-off between accuracy and inference speed has become the core issue for real-time semantic segmentation applications, which plays a vital role in real-world scenarios such as autonomous driving systems and drones.
Generalized non-stationary bandits
On top of the switching bandit problem (\textbf{Case a}), we are interested in three concrete examples: (\textbf{b}) the means of the arms are local polynomials, (\textbf{c}) the means of the arms are locally smooth, and (\textbf{d}) the gaps of the arms have a bounded number of inflexion points and where the highest arm mean cannot vary too much in a short range.
Optimal network online change point localisation
The goal is to detect the change point as quickly as possible, if it exists, subject to a constraint on the number or probability of false alarms.
MusicTM-Dataset for Joint Representation Learning among Sheet Music, Lyrics, and Musical Audio
This work present a music dataset named MusicTM-Dataset, which is utilized in improving the representation learning ability of different types of cross-modal retrieval (CMR).
Functional Linear Regression with Mixed Predictors
We derive finite sample theoretical guarantees and show that the excess prediction risk of our estimator is minimax optimal.
Statistics Theory Methodology Statistics Theory
mask-Net: Learning Context Aware Invariant Features using Adversarial Forgetting (Student Abstract)
Training a robust system, e. g., Speech to Text (STT), requires large datasets.
Automatic Neural Lyrics and Melody Composition
In this paper, we propose a technique to address the most challenging aspect of algorithmic songwriting process, which enables the human community to discover original lyrics, and melodies suitable for the generated lyrics.
Conditional Hybrid GAN for Sequence Generation
Conditional sequence generation aims to instruct the generation procedure by conditioning the model with additional context information, which is a self-supervised learning issue (a form of unsupervised learning with supervision information from data itself).
Sparsity-Aware SSAF Algorithm with Individual Weighting Factors for Acoustic Echo Cancellation
In this paper, we propose and analyze the sparsity-aware sign subband adaptive filtering with individual weighting factors (S-IWF-SSAF) algorithm, and consider its application in acoustic echo cancellation (AEC).
PAI-BPR: Personalized Outfit Recommendation Scheme with Attribute-wise Interpretability
The rise in the fashion industry and its effect on social influencing have made outfit compatibility a need.
Ranked #1 on
Preference Mapping
on IQOON3000
Unsupervised Generative Adversarial Alignment Representation for Sheet music, Audio and Lyrics
In this paper, we propose an unsupervised generative adversarial alignment representation (UGAAR) model to learn deep discriminative representations shared across three major musical modalities: sheet music, lyrics, and audio, where a deep neural network based architecture on three branches is jointly trained.
End-to-end Named Entity Recognition from English Speech
Named entity recognition (NER) from text has been a widely studied problem and usually extracts semantic information from text.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
C3VQG: Category Consistent Cyclic Visual Question Generation
In this paper, we try to exploit the different visual cues and concepts in an image to generate questions using a variational autoencoder (VAE) without ground-truth answers.
Investigation of Singing Voice Separation for Singing Voice Detection in Polyphonic Music
Singing voice detection (SVD), to recognize vocal parts in the song, is an essential task in music information retrieval (MIR).
Text2FaceGAN: Face Generation from Fine Grained Textual Descriptions
We then model the highly multi-modal problem of text to face generation as learning the conditional distribution of faces (conditioned on text) in same latent space.
Conditional LSTM-GAN for Melody Generation from Lyrics
Melody generation from lyrics has been a challenging research issue in the field of artificial intelligence and music, which enables to learn and discover latent relationship between interesting lyrics and accompanying melody.
Personalized Music Recommendation with Triplet Network
Since many online music services emerged in recent years so that effective music recommendation systems are desirable.
Deep Triplet Neural Networks with Cluster-CCA for Audio-Visual Cross-modal Retrieval
In particular, two significant contributions are made: i) a better representation by constructing deep triplet neural network with triplet loss for optimal projections can be generated to maximize correlation in the shared subspace.
Audio-Visual Embedding for Cross-Modal MusicVideo Retrieval through Supervised Deep CCA
ii) We propose an end-to-end deep model for cross-modal audio-visual learning where S-DCCA is trained to learn the semantic correlation between audio and visual modalities.
Social Influence-based Attentive Mavens Mining and Aggregative Representation Learning for Group Recommendation
To tackle the issue of preference aggregation for group recommendation, we propose a novel attentive aggregation representation learning method based on sociological theory for group recommendation, namely SIAGR (short for "Social Influence-based Attentive Group Recommendation"), which takes attention mechanisms and the popular method (BERT) as the aggregation representation for group profile modeling.
Ensemble Super-Resolution with A Reference Dataset
In this paper, we present a simple but effective single image SR method based on ensemble learning, which can produce a better performance than that could be obtained from any of SR methods to be ensembled (or called component super-resolvers).
Deep Knowledge Tracing and Dynamic Student Classification for Knowledge Tracing
In Intelligent Tutoring System (ITS), tracing the student's knowledge state during learning has been studied for several decades in order to provide more supportive learning instructions.
Context-Patch Face Hallucination Based on Thresholding Locality-constrained Representation and Reproducing Learning
To this end, this study incorporates the contextual information of image patch and proposes a powerful and efficient context-patch based face hallucination approach, namely Thresholding Locality-constrained Representation and Reproducing learning (TLcR-RL).
Deep CNN Denoiser and Multi-layer Neighbor Component Embedding for Face Hallucination
Most of the current face hallucination methods, whether they are shallow learning-based or deep learning-based, all try to learn a relationship model between Low-Resolution (LR) and High-Resolution (HR) spaces with the help of a training set.
Category-Based Deep CCA for Fine-Grained Venue Discovery from Multimodal Data
Given a photo as input, this model performs (i) exact venue search (find the venue where the photo was taken), and (ii) group venue search (find relevant venues with the same category as that of the photo), by the cross-modal correlation between the input photo and textual description of venues.
Towards Deep Modeling of Music Semantics using EEG Regularizers
Modeling of music audio semantics has been previously tackled through learning of mappings from audio data to high-level tags or latent unsupervised spaces.
How Many Communities Are There?
Stochastic blockmodels and variants thereof are among the most widely used approaches to community detection for social networks and relational data.
APPLE: Approximate Path for Penalized Likelihood Estimators
In high-dimensional data analysis, penalized likelihood estimators are shown to provide superior results in both variable selection and parameter estimation.
