Search Results for author:
Found 42 papers, 16 papers with code
Steering No-Regret Agents in MFGs under Model Uncertainty
Our work presents an effective framework for steering agents behaviors in large-population systems under uncertainty.
Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting
Existing 3D OCF approaches struggle to predict plausible spatial details for movable objects and suffer from slow inference speeds due to neglecting the bias and uneven distribution of changing occupancy states in both space and time.
MimicTalk: Mimicking a personalized and expressive 3D talking face in minutes
To be specific, (1) we first come up with a person-agnostic 3D TFG model as the base model and propose to adapt it into a specific identity; (2) we propose a static-dynamic-hybrid adaptation pipeline to help the model learn the personalized static appearance and facial dynamic features; (3) To generate the facial motion of the personalized talking style, we propose an in-context stylized audio-to-motion model that mimics the implicit talking style provided in the reference video without information loss by an explicit style representation.
MulliVC: Multi-lingual Voice Conversion With Cycle Consistency
Specifically, each training step of MulliVC contains three substeps: In step one the model is trained with monolingual speech data; then, steps two and three take inspiration from back translation, construct a cyclical process to disentangle the timbre and other information (content, prosody, and other language-related information) in the absence of multi-lingual data from the same speaker.
Knowledge-Guided Prompt Learning for Lifespan Brain MR Image Segmentation
Automatic and accurate segmentation of brain MR images throughout the human lifespan into tissue and structure is crucial for understanding brain development and diagnosing diseases.
Learning to Steer Markovian Agents under Model Uncertainty
We introduce a model-based non-episodic Reinforcement Learning (RL) formulation for our steering problem.
Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching
By employing a non-autoregressive vector field estimator based on a feed-forward transformer and channel-level cross-modal feature fusion with strong temporal alignment, our model generates audio that is highly synchronized with the input video.
Ranked #4 on
Video-to-Sound Generation
on VGG-Sound
LoCI-DiffCom: Longitudinal Consistency-Informed Diffusion Model for 3D Infant Brain Image Completion
The infant brain undergoes rapid development in the first few years after birth. Compared to cross-sectional studies, longitudinal studies can depict the trajectories of infants brain development with higher accuracy, statistical power and flexibility. However, the collection of infant longitudinal magnetic resonance (MR) data suffers a notorious dropout problem, resulting in incomplete datasets with missing time points.
Cas-DiffCom: Cascaded diffusion model for infant longitudinal super-resolution 3D medical image completion
Early infancy is a rapid and dynamic neurodevelopmental period for behavior and neurocognition.
Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RL
We study the sample complexity of reinforcement learning (RL) in Mean-Field Games (MFGs) with model-based function approximation that requires strategic exploration to find a Nash Equilibrium policy.
Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis
One-shot 3D talking portrait generation aims to reconstruct a 3D avatar from an unseen image, and then animate it with a reference video or audio to generate a talking portrait video.
On Robust Wasserstein Barycenter: The Model and Algorithm
Firstly, we improve efficiency through model reducing; we reduce RWB as an augmented Wasserstein barycenter problem, which works for both fixed-RWB and free-RWB.
Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications
Furthermore, the standardized evaluation protocol for preset multiple tasks is also provided to compare the performance of all the proposed baselines on present and future occupancy estimation with respect to objects of interest in autonomous driving scenarios.
Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation
Finally, we use LLMs to augment and transform a large amount of audio-label data into audio-text datasets to alleviate the problem of scarcity of temporal data.
Ranked #3 on
Audio Generation
on AudioCaps
(FD metric)
On the Statistical Efficiency of Mean-Field Reinforcement Learning with General Function Approximation
In this paper, we study the fundamental statistical efficiency of Reinforcement Learning in Mean-Field Control (MFC) and Mean-Field Game (MFG) with general model-based function approximation.
GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation
Recently, neural radiance field (NeRF) has become a popular rendering technique in this field since it could achieve high-fidelity and 3D-consistent talking face generation with a few-minute-long training video.
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
In this work, we propose a multi-modal AI system named AudioGPT, which complements LLMs (i. e., ChatGPT) with 1) foundation models to process complex audio information and solve numerous understanding and generation tasks; and 2) the input/output interface (ASR, TTS) to support spoken dialogue.
Company Competition Graph
Based on the output from our system, we construct a knowledge graph with more than 700 nodes and 1200 edges.
Online Neural Path Guiding with Normalized Anisotropic Spherical Gaussians
The variance reduction speed of physically-based rendering is heavily affected by the adopted importance sampling technique.
Structure-aware registration network for liver DCE-CT images
In addition, such paired organ segmentations are not always available in DCE-CT images due to the flow of contrast agents.
Robust Knowledge Transfer in Tiered Reinforcement Learning
In this paper, we study the Tiered Reinforcement Learning setting, a parallel transfer learning framework, where the goal is to transfer knowledge from the low-tier (source) task to the high-tier (target) task to reduce the exploration risk of the latter while solving the two tasks in parallel.
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models
Its application to audio still lags behind for two main reasons: the lack of large-scale datasets with high-quality text-audio pairs, and the complexity of modeling long continuous audio data.
Ranked #7 on
Audio Generation
on AudioCaps
(FD metric)
Coresets for Wasserstein Distributionally Robust Optimization Problems
Though it is challenging to obtain a conventional coreset for \textsf{WDRO} due to the uncertainty issue of ambiguous data, we show that we can compute a ``dual coreset'' by using the strong duality property of \textsf{WDRO}.
Learning towards Synchronous Network Memorizability and Generalizability for Continual Segmentation across Multiple Sites
In clinical practice, a segmentation network is often required to continually learn on a sequential data stream from multiple sites rather than a consolidated set, due to the storage cost and privacy restriction.
Tiered Reinforcement Learning: Pessimism in the Face of Uncertainty and Constant Regret
We propose a new learning framework that captures the tiered structure of many real-world user-interaction applications, where the users can be divided into two groups based on their different tolerance on exploration risks and should be treated separately.
Kernel Extreme Learning Machine Optimized by the Sparrow Search Algorithm for Hyperspectral Image Classification
To improve the classification performance and generalization ability of the hyperspectral image classification algorithm, this paper uses Multi-Scale Total Variation (MSTV) to extract the spectral features, local binary pattern (LBP) to extract spatial features, and feature superposition to obtain the fused features of hyperspectral images.
Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality
Deployment efficiency is an important criterion for many real-world applications of reinforcement learning (RL).
A Novel Sequential Coreset Method for Gradient Descent Algorithms
A wide range of optimization problems arising in machine learning can be solved by gradient descent algorithms, and a central question in this area is how to efficiently compress a large-scale dataset so as to reduce the computational complexity.
A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes
In this work, we first propose novel identification methods for OPE in POMDPs with latent confounders, by introducing bridge functions that link the target policy's value and the observed data distribution.
On the Convergence Rate of Off-Policy Policy Optimization Methods with Density-Ratio Correction
In this paper, we study the convergence properties of off-policy policy improvement algorithms with state-action density ratio correction under function approximation setting, where the objective function is formulated as a max-max-min optimization problem.
Is Simple Uniform Sampling Effective for Center-Based Clustering with Outliers: When and Why?
Real-world datasets often contain outliers, and the presence of outliers can make the clustering problems to be much more challenging.
WeightNet: Revisiting the Design Space of Weight Networks
WeightNet is easy and memory-conserving to train, on the kernel space instead of the feature space.
Defending SVMs against Poisoning Attacks: the Hardness and DBSCAN Approach
For the data sanitization defense, we link it to the intrinsic dimensionality of data; in particular, we provide a sampling theorem in doubling metrics for explaining the effectiveness of DBSCAN (as a density-based outlier removal method) for defending against poisoning attacks.
The Effectiveness of Johnson-Lindenstrauss Transform for High Dimensional Optimization With Adversarial Outliers, and the Recovery
We focus on two fundamental optimization problems: {\em SVM with outliers} and {\em $k$-center clustering with outliers}.
Minimax Value Interval for Off-Policy Evaluation and Policy Optimization
By slightly altering the derivation of previous methods (one from each style; Uehara et al., 2020), we unify them into a single value interval that comes with a special type of double robustness: when either the value-function or the importance-weight class is well specified, the interval is valid and its length quantifies the misspecification of the other class.
Minimax Weight and Q-Function Learning for Off-Policy Evaluation
We provide theoretical investigations into off-policy evaluation in reinforcement learning using function approximators for (marginalized) importance weights and value functions.
From Importance Sampling to Doubly Robust Policy Gradient
We show that on-policy policy gradient (PG) and its variance reduction variants can be derived by taking finite difference of function evaluations supplied by estimators from the importance sampling (IS) family for off-policy evaluation (OPE).
The Effectiveness of Uniform Sampling for Center-Based Clustering with Outliers
The experiments suggest that the uniform sampling method can achieve comparable clustering results with other existing methods, but greatly reduce the running times.
Learning to Cluster for Proposal-Free Instance Segmentation
We utilize the most fundamental property of instance labeling -- the pairwise relationship between pixels -- as the supervision to formulate the learning objective, then apply it to train a fully convolutional network (FCN) for learning to perform pixel-wise clustering.
Ranked #14 on
Lane Detection
on TuSimple
Training Shallow and Thin Networks for Acceleration via Knowledge Distillation with Conditional Adversarial Networks
There is an increasing interest on accelerating neural networks for real-time applications.
Robust Lane Tracking with Multi-mode Observation Model and Particle Filtering
Automatic lane tracking involves estimating the underlying signal from a sequence of noisy signal observations.
Towards Full Automated Drive in Urban Environments: A Demonstration in GoMentum Station, California
Each year, millions of motor vehicle traffic accidents all over the world cause a large number of fatalities, injuries and significant material loss.
