Search Results for author: Zeke Xie

Found 26 papers, 11 papers with code

Learning from Ambiguous Data with Hard Labels

no code implementations3 Jan 2025 Zeke Xie, Zheng He, Nan Lu, Lichen Bai, Bao Li, Shuo Yang, Mingming Sun, Ping Li

Real-world data often contains intrinsic ambiguity that the common single-hard-label annotation paradigm ignores.

A Simple and Efficient Baseline for Zero-Shot Generative Classification

no code implementations17 Dec 2024 Zipeng Qi, Buhua Liu, Shiyan Zhang, Bao Li, Zhiqiang Xu, Haoyi Xiong, Zeke Xie

While recent zero-shot diffusion-based classifiers have made performance advancement on benchmark datasets, they still suffered badly from extremely slow classification speed (e. g., ~1000 seconds per classifying single image on ImageNet).

Zero-Shot Learning

Zigzag Diffusion Sampling: Diffusion Models Can Self-Improve via Self-Reflection

1 code implementation14 Dec 2024 Lichen Bai, Shitong Shao, Zikai Zhou, Zipeng Qi, Zhiqiang Xu, Haoyi Xiong, Zeke Xie

Diffusion models, the most popular generative paradigm so far, can inject conditional information into the generation path to guide the latent towards desired directions.

Denoising

Bag of Design Choices for Inference of High-Resolution Masked Generative Transformer

2 code implementations16 Nov 2024 Shitong Shao, Zikai Zhou, Tian Ye, Lichen Bai, Zhiqiang Xu, Zeke Xie

Text-to-image diffusion models (DMs) develop at an unprecedented pace, supported by thorough theoretical exploration and empirical analysis.

Text Generation

Golden Noise for Diffusion Models: A Learning Framework

1 code implementation14 Nov 2024 Zikai Zhou, Shitong Shao, Lichen Bai, Zhiqiang Xu, Bo Han, Zeke Xie

With the prepared NPD as the training dataset, we trained a small \textit{noise prompt network}~(NPNet) that can directly learn to transform a random noise into a golden noise.

Pre-trained Molecular Language Models with Random Functional Group Masking

no code implementations3 Nov 2024 Tianhao Peng, Yuchen Li, Xuhong LI, Jiang Bian, Zeke Xie, Ning Sui, Shahid Mumtaz, Yanwu Xu, Linghe Kong, Haoyi Xiong

Recent advancements in computational chemistry have leveraged the power of trans-former-based language models, such as MoLFormer, pre-trained using a vast amount of simplified molecular-input line-entry system (SMILES) sequences, to understand and predict molecular properties and activities, a critical step in fields like drug discovery and materials science.

Computational chemistry Drug Discovery

IV-Mixed Sampler: Leveraging Image Diffusion Models for Enhanced Video Synthesis

1 code implementation5 Oct 2024 Shitong Shao, Zikai Zhou, Lichen Bai, Haoyi Xiong, Zeke Xie

While there have been rapid advancements in developing inference-heavy algorithms for improved image generation, relatively little work has explored inference scaling laws in video diffusion models (VDMs).

Text-to-Video Generation

Alignment of Diffusion Models: Fundamentals, Challenges, and Future

1 code implementation11 Sep 2024 Buhua Liu, Shitong Shao, Bao Li, Lichen Bai, Zhiqiang Xu, Haoyi Xiong, James Kwok, Sumi Helal, Zeke Xie

Diffusion models have emerged as the leading paradigm in generative modeling, excelling in various applications.

Channel-wise Influence: Estimating Data Influence for Multivariate Time Series

no code implementations27 Aug 2024 Muyao Wang, Zeke Xie, Bo Chen

To fill this gap, we propose a channel-wise influence function, which is the first method that can estimate the influence of different channels in MTS, utilizing a first-order gradient approximation that leverages the more informative average gradient of the data set.

Anomaly Detection Time Series

Not All Noises Are Created Equally:Diffusion Noise Selection and Optimization

no code implementations19 Jul 2024 Zipeng Qi, Lichen Bai, Haoyi Xiong, Zeke Xie

We are the first to hypothesize and empirically observe that the generation quality of diffusion models significantly depend on the noise inversion stability.

Converging Paradigms: The Synergy of Symbolic and Connectionist AI in LLM-Empowered Autonomous Agents

no code implementations11 Jul 2024 Haoyi Xiong, Zhiyuan Wang, Xuhong LI, Jiang Bian, Zeke Xie, Shahid Mumtaz, Anwer Al-Dulaimi, Laura E. Barnes

This article explores the convergence of connectionist and symbolic artificial intelligence (AI), from historical debates to contemporary advancements.

Decision Making Knowledge Graphs

SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior

no code implementations29 Mar 2024 Zhongrui Yu, Haoran Wang, Jinze Yang, Hanzhang Wang, Zeke Xie, Yunfeng Cai, Jiale Cao, Zhong Ji, Mingming Sun

To tackle this problem, we propose a novel approach that enhances the capacity of 3DGS by leveraging prior from a Diffusion Model along with complementary multi-modal data.

3DGS Autonomous Driving +3

Neural Field Classifiers via Target Encoding and Classification Loss

no code implementations2 Mar 2024 Xindi Yang, Zeke Xie, Xiong Zhou, Boyu Liu, Buhua Liu, Yi Liu, Haoran Wang, Yunfeng Cai, Mingming Sun

We successfully propose a novel Neural Field Classifier (NFC) framework which formulates existing neural field methods as classification tasks rather than regression tasks.

Classification Multi-Label Classification +4

HiCAST: Highly Customized Arbitrary Style Transfer with Adapter Enhanced Diffusion Models

no code implementations11 Jan 2024 Hanzhang Wang, Haoran Wang, Jinze Yang, Zhongrui Yu, Zeke Xie, Lei Tian, Xinyan Xiao, Junjun Jiang, Xianming Liu, Mingming Sun

In the specific, our model is constructed based on Latent Diffusion Model (LDM) and elaborately designed to absorb content and style instance as conditions of LDM.

Style Transfer

S3IM: Stochastic Structural SIMilarity and Its Unreasonable Effectiveness for Neural Fields

1 code implementation ICCV 2023 Zeke Xie, Xindi Yang, Yujie Yang, Qi Sun, Yixiang Jiang, Haoran Wang, Yunfeng Cai, Mingming Sun

Recently, Neural Radiance Field (NeRF) has shown great success in rendering novel-view images of a given scene by learning an implicit representation with only posed RGB images.

NeRF Novel View Synthesis +1

Sparse Double Descent: Where Network Pruning Aggravates Overfitting

1 code implementation17 Jun 2022 Zheng He, Zeke Xie, Quanzhi Zhu, Zengchang Qin

People usually believe that network pruning not only reduces the computational cost of deep networks, but also prevents overfitting by decreasing model capacity.

Network Pruning

Dataset Pruning: Reducing Training Data by Examining Generalization Influence

no code implementations19 May 2022 Shuo Yang, Zeke Xie, Hanyu Peng, Min Xu, Mingming Sun, Ping Li

To answer these, we propose dataset pruning, an optimization-based sample selection method that can (1) examine the influence of removing a particular set of training samples on model's generalization ability with theoretical guarantee, and (2) construct the smallest subset of training data that yields strictly constrained generalization gap.

On the Power-Law Hessian Spectrums in Deep Learning

no code implementations31 Jan 2022 Zeke Xie, Qian-Yuan Tang, Yunfeng Cai, Mingming Sun, Ping Li

It is well-known that the Hessian of deep loss landscape matters to optimization, generalization, and even robustness of deep learning.

Deep Learning

Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum

no code implementations29 Sep 2021 Zeke Xie, Xinrui Wang, Huishuai Zhang, Issei Sato, Masashi Sugiyama

Specifically, we disentangle the effects of Adaptive Learning Rate and Momentum of the Adam dynamics on saddle-point escaping and flat minima selection.

Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization

1 code implementation31 Mar 2021 Zeke Xie, Li Yuan, Zhanxing Zhu, Masashi Sugiyama

It is well-known that stochastic gradient noise (SGN) acts as implicit regularization for deep learning and is essentially important for both optimization and generalization of deep networks.

On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective

1 code implementation NeurIPS 2023 Zeke Xie, Zhiqiang Xu, Jingzhao Zhang, Issei Sato, Masashi Sugiyama

Weight decay is a simple yet powerful regularization technique that has been very widely used in training of deep neural networks (DNNs).

Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting

1 code implementation12 Nov 2020 Zeke Xie, Fengxiang He, Shaopeng Fu, Issei Sato, DaCheng Tao, Masashi Sugiyama

Thus it motivates us to design a similar mechanism named {\it artificial neural variability} (ANV), which helps artificial neural networks learn some advantages from ``natural'' neural networks.

Memorization

Stable Weight Decay Regularization

no code implementations28 Sep 2020 Zeke Xie, Issei Sato, Masashi Sugiyama

\citet{loshchilov2018decoupled} demonstrated that $L_{2}$ regularization is not identical to weight decay for adaptive gradient methods, such as Adaptive Momentum Estimation (Adam), and proposed Adam with Decoupled Weight Decay (AdamW).

Adai: Separating the Effects of Adaptive Learning Rate and Momentum Inertia

1 code implementation29 Jun 2020 Zeke Xie, Xinrui Wang, Huishuai Zhang, Issei Sato, Masashi Sugiyama

Specifically, we disentangle the effects of Adaptive Learning Rate and Momentum of the Adam dynamics on saddle-point escaping and minima selection.

A Quantum-Inspired Ensemble Method and Quantum-Inspired Forest Regressors

no code implementations22 Nov 2017 Zeke Xie, Issei Sato

The contribution of this work is two-fold, a novel ensemble regression algorithm inspired by quantum mechanics and the theoretical connection between quantum interpretations and machine learning algorithms.

Diversity regression

Cannot find the paper you are looking for? You can Submit a new open access paper.