Search Results for author:
Found 34 papers, 15 papers with code
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
The idea of rectified flow is to learn the ODE to follow the straight paths connecting the points drawn from \pi_0 and \pi_1 as much as possible.
FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimization
We approach text-to-image generation by combining the power of the retrained CLIP representation with an off-the-shelf image generator (GANs), optimizing in the latent space of GAN to find images that achieve maximum CLIP score with the given input text.
Ranked #48 on
Text-to-Image Generation
on MS COCO
FRAGE: Frequency-Agnostic Word Representation
Continuous word representation (aka word embedding) is a basic building block in many neural network-based models used in natural language processing tasks.
Ranked #3 on
Machine Translation
on IWSLT2015 German-English
AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling
Our discovered model family, AttentiveNAS models, achieves top-1 accuracy from 77. 3% to 80. 7% on ImageNet, and outperforms SOTA models, including BigNAS and Once-for-All networks.
Ranked #21 on
Neural Architecture Search
on ImageNet
AlphaNet: Improved Training of Supernets with Alpha-Divergence
Weight-sharing NAS builds a supernet that assembles all the architectures as its sub-networks and jointly trains the supernet with the sub-networks.
Ranked #12 on
Neural Architecture Search
on ImageNet
Vision Transformers with Patch Diversification
To alleviate this problem, in this work, we introduce novel loss functions in vision transformer training to explicitly encourage diversity across patch representations for more discriminative feature extraction.
Ranked #19 on
Semantic Segmentation
on Cityscapes val
NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training
In this work, we observe that the poor performance is due to a gradient conflict issue: the gradients of different sub-networks conflict with that of the supernet more severely in ViTs than CNNs, which leads to early saturation in training and inferior convergence.
Ranked #7 on
Neural Architecture Search
on ImageNet
Improving Neural Language Modeling via Adversarial Training
Theoretically, we show that our adversarial mechanism effectively encourages the diversity of the embedding vectors, helping to increase the robustness of models.
Ranked #5 on
Language Modelling
on Penn Treebank (Word Level)
Fast Point Cloud Generation with Straight Flows
We perform evaluations on multiple 3D tasks and find that our PSF performs comparably to the standard diffusion model, outperforming other efficient 3D point cloud generation methods.
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
For security reasons, it is of critical importance to develop models with certified robustness that can provably guarantee that the prediction is can not be altered by any possible synonymous word substitution.
Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection
This differs from the existing methods based on backward elimination, which remove redundant neurons from the large network.
KeepAugment: A Simple Information-Preserving Data Augmentation Approach
Data augmentation (DA) is an essential technique for training state-of-the-art deep learning systems.
Knowing More About Questions Can Help: Improving Calibration in Question Answering
We study calibration in question answering, estimating whether model correctly predicts answer for each question.
Learning with Different Amounts of Annotation: From Zero to Many Labels
Introducing such multi label examples at the cost of annotating fewer examples brings clear gains on natural language inference task and entity typing task, even when we simply first train with a single label data and then fine tune with multi label examples.
MaxUp: A Simple Way to Improve Generalization of Neural Network Training
The idea is to generate a set of augmented data with some random perturbations or transforms and minimize the maximum, or worst case loss over the augmented data.
Ranked #185 on
Image Classification
on ImageNet
Sentence-wise Smooth Regularization for Sequence to Sequence Learning
Maximum-likelihood estimation (MLE) is widely used in sequence to sequence tasks for model training.
Deep Dynamic Poisson Factorization Model
A new model, named as deep dynamic poisson factorization model, is proposed in this paper for analyzing sequential count vectors.
Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Framework
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning.
Network Pruning by Greedy Subnetwork Selection
Theoretically, we show that the small networks pruned using our method achieve provably lower loss than small networks trained from scratch with the same size.
Fast Training of Contrastive Learning with Intermediate Contrastive Loss
We apply our method to recently-proposed MOCO, SimCLR, SwAV and notice that we can reduce the computational cost with little loss on the performance of ImageNet linear classification and other downstream tasks.
AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence
Semi-supervised learning (SSL) is a key approach toward more data-efficient machine learning by jointly leverage both labeled and unlabeled data.
Capturing Label Distribution: A Case Study in NLI
We depart from the standard practice of collecting a single reference per each training example, and find that collecting multiple references can achieve better accuracy under the fixed annotation budget.
MaxUp: Lightweight Adversarial Training With Data Augmentation Improves Neural Network Training
The idea is to generate a set of augmented data with some random perturbations or transforms, and minimize the maximum, or worst case loss over the augmented data.
argmax centroid
We propose a general method to construct centroid approximation for the distribution of maximum points of a random function (a. k. a.
Automatic and Harmless Regularization with Constrained and Lexicographic Optimization: A Dynamic Barrier Approach
In this work, we consider constrained optimization as a more principled approach for trading off two losses, with a special emphasis on lexicographic optimization, a degenerated limit of constrained optimization which optimizes a secondary loss inside the optimal set of the main loss.
Automatic and Harmless Regularization with Constrained and Lexicographic Optimization: A Dynamic Barrier Approach
In this work, we consider constrained optimization as a more principled approach for trading off two losses, with a special emphasis on lexicographic optimization, a degenerated limit of constrained optimization which optimizes a secondary loss inside the optimal set of the main loss.
How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity
Although traditional optimization methods focus on finding a single optimal solution, most objective functions in modern machine learning problems, especially those in deep learning, often have multiple or infinite numbers of optima.
ALLSH: Active Learning Guided by Local Sensitivity and Hardness
Active learning, which effectively collects informative unlabeled data for annotation, reduces the demand for labeled data.
Harmless Transfer Learning for Item Embeddings
On the definition side, we reduce the bias in transfer loss by focusing on the items to which information from high-frequency items can be efficiently transferred.
Diffusion-based Molecule Generation with Informative Prior Bridges
AI-based molecule generation provides a promising approach to a large area of biomedical sciences and engineering, such as antibody design, hydrolase engineering, or vaccine development.
Passage-Mask: A Learnable Regularization Strategy for Retriever-Reader Models
Experiments on different tasks across open question answering, dialogue conversation, and fact verification show that our method consistently outperforms its baselines.
AutoML-GPT: Automatic Machine Learning with GPT
Ultimately, with this prompt paragraph, AutoML-GPT will automatically conduct the experiments from data processing to model architecture, hyperparameter tuning, and predicted training log.
FlowGrad: Controlling the Output of Generative ODEs With Gradients
To further accelerate the computation of the back-propagation, we propose to use a non-uniform discretization to approximate the ODE trajectory, where we measure how straight the trajectory is and gather the straight parts into one discretization step.
Language Rectified Flow: Advancing Diffusion Language Generation with Probabilistic Flows
Extensive experiments and ablation studies demonstrate that our method can be general, effective, and beneficial for many NLP tasks.
