Search Results for author:
Found 14 papers, 9 papers with code
TextCraftor: Your Text Encoder Can be Image Quality Controller
Our findings reveal that, instead of replacing the CLIP text encoder used in Stable Diffusion with other large language models, we can enhance it through our proposed fine-tuning approach, TextCraftor, leading to substantial improvements in quantitative benchmarks and human assessments.
Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis
Since video content is highly redundant, we argue that naively bringing advances of image models to the video generation domain reduces motion fidelity, visual quality and impairs scalability.
Ranked #1 on
Text-to-Video Generation
on MSR-VTT
Scaffolding a Student to Instill Knowledge
We propose a novel knowledge distillation (KD) method to selectively instill teacher knowledge into a student model motivated by situations where the student's capacity is significantly smaller than that of the teachers.
Efficient Edge Inference by Selective Query
Training a hybrid learner is difficult since we lack annotations of hard edge-examples.
Condensing CNNs With Partial Differential Equations
Convolutional neural networks (CNNs) rely on the depth of the architecture to obtain complex features.
Online Selective Classification with Limited Feedback
For example, this may model an adaptive decision to invoke more resources on this instance.
Hybrid Cloud-Edge Networks for Efficient Inference
The first network is a low-capacity network that can be deployed on an edge device, whereas the second is a high-capacity network deployed in the cloud.
Training Recurrent Neural Networks via Forward Propagation Through Time
BPTT updates RNN parameters on an instance by back-propagating the error in time over the entire sequence length, and as a result, leads to poor trainability due to the well-known gradient explosion/decay phenomena.
Time Adaptive Recurrent Neural Network
We propose a learning method that, dynamically modifies the time-constants of the continuous-time counterpart of a vanilla RNN.
Selective Classification via One-Sided Prediction
We propose a novel method for selective classification (SC), a problem which allows a classifier to abstain from predicting some instances, thus trading off accuracy against coverage (the fraction of instances predicted).
RNNs Incrementally Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients?
Recurrent neural networks (RNNs) are particularly well-suited for modeling long-term dependencies in sequential data, but are notoriously hard to train because the error backpropagated in time either vanishes or explodes at an exponential rate.
RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients?
Recurrent neural networks (RNNs) are particularly well-suited for modeling long-term dependencies in sequential data, but are notoriously hard to train because the error backpropagated in time either vanishes or explodes at an exponential rate.
Equilibrated Recurrent Neural Network: Neuronal Time-Delayed Self-Feedback Improves Accuracy and Stability
We show that such self-feedback helps stabilize the hidden state transitions leading to fast convergence during training while efficiently learning discriminative latent features that result in state-of-the-art results on several benchmark datasets at test-time.
Learning Compact Networks via Adaptive Network Regularization
We describe a training strategy that grows the number of units during training, and show on several benchmark datasets that our model yields architectures that are smaller than those obtained when tuning the number of hidden units on a standard fixed architecture.
