Search Results for author:
Found 18 papers, 7 papers with code
DreamRunner: Fine-Grained Storytelling Video Generation with Retrieval-Augmented Motion Adaptation
To address these challenges, we propose DreamRunner, a novel story-to-video generation method: First, we structure the input script using a large language model (LLM) to facilitate both coarse-grained scene planning as well as fine-grained object-level layout and motion planning.
VEDIT: Latent Prediction Architecture For Procedural Video Representation Learning
In this work, we show that a strong off-the-shelf frozen pretrained visual encoder, along with a well designed prediction model, can achieve state-of-the-art (SoTA) performance in forecasting and procedural planning without the need for pretraining the prediction model, nor requiring additional supervision from language or ASR.
Fast Tree-Field Integrators: From Low Displacement Rank to Topological Transformers
We present a new class of fast polylog-linear algorithms based on the theory of structured matrices (in particular low displacement rank) for integrating tensor fields defined on weighted trees.
MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI
Large Vision-Language Models (LVLMs) show significant strides in general-purpose multimodal applications such as visual dialogue and embodied navigation.
MCM: Multi-condition Motion Synthesis Framework
Conditional human motion synthesis (HMS) aims to generate human motion sequences that conform to specific conditions.
Ranked #7 on
Motion Synthesis
on HumanML3D
Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model
ControlNets are widely used for adding spatial control to text-to-image diffusion models with different conditions, such as depth maps, scribbles/sketches, and human poses.
EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents
Then, we enable the LLM to continuously adapt the generated environments to progressively improve the skills that the agent is weak at, by providing feedback to the LLM in the form of the agent's performance.
DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning
In the second stage, we use a diagram generator, DiagramGLIGEN, and a text label rendering module to generate diagrams (with clear text labels) following the diagram plans.
VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning
Our experiments demonstrate that our proposed VideoDirectorGPT framework substantially improves layout and movement control in both single- and multi-scene video generation and can generate multi-scene videos with consistency, while achieving competitive performance with SOTAs in open-domain single-scene T2V generation.
Supervised Masked Knowledge Distillation for Few-Shot Transformers
Vision Transformers (ViTs) emerge to achieve impressive performance on many data-abundant computer vision tasks by capturing long-range dependencies among local features.
Efficient Graph Field Integrators Meet Point Clouds
We present two new classes of algorithms for efficient field integration on graphs encoding point clouds.
Self-supervised Learning for Segmentation and Quantification of Dopamine Neurons in Parkinson's Disease
To this end, we propose an end-to-end deep learning framework based on self-supervised learning for the segmentation and quantification of dopaminergic neurons in PD animal models.
TANDEM3D: Active Tactile Exploration for 3D Object Recognition
In this work, we propose TANDEM3D, a method that applies a co-training framework for exploration and decision making to 3D object recognition with tactile signals.
Hybrid Random Features
We propose a new class of random feature methods for linearizing softmax and Gaussian kernels called hybrid random features (HRFs) that automatically adapt the quality of kernel estimation to provide most accurate approximation in the defined regions of interest.
From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers
In this paper we provide, to the best of our knowledge, the first comprehensive approach for incorporating various masking mechanisms into Transformers architectures in a scalable way.
SN 2017hpa: A Nearby Carbon-Rich Type Ia Supernova with a Large Velocity Gradient
We present extensive, well-sampled optical and ultraviolet photometry and optical spectra of the Type Ia supernova (SN Ia) 2017hpa.
High Energy Astrophysical Phenomena Solar and Stellar Astrophysics
The Tsinghua University-Ma Huateng Telescopes for Survey: Overview and Performance of the System
The TMTS system can have a FoV of about 9 deg2 when monitoring the sky with two bands (i. e., SDSS g and r filters) at the same time, and a maximum FoV of ~18 deg2 when four telescopes monitor different sky areas in monochromatic filter mode.
Instrumentation and Methods for Astrophysics
Demystifying Orthogonal Monte Carlo and Beyond
Orthogonal Monte Carlo (OMC) is a very effective sampling algorithm imposing structural geometric conditions (orthogonality) on samples for variance reduction.
