Search Results for author:
Found 15 papers, 7 papers with code
VideoPoet: A Large Language Model for Zero-Shot Video Generation
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals.
Ranked #3 on
Text-to-Video Generation
on MSR-VTT
Pixel Aligned Language Models
When taking locations as inputs, the model performs location-conditioned captioning, which generates captions for the indicated object or region.
CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor
Mask labels are labor-intensive, which limits the number of categories in segmentation datasets.
Photorealistic Video Generation with Diffusion Models
We present W. A. L. T, a transformer-based approach for photorealistic video generation via diffusion modeling.
Ranked #1 on
Video Prediction
on Kinetics-600 12 frames, 64x64
PolyMaX: General Dense Prediction with Mask Transformer
Despite this shift, methods based on the per-pixel prediction paradigm still dominate the benchmarks on the other dense prediction tasks that require continuous outputs, such as depth estimation and surface normal prediction.
Ranked #1 on
Surface Normals Estimation
on NYU Depth v2
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
While Large Language Models (LLMs) are the dominant models for generative tasks in language, they do not perform as well as diffusion models on image and video generation.
Ranked #2 on
Video Prediction
on Kinetics-600 12 frames, 64x64
A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models
In particular, we ask "Given a large pool of prompts, can we automatically score the prompts and ensemble those that are most suitable for a particular downstream dataset, without needing access to labeled validation data?".
Open-Vocabulary Temporal Action Detection with Off-the-Shelf Image-Text Features
Detecting actions in untrimmed videos should not be limited to a small, closed set of classes.
F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models
We present F-VLM, a simple open-vocabulary object detection method built upon Frozen Vision and Language Models.
Scaling Open-Vocabulary Image Segmentation with Image-Level Labels
We propose OpenSeg to address the above issue while still making use of scalable image-level supervision of captions.
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation
On COCO, ViLD outperforms the previous state-of-the-art by 4. 8 on novel AP and 11. 4 on overall AP.
Ranked #2 on
Open Vocabulary Object Detection
on Objects365
Password-conditioned Anonymization and Deanonymization with Face Identity Transformers
Cameras are prevalent in our daily lives, and enable many useful systems built upon computer vision technologies such as smart cameras and home robots for service applications.
HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds
We present a novel deep neural network architecture for end-to-end scene flow estimation that directly operates on large-scale 3D point clouds.
A Revisit on Deep Hashings for Large-scale Content Based Image Retrieval
However, there are serious flaws in the evaluations of existing deep hashing papers: (1) The datasets they used are too small and simple to simulate the real CBIR situation.
Interspecies Knowledge Transfer for Facial Keypoint Detection
Instead of directly finetuning a network trained to detect keypoints on human faces to animal faces (which is sub-optimal since human and animal faces can look quite different), we propose to first adapt the animal images to the pre-trained human detection network by correcting for the differences in animal and human face shape.
