Search Results for author:
Found 20 papers, 10 papers with code
Movie Gen: A Cast of Media Foundation Models
Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation.
TLDR: Token-Level Detective Reward Model for Large Vision Language Models
Although reward models have been successful in improving multimodal large language models, the reward models themselves remain brutal and contain minimal information.
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #3 on
Question Answering
on PeerQA
Layout Agnostic Scene Text Image Synthesis with Diffusion Models
While diffusion models have significantly advanced the quality of image generation their capability to accurately and coherently render text within these images remains a substantial challenge.
Animated Stickers: Bringing Stickers to Life with Video Diffusion
Our model is built on top of the state-of-the-art Emu text-to-image model, with the addition of temporal layers to model motion.
LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning
Additionally, existing diffusion-based image manipulation models are sub-optimal in controlling the state transition of an action in egocentric image pixel space because of the domain gap.
DISGO: Automatic End-to-End Evaluation for Scene Text OCR
This paper discusses the challenges of optical character recognition (OCR) on natural scenes, which is harder than OCR on documents due to the wild content and various image backgrounds.
Text-Conditional Contextualized Avatars For Zero-Shot Personalization
Recent large-scale text-to-image generation models have made significant improvements in the quality, realism, and diversity of the synthesized images and enable users to control the created content through language.
MUGEN: A Playground for Video-Audio-Text Multimodal Understanding and GENeration
Altogether, MUGEN can help progress research in many tasks in multimodal understanding and generation.
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer
Videos are created to express emotion, exchange information, and share experiences.
Ranked #22 on
Video Generation
on UCF-101
Revisiting Linear Decision Boundaries for Few-Shot Learning with Transformer Hypernetworks
Few-shot learning (FSL) methods aim to generalize a model to new unseen classes using only a small number of support examples.
TextStyleBrush: Transfer of Text Aesthetics from a Single Example
We present a novel approach for disentangling the content of a text image from all aspects of its appearance.
TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text
A crucial component for the scene text based reasoning required for TextVQA and TextCaps datasets involve detecting and recognizing text present in the images using an optical character recognition (OCR) system.
Optical Character Recognition
Optical Character Recognition (OCR)
+3
A Multiplexed Network for End-to-End, Multilingual OCR
Recent advances in OCR have shown that an end-to-end (E2E) training pipeline that includes both detection and recognition leads to the best results.
img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation
Tests on AFLW2000-3D and BIWI show that our method runs at real-time and outperforms state of the art (SotA) face pose estimators.
Ranked #10 on
Head Pose Estimation
on BIWI
Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spotting
Recent end-to-end trainable methods for scene text spotting, integrating detection and recognition, showed much progress.
Ranked #11 on
Text Spotting
on Total-Text
From Satellite Imagery to Disaster Insights
The use of satellite imagery has become increasingly popular for disaster monitoring and response.
Improving Rotated Text Detection with Rotation Region Proposal Networks
In this work, we extend the scene-text extraction system at Facebook, Rosetta, to efficiently handle text in various orientations.
DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images
We present the DeepGlobe 2018 Satellite Image Understanding Challenge, which includes three public competitions for segmentation, detection, and classification tasks on satellite images.
End-to-end Planning of Fixed Millimeter-Wave Networks
This article discusses a framework to support the design and end-to-end planning of fixed millimeter-wave networks.
