Search Results for author:
Found 10 papers, 5 papers with code
Visual Concept-driven Image Generation with Text-to-Image Diffusion Model
We illustrate that such joint alternating refinement leads to the learning of better tokens for concepts and, as a bi-product, latent masks.
Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models
Further, we leverage the findings that different timesteps of the diffusion process cater to different levels of detail in an image.
Make-A-Story: Visual Memory Conditioned Consistent Story Generation
Our experiments for story generation on the MUGEN, the PororoSV and the FlintstonesSV dataset show that our method not only outperforms prior state-of-the-art in generating frames with high visual quality, which are consistent with the story, but also models appropriate correspondences between the characters and the background.
TriBERT: Human-centric Audio-visual Representation Learning
In this work, we introduce TriBERT -- a transformer-based architecture, inspired by ViLBERT, which enables contextual feature learning across three modalities: vision, pose, and audio, with the use of flexible co-attention.
TriBERT: Full-body Human-centric Audio-visual Representation Learning for Visual Sound Separation
In this work, we introduce TriBERT -- a transformer-based architecture, inspired by ViLBERT, which enables contextual feature learning across three modalities: vision, pose, and audio, with the use of flexible co-attention.
Weakly-supervised Audio-visual Sound Source Detection and Separation
Learning how to localize and separate individual object sounds in the audio channel of the video is a difficult task.
An Improved Attention for Visual Question Answering
We also propose multimodal fusion module to combine both visual and textual information.
Watch, Listen and Tell: Multi-modal Weakly Supervised Dense Event Captioning
Multi-modal learning, particularly among imaging and linguistic modalities, has made amazing strides in many high-level fundamental visual understanding problems, ranging from language grounding to dense event captioning.
Convolutional Temporal Attention Model for Video-based Person Re-identification
A common approach for person re-identification is to first extract image features for all frames in the video, then aggregate all the features to form a video-level feature.
Video-based Person Re-identification Using Spatial-Temporal Attention Networks
The goal is to identify a person from videos captured under different cameras.
