Search Results for author:
Found 14 papers, 11 papers with code
OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting
Diffusion-based generative models have revolutionized object-oriented image editing, yet their deployment in realistic object removal and insertion remains hampered by challenges such as the intricate interplay of physical effects and insufficient paired training data.
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models
With MMCOMPOSITION, we can quantify and explore the compositionality of the mainstream VLMs.
PromptFix: You Prompt and We Fix the Photo
To address these limitations, we propose PromptFix, a comprehensive framework that enables diffusion models to follow human instructions to perform a wide variety of image-processing tasks.
GMMFormer: Gaussian-Mixture-Model Based Transformer for Efficient Partially Relevant Video Retrieval
Current PRVR methods adopt scanning-based clip construction to achieve explicit clip modeling, which is information-redundant and requires a large storage overhead.
Making LLaMA SEE and Draw with SEED Tokenizer
We identify two crucial design principles: (1) Image tokens should be independent of 2D physical patch positions and instead be produced with a 1D causal dependency, exhibiting intrinsic interdependence that aligns with the left-to-right autoregressive prediction mechanism in LLMs.
VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation
Existing approaches to unsupervised video instance segmentation typically rely on motion estimates and experience difficulties tracking small or divergent motions.
MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation
We propose MISSRec, a multi-modal pre-training and transfer learning framework for SR. On the user side, we design a Transformer-based encoder-decoder model, where the contextual encoder learns to capture the sequence-level multi-modal user interests while a novel interest-aware decoder is developed to grasp item-modality-interest relations for better sequence representation.
Planting a SEED of Vision in Large Language Model
Research on image tokenizers has previously reached an impasse, as frameworks employing quantized visual tokens have lost prominence due to subpar performance and convergence in multimodal comprehension (compared to BLIP-2, etc.)
TVTSv2: Learning Out-of-the-box Spatiotemporal Visual Representations at Scale
We argue that tuning a text encoder end-to-end, as done in previous work, is suboptimal since it may overfit in terms of styles, thereby losing its original generalization ability to capture the semantics of various language registers.
Learning Transferable Spatiotemporal Representations from Natural Script Knowledge
Pre-training on large-scale video data has become a common recipe for learning transferable spatiotemporal representations in recent years.
Hybrid Contrastive Quantization for Efficient Cross-View Video Retrieval
By performing Asymmetric-Quantized Contrastive Learning (AQ-CL) across views, HCQ aligns texts and videos at coarse-grained and multiple fine-grained levels.
PHPQ: Pyramid Hybrid Pooling Quantization for Efficient Fine-Grained Image Retrieval
To improve fine-grained image hashing, we propose Pyramid Hybrid Pooling Quantization (PHPQ).
Contrastive Quantization with Code Memory for Unsupervised Image Retrieval
The high efficiency in computation and storage makes hashing (including binary hashing and quantization) a common strategy in large-scale retrieval systems.
