Search Results for author:
Found 20 papers, 16 papers with code
Motion-aware Contrastive Learning for Temporal Panoptic Scene Graph Generation
To overcome this limitation, we introduce a contrastive representation learning framework that focuses on motion pattern for temporal scene graph generation.
Multi-Scale Contrastive Learning for Video Temporal Grounding
To resolve this problem, we propose a contrastive learning framework to capture salient semantics among video moments.
Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping
Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across multimodal tasks such as visual perception and reasoning, leading to good performance on various multimodal evaluation benchmarks.
PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization
In this work, we propose PrefixQuant, a novel quantization method that achieves state-of-the-art performance across various precision levels (W4A4KV4 and W4A8KV4) and granularities (dynamic and static quantization) by effectively isolating token-wise outliers.
MM-Forecast: A Multimodal Approach to Temporal Event Forecasting with Large Language Models
We study an emerging and intriguing problem of multimodal temporal event forecasting with large language models.
Leveraging Weak Cross-Modal Guidance for Coherence Modelling via Iterative Learning
The iterative cross-modal boosting also functions in inference to further enhance coherence prediction in each modality.
GalleryGPT: Analyzing Paintings with Large Multimodal Models
Specifically, we first propose a task of composing paragraph analysis for artworks, i. e., painting in this paper, only focusing on visual characteristics to formulate more comprehensive understanding of artworks.
Exploring Deeper! Segment Anything Model with Depth Perception for Camouflaged Object Detection
It maximizes the utilization of depth features while synergizing with RGB features to achieve multimodal complementarity, thereby overcoming the segmentation limitations of SAM and improving its accuracy in COD.
MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning
To address these problems, we propose MAMA, a new approach to learning video-language representations by utilizing a contrastive objective with a subtractive angular margin to regularize cross-modal representations in their effort to reach perfect similarity.
Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models
To bridge this gap, we address the lack of high-quality, diverse multimodal mathematical datasets by collecting 40K high-quality images with question-answer pairs from 24 existing datasets and synthesizing 320K new pairs, creating the MathV360K dataset, which enhances both the breadth and depth of multimodal mathematical questions.
Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives
Humans use multiple senses to comprehend the environment.
Ensemble Diversity Facilitates Adversarial Transferability
With the advent of ensemble-based attacks the transferability of generated adversarial examples is elevated by a noticeable margin despite many methods only employing superficial integration yet ignoring the diversity between ensemble models.
Solving Math Word Problems with Reexamination
Math word problem (MWP) solving aims to understand the descriptive math problem and calculate the result, for which previous efforts are mostly devoted to upgrade different technical modules.
Unifying Two-Stream Encoders with Transformers for Cross-Modal Retrieval
Specifically, on two key tasks, \textit{i. e.}, image-to-text and text-to-image retrieval, HAT achieves 7. 6\% and 16. 7\% relative score improvement of Recall@1 on MSCOCO, and 4. 4\% and 11. 6\% on Flickr30k respectively.
Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination
Most existing image-text matching methods adopt triplet loss as the optimization objective, and choosing a proper negative sample for the triplet of <anchor, positive, negative> is important for effectively training the model, e. g., hard negatives make the model learn efficiently and effectively.
Non-Autoregressive Math Word Problem Solver with Unified Tree Structure
For evaluating the possible expression variants, we design a path-based metric to evaluate the partial accuracy of expressions of a unified tree.
NTIRE 2021 Challenge on Perceptual Image Quality Assessment
This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021.
Graph-to-Tree Learning for Solving Math Word Problems
While the recent tree-based neural models have demonstrated promising results in generating solution expression for the math word problem (MWP), most of these models do not capture the relationships and order information among the quantities well.
Ranked #11 on
Math Word Problem Solving
on Math23K
Bidirectional Long-Short Term Memory for Video Description
Video captioning has been attracting broad research attention in multimedia community.
