Search Results for author:
Found 36 papers, 21 papers with code
Temporal-Oriented Recipe for Transferring Large Vision-Language Model to Video Understanding
In this work, we conduct a thorough empirical study to demystify crucial components that influence the temporal understanding of LVLMs.
Effective Inference-Free Retrieval for Learned Sparse Representations
In this paper, we conduct an extended evaluation of regularization approaches for LSR where we discuss their effectiveness, efficiency, and out-of-domain generalization capabilities.
Leveraging Decoder Architectures for Learned Sparse Retrieval
Learned Sparse Retrieval (LSR) has traditionally focused on small-scale encoder-only transformer architectures.
TI-JEPA: An Innovative Energy-based Joint Embedding Strategy for Text-Image Multimodal Systems
This paper focuses on multimodal alignment within the realm of Artificial Intelligence, particularly in text and image modalities.
CutPaste&Find: Efficient Multimodal Hallucination Detector with Visual-aid Knowledge Base
Large Vision-Language Models (LVLMs) have demonstrated impressive multimodal reasoning capabilities, but they remain susceptible to hallucination, particularly object hallucination where non-existent objects or incorrect attributes are fabricated in generated descriptions.
Enhancing Multimodal Entity Linking with Jaccard Distance-based Conditional Contrastive Learning and Contextual Visual Augmentation
Previous research on multimodal entity linking (MEL) has primarily employed contrastive learning as the primary objective.
DyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities
Learned Sparse Retrieval (LSR) models use vocabularies from pre-trained transformers, which often split entities into nonsensical fragments.
Topic-aware Causal Intervention for Counterfactual Detection
Hence, we consider the problem of counterfactual detection (CFD) and seek to enhance the CFD models.
MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning
To address these problems, we propose MAMA, a new approach to learning video-language representations by utilizing a contrastive objective with a subtractive angular margin to regularize cross-modal representations in their effort to reach perfect similarity.
Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives
Humans use multiple senses to comprehend the environment.
FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model
We further propose a novel Embedding Transport Plan (ETP) method.
Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion
However, existing models suffer from repetitive topic and unassociated topic issues, failing to reveal the evolution and hindering further applications.
KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learning
Previous work on multimodal sentence embedding has proposed multimodal contrastive learning and achieved promising results.
Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control
Our proposed approach efficiently transforms dense vectors from a frozen dense model into sparse lexical vectors.
Topic Modeling as Multi-Objective Contrastive Optimization
Secondly, we explicitly cast contrastive topic modeling as a gradient-based multi-objective optimization problem, with the goal of achieving a Pareto stationary solution that balances the trade-off between the ELBO and the contrastive objective.
Multimodal Learned Sparse Retrieval for Image Suggestion
Motivated by this, in this work, we explore the application of LSR in the multi-modal domain, i. e., we focus on Multi-Modal Learned Sparse Retrieval (MLSR).
A Survey on Neural Topic Models: Methods, Applications, and Challenges
In this paper, we present a comprehensive survey on neural topic models concerning methods, applications, and challenges.
READ: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling
Fully fine-tuning pretrained large-scale transformer models has become a popular paradigm for video-language modeling tasks, such as temporal language grounding and video-language summarization.
DemaFormer: Damped Exponential Moving Average Transformer with Energy-Based Modeling for Temporal Language Grounding
Temporal Language Grounding seeks to localize video moments that semantically correspond to a natural language query.
Improving Multimodal Sentiment Analysis: Supervised Angular Margin-based Contrastive Learning for Enhanced Fusion Representation
The effectiveness of a model is heavily reliant on the quality of the fusion representation of multiple modalities in multimodal sentiment analysis.
Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignment
Language models have been supervised with both language-only objective and visual grounding in existing studies of visual-grounded language learning.
Generative Retrieval as Dense Retrieval
Generative retrieval is a promising new neural retrieval paradigm that aims to optimize the retrieval pipeline by performing both indexing and retrieval with a single transformer model.
Effective Neural Topic Modeling with Embedding Clustering Regularization
Topic models have been prevalent for decades with various applications.
Adapting Learned Sparse Retrieval for Long Documents
We investigate existing aggregation approaches for adapting LSR to longer documents and find that proximal scoring is crucial for LSR to handle long documents.
Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction
Multimodal Review Helpfulness Prediction (MRHP) aims to rank product reviews based on predicted helpfulness scores and has been widely applied in e-commerce via presenting customers with useful reviews.
InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling
Instead of the direct alignment in previous work, we propose a topic alignment with mutual information method.
A Unified Framework for Learned Sparse Retrieval
We then reproduce all prominent methods using a common codebase and re-train them in the same environment, which allows us to quantify how components of the framework affect effectiveness and efficiency.
Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictions
To overcome the aforementioned issues, we propose Multimodal Contrastive Learning for Multimodal Review Helpfulness Prediction (MRHP) problem, concentrating on mutual information between input modalities to explicitly elaborate cross-modal relations.
Vision-and-Language Pretraining
Moreover, a list of training datasets and downstream tasks is supplied to further polish the perspective into V\&L pretraining.
Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires
In dataset-transfer experiments on three social media datasets, we find that grounding the model in PHQ9's symptoms substantially improves its ability to generalize to out-of-distribution data compared to a standard BERT-based approach.
Improving Neural Cross-Lingual Summarization via Employing Optimal Transport Distance for Knowledge Distillation
Current state-of-the-art cross-lingual summarization models employ multi-task learning paradigm, which works on a shared vocabulary module and relies on the self-attention mechanism to attend among tokens in two languages.
Contrastive Learning for Neural Topic Model
Recent empirical studies show that adversarial topic models (ATM) can successfully capture semantic patterns of the document by differentiating a document with another dissimilar sample.
Enriching and Controlling Global Semantics for Text Summarization
Recently, Transformer-based models have been proven effective in the abstractive summarization task by creating fluent and informative summaries.
Adaptive Name Entity Recognition under Highly Unbalanced Data
For several purposes in Natural Language Processing (NLP), such as Information Extraction, Sentiment Analysis or Chatbot, Named Entity Recognition (NER) holds an important role as it helps to determine and categorize entities in text into predefined groups such as the names of persons, locations, quantities, organizations or percentages, etc.
Fast Transient Simulation of High-Speed Channels Using Recurrent Neural Network
In this paper, we leverage machine learning methods, to be specific, the recurrent neural network (RNN), to generate black-box macromodels and achieve significant reduction of computation time.
