Search Results for author:
Found 303 papers, 191 papers with code
Learning to Imagine: Integrating Counterfactual Thinking in Neural Discrete Reasoning
Neural discrete reasoning (NDR) has shown remarkable progress in combining deep models with discrete reasoning.
Re-examining the Role of Schema Linking in Text-to-SQL
By providing a schema linking corpus based on the Spider text-to-SQL dataset, we systematically study the role of schema linking.
Dynamic Multimodal Fusion via Meta-Learning Towards Micro-Video Recommendation
In this paper, we develop a novel meta-learning-based multimodal fusion framework called Meta Multimodal Fusion (MetaMMF), which dynamically assigns parameters to the multimodal fusion function for each micro-video during its representation learning.
How to Enable Effective Cooperation Between Humans and NLP Models: A Survey of Principles, Formalizations, and Beyond
With the advancement of large language models (LLMs), intelligent models have evolved from mere tools to autonomous agents with their own goals and strategies for cooperating with humans.
Optimize Incompatible Parameters through Compatibility-aware Knowledge Integration
The integrated model can be used directly for inference or for further fine-tuning.
Aligning Large Language Models for Faithful Integrity Against Opposing Argument
In this work, we propose a novel framework, named Alignment for Faithful Integrity with Confidence Estimation (AFICE), which aims to align the LLM responses with faithful integrity.
Towards Modality Generalization: A Benchmark and Prospective Analysis
Multi-modal learning has achieved remarkable success by integrating information from various modalities, achieving superior performance in tasks like recognition and retrieval compared to uni-modal approaches.
Length Controlled Generation for Black-box LLMs
Large language models (LLMs) have demonstrated impressive instruction following capabilities, while still struggling to accurately manage the length of the generated text, which is a fundamental requirement in many real-world applications.
Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
Large vision-language models (LVLMs) have made substantial progress in integrating large language models (LLMs) with visual inputs, enabling advanced multimodal reasoning.
LLaVA-UHD v2: an MLLM Integrating High-Resolution Feature Pyramid via Hierarchical Window Transformer
To address this issue, we present LLaVA-UHD v2, an advanced MLLM centered around a Hierarchical window transformer that enables capturing diverse visual granularity by constructing and integrating a high-resolution feature pyramid.
Knowledge Boundary of Large Language Models: A Survey
Although large language models (LLMs) store vast amount of knowledge in their parameters, they still have limitations in the memorization and utilization of certain knowledge, leading to undesired behaviors such as generating untruthful and inaccurate responses.
Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning
Our framework systematically addresses potential issues in both visual and textual inputs by verifying and integrating perception-level information with cognition-level commonsense knowledge, ensuring more reliable outputs.
SILMM: Self-Improving Large Multimodal Models for Compositional Text-to-Image Generation
To adapt SILMM to LMMs with continuous features, we propose a diversity mechanism to obtain diverse representations and a kernel-based continuous DPO for alignment.
STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training
Video Large Language Models (Video-LLMs) have recently shown strong performance in basic video understanding tasks, such as captioning and coarse-grained question answering, but struggle with compositional reasoning that requires multi-step spatio-temporal inference across object relations, interactions, and events.
Headache to Overstock? Promoting Long-tail Items through Debiased Product Bundling
The inherent popularity bias in the pre-extracted user feedback features and the insufficient utilization of other popularity-independent knowledge may force the conventional bundling methods to find more popular items, thereby struggling with this long-tail bundling scenario.
Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Models
Recent advances in Large Vision-Language Models (LVLMs) have showcased strong reasoning abilities across multiple modalities, achieving significant breakthroughs in various real-world applications.
Self-Calibrated Listwise Reranking with Large Language Models
Large language models (LLMs), with advanced linguistic capabilities, have been employed in reranking tasks through a sequence-to-sequence approach.
Show Me What and Where has Changed? Question Answering and Grounding for Remote Sensing Change Detection
Furthermore, we present VisTA, a simple yet effective baseline method that unifies the tasks of question answering and grounding by delivering both visual and textual answers.
Causality-Enhanced Behavior Sequence Modeling in LLMs for Personalized Recommendation
Recent advancements in recommender systems have focused on leveraging Large Language Models (LLMs) to improve user preference modeling, yielding promising outcomes.
Effective and Efficient Adversarial Detection for Vision-Language Models via A Single Vector
With the new RADAR dataset, we further develop a novel and effective iN-time Embedding-based AdveRSarial Image DEtection (NEARSIDE) method, which exploits a single vector that distilled from the hidden states of VLMs, which we call the attacking direction, to achieve the detection of adversarial images against benign ones in the input.
Large Language Models Empowered Personalized Web Agents
To overcome the limitation, we first formulate the task of LLM-empowered personalized Web agents, which integrate personalized data and user instructions to personalize instruction comprehension and action execution.
Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment
The recent advancements in large language models (LLMs) and pre-trained vision models have accelerated the development of vision-language large models (VLLMs), enhancing the interaction between visual and linguistic modalities.
Preference Diffusion for Recommendation
Recommender systems predict personalized item rankings based on user preference distributions derived from historical behavior data.
Addressing Heterogeneity and Heterophily in Graphs: A Heterogeneous Heterophilic Spectral Graph Neural Network
Specifically, we propose a Heterogeneous Heterophilic Spectral Graph Neural Network (H2SGNN), which employs a dual-module approach: local independent filtering and global hybrid filtering.
PSVMA+: Exploring Multi-granularity Semantic-visual Adaption for Generalized Zero-shot Learning
Generalized zero-shot learning (GZSL) endeavors to identify the unseen categories using knowledge from the seen domain, necessitating the intrinsic interactions between the visual features and attribute semantic features.
Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing
Recent developments of vision large language models (LLMs) have seen remarkable progress, yet still encounter challenges towards multimodal generalists, such as coarse-grained instance-level understanding, lack of unified support for both images and videos, and insufficient coverage across various vision tasks.
Efficient Inference for Large Language Model-based Generative Recommendation
To alleviate this, we consider 1) boosting top-K sequence alignment between the draft model and the target LLM, and 2) relaxing the verification strategy to reduce trivial LLM calls.
Temporal Relational Reasoning of Large Language Models for Detecting Stock Portfolio Crashes
Stock portfolios are often exposed to rare consequential events (e. g., 2007 global financial crisis, 2020 COVID-19 stock market crash), as they do not have enough historical information to learn from.
AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
To address this, we introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters.
Grammar Induction from Visual, Speech and Text
Grammar Induction could benefit from rich heterogeneous signals, such as text, vision, and acoustics.
MASKDROID: Robust Android Malware Detection with Masked Graph Representations
Among the various tools employed in malware detection, graph representations (e. g., function call graphs) have played a pivotal role in characterizing the behaviors of Android apps.
Scene-Text Grounding for Text-Based Video Question Answering
In this paper, we propose to study Grounded TextVideoQA by forcing models to answer questions and spatio-temporally localize the relevant scene-text regions, thus decoupling QA from scenetext recognition and promoting research towards interpretable QA.
Beyond Persuasion: Towards Conversational Recommender System with Credible Explanations
With the aid of large language models, current conversational recommender system (CRS) has gaining strong abilities to persuade users to accept recommended items.
ExpLLM: Towards Chain of Thought for Facial Expression Recognition
Specifically, we have designed the CoT mechanism from three key perspectives: key observations, overall emotional interpretation, and conclusion.
Facial Expression Recognition
Facial Expression Recognition (FER)
Chain-of-Exemplar: Enhancing Distractor Generation for Multimodal Educational Question Generation
Multiple-choice questions (MCQs) are important in enhancing concept learning and student engagement for educational purposes.
VideoQA in the Era of LLMs: An Empirical Study
Video Large Language Models (Video-LLMs) are flourishing and has advanced many video-language tasks.
MM-Forecast: A Multimodal Approach to Temporal Event Forecasting with Large Language Models
We study an emerging and intriguing problem of multimodal temporal event forecasting with large language models.
Revolutionizing Text-to-Image Retrieval as Autoregressive Token-to-Voken Generation
Despite its great potential, existing generative approaches are limited due to the following issues: insufficient visual information in identifiers, misalignment with high-level semantics, and learning gap towards the retrieval target.
Fine-tuning Multimodal Large Language Models for Product Bundling
Recent advances in product bundling have leveraged multimodal information through sophisticated encoders, but remain constrained by limited semantic understanding and a narrow scope of knowledge.
A Comprehensive Evaluation of Large Language Models on Temporal Event Forecasting
However, the reasoning capability of LLMs on temporal event forecasting has been under-explored.
Disentangling Masked Autoencoders for Unsupervised Domain Generalization
Domain Generalization (DG), designed to enhance out-of-distribution (OOD) generalization, is all about learning invariance against domain shifts utilizing sufficient supervision signals.
Language Representations Can be What Recommenders Need: Findings and Potentials
Contrary to prevailing understanding that LMs and traditional recommenders learn two distinct representation spaces due to the huge gap in language and behavior modeling objectives, this work re-examines such understanding and explores extracting a recommendation space directly from the language representation space.
Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment
Then, an SG-based framework is built, where the textual SG (TSG) is encoded with a graph Transformer, while the video dynamic SG (DSG) and the HSG are modeled with a novel recurrent graph Transformer for spatial and temporal feature propagation.
LARP: Language Audio Relational Pre-training for Cold-Start Playlist Continuation
As online music consumption increasingly shifts towards playlist-based listening, the task of playlist continuation, in which an algorithm suggests songs to extend a playlist in a personalized and musically cohesive manner, has become vital to the success of music streaming.
Ask-before-Plan: Proactive Language Agents for Real-World Planning
In this work, we introduce a new task, Proactive Agent Planning, which requires language agents to predict clarification needs based on user-agent conversation and agent-environment interaction, invoke external tools to collect valid information, and generate a plan to fulfill the user's demands.
On Softmax Direct Preference Optimization for Recommendation
Specifically, we incorporate multiple negatives in user preference data and devise an alternative version of DPO loss tailored for LM-based recommenders, which is extended from the traditional full-ranking Plackett-Luce (PL) model to partial rankings and connected to softmax sampling strategies.
Unified Text-to-Image Generation and Retrieval
Subsequently, we unify generation and retrieval in an autoregressive generation way and propose an autonomous decision module to choose the best-matched one between generated and retrieved images as the response to the text query.
Hello Again! LLM-powered Personalized Agent for Long-term Dialogue
Crucial to addressing this real-world need are event summary and persona management, which enable reasoning for appropriate long-term dialogue responses.
Towards Semantic Equivalence of Tokenization in Multimodal LLM
The resulting vision tokens effectively preserve semantic integrity and capture both low-frequency and high-frequency visual features.
Ranked #72 on
Visual Question Answering
on MM-Vet
Attend and Enrich: Enhanced Visual Prompt for Zero-Shot Learning
Zero-shot learning (ZSL) endeavors to transfer knowledge from seen categories to recognize unseen categories, which mostly relies on the semantic-visual interactions between image and attribute tokens.
Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding
We refer to the complex events composed of many news articles over an extended period as Temporal Complex Event (TCE).
Causal Distillation for Alleviating Performance Heterogeneity in Recommender Systems
The emerging causal recommendation methods achieve this by modeling the causal effect between user behaviors, however potentially neglect unobserved confounders (\eg, friend suggestions) that are hard to measure in practice.
RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthiness
Traditional feedback learning for hallucination reduction relies on labor-intensive manual labeling or expensive proprietary models.
Ranked #1 on
Visual Question Answering
on AMBER
ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluation
Systematic analysis also validates that the generated test scenarios represent meaningful use cases, as well as integrate enhanced measures to probe long-tail risks.
ReactXT: Understanding Molecular "Reaction-ship" via Reaction-Contextualized Molecule-Text Pretraining
To resolve the challenges above, we propose a new pretraining method, ReactXT, for reaction-text modeling, and a new dataset, OpenExp, for experimental procedure prediction.
Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching
Safety alignment of large language models (LLMs) has been gaining increasing attention.
ProtT3: Protein-to-Text Generation for Text-based Protein Understanding
ProtT3 empowers an LM to understand protein sequences of amino acids by incorporating a PLM as its protein understanding module, enabling effective protein-to-text generation.
CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models
To this end, we introduce CLAMBER, a benchmark for evaluating LLMs using a well-organized taxonomy.
STYLE: Improving Domain Transferability of Asking Clarification Questions in Large Language Model Powered Conversational Agents
However, they still struggle to deliver promising performance on unseen domains, struggling to achieve effective domain transferability.
Co-Matching: Towards Human-Machine Collaborative Legal Case Matching
However, successful legal case matching requires the tacit knowledge of legal practitioners, which is difficult to verbalize and encode into machines.
Learnable Item Tokenization for Generative Recommendation
Utilizing powerful Large Language Models (LLMs) for generative recommendation has attracted much attention.
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Given the powerful abilities of RAG in providing the latest and helpful auxiliary information, Retrieval-Augmented Large Language Models (RA-LLMs) have emerged to harness external and authoritative knowledge bases, rather than solely relying on the model's internal knowledge, to augment the generation quality of LLMs.
Auto-Encoding Morph-Tokens for Multimodal LLM
For multimodal LLMs, the synergy of visual comprehension (textual output) and generation (visual output) presents an ongoing challenge.
A Survey of Generative Search and Recommendation in the Era of Large Language Models
With the information explosion on the Web, search and recommendation are foundational infrastructures to satisfying users' information needs.
Towards Human-centered Proactive Conversational Agents
Recent research on proactive conversational agents (PCAs) mainly focuses on improving the system's capabilities in anticipating and planning action sequences to accomplish tasks and achieve goals before users articulate their requests.
Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales
Their ability to execute intricate compositional reasoning tasks is also constrained, culminating in a stagnation of learning progression for these models.
Concept -- An Evaluation Protocol on Conversational Recommender Systems with System-centric and User-centric Factors
The conversational recommendation system (CRS) has been criticized regarding its user experience in real-world scenarios, despite recent significant progress achieved in academia.
CIRP: Cross-Item Relational Pre-training for Multimodal Product Bundling
However, the cross-item relations have been under-explored in the current multimodal pre-train models.
A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal Reasoning
The study addresses two key challenges: the trivialization of opinions resulting from excessive summarization and the diversion of focus caused by distractor concepts introduced from images.
LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Images
To address the challenges, we present LLaVA-UHD, a large multimodal model that can efficiently perceive images in any aspect ratio and high resolution.
Ranked #5 on
Long-Context Understanding
on MMNeedle
Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection
Self-detection for Large Language Models (LLMs) seeks to evaluate the trustworthiness of the LLM's output by leveraging its own capabilities, thereby alleviating the issue of output hallucination.
Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulation
We investigate non-collaborative dialogue agents, which are expected to engage in strategic conversations with diverse users, for securing a mutual agreement that leans favorably towards the system's objectives.
FashionReGen: LLM-Empowered Fashion Report Generation
Fashion analysis refers to the process of examining and evaluating trends, styles, and elements within the fashion industry to understand and interpret its current state, generating fashion reports.
Discriminative Probing and Tuning for Text-to-Image Generation
We present a discriminative adapter built on T2I models to probe their discriminative abilities on two representative tasks and leverage discriminative fine-tuning to improve their text-image alignment.
Contrastive Pre-training for Deep Session Data Understanding
In this work, we delve into deep session data understanding via scrutinizing the various clues inside the rich information in user sessions.
Uplift Modeling for Target User Attacks on Recommender Systems
UBA estimates the treatment effect on each target user and optimizes the allocation of fake user budgets to maximize the attack performance.
Learning to Ask Critical Questions for Assisting Product Search
In this paper, we propose a dual-learning model that hybrids the best from both implicit session feedback and proactively clarifying with users on the most critical questions.
Abductive Ego-View Accident Video Understanding for Safe Driving Perception
This model involves a contrastive interaction loss to learn the pair co-occurrence of normal, near-accident, accident frames with the corresponding text descriptions, such as accident reasons, prevention advice, and accident categories.
A Survey on Neural Question Generation: Methods, Applications, and Prospects
In this survey, we present a detailed examination of the advancements in Neural Question Generation (NQG), a field leveraging neural network techniques to generate relevant questions from diverse inputs like knowledge bases, texts, and images.
Prospect Personalized Recommendation on Large Language Model-based Agent Platform
Additionally, we prospect the evolution of Rec4Agentverse and conceptualize it into three stages based on the enhancement of the interaction and information exchange among Agent Items, Agent Recommender, and the user.
On the Multi-turn Instruction Following for Conversational Web Agents
Web agents powered by Large Language Models (LLMs) have demonstrated remarkable abilities in planning and executing multi-step interactions within complex web-based environments, fulfilling a wide range of web navigation tasks.
Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations
Despite the remarkable abilities of Large Language Models (LLMs) to answer questions, they often display a considerable level of overconfidence even when the question does not have a definitive answer.
GraphEdit: Large Language Models for Graph Structure Learning
Graph Structure Learning (GSL) focuses on capturing intrinsic dependencies and interactions among nodes in graph-structured data by generating novel graph structures.
Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning
Large Language Models (LLMs) demonstrate remarkable proficiency in comprehending and handling text-based tasks.
Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond
Building upon this capability, we propose to enable multimodal large language models (MLLMs) to memorize and recall images within their parameters.
Distillation Enhanced Generative Retrieval
Generative retrieval is a promising new paradigm in text retrieval that generates identifier strings of relevant passages as the retrieval target.
LLM-based Federated Recommendation
Nevertheless, applying Fed4Rec to LLM-based recommendation presents two main challenges: first, an increase in the imbalance of performance across clients, affecting the system's efficiency over time, and second, a high demand on clients' computational and storage resources for local training and inference of LLMs.
Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models
The training samples for the PPO trainer are also the responses generated during the reflective process, which eliminates the need for human annotators.
Data-efficient Fine-tuning for LLM-based Recommendation
To pursue the two objectives, we propose a novel data pruning method based on two scores, i. e., influence score and effort score, to efficiently identify the influential samples.
Improving Expressive Power of Spectral Graph Neural Networks with Eigenvalue Correction
Moreover, we theoretically establish that the number of distinguishable eigenvalues plays a pivotal role in determining the expressive power of spectral graph neural networks.
Towards 3D Molecule-Text Interpretation in Language Models
Through 3D molecule-text alignment and 3D molecule-centric instruction tuning, 3D-MoLM establishes an integration of 3D molecular encoder and LM.
TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data
In this work, we address question answering (QA) over a hybrid of tabular and textual data that are very common content on the Web (e. g. SEC filings), where discrete reasoning capabilities are often required.
Instilling Multi-round Thinking to Text-guided Image Generation
This paper delves into the text-guided image editing task, focusing on modifying a reference image according to user-specified textual feedback to embody specific attributes.
Denoising Diffusion Recommender Model
To achieve this target, the key lies in offering appropriate guidance to steer the reverse denoising process and providing a proper starting point to start the forward-reverse process during inference.
GOODAT: Towards Test-time Graph Out-of-Distribution Detection
To identify and reject OOD samples with GNNs, recent studies have explored graph OOD detection, often focusing on training a specific model or modifying the data on top of a well-trained GNN.
LASO: Language-guided Affordance Segmentation on 3D Object
With this regard we propose a novel task Language-guided Affordance Segmentation on 3D Object (LASO) which challenges a model to segment a 3D object's part relevant to a given affordance question.
Temporally and Distributionally Robust Optimization for Cold-Start Recommendation
They learn a feature extractor on warm-start items to align feature representations with interactions, and then leverage the feature extractor to extract the feature representations of cold-start items for interaction prediction.
Towards Goal-oriented Intelligent Tutoring Systems in Online Education
In this work, we investigate a new task, named Goal-oriented Intelligent Tutoring Systems (GITS), which aims to enable the student's mastery of a designated concept by strategically planning a customized sequence of exercises and assessment.
SCTc-TE: A Comprehensive Formulation and Benchmark for Temporal Event Forecasting
Temporal complex event forecasting aims to predict the future events given the observed events from history.
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
Multimodal Large Language Models (MLLMs) have recently demonstrated impressive capabilities in multimodal understanding, reasoning, and interaction.
Ranked #1 on
Visual Question Answering
on VQA v2
MultiCBR: Multi-view Contrastive Learning for Bundle Recommendation
It does, however, have two limitations: 1) the two-view formulation does not fully exploit all the heterogeneous relations among users, bundles and items; and 2) the "early contrast and late fusion" framework is less effective in capturing user preference and difficult to generalize to multiple views.
Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching
Navigating drones through natural language commands remains challenging due to the dearth of accessible multi-modal datasets and the stringent precision requirements for aligning visual and textual data.
A Study of Implicit Ranking Unfairness in Large Language Models
Worse still, in this paper, we identify a subtler form of discrimination in LLMs, termed \textit{implicit ranking unfairness}, where LLMs exhibit discriminatory ranking patterns based solely on non-sensitive user profiles, such as user names.
NExT-Chat: An LMM for Chat, Detection and Segmentation
The development of large language models (LLMs) has greatly advanced the field of multimodal understanding, leading to the emergence of large multimodal models (LMMs).
Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents
Proactive dialogues serve as a practical yet challenging dialogue problem in the era of large language models (LLMs), where the dialogue policy planning is the key to improving the proactivity of LLMs.
Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss
To bridge the gap, we delve into the reasons underpinning the success of contrastive loss in CF, and propose a principled Adversarial InfoNCE loss (AdvInfoNCE), which is a variant of InfoNCE, specially tailored for CF methods.
Leveraging Multimodal Features and Item-level User Feedback for Bundle Construction
Specifically, we use self-attention modules to combine the multimodal and multi-item features, and then leverage both item- and bundle-level contrastive learning to enhance the representation learning, thus to counter the modality missing, noise, and sparsity problems.
Rethinking Tokenizer and Decoder in Masked Graph Modeling for Molecules
Our results show that a subgraph-level tokenizer and a sufficiently expressive decoder with remask decoding have a large impact on the encoder's representation learning.
MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter
MolCA enables an LM (e. g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector.
Ranked #5 on
Molecule Captioning
on ChEBI-20
A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction
Large language models (LLMs) have demonstrated great potential for domain-specific applications, such as the law domain.
On Generative Agents in Recommendation
Recommender systems are the cornerstone of today's information dissemination, yet a disconnect between offline metrics and online performance greatly hinders their development.
Robust Collaborative Filtering to Popularity Distribution Shift
The popularity shortcut tricks are good for in-distribution (ID) performance but poorly generalized to out-of-distribution (OOD) data, i. e., when popularity distribution of test data shifts w. r. t.
Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators
Large language models (LLMs) outperform information retrieval techniques for downstream knowledge-intensive tasks when being prompted to generate world knowledge.
Bridging Items and Language: A Transition Paradigm for Large Language Model-Based Recommendation
Harnessing Large Language Models (LLMs) for recommendation is rapidly emerging, which relies on two fundamental steps to bridge the recommendation item space and the language space: 1) item indexing utilizes identifiers to represent items in the language space, and 2) generation grounding associates LLMs' generated token sequences to in-corpus items.
Progressive Text-to-3D Generation for Automatic 3D Prototyping
We aspire for our work to pave the way for automatic 3D prototyping via natural language descriptions.
NExT-GPT: Any-to-Any Multimodal LLM
While recently Multimodal Large Language Models (MM-LLMs) have made exciting strides, they mostly fall prey to the limitation of only input-side multimodal understanding, without the ability to produce content in multiple modalities.
Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs
In this work, we investigate strengthening the awareness of video dynamics for DMs, for high-quality T2V generation.
I3: Intent-Introspective Retrieval Conditioned on Instructions
Recent studies indicate that dense retrieval models struggle to perform well on a wide variety of retrieval tasks that lack dedicated training data, as different retrieval tasks often entail distinct search intents.
Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction
The hierarchical VAE allows us to learn the complex and low-level latent variables for stock prediction, while the diffusion probabilistic model trains the predictor to handle stock price stochasticity by progressively adding random noise to the stock data.
Context-aware Event Forecasting via Graph Disentanglement
The task of event forecasting aims to model the relational and temporal patterns based on historical events and makes forecasting to what will happen in the future.
LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation
Afterward, we propose a fine-grained object-interaction diffusion method to synthesize high-faithfulness images conditioned on the prompt and the automatically generated layout.
Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labeling
A scene-event mapping mechanism is first designed to bridge the gap between the underlying scene structure and the high-level event semantic structure, resulting in an overall hierarchical scene-event (termed ICE) graph structure.
Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition
On the other hand, during the feature fusion stage, we propose a Contribution-aware Fusion Mechanism (CFM) and a Context Refusion Mechanism (CRM) for multimodal and context integration, respectively.
Ranked #8 on
Emotion Recognition in Conversation
on IEMOCAP
Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions
This shortcoming results in MLLMs' underperformance in comprehending demonstrative instructions consisting of multiple, interleaved, and multimodal instructions that demonstrate the required context to complete a task.
Redundancy-aware Transformer for Video Question Answering
This paper identifies two kinds of redundancy in the current VideoQA paradigm.
SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning
More recently, with the rapid development of language models (LMs), researchers have focused on leveraging LMs to facilitate the learning of TGs, either by jointly training them in a computationally intensive framework (merging the two stages), or designing complex self-supervised training tasks for feature extraction (enhancing the first stage).
Ranked #1 on
Node Property Prediction
on ogbn-arxiv
XNLP: An Interactive Demonstration System for Universal Structured NLP
Structured Natural Language Processing (XNLP) is an important subset of NLP that entails understanding the underlying semantic or syntactic structure of texts, which serves as a foundational component for many downstream applications.