Search Results for author:
Found 67 papers, 37 papers with code
Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text
Machine reading comprehension (MRC) has achieved significant progress on the open domain in recent years, mainly due to large-scale pre-trained language models.
Glyph Features Matter: A Multimodal Solution for EvaHan in LT4HALA2022
We participate in the LT4HALA2022 shared task EvaHan.
Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models
Large Multimodal Reasoning Models (LMRMs) have emerged as a promising paradigm, integrating modalities such as text, images, audio, and video to support complex reasoning capabilities and aiming to achieve comprehensive perception, precise understanding, and deep reasoning.
VideoVista-CulturalLingo: 360$^\circ$ Horizons-Bridging Cultures, Languages, and Domains in Video Comprehension
Assessing the video comprehension capabilities of multimodal AI systems can effectively measure their understanding and reasoning abilities.
A Unified Agentic Framework for Evaluating Conditional Image Generation
This paper introduces CIGEval, a unified agentic framework for comprehensive evaluation of conditional image generation tasks.
Take Off the Training Wheels Progressive In-Context Learning for Effective Alignment
We find that the transformer embeds the task function learned from demonstrations into the separator token representation, which plays an important role in the generation of prior response tokens.
Picking the Cream of the Crop: Visual-Centric Data Selection with Collaborative Agents
Specifically, our approach consists of 1) an image information quantification method via visual agents collaboration to select images with rich visual information, and 2) a visual-centric instruction quality assessment method to select high-quality instruction data related to high-quality images.
Improving Value-based Process Verifier via Structural Prior Injection
In the Large Language Model(LLM) reasoning scenario, people often estimate state value via Monte Carlo sampling.
FilmAgent: A Multi-Agent Framework for End-to-End Film Automation in Virtual 3D Spaces
Virtual film production requires intricate decision-making processes, including scriptwriting, virtual cinematography, and precise actor positioning and actions.
KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Model
As retrieval-augmented generation prevails in large language models, embedding models are becoming increasingly crucial.
RaSeRec: Retrieval-Augmented Sequential Recommendation
Although prevailing supervised and self-supervised learning (SSL)-augmented sequential recommendation (SeRec) models have achieved improved performance with powerful neural network architectures, we argue that they still suffer from two limitations: (1) Preference Drift, where models trained on past data can hardly accommodate evolving user preference; and (2) Implicit Memory, where head patterns dominate parametric learning, making it harder to recall long tails.
Ranked #2 on
Sequential Recommendation
on Amazon-Beauty
CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models
Large Language Models (LLMs) need to adapt to the continuous changes in data, tasks, and user preferences.
SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generation
Recent studies in Retrieval-Augmented Generation (RAG) have investigated extracting evidence from retrieved passages to reduce computational costs and enhance the final RAG performance, yet it remains challenging.
Enhancing Attributed Graph Networks with Alignment and Uniformity Constraints for Session-based Recommendation
In this paper, we propose a model-agnostic framework, named AttrGAU (Attributed Graph Networks with Alignment and Uniformity Constraints), to bring the MIA's superiority into existing attribute-agnostic models, to improve their accuracy and robustness for recommendation.
FunnelRAG: A Coarse-to-Fine Progressive Retrieval Paradigm for RAG
In this work, we propose a progressive retrieval paradigm with coarse-to-fine granularity for RAG, termed FunnelRAG, so as to balance effectiveness and efficiency.
Medico: Towards Hallucination Detection and Correction with Multi-source Evidence Fusion
Therefore, it is necessary to resort to external knowledge to detect and correct the hallucinated content.
Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generation
These images are designed to maintain visual consistency across different scenes using a visual-language prompting method that combines scene descriptions and images of the appearing character and setting.
VideoVista: A Versatile Benchmark for Video Understanding and Reasoning
Through a comprehensive and quantitative evaluation of cutting-edge models, we reveal that: 1) Video-LMMs face difficulties in fine-grained video tasks involving temporal location, object tracking, and anomaly detection; 2) Video-LMMs present inferior logical and relation reasoning abilities; 3) Open-source Video-LMMs' performance is significantly lower than GPT-4o and Gemini-1. 5, lagging by 20 points.
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
Although the Mixture of Experts (MoE) architecture has been employed to efficiently scale large language and image-text models, these efforts typically involve fewer experts and limited modalities.
Ranked #178 on
Visual Question Answering
on MM-Vet
VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context
Large Multimodal Models (LMMs) have achieved impressive success in visual understanding and reasoning, remarkably improving the performance of mathematical reasoning in a visual context.
In-Context Learning State Vector with Inner and Momentum Optimization
In this paper, we address this gap by presenting a comprehensive analysis of these compressed vectors, drawing parallels to the parameters trained with gradient descent, and introduce the concept of state vector.
Improving Attributed Text Generation of Large Language Models via Preference Learning
Large language models have been widely adopted in natural language processing, yet they face the challenge of generating unreliable content.
SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection
In this work, we propose a novel approach, termed SelectIT, that capitalizes on the foundational capabilities of the LLM itself.
Does the Generator Mind its Contexts? An Analysis of Generative Model Faithfulness under Context Transfer
The present study introduces the knowledge-augmented generator, which is specifically designed to produce information that remains grounded in contextual knowledge, regardless of alterations in the context.
A Multimodal In-Context Tuning Approach for E-Commerce Product Description Generation
For this setting, previous methods utilize visual and textual encoders to encode the image and keywords and employ a language model-based decoder to generate the product description.
Cognitive Visual-Language Mapper: Advancing Multimodal Comprehension with Enhanced Visual Knowledge Alignment
Evaluating and Rethinking the current landscape of Large Multimodal Models (LMMs), we observe that widely-used visual-language projection approaches (e. g., Q-former or MLP) focus on the alignment of image-text descriptions yet ignore the visual knowledge-dimension alignment, i. e., connecting visuals to their relevant knowledge.
Towards Faithful Explanations for Text Classification with Robustness Improvement and Explanation Guided Training
Feature attribution methods highlight the important input tokens as explanations to model predictions, which have been widely applied to deep neural networks towards trustworthy AI.
Towards Vision Enhancing LLMs: Empowering Multimodal Knowledge Storage and Sharing in LLMs
These models predominantly map visual information into language representation space, leveraging the vast knowledge and powerful text generation abilities of LLMs to produce multimodal instruction-following responses.
Temporal Knowledge Question Answering via Abstract Reasoning Induction
In this study, we address the challenge of enhancing temporal knowledge reasoning in Large Language Models (LLMs).
Ranked #3 on
Question Answering
on MultiTQ
Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration
Large Language Models (LLMs) have shown remarkable capabilities in general natural language processing tasks but often fall short in complex reasoning tasks.
A Comprehensive Evaluation of GPT-4V on Knowledge-Intensive Visual Question Answering
The emergence of multimodal large models (MLMs) has significantly advanced the field of visual understanding, offering remarkable capabilities in the realm of visual question answering (VQA).
A Survey of Large Language Models Attribution
Open-domain generative systems have gained significant attention in the field of conversational AI (e. g., generative search engines).
Revisiting Sparse Retrieval for Few-shot Entity Linking
Entity linking aims to link ambiguous mentions to their corresponding entities in a knowledge base.
A Read-and-Select Framework for Zero-shot Entity Linking
Zero-shot entity linking (EL) aims at aligning entity mentions to unseen entities to challenge the generalization ability.
Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation
To evaluate the effectiveness of our approach in terms of truthfulness and detoxification, we conduct extensive experiments on LLMs, encompassing additional abilities such as language modeling and mathematical reasoning.
Generative Multimodal Entity Linking
Multimodal Entity Linking (MEL) is the task of mapping mentions with multimodal contexts to the referent entities from a knowledge base.
ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination
As ChatGPT and GPT-4 spearhead the development of Large Language Models (LLMs), more researchers are investigating their performance across various tasks.
A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues
This makes the language model well-suitable for such multi-modal reasoning scenario on joint textual and visual clues.
LMEye: An Interactive Perception Network for Large Language Models
LMEye addresses this issue by allowing the LLM to request the desired visual information aligned with various human instructions, which we term as the dynamic visual information interaction.
A Neural Divide-and-Conquer Reasoning Framework for Image Retrieval from Linguistically Complex Text
Inspired by the Divide-and-Conquer algorithm and dual-process theory, in this paper, we regard linguistically complex texts as compound proposition texts composed of multiple simple proposition sentences and propose an end-to-end Neural Divide-and-Conquer Reasoning framework, dubbed NDCR.
Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation
Moreover, the pipelined approaches of retrieval and generation might result in poor generation performance when retrieval performance is low.
Prompt-based Text Entailment for Low-Resource Named Entity Recognition
To address these challenges, we propose Prompt-based Text Entailment (PTE) for low-resource named entity recognition, which better leverages knowledge in the PLMs.
Low Resource Named Entity Recognition
named-entity-recognition
+3
Calibration Meets Explanation: A Simple and Effective Approach for Model Confidence Estimates
We conduct extensive experiments on six datasets with two popular pre-trained language models in the in-domain and out-of-domain settings.
An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks
Experiments on various knowledge-intensive tasks such as question answering and dialogue datasets show that, simply augmenting parametric models (T5-base) using our method produces more accurate results (e. g., 25. 8 -> 44. 3 EM on NQ) while retaining a high throughput (e. g., 1000 queries/s on NQ).
Ranked #4 on
Question Answering
on KILT: ELI5
Hansel: A Chinese Few-Shot and Zero-Shot Entity Linking Benchmark
We then establish a strong baseline that scores a R@1 of 46. 2% on Few-Shot and 76. 6% on Zero-Shot on our dataset.
Chunk-aware Alignment and Lexical Constraint for Visual Entailment with Natural Language Explanations
CSI), a relation inferrer, and a Lexical Constraint-aware Generator (arr.
Medical Dialogue Response Generation with Pivotal Information Recalling
To mitigate this problem, we propose a medical response generation model with Pivotal Information Recalling (MedPIR), which is built on two components, i. e., knowledge-aware dialogue graph encoder and recall-enhanced generator.
MSDF: A General Open-Domain Multi-Skill Dialog Framework
In this paper, we propose a general Multi-Skill Dialog Framework, namely MSDF, which can be applied in different dialog tasks (e. g. knowledge grounded dialog and persona based dialog).
Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction
Recent works have shown explainability and robustness are two crucial ingredients of trustworthy and reliable text classification.
Sentence-level Online Handwritten Chinese Character Recognition
Single online handwritten Chinese character recognition~(single OLHCCR) has achieved prominent performance.
GlyphCRM: Bidirectional Encoder Representation for Chinese Character with its Glyph
Previous works indicate that the glyph of Chinese characters contains rich semantic information and has the potential to enhance the representation of Chinese characters.
Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning
In this study, we define the spectral graph convolutional network with the high-order dynamic Chebyshev approximation (HDGCN), which augments the multi-hop graph reasoning by fusing messages aggregated from direct and long-term dependencies into one convolutional layer.
You Can Do Better! If You Elaborate the Reason When Making Prediction
The experimental results show that the proposed approach can generate reasonable explanations for its predictions even with a small-scale training corpus.
Transforming Recurrent Neural Networks with Attention and Fixed-point Equations
It has two major parts.
MedWriter: Knowledge-Aware Medical Text Generation
To exploit the domain knowledge to guarantee the correctness of generated text has been a hot topic in recent years, especially for high professional domains such as medical.
Neural Data-to-Text Generation with Dynamic Content Planning
We further design a reconstruction mechanism with a novel objective function that can reconstruct the whole entry of the used data sequentially from the hidden states of the decoder, which aids the accuracy of the generated text.
Text-Guided Neural Image Inpainting
To fulfill such a task, we propose a novel inpainting model named Text-Guided Dual Attention Inpainting Network (TDANet).
Decomposing Word Embedding with the Capsule Network
We propose a Capsule network-based method to Decompose the unsupervised word Embedding of an ambiguous word into context specific Sense embedding, called CapsDecE2S.
Trigger Word Detection and Thematic Role Identification via BERT and Multitask Learning
In this paper, we present our approaches for trigger word detection (task 1) and the identification of its thematic role (task 2) in AGAC track of BioNLP Open Shared Task 2019.
Sentence Simplification with Memory-Augmented Neural Networks
Sentence simplification aims to simplify the content and structure of complex sentences, and thus make them easier to interpret for human readers, and easier to process for downstream NLP applications.
Ranked #1 on
Text Simplification
on PWKP / WikiSmall
Learning to Extract Coherent Summary via Deep Reinforcement Learning
As an effort towards extracting coherent summaries, we propose a neural coherence model to capture the cross-sentence semantic and syntactic coherence patterns.
Ranked #3 on
Text Summarization
on CNN / Daily Mail (Anonymized)
Stroke Sequence-Dependent Deep Convolutional Neural Network for Online Handwritten Chinese Character Recognition
In this paper, we propose a novel model, named Stroke Sequence-dependent Deep Convolutional Neural Network (SSDCNN), using the stroke sequence information and eight-directional features for Online Handwritten Chinese Character Recognition (OLHCCR).
Answer Sequence Learning with Neural Networks for Answer Selection in Community Question Answering
In this paper, the answer selection problem in community question answering (CQA) is regarded as an answer sequence labeling task, and a novel approach is proposed based on the recurrent architecture for this problem.
LCSTS: A Large Scale Chinese Short Text Summarization Dataset
Automatic text summarization is widely regarded as the highly difficult problem, partially because of the lack of large text summarization data set.
Ranked #1 on
Text Summarization
on LCSTS
Convolutional Neural Network Architectures for Matching Natural Language Sentences
Semantic matching is of central importance to many natural language tasks \cite{bordes2014semantic, RetrievalQA}.
Ranked #3 on
Question Answering
on SemEvalCQA
Context-Dependent Translation Selection Using Convolutional Neural Network
We propose a novel method for translation selection in statistical machine translation, in which a convolutional neural network is employed to judge the similarity between a phrase pair in two languages.
