Search Results for author:
Found 108 papers, 74 papers with code
What Does Your Smile Mean? Jointly Detecting Multi-Modal Sarcasm and Sentiment Using Quantum Probability
The QPM framework involves a complex-valued multi-modal representation encoder, a quantum-like fusion subnetwork and a quantum measurement mechanism.
ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation
Recent advances in multimodal generative models have unlocked photorealistic, instruction-aligned image generation, yet leading systems like GPT-4o-Image remain proprietary and inaccessible.
QFFT, Question-Free Fine-Tuning for Adaptive Reasoning
This approach enables the model to adaptively employ both reasoning patterns: it prioritizes the Short CoT patterns and activates the Long CoT patterns only when necessary.
CoRT: Code-integrated Reasoning within Thinking
Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable progress in natural language reasoning with long chain-of-thought (CoT), yet they remain inefficient or inaccurate when handling complex mathematical operations.
FusionAudio-1.2M: Towards Fine-grained Audio Captioning with Multimodal Contextual Fusion
High-quality, large-scale audio captioning is crucial for advancing audio understanding, yet current automated methods often generate captions that lack fine-grained detail and contextual accuracy, primarily due to their reliance on limited unimodal or superficial multimodal information.
Model Unlearning via Sparse Autoencoder Subspace Guided Projections
Large language models (LLMs) store vast amounts of information, making them powerful yet raising privacy and safety concerns when selective knowledge removal is required.
From Word to World: Evaluate and Mitigate Culture Bias via Word Association Test
We extend this test into an LLM-adaptive, free-relation task to assess the alignment of large language models (LLMs) with cross-cultural cognition.
Leveraging Unit Language Guidance to Advance Speech Modeling in Textless Speech-to-Speech Translation
We propose the unit language to overcome the two modeling challenges.
Real-Time Verification of Embodied Reasoning for Generative Skill Acquisition
VERGSA establishes 1) a seamless extension from verification of mathematical reasoning into embodied learning by dynamically incorporating contextually relevant tasks into prompts and defining success metrics for both subtasks and overall tasks, and 2) an automated, scalable reward labeling scheme that synthesizes dense reward signals by iteratively finalizing the contribution of scene configuration and subtask learning to overall skill acquisition.
Learning from Peers in Reasoning Models
Notably, our fine-tuned LeaP-T-7B matches the performance of DeepSeek-R1-Distill-Qwen-14B on AIME 2024.
Video-R1: Reinforcing Video Reasoning in MLLMs
However, directly applying RL training with the GRPO algorithm to video reasoning presents two primary challenges: (i) a lack of temporal modeling for video reasoning, and (ii) the scarcity of high-quality video-reasoning data.
Large Language Models for Outpatient Referral: Problem Definition, Benchmarking and Challenges
Large language models (LLMs) are increasingly applied to outpatient referral tasks across healthcare systems.
S2S-Arena, Evaluating Speech2Speech Protocols on Instruction Following with Paralinguistic Information
The rapid development of large language models (LLMs) has brought significant attention to speech models, particularly recent progress in speech2speech protocols supporting speech input and output.
The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models
By training exclusively on the initial prefix substrings (as few as 8 tokens), UPFT removes the need for labeled data or exhaustive sampling.
NavigateDiff: Visual Predictors are Zero-Shot Navigation Assistants
Navigating unfamiliar environments presents significant challenges for household robots, requiring the ability to recognize and reason about novel decoration and layout.
Soundwave: Less is More for Speech-Text Alignment in LLMs
Existing end-to-end speech large language models (LLMs) usually rely on large-scale annotated data for training, while data-efficient training has not been discussed in depth.
Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
Fine-tuning significantly improves the performance of Large Language Models (LLMs), yet its underlying mechanisms remain poorly understood.
Uncertainty-Aware Search and Value Models: Mitigating Search Scaling Flaws in LLMs
Value model-guided search is effective in steering the generation but suffers from scaling flaws: Its superiority diminishes with larger sample sizes, underperforming non-search baselines.
RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques
In this work, we introduce a new benchmark designed to assess the critique capabilities of LLMs.
Enabling Scalable Oversight via Self-Evolving Critic
Despite their remarkable performance, the development of Large Language Models (LLMs) faces a critical challenge in scalable oversight: providing effective feedback for tasks where human evaluation is difficult or where LLMs outperform humans.
On the Compositional Generalization of Multimodal LLMs for Medical Imaging
Multimodal large language models (MLLMs) hold significant potential in the medical field, but their capabilities are often limited by insufficient data in certain medical domains, highlighting the need for understanding what kinds of images can be used by MLLMs for generalization.
HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
To address this, we propose verifiable medical problems with a medical verifier to check the correctness of model outputs.
BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement
In this paper, we present BlenderLLM, a novel framework for training LLMs specifically for CAD tasks leveraging a self-improvement methodology.
Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion
This paper addresses the critical need for democratizing large language models (LLM) in the Arab world, a region that has seen slower progress in developing models comparable to state-of-the-art offerings like GPT-4 or ChatGPT 3. 5, due to a predominant focus on mainstream languages (e. g., English and Chinese).
Alignment at Pre-training! Towards Native Alignment for Arabic LLMs
The alignment of large language models (LLMs) is critical for developing effective and safe language models.
AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?
Recently, multimodal large language models (MLLMs), such as GPT-4o, Gemini 1. 5 Pro, and Reka Core, have expanded their capabilities to include vision and audio modalities.
Both Text and Images Leaked! A Systematic Analysis of Multimodal LLM Data Contamination
The rapid progression of multimodal large language models (MLLMs) has demonstrated superior performance on various multimodal benchmarks.
Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement
The rapid development of large language models (LLMs), like ChatGPT, has resulted in the widespread presence of LLM-generated content on social media platforms, raising concerns about misinformation, data biases, and privacy violations, which can undermine trust in online discourse.
UCFE: A User-Centric Financial Expertise Benchmark for Large Language Models
Secondly, based on this feedback, we created our dataset that encompasses a wide range of user intents and interactions.
Roadmap towards Superhuman Speech Understanding using Large Language Models
The success of large language models (LLMs) has prompted efforts to integrate speech and audio data, aiming to create general foundation models capable of processing both textual and non-textual inputs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Experts
In order to leverage the generalization capability of multilingual LLMs to efficiently scale to more resource-constrained languages, we explore the internal information flow of LLMs from a multilingual perspective using Mixture of Experts (MoE) modularity.
VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment
As large vision-language models (LVLMs) evolve rapidly, the demand for high-quality and diverse data to align these models becomes increasingly crucial.
Ranked #62 on
Visual Question Answering
on MM-Vet
Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
However, existing benchmarks like GSM8K or MATH are now being solved with high accuracy (e. g., OpenAI o1 achieves 94. 8\% on MATH dataset), indicating their inadequacy for truly challenging these models.
Less is More: A Simple yet Effective Token Reduction Method for Efficient Multi-modal LLMs
The rapid advancement of Multimodal Large Language Models (MLLMs) has led to remarkable performances across various domains.
LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecture
Expanding the long-context capabilities of Multi-modal Large Language Models~(MLLMs) is crucial for video understanding, high-resolution image understanding, and multi-modal agents.
An Empirical Study on Information Extraction using Large Language Models
To demonstrate the latest representative progress in LLMs' information extraction ability, we assess the information extraction ability of GPT-4 (the latest version of GPT at the time of writing this paper) from four perspectives: Performance, Evaluation Criteria, Robustness, and Error Types.
Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications
Financial LLMs hold promise for advancing financial tasks and domain-specific applications.
GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI
Large Vision-Language Models (LVLMs) are capable of handling diverse data types such as imaging, text, and physiological signals, and can be applied in various fields.
CoD, Towards an Interpretable Medical Agent using Chain of Diagnosis
This study introduces Chain-of-Diagnosis (CoD) to enhance the interpretability of LLM-based medical diagnostics.
HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale
The rapid development of multimodal large language models (MLLMs), such as GPT-4V, has led to significant advancements.
LLMs for Doctors: Leveraging Medical LLMs to Assist Doctors, Not Replace Them
Based on this, we construct a Chinese medical dataset called DoctorFLAN to support the entire workflow of doctors, which includes 92K Q\&A samples from 22 tasks and 27 specialists.
MedBench: A Comprehensive, Standardized, and Reliable Benchmarking System for Evaluating Chinese Medical Large Language Models
In this work, we introduce "MedBench", a comprehensive, standardized, and reliable benchmarking system for Chinese medical LLM.
LLMs Could Autonomously Learn Without External Supervision
In the quest for super-human performance, Large Language Models (LLMs) have traditionally been tethered to human-annotated datasets and predefined training objectives-a process that is both labor-intensive and inherently limited.
MotionLLM: Understanding Human Behaviors from Human Motions and Videos
This study delves into the realm of multi-modality (i. e., video and motion modalities) human behavior understanding by leveraging the powerful capabilities of Large Language Models (LLMs).
Boosting Protein Language Models with Negative Sample Mining
We introduce a pioneering methodology for boosting large language models in the domain of protein representation learning.
ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling
This work also introduces IndustryOR, the first industrial benchmark for evaluating LLMs in solving practical OR problems.
Mixture of Latent Experts Using Tensor Products
For \textit{routing function}, we tailor two innovative routing functions according to the granularity: \texttt{TensorPoly-I} which directs to each rank within the entangled tensor while \texttt{TensorPoly-II} offers a finer-grained routing approach targeting each order of the entangled tensor.
Mamo: a Mathematical Modeling Benchmark with Solvers
By focusing on the processes LLMs undertake rather than the correctness of their final solutions, Mamo pioneers a novel evaluation paradigm.
Is Your LLM Outdated? Evaluating LLMs at Temporal Generalization
The rapid advancement of Large Language Models (LLMs) highlights the urgent need for evolving evaluation methodologies that keep pace with improvements in language comprehension and information processing.
Smurfs: Leveraging Multiple Proficiency Agents with Context-Efficiency for Tool Planning
The emergence of large language models (LLMs) has opened up unprecedented possibilities for automating complex tasks that are often comparable to human performance.
MileBench: Benchmarking MLLMs in Long Context
Despite the advancements and impressive performance of Multimodal Large Language Models (MLLMs) on benchmarks, their effectiveness in real-world, long-context, and multi-image tasks is unclear due to the benchmarks' limited scope.
No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarks
While the progression of Large Language Models (LLMs) has notably propelled financial analysis, their application has largely been confined to singular language realms, leaving untapped the potential of bilingual Chinese-English capacity.
Apollo: A Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B People
Despite the vast repository of global medical knowledge predominantly being in English, local languages are crucial for delivering tailored healthcare services, particularly in areas with limited medical resources.
Online Training of Large Language Models: Learn while chatting
Large Language Models(LLMs) have dramatically revolutionized the field of Natural Language Processing(NLP), offering remarkable capabilities that have garnered widespread usage.
Rethinking The Uniformity Metric in Self-Supervised Learning
Uniformity plays an important role in evaluating learned representations, providing insights into self-supervised learning.
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
We introduce Generalized Instruction Tuning (called GLAN), a general and scalable method for instruction tuning of Large Language Models (LLMs).
FinBen: A Holistic Financial Benchmark for Large Language Models
Our evaluation of 15 representative LLMs, including GPT-4, ChatGPT, and the latest Gemini, reveals several key findings: While LLMs excel in IE and textual analysis, they struggle with advanced reasoning and complex tasks like text generation and forecasting.
ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Models
Large vision-language models (LVLMs) have shown premise in a broad range of vision-language tasks with their strong reasoning and generalization capabilities.
Humans or LLMs as the Judge? A Study on Judgement Biases
We further exploit these biases to conduct attacks on LLM judges.
Pushing The Limit of LLM Capacity for Text Classification
The value of text classification's future research has encountered challenges and uncertainties, due to the extraordinary efficacy demonstrated by large language models (LLMs) across numerous downstream NLP tasks.
Silkie: Preference Distillation for Large Visual Language Models
This paper explores preference distillation for large vision language models (LVLMs), improving their ability to generate helpful and faithful responses anchoring the visual context.
Ranked #66 on
Visual Question Answering
on MM-Vet
MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
To this end, in our paper, we propose a new evaluation paradigm for MLLMs, which is evaluating MLLMs with per-sample criteria using potent MLLM as the judge.
OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning
These findings offer a novel perspective on the role of outcome supervision in training value models for multi-step reasoning tasks and provide theoretical justification for its advantage in value estimation for guided decoding.
Ranked #47 on
Arithmetic Reasoning
on GSM8K
HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs
We validate the new protocol in the domains where proprietary LLMs like ChatGPT perform relatively poorly, such as Traditional Chinese Medicine.
On Elastic Language Models
To this end, we propose an elastic language model (ElasticLM) that elastically adjusts the tradeoff according to the request stream.
Quantifying Self-diagnostic Atomic Knowledge in Chinese Medical Foundation Model: A Computational Analysis
Recent studies primarily focused on the quality of FMs evaluated by GPT-4 or their ability to pass medical exams, no studies have quantified the extent of self-diagnostic atomic knowledge stored in FMs' memory, which is the basis of foundation models to provide factual and reliable suggestions.
DialogueLLM: Context and Emotion Knowledge-Tuned Large Language Models for Emotion Recognition in Conversations
Large language models (LLMs) and their variants have shown extraordinary efficacy across numerous downstream natural language processing (NLP) tasks, which has presented a new vision for the development of NLP.
AceGPT, Localizing Large Language Models in Arabic
This paper is devoted to the development of a localized Large Language Model (LLM) specifically for Arabic, a language imbued with unique cultural characteristics inadequately addressed by current mainstream models.
PlatoLM: Teaching LLMs in Multi-Round Dialogue via a User Simulator
The unparalleled performance of closed-sourced ChatGPT has sparked efforts towards its democratization, with notable strides made by leveraging real user and ChatGPT dialogues, as evidenced by Vicuna.
CMB: A Comprehensive Medical Benchmark in Chinese
We hope this benchmark provide first-hand experience in existing LLMs for medicine and also facilitate the widespread adoption and enhancement of medical LLMs within China.
On the Difference of BERT-style and CLIP-style Text Encoders
Masked language modeling (MLM) has been one of the most popular pretraining recipes in natural language processing, e. g., BERT, one of the representative models.
A Survey of Quantum-Cognitively Inspired Sentiment Analysis Models
Quantum theory, originally proposed as a physical theory to describe the motions of microscopic particles, has been applied to various non-physics domains involving human cognition and decision-making that are inherently uncertain and exhibit certain non-classical, quantum-like characteristics.
Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias
Med-UniC reaches superior performance across 5 medical image tasks and 10 datasets encompassing over 30 diseases, offering a versatile framework for unifying multi-modal medical data within diverse linguistic communities.
HuatuoGPT, towards Taming Language Model to Be a Doctor
Experimental results demonstrate that HuatuoGPT achieves state-of-the-art results in performing medical consultation among open-source LLMs in GPT-4 evaluation, human evaluation, and medical benchmark datasets.
Injecting Knowledge into Biomedical Pre-trained Models via Polymorphism and Synonymous Substitution
This gives us a hint that relational knowledge might not be redundant to the stored knowledge of PLMs, but rather be complementary.
An Empirical Study on Information Extraction using Large Language Models
To demonstrate the latest representative progress in LLMs' information extraction ability, we assess the information extraction ability of GPT-4 (the latest version of GPT at the time of writing this paper) from four perspectives: Performance, Evaluation Criteria, Robustness, and Error Types.
Lifting the Curse of Capacity Gap in Distilling Language Models
However, when the capacity gap between the teacher and the student is large, a curse of capacity gap appears, invoking a deficiency in distilling LMs.
Word Grounded Graph Convolutional Network
An inductive Word-grounded Graph Convolutional Network (WGCN) is proposed to learn word and document representations based on WGraph in a supervised manner.
Huatuo-26M, a Large-scale Chinese Medical QA Dataset
Moreover, we also experimentally show the benefit of the proposed dataset in many aspects: (i) trained models for other QA datasets in a zero-shot fashion; and (ii) as external knowledge for retrieval-augmented generation (RAG); and (iii) improving existing pre-trained language models by using the QA pairs as a pre-training corpus in continued training manner.
Effective Open Intent Classification with K-center Contrastive Learning and Adjustable Decision Boundary
In this paper, we introduce novel K-center contrastive learning and adjustable decision boundary learning (CLAB) to improve the effectiveness of open intent classification.
Phoenix: Democratizing ChatGPT across Languages
This paper presents our efforts to democratize ChatGPT across language.
A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Models
In this paper, we present a systematic review of recent advancements in BTS, leveraging cutting-edge NLP techniques from PLMs to LLMs, to help understand the latest progress, challenges, and future directions.
Natural Language Reasoning, A Survey
This survey paper proposes a clearer view of natural language reasoning in the field of Natural Language Processing (NLP), both conceptually and practically.
Modular Retrieval for Generalization and Interpretation
Built upon the paradigm, we propose a retrieval model with modular prompt tuning named REMOP.
Adapting Pre-trained Language Models for Quantum Natural Language Processing
The emerging classical-quantum transfer learning paradigm has brought a decent performance to quantum computational models in many tasks, such as computer vision, by enabling a combination of quantum models and classical pre-trained neural networks.
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts
Medical vision-and-language pre-training (Med-VLP) has shown promising improvements on many downstream medical tasks owing to its applicability to extracting generic representations from medical images and texts.
Document-level Relation Extraction with Relation Correlations
Document-level relation extraction faces two overlooked challenges: long-tail problem and multi-label problem.
MorphTE: Injecting Morphology in Tensorized Embeddings
In the era of deep learning, word embeddings are essential when dealing with text tasks.
Spatio-Temporal Contrastive Learning Enhanced GNNs for Session-based Recommendation
The idea is to supplement the GNN-based main supervised recommendation task with the temporal representation via an auxiliary cross-view contrastive learning mechanism.
DPTDR: Deep Prompt Tuning for Dense Passage Retrieval
We believe this work facilitates the industry, as it saves enormous efforts and costs of deployment and increases the utility of computing resources.
Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets
To uncover the domain-general LM, we propose to identify domain-general parameters by playing lottery tickets (dubbed doge tickets).
Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk
However, the humor aspect of natural language is relatively under-investigated, especially in the age of pre-trained language models.
Exploring Extreme Parameter Compression for Pre-trained Language Models
A tiny version achieves $96. 7\%$ performance of BERT-base with $ {1}/{48} $ encoder parameters (i. e., less than 2M parameters excluding the embedding layer) and $2. 7 \times$ faster on inference.
Word2Fun: Modelling Words as Functions for Diachronic Word Representation
Word meaning may change over time as a reflection of changes in human society.
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
In this paper, we summarize the recent progress of pre-trained language models in the biomedical domain and their applications in biomedical downstream tasks.
On Position Embeddings in BERT
Various Position Embeddings (PEs) have been proposed in Transformer based architectures~(e. g. BERT) to model word order.
Meta-Learning for Neural Relation Classification with Distant Supervision
Distant supervision provides a means to create a large number of weakly labeled data at low cost for relation classification.
A Multi-task Learning Framework for Opinion Triplet Extraction
The state-of-the-art Aspect-based Sentiment Analysis (ABSA) approaches are mainly based on either detecting aspect terms and their corresponding sentiment polarities, or co-extracting aspect and opinion terms.
Aspect-Based Sentiment Analysis
Aspect Sentiment Triplet Extraction
+3
Leveraging Long and Short-Term Information in Content-Aware Movie Recommendation via Adversarial Training
Movie recommendation systems provide users with ranked lists of movies based on individual’s preferences and constraints.
Encoding word order in complex embeddings
The benefit of continuous functions over variable positions is that word representations shift smoothly with increasing positions.
Leveraging Entanglement Entropy for Deep Understanding of Attention Matrix in Text Matching
We prove that by a mapping (via the trace operator) on the high-dimensional matching matrix, a low-dimensional attention matrix can be derived.
CNM: An Interpretable Complex-valued Network for Matching
This paper seeks to model human language by the mathematical framework of quantum physics.
Semantic Hilbert Space for Text Representation Learning
To address this issue, we propose a new framework that models different levels of semantic units (e. g. sememe, word, sentence, and semantic abstraction) on a single \textit{Semantic Hilbert Space}, which naturally admits a non-linear semantic composition by means of a complex-valued vector word representation.
A Quantum Many-body Wave Function Inspired Language Modeling Approach
The recently proposed quantum language model (QLM) aimed at a principled approach to modeling term dependency by applying the quantum probability theory.
Quantum-inspired Complex Word Embedding
A challenging task for word embeddings is to capture the emergent meaning or polarity of a combination of individual words.
TextZoo, a New Benchmark for Reconsidering Text Classification
Text representation is a fundamental concern in Natural Language Processing, especially in text classification.
IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models
This paper provides a unified account of two schools of thinking in information retrieval modelling: the generative retrieval focusing on predicting relevant documents given a query, and the discriminative retrieval focusing on predicting relevancy given a query-document pair.
