Search Results for author:
Found 427 papers, 307 papers with code
BMInf: An Efficient Toolkit for Big Model Inference and Tuning
In recent years, large-scale pre-trained language models (PLMs) containing billions of parameters have achieved promising results on various NLP tasks.
Going “Deeper”: Structured Sememe Prediction via Transformer with Tree Attention
However, all existing sememe prediction studies ignore the hierarchical structures of sememes, which are important in the sememe-based semantic description system.
Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages
In this paper, we propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages.
Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach
In recent years, pre-trained language models (PLMs) have been shown to capture factual knowledge from massive texts, which encourages the proposal of PLM-based knowledge graph completion (KGC) models.
LEGO-ABSA: A Prompt-based Task Assemblable Unified Generative Framework for Multi-task Aspect-based Sentiment Analysis
This paper proposes a unified generative multi-task framework that can solve multiple ABSA tasks by controlling the type of task prompts consisting of multiple element prompts.
Ranked #5 on
Aspect-Based Sentiment Analysis (ABSA)
on TASD
(using extra training data)
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
CodRED: A Cross-Document Relation Extraction Dataset for Acquiring Knowledge in the Wild
Existing relation extraction (RE) methods typically focus on extracting relational facts between entity pairs within single sentences or documents.
Pass off Fish Eyes for Pearls: Attacking Model Selection of Pre-trained Models
To validate our viewpoints, we design two methods to evaluate the robustness of FMS: (1) model disguise attack, which post-trains an inferior PTM with a contrastive objective, and (2) evaluation data selection, which selects a subset of the data points for FMS evaluation based on K-means clustering.
LLM$\times$MapReduce-V2: Entropy-Driven Convolutional Test-Time Scaling for Generating Long-Form Articles from Extremely Long Resources
Long-form generation is crucial for a wide range of practical applications, typically categorized into short-to-long and long-to-long generation.
AIR: A Systematic Analysis of Annotations, Instructions, and Response Pairs in Preference Dataset
Preference learning is critical for aligning large language models (LLMs) with human values, yet its success hinges on high-quality datasets comprising three core components: Preference \textbf{A}nnotations, \textbf{I}nstructions, and \textbf{R}esponse Pairs.
XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?
On top of the XLRS-Bench, 16 sub-tasks are defined to evaluate MLLMs' 10 kinds of perceptual capabilities and 6 kinds of reasoning capabilities, with a primary emphasis on advanced cognitive processes that facilitate real-world decision-making and the capture of spatiotemporal changes.
UltraRAG: A Modular and Automated Toolkit for Adaptive Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) significantly enhances the performance of large language models (LLMs) in downstream tasks by integrating external knowledge.
OThink-MR1: Stimulating multimodal generalized reasoning capabilities via dynamic reinforcement learning
Multimodal Large Language Models (MLLMs) have gained significant traction for their ability to process diverse input data types and generate coherent, contextually relevant outputs across various applications.
Towards Unified Latent Space for 3D Molecular Latent Diffusion Modeling
In this work, we propose \textbf{U}nified Variational \textbf{A}uto-\textbf{E}ncoder for \textbf{3D} Molecular Latent Diffusion Modeling (\textbf{UAE-3D}), a multi-modal VAE that compresses 3D molecules into latent sequences from a unified latent space, while maintaining near-zero reconstruction error.
A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules
Training large models is both resource-intensive and time-consuming, making it crucial to understand the quantitative relationship between model performance and hyperparameters.
WiFi-Diffusion: Achieving Fine-Grained WiFi Radio Map Estimation With Ultra-Low Sampling Rate by Diffusion Models
Fine-grained radio map presents communication parameters of interest, e. g., received signal strength, at every point across a large geographical region.
Multimodal Language Modeling for High-Accuracy Single Cell Transcriptomics Analysis and Generation
Pre-trained language models (PLMs) have revolutionized scientific research, yet their application to single-cell analysis remains limited.
Cost-Optimal Grouped-Query Attention for Long-Context LLMs
However, they overlook the impacts of context length and attention head configuration (the number of query and key-value heads in grouped-query attention) on training and inference.
AgentRM: Enhancing Agent Generalization with Reward Modeling
Based on this finding, we propose AgentRM, a generalizable reward model, to guide the policy model for effective test-time search.
PIP-KAG: Mitigating Knowledge Conflicts in Knowledge-Augmented Generation via Parametric Pruning
Knowledge-Augmented Generation (KAG) has shown great promise in updating the internal memory of Large Language Models (LLMs) by integrating external knowledge.
LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models
Moreover, to achieve long outputs that maintain high-fidelity to the input images, we employ Direct Preference Optimization (DPO) to the SFT model.
TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators
Despite advances in large language models (LLMs) for conventional code generation, these models struggle to generate accurate, performance-optimized Triton code, as they lack awareness of its specifications and the complexities of GPU programming.
FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling
Speculative sampling has emerged as an important technique for accelerating the auto-regressive generation process of large language models (LLMs) by utilizing a draft-then-verify mechanism to produce multiple tokens per forward pass.
Craw4LLM: Efficient Web Crawling for LLM Pretraining
Web crawl is a main source of large language models' (LLMs) pretraining data, but the majority of crawled web pages are discarded in pretraining due to low data quality.
NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation
To combine these advantages for 3D molecule generation, we propose a foundation model -- NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation.
Learning More Effective Representations for Dense Retrieval through Deliberate Thinking Before Search
Recent dense retrievers usually thrive on the emergency capabilities of Large Language Models (LLMs), using them to encode queries and documents into an embedding space for retrieval.
Exploring LLM-based Student Simulation for Metacognitive Cultivation
Metacognitive education plays a crucial role in cultivating students' self-regulation and reflective thinking, providing essential support for those with learning difficulties through academic advising.
APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs
While long-context inference is crucial for advancing large language model (LLM) applications, its prefill speed remains a significant bottleneck.
Diffusion Models for Molecules: A Survey of Methods and Tasks
This survey aims to facilitate understanding and further flourishing development in this area.
Process Reinforcement through Implicit Rewards
While dense rewards also offer an appealing choice for the reinforcement learning (RL) of LLMs since their fine-grained rewards have the potential to address some inherent issues of outcome rewards, such as training efficiency and credit assignment, this potential remains largely unrealized.
ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation
: (1) Low executability and poor restoration of chart details in the generated code and (2) Lack of large-scale and diverse training data.
Migician: Revealing the Magic of Free-Form Multi-Image Grounding in Multimodal Large Language Models
The recent advancement of Multimodal Large Language Models (MLLMs) has significantly improved their fine-grained perception of single images and general comprehension across multiple images.
LLaVA-UHD v2: an MLLM Integrating High-Resolution Feature Pyramid via Hierarchical Window Transformer
To address this issue, we present LLaVA-UHD v2, an advanced MLLM centered around a Hierarchical window transformer that enables capturing diverse visual granularity by constructing and integrating a high-resolution feature pyramid.
Graph Coarsening via Supervised Granular-Ball for Scalable Graph Neural Network Training
Experimental results demonstrate that our method achieves accuracy comparable to training on the original graph.
ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformer
The analysis of the trade-off between autoregressive modeling and diffusion demonstrates the potential of ACDiT to be used in long-horizon visual generation tasks.
Ranked #8 on
Video Generation
on UCF-101
Intelligent System for Automated Molecular Patent Infringement Assessment
Automated drug discovery offers significant potential for accelerating the development of novel therapeutics by substituting labor-intensive human workflows with machine-driven processes.
Densing Law of LLMs
This paper introduces the concept of ``\textit{capacity density}'' as a new metric to evaluate the quality of the LLMs across different scales and describes the trend of LLMs in terms of both effectiveness and efficiency.
Free Process Rewards without Process Labels
The only assumption is to parameterize the outcome reward as the log-likelihood ratios of the policy and reference models, which can be optimized regardless of the specific choice of loss objectives.
Yi-Lightning Technical Report
This technical report presents Yi-Lightning, our latest flagship large language model (LLM).
KBAlign: Efficient Self Adaptation on Specific Knowledge Bases
Humans can utilize techniques to quickly acquire knowledge from specific materials in advance, such as creating self-assessment questions, enabling us to achieving related tasks more efficiently.
ENAT: Rethinking Spatial-temporal Interactions in Token-based Image Synthesis
At the spatial level, we disentangle the computations of visible and mask tokens by encoding visible tokens independently, while decoding mask tokens conditioned on the fully encoded visible tokens.
WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models
Finally, we merge the synthetic samples that pass quality confirmation with the collected samples to obtain the WorkflowBench.
Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
These demonstrate that ReLU is more efficient as the activation function than SiLU and can leverage more training data to improve activation sparsity.
MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning
Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA have significantly improved the adaptation of LLMs to downstream tasks in a resource-efficient manner.
RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards
Our experiments on various knowledge-intensive tasks demonstrate that DDR significantly outperforms the SFT method, particularly for LLMs with smaller-scale parameters that depend more on the retrieved knowledge.
Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance
The labeled data is used to train a reward model that simulates human judgment and serves as an automatic evaluator of the proactiveness of LLM agents.
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents
In this pipeline, instead of first parsing the document to obtain text, the document is directly embedded using a VLM as an image and then retrieved to enhance the generation of a VLM.
LLM$\times$MapReduce: Simplified Long-Sequence Processing using Large Language Models
The proposed LLM$\times$MapReduce framework splits the entire document into several chunks for LLMs to read and then aggregates the intermediate answers to produce the final output.
Retriever-and-Memory: Towards Adaptive Note-Enhanced Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) mitigates issues of the factual errors and hallucinated outputs generated by Large Language Models (LLMs) in open-domain question-answering tasks (OpenQA) via introducing external knowledge.
Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System
Large Language Model (LLM) based multi-agent systems (MAS) show remarkable potential in collaborative problem-solving, yet they still face critical challenges: low communication efficiency, poor scalability, and a lack of effective parameter-updating optimization methods.
Stuffed Mamba: State Collapse and State Capacity of RNN-Based Long-Context Modeling
For the second concern, we train a series of Mamba-2 models on long documents to empirically estimate the recurrent state capacity in language modeling and passkey retrieval.
Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs
Besides, the construction method can also be applied to civil cases and achieve promising results.
DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models
The performance of Large Language Models (LLMs) is substantially influenced by the pretraining corpus, which consists of vast quantities of unsupervised data processed by the models.
Exploring the Benefit of Activation Sparsity in Pre-training
SSD adaptively switches between the Mixtures-of-Experts (MoE) based sparse training and the conventional dense training during the pre-training process, leveraging the efficiency of sparse training and avoiding the static activation correlation of sparse training.
Text-guided Diffusion Model for 3D Molecule Generation
The de novo generation of molecules with targeted properties is crucial in biology, chemistry, and drug discovery.
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
Scaling the input context length of a large language model (LLM) incurs a significant increase in computation cost and memory footprint to maintain the attention key-value (KV) cache.
Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models
Our results underscore that the capabilities of LLMs in handling structured data are still under-explored, and show the effectiveness of LLM4Graph in enhancing LLMs' proficiency of graph analysis.
Enabling Real-Time Conversations with Minimal Training Costs
To address this limitation, researchers have proposed duplex models.
Awaking the Slides: A Tuning-free and Knowledge-regulated AI Tutoring System via Language Model Coordination
We develop Slide2Lecture, a tuning-free and knowledge-regulated intelligent tutoring system that can (1) effectively convert an input lecture slide into a structured teaching agenda consisting of a set of heterogeneous teaching actions; (2) create and manage an interactive lecture that generates responsive interactions catering to student learning demands while regulating the interactions to follow teaching actions.
From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents
Since the first instances of online education, where courses were uploaded to accessible and shared online platforms, this form of scaling the dissemination of human knowledge to reach a broader audience has sparked extensive discussion and widespread adoption.
Configurable Foundation Models: Building LLMs from a Modular Perspective
We first formalize modules into emergent bricks - functional neuron partitions that emerge during the pre-training phase, and customized bricks - bricks constructed via additional post-training to improve the capabilities and knowledge of LLMs.
Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slip Scripts
This study presents a multi-modal multi-granularity tokenizer specifically designed for analyzing ancient Chinese scripts, focusing on the Chu bamboo slip (CBS) script used during the Spring and Autumn and Warring States period (771-256 BCE) in Ancient China.
AdaNAT: Exploring Adaptive Policy for Token-Based Image Generation
As a representative work, non-autoregressive Transformers (NATs) are able to synthesize images with decent quality in a small number of steps.
AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems
However, configuring an MAS for a task remains challenging, with performance only observable post-execution.
COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis
In this paper, we introduce DEBUGEVAL, a comprehensive benchmark for evaluating the debugging abilities of LLMs by emulating the multi-stage human debugging process.
MiniCPM-V: A GPT-4V Level MLLM on Your Phone
The recent surge of Multimodal Large Language Models (MLLMs) has fundamentally reshaped the landscape of AI research and industry, shedding light on a promising path toward the next AI milestone.
Ranked #7 on
Multiple-choice
on Neptune-Full
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework
However, evaluating the effectiveness of RAG systems in specialized scenarios remains challenging due to the high costs of data construction and the lack of suitable evaluation metrics.
Interior Object Geometry via Fitted Frames
We describe a representation targeted for anatomic objects which is designed to enable strong locational correspondence within object populations and thus to provide powerful object statistics.
PersLLM: A Personified Training Approach for Large Language Models
Large language models exhibit aspects of human-level intelligence that catalyze their application as human-like agents in domains such as social simulations, human-machine interactions, and collaborative multi-agent systems.
States Hidden in Hidden States: LLMs Emerge Discrete State Representations Implicitly
Our work presents a novel exploration of LLMs' symbolic calculation abilities and the underlying mechanisms.
Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence
The rapid advancement of large language models (LLMs) has paved the way for the development of highly capable autonomous agents.
Simulating Classroom Education with LLM-Empowered Agents
In this work, we propose SimClass, a multi-agent classroom simulation teaching framework.
Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models
Specifically, we divide the queries and responses of conversations into several time slices and then adopt a time-division-multiplexing (TDM) encoding-decoding strategy to pseudo-simultaneously process these slices.
PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes
Current approaches, however, often neglect the critical role of multiple molecule graph interaction in understanding chemical reactions, leading to suboptimal performance in synthetic chemistry tasks.
Scaling Efficient Masked Image Modeling on Large Remote Sensing Dataset
To address these issues, we present a new pre-training pipeline for RS models, featuring the creation of a large-scale RS dataset and an efficient MIM approach.
GUICourse: From General Vision Language Models to Versatile GUI Agents
Utilizing Graphic User Interface (GUI) for human-computer interaction is essential for accessing a wide range of digital tools.
Ranked #15 on
Natural Language Visual Grounding
on ScreenSpot
Natural Language Visual Grounding
Optical Character Recognition (OCR)
Zero-Shot Generalization during Instruction Tuning: Insights from Similarity and Granularity
For the first time, we show that zero-shot generalization during instruction tuning is a form of similarity-based generalization between training and test data at the instance level.
Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models
Motivated by the long-tail distribution of singular values in the delta weights, we propose a delta quantization approach using mixed-precision.
Scaling Large Language Model-based Multi-Agent Collaboration
Recent breakthroughs in large language model-driven autonomous agents have revealed that multi-agent collaboration often surpasses each individual through collective reasoning.
Revisiting Non-Autoregressive Transformers for Efficient Image Synthesis
In this paper, we aim to re-evaluate the full potential of NATs by revisiting the design of their training and inference strategies.
UltraMedical: Building Specialized Generalists in Biomedicine
Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains and are moving towards more specialized areas.
RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthiness
Traditional feedback learning for hallucination reduction relies on labor-intensive manual labeling or expensive proprietary models.
Ranked #1 on
Visual Question Answering
on AMBER
ReactXT: Understanding Molecular "Reaction-ship" via Reaction-Contextualized Molecule-Text Pretraining
To resolve the challenges above, we propose a new pretraining method, ReactXT, for reaction-text modeling, and a new dataset, OpenExp, for experimental procedure prediction.
ProtT3: Protein-to-Text Generation for Text-based Protein Understanding
ProtT3 empowers an LM to understand protein sequences of amino acids by incorporating a PLM as its protein understanding module, enabling effective protein-to-text generation.
Iterative Experience Refinement of Software-Developing Agents
We propose two fundamental patterns: the successive pattern, refining based on nearest experiences within a task batch, and the cumulative pattern, acquiring experiences across all previous task batches.
LEGENT: Open Platform for Embodied Agents
Despite advancements in Large Language Models (LLMs) and Large Multimodal Models (LMMs), their integration into language-grounded, human-like embodied agents remains incomplete, hindering complex real-life task performance in physical environments.
PreGSU-A Generalized Traffic Scene Understanding Model for Autonomous Driving based on Pre-trained Graph Attention Network
In this study, we propose PreGSU, a generalized pre-trained scene understanding model based on graph attention network to learn the universal interaction and reasoning of traffic scenes to support various downstream tasks.
UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs
The rapid development of LLMs calls for a lightweight and easy-to-use framework for swift evaluation deployment.
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
For data scaling, we introduce a Warmup-Stable-Decay (WSD) learning rate scheduler (LRS), conducive to continuous training and domain adaptation.
Advancing LLM Reasoning Generalists with Preference Trees
We introduce Eurus, a suite of large language models (LLMs) optimized for reasoning.
Robust and Scalable Model Editing for Large Language Models
Large language models (LLMs) can make predictions using parametric knowledge--knowledge encoded in the model weights--or contextual knowledge--knowledge presented in the context.
LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Images
To address the challenges, we present LLaVA-UHD, a large multimodal model that can efficiently perceive images in any aspect ratio and high resolution.
Ranked #5 on
Long-Context Understanding
on MMNeedle
BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences
Effective attention modules have played a crucial role in the success of Transformer-based large language models (LLMs), but the quadratic time and memory complexities of these attention modules also pose a challenge when processing long sequences.
Mastering Text, Code and Math Simultaneously via Fusing Highly Specialized Language Models
Underlying data distributions of natural language, programming code, and mathematical symbols vary vastly, presenting a complex challenge for large language models (LLMs) that strive to achieve high performance across all three domains simultaneously.
StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models
The virtual API server contains a caching system and API simulators which are complementary to alleviate the change in API status.
Yi: Open Foundation Models by 01.AI
The Yi model family is based on 6B and 34B pretrained language models, then we extend them to chat models, 200K long context models, depth-upscaled models, and vision-language models.
Ranked #1 on
Chatbot
on AlpacaEval
(using extra training data)
LLM-Oriented Retrieval Tuner
Dense Retrieval (DR) is now considered as a promising tool to enhance the memorization capacity of Large Language Models (LLM) such as GPT3 and GPT-4 by incorporating external memories.
Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment
In practice, the multifaceted nature of human preferences inadvertently introduces what is known as the "alignment tax" -a compromise where enhancements in alignment within one objective (e. g., harmlessness) can diminish performance in others (e. g., helpfulness).
Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication
Natural language (NL) has long been the predominant format for human cognition and communication, and by extension, has been similarly pivotal in the development and application of Large Language Models (LLMs).
RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation
Generative models have demonstrated considerable potential in software engineering, particularly in tasks such as code generation and debugging.
Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition
Recent studies have uncovered intriguing phenomena in deep learning, such as grokking, double descent, and emergent abilities in large language models, which challenge human intuition and are crucial for a deeper understanding of neural models.
Cleaner Pretraining Corpus Curation with Neural Web Scraping
The web contains large-scale, diverse, and abundant information to satisfy the information-seeking needs of humans.
ActiveRAG: Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents
Retrieval-Augmented Generation (RAG) enables Large Language Models (LLMs) to leverage external knowledge, enhancing their performance on knowledge-intensive tasks.
$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
Processing and reasoning over long contexts is crucial for many practical applications of Large Language Models (LLMs), such as document comprehension and agent construction.
Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding
To achieve this, we introduce Ouroboros, which can generate draft phrases to parallelize the drafting process and meanwhile lengthen drafts in a training-free manner.
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
Notably, the best-performing model, GPT-4V, attains an average score of 17. 97% on OlympiadBench, with a mere 10. 74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning.
ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
Some recent efforts have explored introducing ReLU or its variants as the substitutive activation function to help LLMs achieve activation sparsity and inference acceleration, but few can simultaneously obtain high sparsity and comparable model performance.
Large Language Model-based Human-Agent Collaboration for Complex Task Solving
We construct a human-agent collaboration dataset to train this policy model in an offline reinforcement learning environment.
MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization
Scientific data visualization plays a crucial role in research by enabling the direct display of complex information and assisting researchers in identifying implicit patterns.
LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks
Most prior works on LoRA combination primarily rely on task-level weights for each involved LoRA, making different examples and tokens share the same LoRA weights.
OneBit: Towards Extremely Low-bit Large Language Models
Model quantification uses low bit-width values to represent the weight matrices of existing models to be quantized, which is a promising approach to reduce both storage and computational overheads of deploying highly anticipated LLMs.
Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents
Current language model-driven agents often lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions.
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
In this paper, we unveil the intrinsic capacity of LLMs for understanding extremely long sequences without any fine-tuning.
UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset
Different from previous works that simply translate English instructions, we consider both the language-specific and language-agnostic abilities of LLMs.
ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
To find the most efficient activation function for sparse computation, we propose a systematic framework to examine the sparsity of LLMs from three aspects: the trade-off between sparsity and performance, the predictivity of sparsity, and the hardware affinity.
MolTC: Towards Molecular Relational Modeling In Language Models
Molecular Relational Learning (MRL), aiming to understand interactions between molecular pairs, plays a pivotal role in advancing biochemical research.
UniMem: Towards a Unified View of Long-Context Large Language Models
Distinguished by its four core dimensions-Memory Management, Memory Writing, Memory Reading, and Memory Injection, UniMem empowers researchers to conduct systematic exploration of long-context methods.
Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-Evolution
This paper introduces Investigate-Consolidate-Exploit (ICE), a novel strategy for enhancing the adaptability and flexibility of AI agents through inter-task self-evolution.
Towards 3D Molecule-Text Interpretation in Language Models
Through 3D molecule-text alignment and 3D molecule-centric instruction tuning, 3D-MoLM establishes an integration of 3D molecular encoder and LM.
DebugBench: Evaluating Debugging Capability of Large Language Models
Previous evaluations of LLMs' debugging ability are significantly limited by the risk of data leakage, the scale of the dataset, and the variety of tested bugs.
GitAgent: Facilitating Autonomous Agent with GitHub by Tool Extension
As GitHub has hosted a multitude of repositories which can be seen as a good resource for tools, a promising solution is that LLM-based agents can autonomously integrate the repositories in GitHub according to the user queries to extend their tool set.
Experiential Co-Learning of Software-Developing Agents
Recent advancements in large language models (LLMs) have brought significant changes to various domains, especially through LLM-driven autonomous agents.
D-Bot: Database Diagnosis System using Large Language Models
Database administrators (DBAs) play an important role in managing, maintaining and optimizing database systems.
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
Multimodal Large Language Models (MLLMs) have recently demonstrated impressive capabilities in multimodal understanding, reasoning, and interaction.
Ranked #1 on
Visual Question Answering
on VQA v2
Sparse Low-rank Adaptation of Pre-trained Language Models
Recognizing the need for more flexible adaptation, we extend the methodology of LoRA to an innovative approach we call sparse low-rank adaptation (SoRA) that enables dynamic adjustments to the intrinsic rank during the adaptation process.
INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair
INTERVENOR prompts Large Language Models (LLMs) to play distinct roles during the code repair process, functioning as both a Code Learner and a Code Teacher.
Ranked #29 on
Code Generation
on MBPP
MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation
Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-event relationships.
NExT-Chat: An LMM for Chat, Detection and Segmentation
The development of large language models (LLMs) has greatly advanced the field of multimodal understanding, leading to the emergence of large multimodal models (LMMs).
ProAgent: From Robotic Process Automation to Agentic Process Automation
Empirical experiments are conducted to detail its construction and execution procedure of workflow, showcasing the feasibility of APA, unveiling the possibility of a new paradigm of automation driven by agents.
Enhancing Dense Retrievers' Robustness with Group-level Reweighting
In this paper, we introduce WebDRO, an efficient approach for clustering the web graph data and optimizing group weights to enhance the robustness of dense retrieval models.
MUSER: A Multi-View Similar Case Retrieval Dataset
Furthermore, the case similarities are typically measured solely by the textual semantics of the fact descriptions, which may fail to capture the full complexity of legal cases from the perspective of legal knowledge.
Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
Pre-trained language models (PLMs) have achieved remarkable results on NLP tasks but at the expense of huge parameter sizes and the consequent computational costs.
Rethinking Tokenizer and Decoder in Masked Graph Modeling for Molecules
Our results show that a subgraph-level tokenizer and a sufficiently expressive decoder with remask decoding have a large impact on the encoder's representation learning.
MARVEL: Unlocking the Multi-Modal Capability of Dense Retrieval via Visual Module Plugin
To facilitate the multi-modal retrieval tasks, we build the ClueWeb22-MM dataset based on the ClueWeb22 dataset, which regards anchor texts as queries, and extracts the related text and image documents from anchor-linked web pages.
Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language
With the thriving of pre-trained language model (PLM) widely verified in various of NLP tasks, pioneer efforts attempt to explore the possible cooperation of the general textual information in PLM with the personalized behavioral information in user historical behavior sequences to enhance sequential recommendation (SR).
ReLM: Leveraging Language Models for Enhanced Chemical Reaction Prediction
Predicting chemical reactions, a fundamental challenge in chemistry, involves forecasting the resulting products from a given reaction process.
MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter
MolCA enables an LM (e. g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector.
Ranked #7 on
Molecule Captioning
on ChEBI-20
Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models
Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments, enabling substantial reductions in model storage and memory costs without significant performance compromise.
Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model
We first validate the efficacy of Toolink in harnessing the model's creativity and CoS ability on ChatGPT.
Predicting Emergent Abilities with Infinite Resolution Evaluation
With PassUntil, we conduct a quantitative investigation into the scaling law of task performance.
UltraFeedback: Boosting Language Models with Scaled AI Feedback
Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models, serving as a solid foundation for future feedback learning research.
Reformulating Vision-Language Foundation Models and Datasets Towards Universal Multimodal Assistants
The capabilities of MLLMs depend on two crucial factors: the model architecture to facilitate the feature alignment of visual modules and large language models; the multimodal instruction tuning datasets for human instruction following.
ConPET: Continual Parameter-Efficient Tuning for Large Language Models
First, Static ConPET can adapt former continual learning methods originally designed for relatively smaller models to LLMs through PET and a dynamic replay strategy, which largely reduces the tuning costs and alleviates the over-fitting and forgetting issue.
QASnowball: An Iterative Bootstrapping Framework for High-Quality Question-Answering Data Generation
Recent years have witnessed the success of question answering (QA), especially its potential to be a foundation paradigm for tackling diverse NLP tasks.
Universal Photorealistic Style Transfer: A Lightweight and Adaptive Approach
Photorealistic style transfer aims to apply stylization while preserving the realism and structure of input content.
Empowering Private Tutoring by Chaining Large Language Models
Artificial intelligence has been applied in various aspects of online education to facilitate teaching and learning.
Text Matching Improves Sequential Recommendation by Reducing Popularity Biases
TASTE alleviates the cold start problem by representing long-tail items using full-text modeling and bringing the benefits of pretrained language models to recommendation systems.
Rational Decision-Making Agent with Internalized Utility Judgment
Central to the development of rationality is the construction of an internalized utility judgment, capable of assigning numerical utilities to each decision.
Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages
Building a competitive counterpart in other languages is highly challenging due to the low-resource nature of non-English multimodal data (i. e., lack of large-scale, high-quality image-text data).
AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
Autonomous agents empowered by Large Language Models (LLMs) have undergone significant improvements, enabling them to generalize across a broad spectrum of tasks.
ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
Text evaluation has historically posed significant challenges, often demanding substantial labor and time cost.
LLM As DBA
Database administrators (DBAs) play a crucial role in managing, maintaining and optimizing a database system to ensure data availability, performance, and reliability.
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
Based on ToolBench, we fine-tune LLaMA to obtain an LLM ToolLLaMA, and equip it with a neural API retriever to recommend appropriate APIs for each instruction.
Ranked #3 on
Trajectory Planning
on ToolBench
Exploring Format Consistency for Instruction Tuning
Instruction tuning has emerged as a promising approach to enhancing large language models in following human instructions.
ChatDev: Communicative Agents for Software Development
Numerous studies used deep learning to improve specific phases in a waterfall model, such as design, coding, and testing.
CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices
To address this, there are two key methods available: the first is model compression, which compresses LLMs into smaller sizes; the second is LoRA, which can transfer an LLM to other tasks with very few parameters, avoiding the storage of multiple model variants in multi-task scenarios by only preserving LoRAs.
Won't Get Fooled Again: Answering Questions with False Premises
In this paper, we find that the PLMs already possess the knowledge required to rebut such questions, and the key is how to activate the knowledge.
OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models
The scale of large pre-trained models (PTMs) poses significant challenges in adapting to downstream tasks due to the high optimization overhead and storage costs associated with full-parameter fine-tuning.
Automatic Truss Design with Reinforcement Learning
Directly applying end-to-end reinforcement learning (RL) methods to truss layout design is infeasible either, since only a tiny portion of the entire layout space is valid under the physical constraints, leading to particularly sparse rewards for RL training.
Interactive Molecular Discovery with Natural Language
Natural language is expected to be a key medium for various human-machine interactions in the era of large language models.
KoLA: Carefully Benchmarking World Knowledge of Large Language Models
The unprecedented performance of large language models (LLMs) necessitates improvements in evaluations.
The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation
In this paper, we check the reliability of EE evaluations and identify three major pitfalls: (1) The data preprocessing discrepancy makes the evaluation results on the same dataset not directly comparable, but the data preprocessing details are not widely noted and specified in papers.
Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations
Then we introduce BOSS, a Benchmark suite for Out-of-distribution robustneSS evaluation covering 5 tasks and 20 datasets.
Exploring the Impact of Model Scaling on Parameter-Efficient Tuning
Our investigations reveal that model scaling (1) mitigates the effects of the positions of tunable parameters on performance, and (2) enables tuning methods to achieve performance comparable to full-parameter fine-tuning by optimizing fewer tunable parameters.
Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data
SANTA proposes two pretraining methods to make language models structure-aware and learn effective representations for structured data: 1) Structured Data Alignment, which utilizes the natural alignment relations between structured data and unstructured data for structure-aware pretraining.
Exploring Lottery Prompts for Pre-trained Language Models
Consistently scaling pre-trained language models (PLMs) imposes substantial burdens on model adaptation, necessitating more efficient alternatives to conventional fine-tuning.
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework
In our experiments, we conduct a robustness evaluation of RoBERTa models to demonstrate the effectiveness of our evaluation framework, and further show the rationality of each component in the framework.
Emergent Modularity in Pre-trained Transformers
In analogy to human brains, we consider two main characteristics of modularity: (1) functional specialization of neurons: we evaluate whether each neuron is mainly specialized in a certain function, and find that the answer is yes.
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Experimental results on three knowledge-driven NLP tasks show that existing injection methods are not suitable for the new paradigm, while map-tuning effectively improves the performance of downstream models.
Plug-and-Play Document Modules for Pre-trained Models
By inserting document plugins into the backbone PTM for downstream tasks, we can encode a document one time to handle multiple tasks, which is more efficient than conventional encoding-task coupling methods that simultaneously encode documents and input queries using task-specific encoders.
Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning
Since it is non-trivial to directly model the intermediate states and design a running cost function, we propose to use latent stochastic bridges to regularize the intermediate states and use the regularization as the running cost of PETs.
Fusion-in-T5: Unifying Document Ranking Signals for Improved Information Retrieval
Common document ranking pipelines in search systems are cascade systems that involve multiple ranking layers to integrate different information step-by-step.
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations
Fine-tuning on instruction data has been widely validated as an effective practice for implementing chat language models like ChatGPT.
Efficient Cross-Lingual Transfer for Chinese Stable Diffusion with Images as Pivots
IAP optimizes only a separate Chinese text encoder with all other parameters fixed to align Chinese semantics space to the English one in CLIP.
Recyclable Tuning for Continual Pre-training
In pilot studies, we find that after continual pre-training, the upgraded PLM remains compatible with the outdated adapted weights to some extent.
WebCPM: Interactive Web Search for Chinese Long-form Question Answering
We recruit annotators to search for relevant information using our interface and then answer questions.
Revisiting Graph Contrastive Learning for Anomaly Detection
In the paper, we delve into the misconception and propose Multi-GNN and Augmented Graph contrastive framework MAG, which unified the existing GCAD methods in the contrastive self-supervised perspective.
VPGTrans: Transfer Visual Prompt Generator across LLMs
While developing a new multimodal LLM (MLLM) by pre-training on tremendous image-text pairs from scratch can be exceedingly resource-consuming, connecting an existing LLM with a comparatively lightweight visual prompt generator (VPG) becomes a feasible paradigm.
Tool Learning with Foundation Models
Considering the lack of a systematic tool learning evaluation in prior works, we experiment with 18 representative tools and show the potential of current foundation models in skillfully utilizing tools.
Rethinking Dense Retrieval's Few-Shot Ability
Moreover, the dataset is disjointed into base and novel classes, allowing DR models to be continuously trained on ample data from base classes and a few samples in novel classes.
Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder
Grounded on our analysis, we propose a novel partial attention language model to solve the attention degeneration problem.
Language-Specific Representation of Emotion-Concept Knowledge Causally Supports Emotion Inference
Humans no doubt use language to communicate about their emotional experiences, but does language in turn help humans understand emotions, or is language just a vehicle of communication?
READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises
In order to fill this important gap, we construct READIN: a Chinese multi-task benchmark with REalistic And Diverse Input Noises.
Semi-supervised Large-scale Fiber Detection in Material Images with Synthetic Data
Accurate detection of large-scale, elliptical-shape fibers, including their parameters of center, orientation and major/minor axes, on the 2D cross-sectioned image slices is very important for characterizing the underlying cylinder 3D structures in microscopic material images.
Decoder Tuning: Efficient Language Understanding as Decoding
With the evergrowing sizes of pre-trained models (PTMs), it has been an emerging practice to only provide the inference APIs for users, namely model-as-a-service (MaaS) setting.
Mul-GAD: a semi-supervised graph anomaly detection framework via aggregating multi-view information
For a more comprehensive conclusion, we further investigate the effect of the objective function and the number of fused views on detection performance.
Visually Grounded Commonsense Knowledge Acquisition
In this work, we present CLEVER, which formulates CKE as a distantly supervised multi-instance learning problem, where models learn to summarize commonsense relations from a bag of images about an entity pair without any human annotation on image instances.
MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction
It contains 103, 193 event coreference chains, 1, 216, 217 temporal relations, 57, 992 causal relations, and 15, 841 subevent relations, which is larger than existing datasets of all the ERE tasks by at least an order of magnitude.
Finding Skill Neurons in Pre-trained Transformer-based Language Models
Furthermore, we demonstrate the skill neurons are most likely generated in pre-training rather than fine-tuning by showing that the skill neurons found with prompt tuning are also crucial for other fine-tuning methods freezing neuron weights, such as the adapter-based tuning and BitFit.
FPT: Improving Prompt Tuning Efficiency via Progressive Training
Inspired by these observations, we propose Fast Prompt Tuning (FPT), which starts by conducting PT using a small-scale partial PLM, and then progressively expands its depth and width until the full-model size.
Few-shot Classification with Hypersphere Modeling of Prototypes
Moreover, it is more convenient to perform metric-based classification with hypersphere prototypes than statistical modeling, as we only need to calculate the distance from a data point to the surface of the hypersphere.
COPEN: Probing Conceptual Knowledge in Pre-trained Language Models
We believe this is a critical bottleneck for realizing human-like cognition in PLMs.
Sparse Structure Search for Delta Tuning
Generally, DT methods exquisitely design delta modules (DT modules) which could be applied to arbitrary fine-grained positions inside PTMs.
Reduce Catastrophic Forgetting of Dense Retrieval Training with Teleportation Negatives
In this paper, we investigate the instability in the standard dense retrieval training, which iterates between model training and hard negative selection using the being-trained model.
A Close Look into the Calibration of Pre-trained Language Models
We observe a consistent change in calibration performance across six factors.
Exploring Mode Connectivity for Pre-trained Language Models
(3) How does the PLM's task knowledge change along the path connecting two minima?
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning
To fathom the mystery, we hypothesize that the adaptations of different DETs could all be reparameterized as low-dimensional optimizations in a unified optimization subspace, which could be found by jointly decomposing independent solutions of different DETs.
