Search Results for author:
Found 26 papers, 13 papers with code
Can Language Models Follow Multiple Turns of Entangled Instructions?
This work presents a systematic investigation of LLMs' capabilities in handling multiple turns of instructions, covering three levels of difficulty: (1) retrieving information from instructions, (2) tracking and reasoning across turns, and (3) resolving conflicts among instructions.
The Law of Knowledge Overshadowing: Towards Understanding, Predicting, and Preventing LLM Hallucination
To address it, we propose a novel concept: knowledge overshadowing, where model's dominant knowledge can obscure less prominent knowledge during text generation, causing the model to fabricate inaccurate details.
SyncMind: Measuring Agent Out-of-Sync Recovery in Collaborative Software Engineering
Besides substantial performance gaps among agents (from Llama-3. 1 agent <= 3. 33% to Claude-3. 5-Sonnet >= 28. 18%), their consistently low collaboration willingness (<= 4. 86%) suggests fundamental limitations of existing LLM in CSE.
Learning to Generate Research Idea with Dynamic Control
Our framework provides a balanced approach to research ideation, achieving high-quality outcomes by dynamically navigating the trade-offs among novelty, feasibility, and effectiveness.
Schema-Guided Culture-Aware Complex Event Simulation with Multi-Agent Role-Play
Complex news events, such as natural disasters and socio-political conflicts, require swift responses from the government and society.
MentalArena: Self-play Training of Language Models for Diagnosis and Treatment of Mental Health Disorders
In this paper, we introduce MentalArena, a self-play framework to train language models by generating domain-specific personalized data, where we obtain a better model capable of making a personalized diagnosis and treatment (as a therapist) and providing information (as a patient).
Towards LifeSpan Cognitive Systems
In this paper we focus on the domain of Large Language Models (LLMs), where we identify two major challenges: (1) Abstraction and Experience Merging, and (2) Long-term Retention with Accurate Recall.
Why Does New Knowledge Create Messy Ripple Effects in LLMs?
Extensive previous research has focused on post-training knowledge editing (KE) for language models (LMs) to ensure that knowledge remains accurate and up-to-date.
Eliminating Position Bias of Language Models: A Mechanistic Approach
Position bias has proven to be a prevalent issue of modern language models (LMs), where the models prioritize content based on its position within the given context.
LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
Attempting to complement this deficiency, we investigate the layerwise properties of LoRA on fine-tuning tasks and observe an unexpected but consistent skewness of weight norms across different layers.
Large Language Models on Graphs: A Comprehensive Survey
Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i. e., graph-based reasoning).
InfoPattern: Unveiling Information Propagation Patterns in Social Media
Social media play a significant role in shaping public opinion and influencing ideological communities through information propagation.
Defining a New NLP Playground
The recent explosion of performance of large language models (LLMs) has changed the field of Natural Language Processing (NLP) more abruptly and seismically than any other shift in the field's 80-year history.
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models
As a result, their performance suffers drastically on inputs longer than those encountered during training, substantially limiting their applications in real-world tasks involving long contexts such as encoding scientific articles, code repositories, or long dialogues.
Logical Entity Representation in Knowledge-Graphs for Differentiable Rule Learning
A LERP is designed as a vector of probabilistic logical functions on the entity's neighboring sub-graph.
Ranked #11 on
Link Prediction
on WN18RR
Explaining Emergent In-Context Learning as Kernel Regression
Then, we empirically investigate the in-context behaviors of language models.
Word Embeddings Are Steers for Language Models
In this work, we theoretically and empirically revisit output word embeddings and find that their linear transformations are equivalent to steering language model generation styles.
Understanding the Effect of Data Augmentation on Knowledge Distillation
However, such data augmentation methods are sub-optimal for knowledge distillation since the teacher model could provide label distributions and is more tolerant to semantic shifts.
RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thought
To detect factual inconsistency, RCoT first asks LLMs to reconstruct the problem based on generated solutions.
Tool Learning with Foundation Models
Considering the lack of a systematic tool learning evaluation in prior works, we experiment with 18 representative tools and show the potential of current foundation models in skillfully utilizing tools.
Zero-Shot Classification by Logical Reasoning on Natural Language Explanations
To this end, we propose the framework CLORE (Classification by LOgical Reasoning on Explanations).
Learning Homophilic Incentives in Sequential Social Dilemmas
Promoting cooperation among self-interested agents is a long-standing and interdisciplinary problem, but receives less attention in multi-agent reinforcement learning (MARL).
Learning Shared Semantic Space for Speech-to-Text Translation
By projecting audio and text features to a common semantic representation, Chimera unifies MT and ST tasks and boosts the performance on ST benchmarks, MuST-C and Augmented Librispeech, to a new state-of-the-art.
Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL
We propose a novel learning framework to encourage homophilic incentives and show that it achieves stable cooperation in both SSDs of public goods and tragedy of the commons.
Visual Concept-Metaconcept Learning
Humans reason with concepts and metaconcepts: we recognize red and green from visual input; we also understand that they describe the same property of objects (i. e., the color).
