Search Results for author:
Found 32 papers, 28 papers with code
Structured Self-AttentionWeights Encode Semantics in Sentiment Analysis
Very recent work suggests that the self-attention in the Transformer encodes syntactic information; Here, we show that self-attention scores encode semantics by considering sentiment analysis tasks.
Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language Models
Surprisingly, we find that less is more, as training ReSet with high-quality, yet substantially smaller data (three-fold less) yields superior results.
ReFT: Representation Finetuning for Language Models
We define a strong instance of the ReFT family, Low-rank Linear Subspace ReFT (LoReFT), and we identify an ablation of this method that trades some performance for increased efficiency.
Mapping the Increasing Use of LLMs in Scientific Papers
To address this gap, we conduct the first systematic, large-scale analysis across 950, 965 papers published between January 2020 and February 2024 on the arXiv, bioRxiv, and Nature portfolio journals, using a population-level statistical framework to measure the prevalence of LLM-modified content over time.
pyvene: A Library for Understanding and Improving PyTorch Models via Interventions
Interventions on model-internal states are fundamental operations in many areas of AI, including model editing, steering, robustness, and interpretability.
In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation
Large language models (LLMs) frequently hallucinate and produce factual errors, yet our understanding of why they make these errors remains limited.
RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations
Individual neurons participate in the representation of multiple high-level concepts.
A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Arguments
We respond to the recent paper by Makelov et al. (2023), which reviews subspace interchange intervention methods like distributed alignment search (DAS; Geiger et al. 2023) and claims that these methods potentially cause "interpretability illusions".
Rigorously Assessing Natural Language Explanations of Neurons
Natural language is an appealing medium for explaining how large language models process and store information, but evaluating the faithfulness of such explanations is challenging.
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option.
Interpretability at Scale: Identifying Causal Mechanisms in Alpaca
With Boundless DAS, we discover that Alpaca does this by implementing a causal model with two interpretable boolean variables.
ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretation
We argue that this concern is realized for the COGS benchmark.
Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations
In DAS, we find the alignment between high-level and low-level models using gradient descent rather than conducting a brute-force search, and we allow individual neurons to play multiple distinct roles by analyzing representations in non-standard bases-distributed representations.
Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability
Causal abstraction provides a theoretical foundation for mechanistic interpretability, the field concerned with providing intelligible algorithms that are faithful simplifications of the known, but opaque low-level details of black box AI models.
Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Training
Language tasks involving character-level manipulations (e. g., spelling corrections, arithmetic operations, word games) are challenging for models operating on subword units.
Causal Proxy Models for Concept-Based Model Explanations
The core of our proposal is the Causal Proxy Model (CPM).
ZeroC: A Neuro-Symbolic Model for Zero-shot Concept Recognition and Acquisition at Inference Time
In this work, we introduce Zero-shot Concept Recognition and Acquisition (ZeroC), a neuro-symbolic architecture that can recognize and acquire novel concepts in a zero-shot way.
CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behavior
We introduce CEBaB, a new benchmark dataset for assessing concept-based explanation methods in Natural Language Processing (NLP).
Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies
When we transfer a pretrained language model to a new language, there are many axes of variation that change at once.
Causal Distillation for Language Models
Distillation efforts have led to language models that are more compact and efficient without serious drops in performance.
Inducing Causal Structure for Interpretable Neural Networks
In IIT, we (1) align variables in a causal model (e. g., a deterministic program or Bayesian network) with representations in a neural model and (2) train the neural model to match the counterfactual behavior of the causal model on a base input when aligned representations in both models are set to be the value they would be for a source input.
ReaSCAN: Compositional Reasoning in Language Grounding
The ability to compositionally map language to referents, relations, and actions is an essential component of language understanding.
Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models
There is growing evidence that pretrained language models improve task-specific fine-tuning not just for the languages seen in pretraining, but also for new languages and even non-linguistic data.
Dynabench: Rethinking Benchmarking in NLP
We introduce Dynabench, an open-source platform for dynamic dataset creation and model benchmarking.
On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification
In this paper, we adapt existing attribution methods on explaining decision makings of BERT in sequence classification tasks.
DynaSent: A Dynamic Benchmark for Sentiment Analysis
We introduce DynaSent ('Dynamic Sentiment'), a new English-language benchmark task for ternary (positive/negative/neutral) sentiment analysis.
Context-Guided BERT for Targeted Aspect-Based Sentiment Analysis
We train both models with pretrained BERT on two (T)ABSA datasets: SentiHood and SemEval-2014 (Task 4).
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
Structured Self-Attention Weights Encode Semantics in Sentiment Analysis
Very recent work suggests that the self-attention in the Transformer encodes syntactic information; Here, we show that self-attention scores encode semantics by considering sentiment analysis tasks.
Pragmatically Informative Color Generation by Grounding Contextual Modifiers
One important task that involves grounding contextual modifiers is color generation.
Modeling emotion in complex stories: the Stanford Emotional Narratives Dataset
We begin by assessing the state-of-the-art in time-series emotion recognition, and we review contemporary time-series approaches in affective computing, including discriminative and generative models.
Disentangling Latent Emotions of Word Embeddings on Complex Emotional Narratives
We showed that, in the proposed emotion space, we were able to better disentangle emotions than using raw GloVe vectors alone.
Attending to Emotional Narratives
Attention mechanisms in deep neural networks have achieved excellent performance on sequence-prediction tasks.
