Search Results for author:
Found 9 papers, 7 papers with code
Compressing Language Models for Specialized Domains
Compression techniques such as pruning and quantization offer a solution for more efficient deployment of language models (LMs), albeit with small performance drops in benchmark performance.
Self-calibration for Language Model Quantization and Pruning
In a post-training setting, state-of-the-art quantization and pruning methods require calibration data, a small set of unlabeled examples.
Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization
Despite the remarkable performance of generative large language models (LLMs) on abstractive summarization, they face two significant challenges: their considerable size and tendency to hallucinate.
On the Impact of Temporal Concept Drift on Model Explanations
Explanation faithfulness of model predictions in natural language processing is typically evaluated on held-out data from the same temporal distribution as the training data (i. e. synchronous settings).
An Empirical Study on Explanations in Out-of-Domain Settings
Recent work in Natural Language Processing has focused on developing approaches that extract faithful explanations, either via identifying the most important tokens in the input (i. e. post-hoc explanations) or by designing inherently faithful models that first select the most important tokens and then use them to predict the correct label (i. e. select-then-predict models).
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling
Masked language modeling (MLM), a self-supervised pretraining objective, is widely used in natural language processing for learning text representations.
Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience
In this paper, we hypothesize that salient information extracted a priori from the training data can complement the task-specific information learned by the model during fine-tuning on a downstream task.
Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification
In this paper, we seek to improve the faithfulness of attention-based explanations for text classification.
Flexible Instance-Specific Rationalization of NLP Models
Recent research on model interpretability in natural language processing extensively uses feature scoring methods for identifying which parts of the input are the most important for a model to make a prediction (i. e. explanation or rationale).
