Search Results for author:
Found 26 papers, 7 papers with code
Scaling Parameter-Constrained Language Models with Quality Data
Scaling laws in language modeling traditionally quantify training loss as a function of dataset size and model parameters, providing compute-optimal estimates but often neglecting the impact of data quality on model generalization.
CoDi: Conversational Distillation for Grounded Question Answering
This is a typical on-device scenario for specialist SLMs, allowing for open-domain model responses, without requiring the model to "memorize" world knowledge in its limited weights.
Small But Funny: A Feedback-Driven Approach to Humor Distillation
The emergence of Large Language Models (LLMs) has brought to light promising language generation capabilities, particularly in performing tasks like complex reasoning and creative writing.
Large Language Models as Zero-shot Dialogue State Tracker through Function Calling
We also show that by fine-tuning on a small collection of diverse task-oriented dialogues, we can equip modestly sized models, specifically a 13B parameter LLaMA2-Chat model, with function-calling capabilities and DST performance comparable to ChatGPT while maintaining their chat capabilities.
FLEE-GNN: A Federated Learning System for Edge-Enhanced Graph Neural Network in Analyzing Geospatial Resilience of Multicommodity Food Flows
This paper proposes FLEE-GNN, a novel Federated Learning System for Edge-Enhanced Graph Neural Network, designed to overcome these challenges and enhance the analysis of geospatial resilience of multicommodity food flow network, which is one type of spatial networks.
Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues
As a result, we explore approaches to build discourse structures for dialogues, based on attention matrices from Pre-trained Language Models (PLMs).
Large Discourse Treebanks from Scalable Distant Supervision
Discourse parsing is an essential upstream task in Natural Language Processing with strong implications for many real-world applications.
Unsupervised Inference of Data-Driven Discourse Structures using a Tree Auto-Encoder
With a growing need for robust and general discourse structures in many downstream tasks and real-world applications, the current lack of high-quality, high-quantity discourse trees poses a severe shortcoming.
Towards Domain-Independent Supervised Discourse Parsing Through Gradient Boosting
Discourse analysis and discourse parsing have shown great impact on many important problems in the field of Natural Language Processing (NLP).
Improving Topic Segmentation by Injecting Discourse Dependencies
Recent neural supervised topic segmentation models achieve distinguished superior effectiveness over unsupervised methods, with the availability of large-scale training corpora sampled from Wikipedia.
Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models
With a growing number of BERTology work analyzing different components of pre-trained language models, we extend this line of research through an in-depth analysis of discourse information in pre-trained and fine-tuned language models.
Predicting Above-Sentence Discourse Structure using Distant Supervision from Topic Segmentation
RST-style discourse parsing plays a vital role in many NLP tasks, revealing the underlying semantic/pragmatic structure of potentially complex and diverse documents.
CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training
Consequently, we propose a novel QA dataset based on the Common Crawl project in this paper.
W-RST: Towards a Weighted RST-style Discourse Framework
Aiming for a better integration of data-driven and linguistically-inspired approaches, we explore whether RST Nuclearity, assigning a binary assessment of importance between text segments, can be replaced by automatically generated, real-valued scores, in what we call a Weighted-RST framework.
Predicting Discourse Trees from Transformer-based Neural Summarizers
Previous work indicates that discourse information benefits summarization.
Dynamics of Water Confined in Mesopores with Variable Surface Interaction
We have investigated the dynamics of liquid water confined in mesostructured porous silica (MCM-41) and periodic mesoporous organosilicas (PMOs) by incoherent quasielastic neutron scattering experiments.
Chemical Physics
Unsupervised Learning of Discourse Structures using a Tree Autoencoder
In this paper we are inferring general tree structures of natural text in multiple domains, showing promising results on a diverse set of tasks.
Do We Really Need That Many Parameters In Transformer For Extractive Summarization? Discourse Can Help !
The multi-head self-attention of popular transformer models is widely used within Natural Language Processing (NLP), including for the task of extractive summarization.
Unleashing the Power of Neural Discourse Parsers - A Context and Structure Aware Approach Using Large Scale Pretraining
RST-based discourse parsing is an important NLP task with numerous downstream applications, such as summarization, machine translation and opinion mining.
Ranked #18 on
Discourse Parsing
on RST-DT
(Standard Parseval (Span) metric)
Unleashing the Power of Neural Discourse Parsers -- A Context and Structure Aware Approach Using Large Scale Pretraining
RST-based discourse parsing is an important NLP task with numerous downstream applications, such as summarization, machine translation and opinion mining.
Ranked #9 on
Discourse Parsing
on Instructional-DT (Instr-DT)
MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision
The lack of large and diverse discourse treebanks hinders the application of data-driven approaches, such as deep-learning, to RST-style discourse parsing.
From Sentiment Annotations to Sentiment Prediction through Discourse Augmentation
Sentiment analysis, especially for long documents, plausibly requires methods capturing complex linguistics structures.
Predicting Discourse Structure using Distant Supervision from Sentiment
Results indicate that while our parser does not yet match the performance of a parser trained and tested on the same dataset (intra-domain), it does perform remarkably well on the much more difficult and arguably more useful task of inter-domain discourse structure prediction, where the parser is trained on one domain and tested/applied on another one.
A Hierarchical Approach to Neural Context-Aware Modeling
Our approach overcomes recent limitations with extended narratives through a multi-layered computational approach to generate an abstract context representation.
Automated Evaluation of Out-of-Context Errors
We present a new approach to evaluate computational models for the task of text understanding by the means of out-of-context error detection.
On the determination of anti-neutrino spectra from nuclear reactors
We also perform a critical evaluation of the errors associated with our method to extract the anti-neutrino spectrum using synthetic beta spectra.
High Energy Physics - Phenomenology High Energy Physics - Experiment Nuclear Experiment Nuclear Theory
