Search Results for author:
Found 28 papers, 20 papers with code
T3-Vis: visual analytic for Training and fine-Tuning Transformers in NLP
Transformers are the dominant architecture in NLP, but their training and fine-tuning is still very challenging.
KW-ATTN: Knowledge Infused Attention for Accurate and Interpretable Text Classification
Text classification has wide-ranging applications in various domains.
HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading
Extending the context length has disproportionately shifted the memory footprint of LLMs during inference to the key-value cache (KV cache).
Environmental Factors Can Have Opposite Biodiversity Influences on the Community Temporal Stability In Aquatic Ecosystems
However, we know little about how local environmental conditions can influence these biodiversity drivers, and consequently how they indirectly shape the ecological stability of ecosystems.
Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey
As Large Language Models (LLMs) have advanced to unify understanding and generation tasks within the textual modality, recent research has shown that tasks from different modalities can also be effectively encapsulated within the NTP framework, transforming the multimodal information into tokens and predict the next one given the context.
Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning
Key-Value (KV) caching is a common technique to enhance the computational efficiency of Large Language Models (LLMs), but its memory overhead grows rapidly with input length.
Integrative Decoding: Improve Factuality via Implicit Self-consistency
Self-consistency-based approaches, which involve repeatedly sampling multiple outputs and selecting the most consistent one as the final response, prove to be remarkably effective in improving the factual accuracy of large language models.
Towards a Unified View of Preference Learning for Large Language Models: A Survey
Finally, based on our unified perspective, we explore the challenges and future research directions for aligning large language models with human preferences.
LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback
In recent progress, mathematical verifiers have achieved success in mathematical reasoning tasks by validating the correctness of solutions generated by policy models.
PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
Our experimental evaluations, utilizing the LongBench benchmark, show that PyramidKV matches the performance of models with a full KV cache while retaining only 12% of the KV cache, thus significantly reducing memory usage.
Cross-Task Defense: Instruction-Tuning LLMs for Content Safety
Recent studies reveal that Large Language Models (LLMs) face challenges in balancing safety with utility, particularly when processing long texts for NLP tasks like summarization and translation.
Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack
Recent developments in balancing the usefulness and safety of Large Language Models (LLMs) have raised a critical question: Are mainstream NLP tasks adequately aligned with safety consideration?
Visual Analytics for Generative Transformer Models
While transformer-based models have achieved state-of-the-art results in a variety of classification and generation tasks, their black-box nature makes them challenging for interpretability.
Personalized Abstractive Summarization by Tri-agent Generation Pipeline
The inference-only large language model (ChatGPT) serves as both the generator and editor, with a smaller model acting as the instructor to guide output generation.
Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues
As a result, we explore approaches to build discourse structures for dialogues, based on attention matrices from Pre-trained Language Models (PLMs).
Attend to the Right Context: A Plug-and-Play Module for Content-Controllable Summarization
Content-Controllable Summarization generates summaries focused on the given controlling signals.
Entity-based SpanCopy for Abstractive Summarization to Improve the Factual Consistency
Despite the success of recent abstractive summarizers on automatic evaluation metrics, the generated summaries still present factual inconsistencies with the source document.
SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds
Each point in the dataset has been labelled with fine-grained semantic annotations, resulting in a dataset that is three times the size of the previous existing largest photogrammetric point cloud dataset.
Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentation
The multi-head self-attention mechanism of the transformer model has been thoroughly investigated recently.
PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization
We introduce PRIMERA, a pre-trained model for multi-document representation with a focus on summarization that reduces the need for dataset-specific architectures and large amounts of fine-tuning labeled data.
Ranked #1 on
Multi-Document Summarization
on Multi-News
T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP
Transformers are the dominant architecture in NLP, but their training and fine-tuning is still very challenging.
W-RST: Towards a Weighted RST-style Discourse Framework
Aiming for a better integration of data-driven and linguistically-inspired approaches, we explore whether RST Nuclearity, assigning a binary assessment of importance between text segments, can be replaced by automatically generated, real-valued scores, in what we call a Weighted-RST framework.
Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning
In news articles the lead bias is a common phenomenon that usually dominates the learning signals for neural extractive summarizers, severely limiting their performance on data with different or even no bias.
Predicting Discourse Trees from Transformer-based Neural Summarizers
Previous work indicates that discourse information benefits summarization.
Do We Really Need That Many Parameters In Transformer For Extractive Summarization? Discourse Can Help !
The multi-head self-attention of popular transformer models is widely used within Natural Language Processing (NLP), including for the task of extractive summarization.
Systematically Exploring Redundancy Reduction in Summarizing Long Documents
Our analysis of large summarization datasets indicates that redundancy is a very serious problem when summarizing long documents.
Ranked #16 on
Text Summarization
on Pubmed
Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
An essential prerequisite for unleashing the potential of supervised deep learning algorithms in the area of 3D scene understanding is the availability of large-scale and richly annotated datasets.
Extractive Summarization of Long Documents by Combining Global and Local Context
In this paper, we propose a novel neural single document extractive summarization model for long documents, incorporating both the global context of the whole document and the local context within the current topic.
Ranked #19 on
Text Summarization
on Arxiv HEP-TH citation graph
