Search Results for author:
Found 26 papers, 11 papers with code
How Much Do Modifications to Transformer Language Models Affect Their Ability to Learn Linguistic Knowledge?
Recent progress in large pretrained language models (LMs) has led to a growth of analyses examining what kinds of linguistic knowledge are encoded by these models.
IGA: An Intent-Guided Authoring Assistant
While large-scale pretrained language models have significantly improved writing assistance functionalities such as autocomplete, more complex and controllable writing assistants have yet to be explored.
SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling
We present a decoder-only Transformer architecture that robustly generalizes to sequence lengths substantially longer than those seen during training.
L0-Reasoning Bench: Evaluating Procedural Correctness in Language Models via Simple Program Execution
Complex reasoning tasks often rely on the ability to consistently and accurately apply simple rules across incremental steps, a foundational capability which we term "level-0" reasoning.
How much do contextualized representations encode long-range context?
We analyze contextual representations in neural autoregressive language models, emphasizing long-range contexts that span several thousand tokens.
nGPT: Normalized Transformer with Representation Learning on the Hypersphere
We propose a novel neural network architecture, the normalized Transformer (nGPT) with representation learning on the hypersphere.
Suri: Multi-constraint Instruction Following for Long-form Text Generation
Existing research on instruction following largely focuses on tasks with simple instructions and short responses.
RULER: What's the Real Context Size of Your Long-Context Language Models?
Despite achieving nearly perfect accuracy in the vanilla NIAH test, almost all models exhibit large performance drops as the context length increases.
TopicGPT: A Prompt-based Topic Modeling Framework
Topic modeling is a well-established technique for exploring text corpora.
Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of RLHF
During the last stage of RLHF, a large language model is aligned to human intents via PPO training, a process that generally requires large-scale computational resources.
PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
PEARL outperforms zero-shot and chain-of-thought prompting on this dataset, and ablation experiments show that each stage of PEARL is critical to its performance.
How Does In-Context Learning Help Prompt Tuning?
This motivates the use of parameter-efficient adaptation methods such as prompt tuning (PT), which adds a small number of tunable embeddings to an otherwise frozen model, and in-context learning (ICL), in which demonstrations of the task are provided to the model in natural language without any additional training.
Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages
With multilingual machine translation (MMT) models continuing to grow in size and number of supported languages, it is natural to reuse and upgrade existing models to save computation as data becomes available in more languages.
Image Coding for Machines with Omnipotent Feature Learning
Learning a kind of feature that is both general (for AI tasks) and compact (for compression) is pivotal for its success.
ChapterBreak: A Challenge Dataset for Long-Range Language Models
While numerous architectures for long-range language models (LRLMs) have recently been proposed, a meaningful evaluation of their discourse-level language understanding capabilities has not yet followed.
Semantically Video Coding: Instill Static-Dynamic Clues into Structured Bitstream for AI Tasks
Traditional media coding schemes typically encode image/video into a semantic-unknown binary stream, which fails to directly support downstream intelligent tasks at the bitstream level.
Alternative Input Signals Ease Transfer in Multilingual Machine Translation
Further, we find that incorporating alternative inputs via self-ensemble can be particularly effective when training set is small, leading to +5 BLEU when only 5% of the total training data is accessible.
IGA : An Intent-Guided Authoring Assistant
While large-scale pretrained language models have significantly improved writing assistance functionalities such as autocomplete, more complex and controllable writing assistants have yet to be explored.
Revisiting Simple Neural Probabilistic Language Models
Recent progress in language modeling has been driven not only by advances in neural architectures, but also through hardware and optimization improvements.
Ranked #59 on
Language Modelling
on WikiText-103
Learning Omni-frequency Region-adaptive Representations for Real Image Super-Resolution
Traditional single image super-resolution (SISR) methods that focus on solving single and uniform degradation (i. e., bicubic down-sampling), typically suffer from poor performance when applied into real-world low-resolution (LR) images due to the complicated realistic degradations.
Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models
To benefit from this observation, we train an energy-based model to mimic the behavior of the task measure (i. e., the energy-based model assigns lower energy to samples with higher BLEU score), which is resulted in a re-ranking algorithm based on the samples drawn from NMT: energy-based re-ranking (EBR).
Multi-scale Grouped Dense Network for VVC Intra Coding
Versatile Video Coding (H. 266/VVC) standard achieves better image quality when keeping the same bits than any other conventional image codec, such as BPG, JPEG, and etc.
Hard-Coded Gaussian Attention for Neural Machine Translation
Recent work has questioned the importance of the Transformer's multi-headed attention for achieving high translation quality.
The Feasibility of Embedding Based Automatic Evaluation for Single Document Summarization
ROUGE is widely used to automatically evaluate summarization systems.
How to Compare Summarizers without Target Length? Pitfalls, Solutions and Re-Examination of the Neural Summarization Literature
We show that plain ROUGE F1 scores are not ideal for comparing current neural systems which on average produce different lengths.
