Search Results for author:
Found 13 papers, 10 papers with code
GIRAFFE: Design Choices for Extending the Context Length of Visual Language Models
Towards this goal, we make the best design choice through extensive experiment settings from data curation to context window extending and utilizing: (1) we analyze data sources and length distributions to construct ETVLM - a data recipe to balance the performance across scenarios; (2) we examine existing position extending methods, identify their limitations and propose M-RoPE++ as an enhanced approach; we also choose to solely instruction-tune the backbone with mixed-source data; (3) we discuss how to better utilize extended context windows and propose hybrid-resolution training.
Why Does the Effective Context Length of LLMs Fall Short?
Advancements in distributed training and efficient attention mechanisms have significantly expanded the context window sizes of large language models (LLMs).
Scaling Diffusion Language Models via Adaptation from Autoregressive Models
Diffusion Language Models (DLMs) have emerged as a promising new paradigm for text generative modeling, potentially addressing limitations of autoregressive (AR) models.
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning
Our work highlights the potential of diffusion-based approaches in advancing AI capabilities for sophisticated language understanding and problem-solving tasks.
Training-Free Long-Context Scaling of Large Language Models
The ability of Large Language Models (LLMs) to process and generate coherent text is markedly weakened when the number of input tokens exceeds their pretraining length.
BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models
Multimodal reasoning stands as a pivotal capability for large vision-language models (LVLMs).
Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models
Recently, diffusion models have garnered significant interest in the field of text processing due to their many potential advantages compared to conventional autoregressive models.
DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models
Diffusion models have gained prominence in generating high-quality sequences of text.
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations with more extensive histories.
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks.
Few-Shot Natural Language Inference Generation with PDD: Prompt and Dynamic Demonstration
Natural Language Inference Generation task is to generate a text hypothesis given a text premise and a logical relation between the two.
Positive, Negative and Neutral: Modeling Implicit Feedback in Session-based News Recommendation
News recommendation for anonymous readers is a useful but challenging task for many news portals, where interactions between readers and articles are limited within a temporary login session.
