Search Results for author:
Found 351 papers, 176 papers with code
[CASPI] Causal-aware Safe Policy Improvement for Task-oriented Dialogue
Further more we demonstrate sample efficiency, where our method trained only on 20% of the data, are comparable to current state of the art method trained on 100% data on two out of there evaluation metrics.
Few-Shot Intent Classification by Gauging Entailment Relationship Between Utterance and Semantic Label
Compared with DNNC, our proposed method is more efficient in both training and serving since it is based upon the entailment between query utterance and labels instead of all the training examples.
Continual Learning via Explicit Structure Learning
During structure learning, the model optimizes for the best structure for the current task.
Are Pre-trained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection
Pre-trained Transformer-based models were reported to be robust in intent classification.
DocQueryNet: Value Retrieval with Arbitrary Queries for Form-like Documents
Unlike previous methods that only address a fixed set of field items, our method predicts target value for an arbitrary query based on the understanding of the layout and semantics of a form.
Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging
The concept of Dialogue Act (DA) is universal across different task-oriented dialogue domains - the act of {``}request{''} carries the same speaker intention whether it is for restaurant reservation or flight booking.
The Thieves on Sesame Street are Polyglots - Extracting Multilingual Models from Monolingual APIs
Pre-training in natural language processing makes it easier for an adversary with only query access to a victim model to reconstruct a local copy of the victim by training with gibberish input data paired with the victim{'}s labels for that data.
Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models
Large reasoning models (LRMs) already possess a latent capacity for long chain-of-thought reasoning.
BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset
Building on our innovative model design, training recipe, and datasets, we develop BLIP3-o, a suite of state-of-the-art unified multimodal models.
xGen-small Technical Report
We introduce xGen-small, a family of 4B and 9B Transformer decoder models optimized for long-context applications.
Scalable Chain of Thoughts via Elastic Reasoning
Large reasoning models (LRMs) have achieved remarkable progress on complex tasks by generating extended chains of thought (CoT).
SweRank: Software Issue Localization with Code Ranking
Software issue localization, the task of identifying the precise code locations (files, classes, or functions) relevant to a natural language issue description (e. g., bug report, feature request), is a critical yet time-consuming aspect of software development.
LZ Penalty: An information-theoretic repetition penalty for autoregressive language models
We demonstrate the LZ penalty enables state-of-the-art open-source reasoning models to operate with greedy (temperature zero) decoding without loss of capability and without instances of degenerate repetition.
DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs
Extensive experiments on image and video understanding tasks demonstrate that DyMU can reduce the average visual token count by 32%-85% while achieving comparable performance to full-length models across diverse VLM architectures, including the recently popularized AnyRes-based visual encoders.
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
Scaling test-time computation, or affording a generator large language model (LLM) extra compute during inference, typically employs the help of external non-generative evaluators (i. e., reward models).
A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
In this work, we revisit GRPO from a reinforce-like algorithm perspective and analyze its core components.
APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay
We open-source 5K synthetic data trajectories and the trained xLAM-2-fc-r models to advance research in AI agents.
ActionStudio: A Lightweight Framework for Data and Training of Large Action Models
However, training large action models remains challenging due to the diversity of agent environments and the complexity of agentic data.
Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models
Time series analysis is crucial for understanding dynamics of complex systems.
Generative Frame Sampler for Long Video Understanding
Despite recent advances in Video Large Language Models (VideoLLMs), effectively understanding long-form videos remains a significant challenge.
BingoGuard: LLM Content Moderation Tools with Risk Levels
Using this framework, we create BingoGuardTrain, a training dataset with 54, 897 examples covering a variety of topics, response severity, styles, and BingoGuardTest, a test set with 988 examples explicitly labeled based on our severity rubrics that enables fine-grained analysis on model behaviors on different severity levels.
A Survey on Post-training of Large Language Models
The emergence of Large Language Models (LLMs) has fundamentally transformed natural language processing, making them indispensable across domains ranging from conversational systems to scientific exploration.
Turning Conversations into Workflows: A Framework to Extract and Evaluate Dialog Workflows for Service AI Agents
Automated service agents require well-structured workflows to provide consistent and accurate responses to customer queries.
Reward Models Identify Consistency, Not Causality
Reward models (RMs) play a crucial role in aligning large language models (LLMs) with human preferences and enhancing reasoning quality.
Why Vision Language Models Struggle with Visual Arithmetic? Towards Enhanced Chart and Geometry Understanding
In this work, we first investigate the root causes of this deficiency through a suite of probing tasks focusing on basic visual arithmetic.
BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillation
This paper introduces a novel approach to enable LLM's LongCoT capacity without distillation from o1-like models or expensive human annotations, where we bootstrap LongCoT (BOLT) from a standard instruct model.
Reward-Guided Speculative Decoding for Efficient LLM Reasoning
We introduce Reward-Guided Speculative Decoding (RSD), a novel framework aimed at improving the efficiency of inference in large language models (LLMs).
Demystifying Domain-adaptive Post-training for Financial LLMs
Domain-adaptive post-training of large language models (LLMs) has emerged as a promising approach for specialized domains such as medicine and finance.
StructTest: Benchmarking LLMs' Reasoning through Compositional Structured Outputs
The rapid development of large language models (LLMs) necessitates robust, unbiased, and scalable methods for evaluating their capabilities.
Bridging the Data Provenance Gap Across Text, Speech and Video
In this work we conduct the largest and first-of-its-kind longitudinal audit across modalities--popular text, speech, and video datasets--from their detailed sourcing trends and use restrictions to their geographical and linguistic representation.
Unanswerability Evaluation for Retrieval Augmented Generation
Existing evaluation frameworks for retrieval-augmented generation (RAG) systems focus on answerable queries, but they overlook the importance of appropriately rejecting unanswerable requests.
GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers
The effectiveness of large language models (LLMs) is closely tied to the design of prompts, making prompt optimization essential for enhancing their performance across a wide range of tasks.
AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials
This work underscores the potential of guided replay with web tutorials as a viable strategy for large-scale GUI agent training, paving the way for more capable and autonomous digital agents.
ViUniT: Visual Unit Tests for More Robust Visual Programming
In our framework, a unit test is represented as a novel image and answer pair meant to verify the logical correctness of a program produced for a given query.
ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Models
Our multi-image instruction data leads to an 8% improvement on Mantis-Eval.
Ranked #107 on
Visual Question Answering
on MM-Vet
TACO: Learning Multi-modal Action Models with Synthetic Chains-of-Thought-and-Action
While open-source multi-modal language models perform well on simple question answering tasks, they often fail on complex questions that require multiple capabilities, such as fine-grained recognition, visual grounding, and reasoning, and that demand multi-step solutions.
Ranked #61 on
Visual Question Answering
on MM-Vet
Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction
Automating GUI tasks remains challenging due to reliance on textual representations, platform-specific action spaces, and limited reasoning capabilities.
Ranked #2 on
Natural Language Visual Grounding
on ScreenSpot
SpecTool: A Benchmark for Characterizing Errors in Tool-Use LLMs
To solve this problem, we introduce SpecTool, a new benchmark to identify error patterns in LLM output on tool-use tasks.
CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval
This gap leaves existing models unable to effectively capture the diversity of programming languages and tasks across different domains, highlighting the need for more focused research in code retrieval.
PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback
Large Language Models (LLMs) are widely adopted for assisting in software development tasks, yet their performance evaluations have narrowly focused on the functional correctness of generated code.
BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions
We introduce BLIP3-KALE, a dataset of 218 million image-text pairs that bridges the gap between descriptive synthetic captions and factual web-scale alt-text.
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
Real-world enterprise text-to-SQL workflows often involve complex cloud or local data across various database systems, multiple SQL queries in various dialects, and diverse operations from data transformation to analytics.
Ranked #1 on
Text-To-SQL
on Spider 2.0
CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models
We comprehensively evaluated CodeTree on 7 code generation benchmarks and demonstrated the significant performance gains of CodeTree against strong baselines.
Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding
Large language models (LLMs) have shown impressive capabilities, but still struggle with complex reasoning tasks requiring multiple steps.
CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments
Customer Relationship Management (CRM) systems are vital for modern enterprises, providing a foundation for managing customer interactions and data.
JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking
Accurate document retrieval is crucial for the success of retrieval-augmented generation (RAG) applications, including open-domain question answering and code completion.
Asynchronous Tool Usage for Real-Time Agents
While frontier large language models (LLMs) are capable tool-using agents, current AI systems still operate in a strict turn-based fashion, oblivious to passage of time.
PRACT: Optimizing Principled Reasoning and Acting of LLM Agent
We introduce the Principled Reasoning and Acting (PRAct) framework, a novel method for learning and enforcing action principles from trajectory data.
Distill-SynthKG: Distilling Knowledge Graph Synthesis Workflow for Improved Coverage and Efficiency
Additionally, there is a gap in evaluation datasets and methodologies for ontology-free KG construction.
xGen-MM-Vid (BLIP-3-Video): You Only Need 32 Tokens to Represent a Video Even in VLMs
We present xGen-MM-Vid (BLIP-3-Video): a multimodal language model for videos, particularly designed to efficiently capture temporal information over multiple frames.
Do RAG Systems Cover What Matters? Evaluating and Optimizing Responses with Sub-Question Coverage
Using this categorization, we introduce a fine-grained evaluation protocol that provides insights into the retrieval and generation characteristics of RAG systems, including three commercial generative answer engines: You. com, Perplexity AI, and Bing Chat.
XForecast: Evaluating Natural Language Explanations for Time Series Forecasting
However, evaluating forecast NLEs is difficult due to the complex causal relationships in time series data.
Trust but Verify: Programmatic VLM Evaluation in the Wild
Next, to evaluate free-form model responses to queries in PROVE, we propose a programmatic evaluation strategy that measures both the helpfulness and truthfulness of a response within a unified scene graph-based framework.
Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts
Frequency-level specialization is too coarse-grained to capture this level of diversity.
GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation
Additionally, we provide a comprehensive analysis of 17 baselines, which includes statistical models, deep learning models, and foundation models.
P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains
We show that human-written reasoning chains significantly boost the logical reasoning capabilities of LLMs via many-shot prompting and fine-tuning.
Automatic Curriculum Expert Iteration for Reliable LLM Reasoning
To mitigate hallucination and laziness in reasoning tasks, we propose Automatic Curriculum Expert Iteration (Auto-CEI) to enhance LLM reasoning and align responses to the model's capabilities--assertively answering within its limits and declining when tasks exceed them.
MathHay: An Automated Benchmark for Long-Context Mathematical Reasoning in LLMs
Although some recent benchmarks have been developed to evaluate the long-context capabilities of LLMs, there is a lack of benchmarks evaluating the mathematical reasoning abilities of LLMs over long contexts, which is crucial for LLMs' application in real-world scenarios.
ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement
We show that ReGenesis achieves superior performance on all in-domain and OOD settings tested compared to existing methods.
FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"
Ensuring faithfulness to context in large language models (LLMs) and retrieval-augmented generation (RAG) systems is crucial for reliable deployment in real-world applications, as incorrect or unsupported information can erode user trust.
Direct Judgement Preference Optimization
Auto-evaluation is crucial for assessing response quality and offering feedback for model development.
xLAM: A Family of Large Action Models to Empower AI Agent Systems
By releasing the xLAM series, we aim to advance the performance of open-source LLMs for autonomous AI agents, potentially accelerating progress and democratizing access to high-performance models for agent tasks.
xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations
We present xGen-VideoSyn-1, a text-to-video (T2V) generation model capable of producing realistic scenes from textual descriptions.
xGen-MM (BLIP-3): A Family of Open Large Multimodal Models
The framework comprises meticulously curated datasets, a training recipe, model architectures, and a resulting suite of LMMs.
Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents
For instance, a group of open-source SWE agents, with a maximum individual resolve rate of 27. 3% on SWE-Bench Lite, can achieve a 34. 3% resolve rate with DEI, making a 25% improvement and beating most closed-source solutions.
Enabling High Data Throughput Reinforcement Learning on GPUs: A Domain Agnostic Framework for Data-Driven Scientific Research
We introduce WarpSci, a domain agnostic framework designed to overcome crucial system bottlenecks encountered in the application of reinforcement learning to intricate environments with vast datasets featuring high-dimensional observation or action spaces.
Personalized Multi-task Training for Recommender System
In the vast landscape of internet information, recommender systems (RecSys) have become essential for guiding users through a sea of choices aligned with their preferences.
ThinK: Thinner Key Cache by Query-Driven Pruning
Large Language Models (LLMs) have revolutionized the field of natural language processing, achieving unprecedented performance across a variety of applications.
Shared Imagination: LLMs Hallucinate Alike
In this paper, we propose a novel setting, imaginary question answering (IQA), to better understand model similarity.
Consent in Crisis: The Rapid Decline of the AI Data Commons
To our knowledge, we conduct the first, large-scale, longitudinal audit of the consent protocols for the web domains underlying AI training corpora.
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
These tasks, derived from real-world use cases, evaluate the ability of a multimodal agent to perform data-related tasks by writing code and managing the GUI in enterprise data software systems.
Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems
The "Summary of a Haystack" (SummHay) task then requires a system to process the Haystack and generate, given a query, a summary that identifies the relevant insights and precisely cites the source documents.
APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets
The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets.
INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulness
In this work, we introduce INDICT: a new framework that empowers LLMs with Internal Dialogues of Critiques for both safety and helpfulness guidance.
MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens
Multimodal interleaved datasets featuring free-form interleaved sequences of images and text are crucial for training frontier large multimodal models (LMMs).
MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Cases
The deployment of Large Language Models (LLMs) and Large Multimodal Models (LMMs) on mobile devices has gained significant attention due to the benefits of enhanced privacy, stability, and personalization.
UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting
However, existing Transformer models often fall short of capturing both intricate dependencies across variate and temporal dimensions in MTS data.
RLHF Workflow: From Reward Modeling to Online RLHF
We present the workflow of Online Iterative Reinforcement Learning from Human Feedback (RLHF) in this technical report, which is widely reported to outperform its offline counterpart by a large margin in the recent large language model (LLM) literature.
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Autonomous agents that accomplish complex computer tasks with minimal human interventions have the potential to transform human-computer interaction, significantly enhancing accessibility and productivity.
What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biases
Vision-language (VL) models, pretrained on colossal image-text datasets, have attained broad VL competence that is difficult to evaluate.
How Much are Large Language Models Contaminated? A Comprehensive Survey and the LLMSanitize Library
With the rise of Large Language Models (LLMs) in recent years, abundant new opportunities are emerging, but also new challenges, among which contamination is quickly becoming critical.
FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability
This paper presents FoFo, a pioneering benchmark for evaluating large language models' (LLMs) ability to follow complex, domain-specific formats, a crucial yet underexamined capability for their application as AI agents.
AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System
Thus, we open-source a new AI agent library, AgentLite, which simplifies this process by offering a lightweight, user-friendly platform for innovating LLM agent reasoning, architectures, and applications with ease.
AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning
It meticulously standardizes and unifies these trajectories into a consistent format, streamlining the creation of a generic data loader optimized for agent training.
Text2Data: Low-Resource Data Generation with Textual Control
Natural language serves as a common and straightforward signal for humans to interact seamlessly with machines.
Unified Training of Universal Time Series Forecasting Transformers
Deep learning for time series forecasting has traditionally operated within a one-model-per-dataset framework, limiting its potential to leverage the game-changing impact of large pre-trained models.
Ranked #13 on
Time Series Forecasting
on ETTh1 (336) Multivariate
Causal Layering via Conditional Entropy
Under appropriate assumptions and conditioning, we can separate the sources or sinks from the remainder of the nodes by comparing their conditional entropy to the unconditional entropy of their noise.
Editing Arbitrary Propositions in LLMs without Subject Labels
On datasets of binary propositions derived from the CounterFact dataset, we show that our method -- without access to subject labels -- performs close to state-of-the-art L\&E methods which has access subject labels.
Parameter-Efficient Detoxification with Contrastive Decoding
DETOXIGEN is an ensemble of a pre-trained language model (generator) and a detoxifier.
TrustLLM: Trustworthiness in Large Language Models
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions
This work presents Moonshot, a new video generation model that conditions simultaneously on multimodal inputs of image and text.
Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding
Large Language Models (LLMs) have demonstrated a powerful ability for text generation.
X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning
To enable this framework, we devise a scalable pipeline that automatically generates high-quality, instruction-tuning datasets from readily available captioning data across different modalities, and contribute 24K QA data for audio and 250K QA data for 3D.
ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?
Upon its release in late 2022, ChatGPT has brought a seismic shift in the entire landscape of AI, both in research and commerce.
Diffusion Model Alignment Using Direct Preference Optimization
Large language models (LLMs) are fine-tuned using human comparison data with Reinforcement Learning from Human Feedback (RLHF) methods to make them better aligned with users' preferences.
Lexical Repetitions Lead to Rote Learning: Unveiling the Impact of Lexical Overlap in Train and Test Reference Summaries
Ideal summarization models should generalize to novel summary-worthy content without remembering reference training summaries by rote.
Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment
The interactive nature of Large Language Models (LLMs) theoretically allows models to refine and improve their answers, yet systematic analysis of the multi-turn behavior of LLMs remains limited.
Fair Abstractive Summarization of Diverse Perspectives
However, current work in summarization metrics and Large Language Models (LLMs) evaluation has not explored fair abstractive summarization.
Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems
Making big purchases requires consumers to research or consult a salesperson to gain domain expertise.
How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations
Through extensive probing and a new pasting experiment, we further reveal several mechanisms within the trained transformers, such as concrete copying behaviors on both the inputs and the representations, linear ICL capability of the upper layers alone, and a post-ICL representation selection mechanism in a harder mixture setting.
OpenAgents: An Open Platform for Language Agents in the Wild
Language agents show potential in being capable of utilizing natural language for varied and intricate tasks in diverse environments, particularly when built upon large language models (LLMs).
Lemur: Harmonizing Natural Language and Code for Language Agents
We introduce Lemur and Lemur-Chat, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents.
L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models
Recently, large language models (LLMs), especially those that are pretrained on code, have demonstrated strong capabilities in generating programs from natural language inputs in a few-shot or even zero-shot manner.
Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
Conversational interfaces powered by Large Language Models (LLMs) have recently become a popular way to obtain feedback during document editing.
Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles
In this paper, we propose a new task of summarizing diverse information encountered in multiple news articles encompassing the same event.
XGen-7B Technical Report
Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context.
Modeling Uncertainty and Using Post-fusion as Fallback Improves Retrieval Augmented Generation with LLMs
This paper aims to fill this gap by investigating different methods of combining retrieved passages with LLMs to enhance answer generation.
Enhancing Performance on Seen and Unseen Dialogue Scenarios using Retrieval-Augmented End-to-End Task-Oriented System
End-to-end task-oriented dialogue (TOD) systems have achieved promising performance by leveraging sophisticated natural language understanding and natural language generation capabilities of pre-trained models.
BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents
The massive successes of large language models (LLMs) encourage the emerging exploration of LLM-augmented Autonomous Agents (LAAs).
Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
This demonstrates that using policy gradient optimization to improve language agents, for which we believe our work is one of the first, seems promising and can be applied to optimize other models in the agent architecture to enhance agent performances over time.
DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI
Despite advancements in conversational AI, language models encounter challenges to handle diverse conversational tasks, and existing dialogue dataset collections often lack diversity and comprehensiveness.
REX: Rapid Exploration and eXploitation for AI Agents
In this paper, we propose an enhanced approach for Rapid Exploration and eXploitation for AI Agents called REX.
Sample-Efficient Learning of POMDPs with Multiple Observations In Hindsight
This paper studies the sample-efficiency of learning in Partially Observable Markov Decision Processes (POMDPs), a challenging problem in reinforcement learning that is known to be exponentially hard in the worst-case.
Preference-grounded Token-level Guidance for Language Model Fine-tuning
Aligning language models (LMs) with preferences is an important problem in natural language generation.
Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning
Large language models (LLMs) have shown impressive performance in following natural language instructions to solve unseen tasks.
SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages
Text simplification research has mostly focused on sentence-level simplification, even though many desirable edits - such as adding relevant background information or reordering content - may require document-level context.
LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
To address this, we propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits.
UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild
Visual generative foundation models such as Stable Diffusion show promise in navigating these goals, especially when prompted with arbitrary languages.
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding
It achieves a new SOTA of 50. 6% (top-1) on Objaverse-LVIS and 84. 7% (top-1) on ModelNet40 in zero-shot classification.
Ranked #10 on
3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training
This paper presents a novel paradigm for the Zero-Shot Item-based Recommendation (ZSIR) task, which pre-trains a model on product knowledge graph (PKG) to refine the item features from PLMs.
HPE:Answering Complex Questions over Text by Hybrid Question Parsing and Execution
It comprises two central pillars: (1) We parse the question of varying complexity into an intermediate representation, named H-expression, which is composed of simple questions as the primitives and symbolic operations representing the relationships among them; (2) To execute the resulting H-expressions, we design a hybrid executor, which integrates the deterministic rules to translate the symbolic operations with a drop-in neural reader network to answer each decomposed simple question.
CodeGen2: Lessons for Training LLMs on Programming and Natural Languages
In this study, we attempt to render the training of LLMs for program synthesis more efficient by unifying four key components: (1) model architectures, (2) learning methods, (3) infill sampling, and, (4) data distributions.
Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning
Our results demonstrate the strong and efficient modeling ability of NLI-based classifiers and the large cross-lingual transfer improvements achieved by our aligned prompts, particularly in few-shot settings.
GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation
Empirical results show that GlueNet can be trained efficiently and enables various capabilities beyond previous state-of-the-art models: 1) multilingual language models such as XLM-Roberta can be aligned with existing T2I models, allowing for the generation of high-quality images from captions beyond English; 2) GlueNet can align multi-modal encoders such as AudioCLIP with the Stable Diffusion model, enabling sound-to-image generation; 3) it can also upgrade the current text encoder of the latent diffusion model for challenging case generation.
HIVE: Harnessing Human Feedback for Instructional Visual Editing
Incorporating human feedback has been shown to be crucial to align text generated by large language models to human preferences.
On the Unlikelihood of D-Separation
We then provide an analytic average case analysis of the PC Algorithm for causal discovery, as well as a variant of the SGS Algorithm we call UniformSGS.
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation
Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics.
Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systems
Prior works mainly focus on adopting advanced RL techniques to train the ToD agents, while the design of the reward function is not well studied.
A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
This study provides a comprehensive review of recent research advancements, challenges, and opportunities for PFMs in text, image, graph, as well as other data modalities.
Designing and Evaluating Interfaces that Highlight News Coverage Diversity Using Discord Questions
In a second usability study, we developed and implemented a reading exercise with 95 novice news readers to measure exposure to coverage diversity.
Improved Online Conformal Prediction via Strongly Adaptive Online Learning
We prove that our methods achieve near-optimal strongly adaptive regret for all interval lengths simultaneously, and approximately valid coverage.
Lower Bounds for Learning in Revealing POMDPs
However, the fundamental limits for learning in revealing POMDPs are much less understood, with existing lower bounds being rather preliminary and having substantial gaps from the current best upper bounds.
Salesforce CausalAI Library: A Fast and Scalable Framework for Causal Analysis of Time Series and Tabular Data
Finally, we provide a user interface (UI) that allows users to perform causal analysis on data without coding.
Hierarchical Point Attention for Indoor 3D Object Detection
By plugging our proposed modules into the state-of-the-art transformer-based 3D detectors, we improve the previous best results on both benchmarks, with more significant improvements on smaller objects.
LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer
Graphic layout designs play an essential role in visual communication.
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
Human evaluation is the foundation upon which the evaluation of both summarization systems and automatic metrics rests.
ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding
Then, ULIP learns a 3D representation space aligned with the common image-text space, using a small number of automatically synthesized triplets.
Ranked #4 on
Training-free 3D Point Cloud Classification
on ModelNet40
(using extra training data)
Best-$k$ Search Algorithm for Neural Text Generation
We first investigate the vanilla best-first search (BFS) algorithm and then propose the Best-$k$ Search algorithm.
SPE: Symmetrical Prompt Enhancement for Fact Probing
In this work, we propose Symmetrical Prompt Enhancement (SPE), a continuous prompt-based method for factual probing in PLMs that leverages the symmetry of the task by constructing symmetrical prompts for subject and object prediction.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We propose to use sentence-compression data to train the post-editing model to take a summary with extrinsic entity errors marked with special tokens and output a compressed, well-formed summary with those errors removed.
Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Database
Parsing natural language questions into executable logical forms is a useful and interpretable way to perform question answering on structured data such as knowledge bases (KB) or databases (DB).
Discord Questions: A Computational Approach To Diversity Analysis in News Coverage
There are many potential benefits to news readers accessing diverse sources.
Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning
Through this way, SESoM inherits the superior generalization of model ensemble approaches and simultaneously captures the sample-specific competence of each source prompt.
Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models
Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks.
Binding Language Models in Symbolic Languages
We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e. g., SQL, Python) to extend its grammar coverage and thus tackle more diverse questions, (2) adopts an LM as both the program parser and the underlying model called by the API during execution, and (3) requires only a few in-context exemplar annotations.
Ranked #5 on
Table-based Fact Verification
on TabFact
FOLIO: Natural Language Reasoning with First-Order Logic
We present FOLIO, a human-annotated, logically complex and diverse dataset for reasoning in natural language (NL), equipped with first-order logic (FOL) annotations.
Generating Negative Samples for Sequential Recommendation
Due to the dynamics of users' interests and model updates during training, considering randomly sampled items from a user's non-interacted item set as negatives can be uninformative.
BigIssue: A Realistic Bug Localization Benchmark
As machine learning tools progress, the inevitable question arises: How can machine learning help us write better code?
Policy Optimization for Markov Games: Unified Framework and Faster Convergence
Next, we show that this framework instantiated with the Optimistic Follow-The-Regularized-Leader (OFTRL) algorithm at each state (and smooth value updates) can find an $\mathcal{\widetilde{O}}(T^{-5/6})$ approximate NE in $T$ iterations, and a similar algorithm with slightly modified value update rule achieves a faster $\mathcal{\widetilde{O}}(T^{-1})$ convergence rate.
MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation
Counterfactual explanation is an important Explainable AI technique to explain machine learning predictions.
Modeling Multi-hop Question Answering as Single Sequence Prediction
Fusion-in-decoder (Fid) (Izacard and Grave, 2020) is a generative question answering (QA) model that leverages passage retrieval with a pre-trained transformer and pushed the state of the art on single-hop QA.
OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrieval
When finetuned on a single rich-resource language pair, be it English-centered or not, our model is able to match the performance of the ones finetuned on all language pairs under the same data budget with less than 2. 0 points decrease in accuracy.
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets
Precisely assessing the progress in natural language generation (NLG) tasks is challenging, and human evaluation to establish a preference in a model's output over another is often necessary.
Quiz Design Task: Helping Teachers Create Quizzes with Automated Question Generation
Question generation (QGen) models are often evaluated with standardized NLG metrics that are based on n-gram overlap.
Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework
Current contrastive learning frameworks focus on leveraging a single supervisory signal to learn representations, which limits the efficacy on unseen data and downstream tasks.
A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis
Our evaluation results on the single-task polarity prediction show that our approach outperforms the previous state-of-the-art (based on BERT) on average performance by a large margins in few-shot and full-shot settings.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+5
ELECRec: Training Sequential Recommenders as Discriminators
A generator, as an auxiliary model, is trained jointly with the discriminator to sample plausible alternative next items and will be thrown out after training.
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
To democratize this, we train and release a family of large language models up to 16. 1B parameters, called CODEGEN, on natural language and programming language data, and open source the training library JAXFORMER.
Improving Contrastive Learning with Model Augmentation
However, existing methods all construct views by adopting augmentation from data perspectives, while we argue that 1) optimal data augmentation methods are hard to devise, 2) data augmentation methods destroy sequential correlations, and 3) data augmentation fails to incorporate comprehensive self-supervised signals.
Converse: A Tree-Based Modular Task-Oriented Dialogue System
At the core of the struggle is the need to script every single turn of interactions between the bot and the human user.
ConTinTin: Continual Learning from Task Instructions
This work defines a new learning paradigm ConTinTin (Continual Learning from Task Instructions), in which a system should learn a sequence of new tasks one by one, each task is explained by a piece of textual instruction.
Long Document Summarization with Top-down and Bottom-up Inference
Critical to the success of a summarization model is the faithful inference of latent representations of words or tokens in the source documents.
Ranked #1 on
Text Summarization
on Pubmed
Structure Extraction in Task-Oriented Dialogues with Slot Clustering
Extracting structure information from dialogue data can help us better understand user and system behaviors.
Efficient and Differentiable Conformal Prediction with General Function Classes
Experiments show that our algorithm is able to learn valid prediction sets and improve the efficiency significantly over existing approaches in several applications such as prediction intervals with improved length, minimum-volume prediction sets for multi-output regression, and label prediction sets for image classification.
Intent Contrastive Learning for Sequential Recommendation
Specifically, we introduce a latent variable to represent users' intents and learn the distribution function of the latent variable via clustering.
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision.
Ranked #3 on
Open Vocabulary Attribute Detection
on OVAD-Box benchmark
(using extra training data)
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases.
Ranked #1 on
Task-Oriented Dialogue Systems
on KVRET
RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems
Robust machine learning is an increasingly important topic that focuses on developing models resilient to various forms of imperfect data.
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization
Factual consistency is an essential quality of text summarization models in practical settings.
Value Retrieval with Arbitrary Queries for Form-like Documents
Unlike previous methods that only address a fixed set of field items, our method predicts target value for an arbitrary query based on the understanding of the layout and semantics of a form.
Combining Data-driven Supervision with Human-in-the-loop Feedback for Entity Resolution
The distribution gap between training datasets and data encountered in production is well acknowledged.
Open Vocabulary Object Detection with Pseudo Bounding-Box Labels
To enlarge the set of base classes, we propose a method to automatically generate pseudo bounding-box annotations of diverse objects from large-scale image-caption pairs.
Dense Hierarchical Retrieval for Open-Domain Question Answering
In this work, we propose Dense Hierarchical Retrieval (DHR), a hierarchical framework that can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic semantics specific to each passage.
Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization
We first show that this chaotic behavior exists even along the training optimization trajectory of a single model, and propose a simple model averaging protocol that both significantly boosts domain generalization and diminishes the impact of stochasticity by improving the rank correlation between the in-domain validation accuracy and out-domain test accuracy, which is crucial for reliable early stopping.
Ranked #5 on
Domain Generalization
on TerraIncognita
Learning Rich Nearest Neighbor Representations from Self-supervised Ensembles
Pretraining convolutional neural networks via self-supervision, and applying them in transfer learning, is an incredibly fast-growing field that is rapidly and iteratively improving performance across practically all image domains.
Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE
Wasserstein autoencoder (WAE) shows that matching two distributions is equivalent to minimizing a simple autoencoder (AE) loss under the constraint that the latent space of this AE matches a pre-specified prior distribution.
Improving Tail-Class Representation with Centroid Contrastive Learning
ICCL interpolates two images from a class-agnostic sampler and a class-aware sampler, and trains the model such that the representation of the interpolative image can be used to retrieve the centroids for both source classes.
Ranked #22 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
MixQG: Neural Question Generation with Mixed Answer Types
Asking good questions is an essential ability for both human and machine intelligence.
DialFact: A Benchmark for Fact-Checking in Dialogue
Fact-checking is an essential tool to mitigate the spread of misinformation and disinformation.
Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting
In this work, we empirically show that catastrophic forgetting occurs in such methods by evaluating them with general NLP tasks in GLUE.
Field Extraction from Forms with Unlabeled Data
We propose a novel framework to conduct field extraction from forms with unlabeled data.
Robustness Evaluation of Transformer-based Form Field Extractors via Form Attacks
We propose a novel framework to evaluate the robustness of transformer-based form field extraction methods via form attacks.
Self-supervised Learning for Sequential Recommendation with Model Augmentation
However, existing methods all construct views by adopting augmentation from data perspectives, while we argue that 1) optimal data augmentation methods are hard to devise, 2) data augmentation methods destroy sequential correlations, and 3) data augmentation fails to incorporate comprehensive self-supervised signals.
Long Document Summarization with Top-Down and Bottom-Up Representation Inference
Critical to the success of a summarization model is the faithful inference of latent representations of words or tokens in the source documents.
Modeling Dynamic Attributes for Next Basket Recommendation
However, properly integrating them into user interest models is challenging since attribute dynamics can be diverse such as time-interval aware, periodic patterns (etc.
Merlion: A Machine Learning Library for Time Series
We introduce Merlion, an open-source machine learning library for time series.
RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering
We present RnG-KBQA, a Rank-and-Generate approach for KBQA, which remedies the coverage issue with a generation model while preserving a strong generalization capability.
Contrastive Self-supervised Sequential Recommendation with Robust Augmentation
In this paper, we investigate the application of contrastive Self-Supervised Learning (SSL) to the sequential recommendation, as a way to alleviate some of these issues.
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Most existing methods employ a transformer-based multimodal encoder to jointly model visual tokens (region-based image features) and word tokens.
Ranked #5 on
Open Vocabulary Attribute Detection
on OVAD-Box benchmark
(using extra training data)
A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning
Although intuitive, such a native label assignment strategy cannot reveal the underlying semantic similarity between a query and its positives and negatives, and impairs performance, since some negatives are semantically similar to the query or even share the same semantic class as the query.
DocNLI: A Large-scale Dataset for Document-level Natural Language Inference
It has been studied intensively in the past few years thanks to the availability of large-scale labeled datasets.
Learning to Play General-Sum Games Against Multiple Boundedly Rational Agents
We study the problem of training a principal in a multi-agent general-sum game using reinforcement learning (RL).
Understanding the Under-Coverage Bias in Uncertainty Estimation
Estimating the data uncertainty in regression tasks is often done by learning a quantile function or a prediction interval of the true label conditioned on the input.
Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning
This offline result is the first that matches the sample complexity lower bound in this setting, and resolves a recent open question in offline RL.
Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection
Pre-trained Transformer-based models were reported to be robust in intent classification.
Evaluating State-of-the-Art Classification Models Against Bayes Optimality
Moreover, we show that by varying the temperature of the learned flow models, we can generate synthetic datasets that closely resemble standard benchmark datasets, but with almost any desired Bayes error.
Unsupervised Out-of-Domain Detection via Pre-trained Transformers
Deployed real-world machine learning applications are often subject to uncontrolled and even potentially malicious inputs.
