Search Results for author:
Found 27 papers, 24 papers with code
SyncMind: Measuring Agent Out-of-Sync Recovery in Collaborative Software Engineering
Besides substantial performance gaps among agents (from Llama-3. 1 agent <= 3. 33% to Claude-3. 5-Sonnet >= 28. 18%), their consistently low collaboration willingness (<= 4. 86%) suggests fundamental limitations of existing LLM in CSE.
OpenOmni: Large Language Models Pivot Zero-shot Omnimodal Alignment across Language with Real-time Self-Aware Emotional Speech Synthesis
In the alignment phase, a pre-trained speech model is further trained on text-image tasks to generalize from vision to speech in a (near) zero-shot manner, outperforming models trained on tri-modal datasets.
Scaling Laws for Predicting Downstream Performance in LLMs
Our two-stage approach consists of first estimating a function that maps computational resources (e. g., FLOPs) to the pre-training Loss using a series of sampling models, followed by mapping the pre-training loss to downstream task Performance after the critical "emergent phase".
SOLO: A Single Transformer for Scalable Vision-Language Modeling
We present SOLO, a single transformer for Scalable visiOn-Language mOdeling.
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales
Large language models (LLMs) often generate inaccurate or fabricated information and generally fail to indicate their confidence, which limits their broader applications.
Executable Code Actions Elicit Better LLM Agents
LLM agents are typically prompted to produce actions by generating JSON or text in a pre-defined format, which is usually limited by constrained action space (e. g., the scope of pre-defined tools) and restricted flexibility (e. g., inability to compose multiple tools).
ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation
We adopt a weakly-supervised approach to directly generate visual event structures from captions for ViStruct training, capitalizing on abundant image-caption pairs from the web.
DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback
The critique NLF identifies the strengths and weaknesses of the responses and is used to align the LVLMs with human preferences.
R-Tuning: Instructing Large Language Models to Say `I Don't Know'
This approach is formalized by first identifying the disparity in knowledge encompassed by pre-trained parameters compared to that of instruction tuning data.
Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge
Can large language models (LLMs) express their uncertainty in situations where they lack sufficient parametric knowledge to generate reasonable responses?
CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets
It creates toolsets specifically curated for the tasks and equips LLMs with a component that retrieves tools from these sets to enhance their capability to solve complex tasks.
MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback
However, current evaluation protocols often emphasize benchmark performance with single-turn exchanges, neglecting the nuanced interactions among the user, LLMs, and external tools, while also underestimating the importance of natural language feedback from users.
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models
Based on this pipeline and the existing coarse-grained annotated dataset, we build the CURE benchmark to measure both the zero-shot reasoning performance and consistency of VLMs.
Making Pre-trained Language Models both Task-solvers and Self-calibrators
In this work, we consider the practical scenario that we need to effectively utilize training samples to make PLMs both task-solvers and self-calibrators.
Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations
Then we introduce BOSS, a Benchmark suite for Out-of-distribution robustneSS evaluation covering 5 tasks and 20 datasets.
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework
In our experiments, we conduct a robustness evaluation of RoBERTa models to demonstrate the effectiveness of our evaluation framework, and further show the rationality of each component in the framework.
A Close Look into the Calibration of Pre-trained Language Models
We observe a consistent change in calibration performance across six factors.
Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP
We discuss the deficiencies in previous work and propose our suggestions that the research on the Security-oriented adversarial NLP (SoadNLP) should: (1) evaluate their methods on security tasks to demonstrate the real-world concerns; (2) consider real-world attackers' goals, instead of developing impractical methods.
A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks
However, we highlight two issues in previous backdoor learning evaluations: (1) The differences between real-world scenarios (e. g. releasing poisoned datasets or models) are neglected, and we argue that each scenario has its own constraints and concerns, thus requires specific evaluation protocols; (2) The evaluation metrics only consider whether the attacks could flip the models' predictions on poisoned samples and retain performances on benign samples, but ignore that poisoned samples should also be stealthy and semantic-preserving.
Exploring the Universal Vulnerability of Prompt-based Learning Paradigm
Prompt-based learning paradigm bridges the gap between pre-training and fine-tuning, and works effectively under the few-shot setting.
Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework
In this paper, we instantiate our framework with an attack algorithm named Textual Projected Gradient Descent (T-PGD).
Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks
In this paper, we find two simple tricks that can make existing textual backdoor attacks much more harmful.
Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer
In this paper, we make the first attempt to conduct adversarial and backdoor attacks based on text style transfer, which is aimed at altering the style of a sentence while preserving its meaning.
Multi-granularity Textual Adversarial Attack with Behavior Cloning
Furthermore, we propose a reinforcement-learning based method to train a multi-granularity attack agent through behavior cloning with the expert knowledge from our MAYA algorithm to further reduce the query times.
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
As far as we know, almost all existing textual backdoor attack methods insert additional contents into normal samples as triggers, which causes the trigger-embedded samples to be detected and the backdoor attacks to be blocked without much effort.
Automatic Construction of Sememe Knowledge Bases via Dictionaries
We use this method to build an English SKB and a French SKB, and conduct comprehensive evaluations from both intrinsic and extrinsic perspectives.
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks
Nevertheless, there are few studies on defending against textual backdoor attacks.
