Search Results for author:
Found 52 papers, 35 papers with code
Categorizing Offensive Language in Social Networks: A Chinese Corpus, Systems and an Explainable Tool
Recently, more and more data have been generated in the online world, filled with offensive language such as threats, swear words or straightforward insults.
LocAgent: Graph-Guided LLM Agents for Code Localization
By parsing codebases into directed heterogeneous graphs, LocAgent creates a lightweight representation that captures code structures (files, classes, functions) and their dependencies (imports, invocations, inheritance), enabling LLM agents to effectively search and locate relevant entities through powerful multi-hop reasoning.
MedAgentsBench: Benchmarking Thinking Models and Agent Frameworks for Complex Medical Reasoning
Large Language Models (LLMs) have shown impressive performance on existing medical question-answering benchmarks.
MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents
Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents, yet existing benchmarks either focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition.
MMVU: Measuring Expert-Level Multi-Discipline Video Understanding
We introduce MMVU, a comprehensive expert-level, multi-discipline benchmark for evaluating foundation models in video understanding.
ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning
To address these challenges, we present ChemAgent, a novel framework designed to improve the performance of LLMs through a dynamic, self-updating library.
ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain
To address this issue in the field of chemistry, we introduce ChemSafetyBench, a benchmark designed to evaluate the accuracy and safety of LLM responses.
FinDVer: Explainable Claim Verification over Long and Hybrid-Content Financial Documents
We introduce FinDVer, a comprehensive benchmark specifically designed to evaluate the explainable claim verification capabilities of LLMs in the context of understanding and analyzing long, hybrid-content financial documents.
OpenHands: An Open Platform for AI Software Developers as Generalist Agents
OpenDevin), a platform for the development of powerful and flexible AI agents that interact with the world in similar ways to those of a human developer: by writing code, interacting with a command line, and browsing the web.
Step-Back Profiling: Distilling User History for Personalized Scientific Writing
To conduct the experiments, we construct a Personalized Scientific Writing (PSW) dataset to study multi-user personalization.
Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediation
This survey serves as a succinct overview of the most recent advancements in data contamination research, providing a straightforward guide for the benefit of future research endeavors.
PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes
Current approaches, however, often neglect the critical role of multiple molecule graph interaction in understanding chemical reactions, leading to suboptimal performance in synthetic chemistry tasks.
Lessons from the Trenches on Reproducible Evaluation of Language Models
Third, we present the Language Model Evaluation Harness (lm-eval): an open source library for independent, reproducible, and extensible evaluation of language models that seeks to address these issues.
MIMIR: A Streamlined Platform for Personalized Agent Tuning in Domain Expertise
Recently, large language models (LLMs) have evolved into interactive agents, proficient in planning, tool use, and task execution across a wide variety of tasks.
StarCoder 2 and The Stack v2: The Next Generation
Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size.
Ranked #35 on
Code Generation
on MBPP
Data Interpreter: An LLM Agent For Data Science
On InfiAgent-DABench, it achieves a 25% performance boost, raising accuracy from 75. 9% to 94. 9%.
ChatCell: Facilitating Single-Cell Analysis with Natural Language
As Large Language Models (LLMs) rapidly evolve, their influence in science is becoming increasingly prominent.
A Survey of Generative AI for de novo Drug Design: New Frontiers in Molecule and Protein Generation
Within each theme, we identify a variety of subtasks and applications, highlighting important datasets, benchmarks, and model architectures and comparing the performance of top models.
Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science
Intelligent agents powered by large language models (LLMs) have demonstrated substantial promise in autonomously conducting experiments and facilitating scientific discoveries across various disciplines.
Weaver: Foundation Models for Creative Writing
Weaver is pre-trained on a carefully selected corpus that focuses on improving the writing capabilities of large language models.
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents
Large language models (LLMs) have dramatically enhanced the field of language intelligence, as demonstrably evidenced by their formidable empirical performance across a spectrum of complex reasoning tasks.
Investigating Data Contamination in Modern Benchmarks for Large Language Models
Recent observations have underscored a disparity between the inflated benchmark scores and the actual performance of LLMs, raising concerns about potential contamination of evaluation benchmarks.
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning
Large language models (LLMs), despite their remarkable progress across various general domains, encounter significant barriers in medicine and healthcare.
DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Long and Specialized Documents
Recent LLMs have demonstrated remarkable performance in solving exam-like math word problems.
ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code
Despite Large Language Models (LLMs) like GPT-4 achieving impressive results in function-level code generation, they struggle with repository-scale code understanding (e. g., coming up with the right arguments for calling routines), requiring a deeper comprehension of complex file interactions.
Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
This survey addresses the crucial issue of factuality in Large Language Models (LLMs).
Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models
Large language models (LLMs) have unveiled remarkable reasoning capabilities by exploiting chain-of-thought (CoT) prompting, which generates intermediate reasoning chains to serve as the rationale for deriving the answer.
Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?
Despite the remarkable capabilities of Large Language Models (LLMs) like GPT-4, producing complex, structured tabular data remains challenging.
Agents: An Open-source Framework for Autonomous Language Agents
Recent advances on large language models (LLMs) enable researchers and developers to build autonomous language agents that can automatically solve various tasks and interact with environments, humans, and other agents using natural language interfaces.
BioCoder: A Benchmark for Bioinformatics Code Generation with Large Language Models
This is evident from the performance gain of GPT-3. 5/4 compared to the smaller models on our benchmark (50% vs. up to 25%).
OctoPack: Instruction Tuning Code Large Language Models
We benchmark CommitPack against other natural and synthetic code instructions (xP3x, Self-Instruct, OASST) on the 16B parameter StarCoder model, and achieve state-of-the-art performance among models not trained on OpenAI outputs, on the HumanEval Python benchmark (46. 2% pass@1).
Ranked #19 on
Code Generation
on HumanEval
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
Based on ToolBench, we fine-tune LLaMA to obtain an LLM ToolLLaMA, and equip it with a neural API retriever to recommend appropriate APIs for each instruction.
Ranked #3 on
Trajectory Planning
on ToolBench
RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations
Despite significant progress having been made in question answering on tabular data (Table QA), it's unclear whether, and to what extent existing Table QA models are robust to task-specific perturbations, e. g., replacing key question entities or shuffling table columns.
Investigating Table-to-Text Generation Capabilities of LLMs in Real-World Information Seeking Scenarios
These include the LogicNLG and our newly-constructed LoTNLG datasets for data insight generation, along with the FeTaQA and our newly-constructed F2WTQ datasets for query-based generation.
QTSumm: Query-Focused Summarization over Tabular Data
Motivated by this, we define a new query-focused table summarization task, where text generation models have to perform human-like reasoning and analysis over the given table to generate a tailored summary.
RWKV: Reinventing RNNs for the Transformer Era
This work presents a significant step towards reconciling trade-offs between computational efficiency and model performance in sequence processing tasks.
Ranked #22 on
Natural Language Inference
on WNLI
GersteinLab at MEDIQA-Chat 2023: Clinical Note Summarization from Doctor-Patient Conversations through Fine-tuning and In-context Learning
This paper presents our contribution to the MEDIQA-2023 Dialogue2Note shared task, encompassing both subtask A and subtask B.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Crosslingual Generalization through Multitask Finetuning
We find finetuning large multilingual language models on English tasks with English prompts allows for task generalization to non-English languages that appear only in the pretraining corpus.
Ranked #1 on
Question Answering
on StoryCloze
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
PromptSource is a system for creating, sharing, and using natural language prompts.
CLICKER: A Computational LInguistics Classification Scheme for Educational Resources
We propose a classification scheme -- CLICKER for CL/NLP based on the analysis of online lectures from 77 university courses on this subject.
CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuning
Using human evaluation and automatic faithfulness metrics, we show that our model significantly reduces all kinds of factual errors on the dialogue summarization, SAMSum corpus.
Surfer100: Generating Surveys From Web Resources, Wikipedia-style
Fast-developing fields such as Artificial Intelligence (AI) often outpace the efforts of encyclopedic sources such as Wikipedia, which either do not completely cover recently-introduced topics or lack such content entirely.
Improving RNA Secondary Structure Design using Deep Reinforcement Learning
Rising costs in recent years of developing new drugs and treatments have led to extensive research in optimization techniques in biomolecular design.
Multi-modal Self-supervised Pre-training for Regulatory Genome Across Cell Types
The core problem is to model how regulatory elements interact with each other and its variability across different cell types.
Multi-modal Self-supervised Pre-training for Large-scale Genome Data
In this work, we propose a simple yet effective approach for pre-training genome data in a multi-modal and self-supervised manner, which we call GeneBERT.
Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries
Current pre-trained models applied to summarization are prone to factual inconsistencies which either misrepresent the source text or introduce extraneous information.
FeTaQA: Free-form Table Question Answering
Existing table question answering datasets contain abundant factual questions that primarily evaluate the query and schema comprehension capability of a system, but they fail to include questions that require complex reasoning and integration of information due to the constraint of the associated short-form answers.
FILM: A Fast, Interpretable, and Low-rank Metric Learning Approach for Sentence Matching
We construct this metric learning problem as a manifold optimization problem and solve it with the Cayley transformation method with the Barzilai-Borwein step size.
Multi-Granularity Modularized Network for Abstract Visual Reasoning
Abstract visual reasoning connects mental abilities to the physical world, which is a crucial factor in cognitive development.
DART: Open-Domain Structured Data Record to Text Generation
Data-to-Text annotations can be a costly process, especially when dealing with tables which are the major source of structured data and contain nontrivial structures.
CUHK at SemEval-2020 Task 4: CommonSense Explanation, Reasoning and Prediction with Multi-task Learning
This paper describes our system submitted to task 4 of SemEval 2020: Commonsense Validation and Explanation (ComVE) which consists of three sub-tasks.
