Search Results for author:
Found 67 papers, 41 papers with code
Evaluating Mathematical Reasoning Beyond Accuracy
To measure reasoning beyond final-answer accuracy, we introduce ReasonEval, a new methodology for evaluating the quality of reasoning steps.
On the Role of Summary Content Units in Text Summarization Evaluation
At the heart of the Pyramid evaluation method for text summarization lie human written summary content units (SCUs).
Medical Unlearnable Examples: Securing Medical Data from Unauthorized Traning via Sparsity-Aware Local Masking
With the rapid growth of artificial intelligence (AI) in healthcare, there has been a significant increase in the generation and storage of sensitive medical data.
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models
Sora is a text-to-video generative AI model, released by OpenAI in February 2024.
Calibrating Long-form Generations from Large Language Models
Addressing this gap, we introduce a unified calibration framework, in which both the correctness of the LLMs' responses and their associated confidence levels are treated as distributions across a range of scores.
GOODAT: Towards Test-time Graph Out-of-Distribution Detection
To identify and reject OOD samples with GNNs, recent studies have explored graph OOD detection, often focusing on training a specific model or modifying the data on top of a well-trained GNN.
TrustLLM: Trustworthiness in Large Language Models
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
Improving Faithfulness for Vision Transformers
However, ViTs suffer from issues with explanation faithfulness, as their focal points are fragile to adversarial attacks and can be easily changed with even slight perturbations on the input image.
Vector-Quantized Prompt Learning for Paraphrase Generation
Therefore, we present vector-quantized prompts as the cues to control the generation of pre-trained models.
MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning
To solve these challenges, we propose MetaCloak, which solves the bi-level poisoning problem with a meta-learning framework with an additional transformation sampling process to craft transferable and robust perturbation.
Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noise
Observing that simply removing the adversarial noise on the training process of the defensive noise can improve the performance of robust unlearnable examples, we identify that solely the surrogate model's robustness contributes to the performance.
DocMath-Eval: Evaluating Numerical Reasoning Capabilities of LLMs in Understanding Long Documents with Tabular Data
This paper introduces DocMath-Eval, a comprehensive benchmark specifically designed to evaluate the numerical reasoning and problem-solving capabilities of LLMs in the context of understanding and analyzing financial documents containing both text and tables.
Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts
This finding indicates potential exploitable security risks in MLLMs; 2) Based on the acquired system prompts, we propose a novel MLLM jailbreaking attack method termed SASP (Self-Adversarial Attack via System Prompt).
Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization
Our study reveals that instruction controllable text summarization remains a challenging task for LLMs, since (1) all LLMs evaluated still make factual and other types of errors in their summaries; (2) all LLM-based evaluation methods cannot achieve a strong alignment with human annotators when judging the quality of candidate summaries; (3) different LLMs show large performance gaps in summary generation and evaluation.
Fair Abstractive Summarization of Diverse Perspectives
However, current work in summarization metrics and Large Language Models (LLMs) evaluation has not explored fair abstractive summarization.
Towards Self-Interpretable Graph-Level Anomaly Detection
In this paper, we investigate a new challenging problem, explainable GLAD, where the learning objective is to predict the abnormality of each graph sample with corresponding explanations, i. e., the vital subgraph that leads to the predictions.
MaRU: A Manga Retrieval and Understanding System Connecting Vision and Language
Manga, a widely celebrated Japanese comic art form, is renowned for its diverse narratives and distinct artistic styles.
PREM: A Simple Yet Effective Approach for Node-Level Graph Anomaly Detection
Comprising two modules - a pre-processing module and an ego-neighbor matching module - PREM eliminates the necessity for message-passing propagation during training, and employs a simple contrastive loss, leading to considerable reductions in training time and memory usage.
Improving Large Language Model Fine-tuning for Solving Math Problems
With these methods, we present a thorough empirical study on a series of PaLM 2 models and find: (1) The quality and style of the step-by-step solutions used for fine-tuning can make a significant impact on the model performance; (2) While solution re-ranking and majority voting are both effective for improving the model performance when used separately, they can also be used together for an even greater performance boost; (3) Multi-task fine-tuning that sequentially separates the solution generation and evaluation tasks can offer improved performance compared with the solution fine-tuning baseline.
Integrating Graphs with Large Language Models: Methods and Prospects
Large language models (LLMs) such as GPT-4 have emerged as frontrunners, showcasing unparalleled prowess in diverse applications, including answering queries, code generation, and more.
MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
However, in scenarios where LLMs serve as intelligent agents, as seen in applications like AutoGPT and MetaGPT, LLMs are expected to engage in intricate decision-making processes that involve deciding whether to employ a tool and selecting the most suitable tool(s) from a collection of available tools to fulfill user requests.
Towards Data-centric Graph Machine Learning: Review and Outlook
Data-centric AI, with its primary focus on the collection, management, and utilization of data to drive AI models and applications, has attracted increasing attention in recent years.
ODSum: New Benchmarks for Open Domain Multi-Document Summarization
Open-domain Multi-Document Summarization (ODMDS) is a critical tool for condensing vast arrays of documents into coherent, concise summaries.
Everybody Compose: Deep Beats To Music
This project presents a deep learning approach to generate monophonic melodies based on input beats, allowing even amateurs to create their own music compositions.
Learning Strong Graph Neural Networks with Weak Information
Accordingly, we propose D$^2$PT, a dual-channel GNN framework that performs long-range information propagation not only on the input graph with incomplete structure, but also on a global graph that encodes global semantic similarities.
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks
Conventional task- and modality-specific artificial intelligence (AI) models are inflexible in real-world deployment and maintenance for biomedicine.
Ranked #1 on
Text Summarization
on MeQSum
QTSumm: Query-Focused Summarization over Tabular Data
Motivated by this, we define a new query-focused table summarization task, where text generation models have to perform human-like reasoning and analysis over the given table to generate a tailored summary.
On Learning to Summarize with Large Language Models as References
Meanwhile, we perform a meta-analysis on this new learning setting that reveals a discrepancy between human and LLM-based evaluation, highlighting the benefits and risks of this LLM-as-reference setting we investigated.
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation
Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics.
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT
The goal of AIGC is to make the content creation process more efficient and accessible, allowing for the production of high-quality content at a faster pace.
Securing Biomedical Images from Unauthorized Training with Anti-Learning Perturbation
The volume of open-source biomedical data has been essential to the development of various spheres of the healthcare community since more `free' data can provide individual researchers more chances to contribute.
Unlearnable Graph: Protecting Graphs from Unauthorized Exploitation
While the use of graph-structured data in various fields is becoming increasingly popular, it also raises concerns about the potential unauthorized exploitation of personal data for training commercial graph neural network (GNN) models, which can compromise privacy.
BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT
Recently, ChatGPT has gained significant attention in research due to its ability to interact with humans effectively.
A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
This study provides a comprehensive review of recent research advancements, challenges, and opportunities for PFMs in text, image, graph, as well as other data modalities.
On Improving Summarization Factual Consistency from Natural Language Feedback
We collect a high-quality dataset, DeFacto, containing human demonstrations and informational natural language feedback consisting of corrective instructions, edited summaries, and explanations with respect to the factual consistency of the summary.
Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization
To prevent the costly and inefficient use of resources on low-quality annotations, we want a method for creating a pool of dependable annotators who can effectively complete difficult tasks, such as evaluating automatic summarization.
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
Human evaluation is the foundation upon which the evaluation of both summarization systems and automatic metrics rests.
Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminating
Node representations are learned through contrasting the dual-channel encodings obtained from the discriminated homophilic and heterophilic edges.
Federated Learning on Non-IID Graphs via Structural Knowledge Sharing
Inspired by this, we propose FedStar, an FGL framework that extracts and shares the common underlying structure information for inter-graph federated learning tasks.
SEAT: Stable and Explainable Attention
Results show that SEAT is more stable against different perturbations and randomness while also keeps the explainability of attention, which indicates it is a more faithful explanation.
GOOD-D: On Unsupervised Graph Out-Of-Distribution Detection
As a pioneering work in unsupervised graph-level OOD detection, we build a comprehensive benchmark to compare our proposed approach with different state-of-the-art methods.
FOLIO: Natural Language Reasoning with First-Order Logic
We present FOLIO, a human-annotated, open-domain, and logically complex and diverse dataset for reasoning in natural language (NL), equipped with first order logic (FOL) annotations.
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
This problem is especially pertinent in natural language generation which requires ever-improving suites of datasets, metrics, and human evaluation to make definitive claims.
R2D2: Robust Data-to-Text with Replacement Detection
Unfaithful text generation is a common problem for text generation systems.
Leveraging Locality in Abstractive Text Summarization
Our experimental results show that our model has a better performance compared with strong baselines with efficient attention modules, and our analysis provides further insights into our locality-aware modeling strategy.
BRIO: Bringing Order to Abstractive Summarization
Abstractive summarization models are commonly trained using maximum likelihood estimation, which assumes a deterministic (one-point) target distribution in which an ideal model will assign all the probability mass to the reference summary.
Ranked #2 on
Text Summarization
on X-Sum
DataLab: A Platform for Data Analysis and Intervention
Despite data's crucial role in machine learning, most existing tools and research tend to focus on systems on top of existing data rather than how to interpret and manipulate data.
Graph Neural Networks for Graphs with Heterophily: A Survey
In the end, we point out the potential directions to advance and stimulate more future research and applications on heterophilic graph learning with GNNs.
From Unsupervised to Few-shot Graph Anomaly Detection: A Multi-scale Contrastive Learning Approach
Anomaly detection from graph data is an important data mining task in many applications such as social networks, finance, and e-commerce.
Towards Unsupervised Deep Graph Structure Learning
To solve the unsupervised GSL problem, we propose a novel StrUcture Bootstrapping contrastive LearnIng fraMEwork (SUBLIME for abbreviation) with the aid of self-supervised contrastive learning.
CLICKER: A Computational LInguistics Classification Scheme for Educational Resources
We propose a classification scheme -- CLICKER for CL/NLP based on the analysis of online lectures from 77 university courses on this subject.
Surfer100: Generating Surveys From Web Resources, Wikipedia-style
Fast-developing fields such as Artificial Intelligence (AI) often outpace the efforts of encyclopedic sources such as Wikipedia, which either do not completely cover recently-introduced topics or lack such content entirely.
Distribution Locational Marginal Pricing Under Uncertainty Considering Coordination of Distribution and Wholesale Markets
An effective distribution electricity market (DEM) is required to manage the rapidly growing small-scale distributed energy resources (DERs) in distribution systems (DSs).
Generative and Contrastive Self-Supervised Learning for Graph Anomaly Detection
While the generative attribute regression module allows us to capture the anomalies in the attribute space, the multi-view contrastive learning module can exploit richer structure information from multiple subgraphs, thus abling to capture the anomalies in the structure space, mixing of structure, and attribute information.
Priority prediction of Asian Hornet sighting report using machine learning methods
In this paper, we propose a method to predict the priority of sighting reports based on machine learning.
Anomaly Detection in Dynamic Graphs via Transformer
Detecting anomalies for dynamic graphs has drawn increasing attention due to their wide applications in social networks, e-commerce, and cybersecurity.
SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization
In this paper, we present a conceptually simple while empirically powerful framework for abstractive summarization, SimCLS, which can bridge the gap between the learning objective and evaluation metrics resulting from the currently dominated sequence-to-sequence learning framework by formulating text generation as a reference-free evaluation problem (i. e., quality estimation) assisted by contrastive learning.
Ranked #4 on
Text Summarization
on X-Sum
RefSum: Refactoring Neural Summarization
Although some recent works show potential complementarity among different state-of-the-art systems, few works try to investigate this problem in text summarization.
ExplainaBoard: An Explainable Leaderboard for NLP
In this paper, we present a new conceptualization and implementation of NLP evaluation: the ExplainaBoard, which in addition to inheriting the functionality of the standard leaderboard, also allows researchers to (i) diagnose strengths and weaknesses of a single system (e. g.~what is the best-performing system bad at?)
Graph Self-Supervised Learning: A Survey
Deep learning on graphs has attracted significant interests recently.
Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning
Our framework fully exploits the local information from network data by sampling a novel type of contrastive instance pair, which can capture the relationship between each node and its neighboring substructure in an unsupervised way.
Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
Federated learning is a new learning paradigm that decouples data collection and model training via multi-party computation and model aggregation.
Cyclic Label Propagation for Graph Semi-supervised Learning
To overcome these limitations, we introduce a novel framework for graph semi-supervised learning termed as Cyclic Label Propagation (CycProp for abbreviation), which integrates GNNs into the process of label propagation in a cyclic and mutually reinforcing manner to exploit the advantages of both GNNs and LPA.
On Learning Text Style Transfer with Direct Rewards
In most cases, the lack of parallel corpora makes it impossible to directly train supervised models for the text style transfer task.
Conditional Automated Channel Pruning for Deep Neural Networks
To address this issue, given a target compression rate for the whole model, one can search for the optimal compression rate for each layer.
Training and Inference Methods for High-Coverage Neural Machine Translation
In this paper, we introduce a system built for the Duolingo Simultaneous Translation And Paraphrase for Language Education (STAPLE) shared task at the 4th Workshop on Neural Generation and Translation (WNGT 2020).
Adversarial Attack and Defense on Graph Data: A Survey
Therefore, this review is intended to provide an overall landscape of more than 100 papers on adversarial attack and defense strategies for graph data, and establish a unified formulation encompassing most graph adversarial learning models.
