Search Results for author:
Found 53 papers, 19 papers with code
Active Learning for Direct Preference Optimization
Direct preference optimization (DPO) is a form of reinforcement learning from human feedback (RLHF) where the policy is learned directly from preferential feedback.
On Mechanistic Circuits for Extractive Question-Answering
In our paper, we extract mechanistic circuits for this real-world language modeling task: context-augmented language modeling for extractive question-answering (QA) tasks and understand the potential benefits of circuits towards downstream applications such as data attribution to context information.
ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual Attribution
Large Language Models (LLMs) can perform chart question-answering tasks but often generate unverified hallucinated responses.
PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback
Scientific data visualization is pivotal for transforming raw data into comprehensible visual representations, enabling pattern recognition, forecasting, and the presentation of data-driven insights.
Do Large Multimodal Models Solve Caption Generation for Scientific Figures? Lessons Learned from SCICAP Challenge 2023
This paper presents an overview of the first SCICAP Challenge and details the performance of various models on its data, capturing a snapshot of the fields state.
PlotEdit: Natural Language-Driven Accessible Chart Editing in PDFs via Multimodal LLM Agents
Chart visualizations, while essential for data interpretation and communication, are predominantly accessible only as images in PDFs, lacking source data tables and stylistic information.
Multi-LLM Collaborative Caption Generation in Scientific Documents
(Diverse Caption Generation) We then employ a strategy of fine-tuning/prompting multiple LLMs on the captioning task to generate candidate captions.
Persona-SQ: A Personalized Suggested Question Generation Framework For Real-world Documents
Suggested questions (SQs) provide an effective initial interface for users to engage with their documents in AI-powered reading applications.
AD-LLM: Benchmarking Large Language Models for Anomaly Detection
Anomaly detection (AD) is an important machine learning task with many real-world uses, including fraud detection, medical diagnosis, and industrial monitoring.
Political-LLM: Large Language Models in Political Science
In recent years, large language models (LLMs) have been widely adopted in political science tasks such as election prediction, sentiment analysis, policy impact assessment, and misinformation detection.
ScaleViz: Scaling Visualization Recommendation Models on Large Data
However, state-of-the art models rely on very large number of expensive statistics and therefore using such models on large datasets become infeasible due to prohibitively large computational time, limiting the effectiveness of such techniques to most real world complex and large datasets.
Optimizing Data Delivery: Insights from User Preferences on Visuals, Tables, and Text
In this work, we research user preferences to see a chart, table, or text given a question asked by the user.
CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement
By learning from both successes and mistakes, CodeLutra provides a scalable and efficient path to high-quality code generation, making smaller open-source models more competitive with leading closed-source alternatives.
VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-use
In this paper, we present VipAct, an agent framework that enhances VLMs by integrating multi-agent collaboration and vision expert models, enabling more precise visual understanding and comprehensive reasoning.
Federated Large Language Models: Current Progress and Future Directions
Large language models are rapidly gaining popularity and have been widely adopted in real-world applications.
X-Reflect: Cross-Reflection Prompting for Multimodal Recommendation
Large Language Models (LLMs) and Large Multimodal Models (LMMs) have been shown to enhance the effectiveness of enriching item descriptions, thereby improving the accuracy of recommendation systems.
MMR: Evaluating Reading Ability of Large Multimodal Models
Large multimodal models (LMMs) have demonstrated impressive capabilities in understanding various types of image, including text-rich images.
Identifying Speakers in Dialogue Transcripts: A Text-based Approach Using Pretrained Language Models
We introduce an approach to identifying speaker names in dialogue transcripts, a crucial task for enhancing content accessibility and searchability in digital media archives.
On Mechanistic Knowledge Localization in Text-to-Image Generative Models
To address this issue, we introduce the concept of Mechanistic Localization in text-to-image models, where knowledge about various visual attributes (e. g., "style", "objects", "facts") can be mechanistically localized to a small fraction of layers in the UNet, thus facilitating efficient model editing.
GLEMOS: Benchmark for Instantaneous Graph Learning Model Selection
The choice of a graph learning (GL) model (i. e., a GL algorithm and its hyperparameter settings) has a significant impact on the performance of downstream tasks.
SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratings
Crafting effective captions for figures is important.
Forward Learning of Graph Neural Networks
To address these limitations, the forward-forward algorithm (FF) was recently proposed as an alternative to BP in the image classification domain, which trains NNs by performing two forward passes over positive and negative data.
Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints
Controllable layout generation refers to the process of creating a plausible visual arrangement of elements within a graphic design (e. g., document and web designs) with constraints representing design intentions.
Continuous Treatment Effects with Surrogate Outcomes
Incorporating surrogates, which are fully observed post-treatment variables related to the primary outcome, can improve estimation in this case.
Augment before You Try: Knowledge-Enhanced Table Question Answering via Table Expansion
Table question answering is a popular task that assesses a model's ability to understand and interact with structured data.
Leveraging Graph Diffusion Models for Network Refinement Tasks
Most real-world networks are noisy and incomplete samples from an unknown target distribution.
GPT-4 as an Effective Zero-Shot Evaluator for Scientific Figure Captions
We first constructed SCICAP-EVAL, a human evaluation dataset that contains human judgments for 3, 600 scientific figure captions, both original and machine-made, for 600 arXiv figures.
Summaries as Captions: Generating Figure Captions for Scientific Documents with Automated Text Summarization
Prior work often treated figure caption generation as a vision-to-language task.
Graph Learning with Localized Neighborhood Fairness
Learning fair graph representations for downstream applications is becoming increasingly important, but existing work has mostly focused on improving fairness at the global level by either modifying the graph structure or objective function without taking into account the local neighborhood of a node.
MetaGL: Evaluation-Free Selection of Graph Learning Models via Meta-Learning
In this work, we develop the first meta-learning approach for evaluation-free graph learning model selection, called MetaGL, which utilizes the prior performances of existing methods on various benchmark graph datasets to automatically select an effective model for the new graph, without any model training or evaluations.
CGC: Contrastive Graph Clustering for Community Detection and Tracking
Especially, deep graph clustering (DGC) methods have successfully extended deep clustering to graph-structured data by learning node representations and cluster assignments in a joint optimization framework.
Neural Point Process for Learning Spatiotemporal Event Dynamics
The key construction of our approach is the nonparametric space-time intensity function, governed by a latent process.
Automatic Unsupervised Outlier Model Selection
Given an unsupervised outlier detection task on a new dataset, how can we automatically select a good outlier detection algorithm and its hyperparameter(s) (collectively called a model)?
Automatic Forecasting via Meta-Learning
In this work, we develop techniques for fast automatic selection of the best forecasting model for a new unseen time-series dataset, without having to first train (or evaluate) all the models on the new time-series data to select the best one.
Asymptotics of Ridge Regression in Convolutional Models
We show the double descent phenomenon in our experiments for convolutional models and show that our theoretical results match the experiments.
Machine Unlearning via Algorithmic Stability
Our key contribution is the design of corresponding efficient unlearning algorithms, which are based on constructing a (maximal) coupling of Markov chains for the noisy SGD procedure.
Neural Point Process for Forecasting Spatiotemporal Events
To the best of our knowledge, this is the first neural point process model that can jointly predict both the space and time of events.
Learning Contextualized Knowledge Graph Structures for Commonsense Reasoning
Recently, neural-symbolic architectures have achieved success on commonsense reasoning through effectively encoding relational structures retrieved from external knowledge graphs (KGs) and obtained state-of-the-art results in tasks such as (commonsense) question answering and natural language inference.
Learning Contextualized Knowledge Structures for Commonsense Reasoning
Recently, knowledge graph (KG) augmented models have achieved noteworthy success on various commonsense reasoning tasks.
Learning to Deceive Knowledge Graph Augmented Models via Targeted Perturbation
Knowledge graphs (KGs) have helped neural models improve performance on various knowledge-intensive tasks, like question answering and item recommendation.
Clustering-based Unsupervised Generative Relation Extraction
To address this issue, we propose a Clustering-based Unsupervised generative Relation Extraction (CURE) framework that leverages an "Encoder-Decoder" architecture to perform self-supervised learning so the encoder can extract relation information.
Reinforcement Learning-based N-ary Cross-Sentence Relation Extraction
In this paper, we relax this strong assumption by a weaker distant supervision assumption to address the second issue and propose a novel sentence distribution estimator model to address the first problem.
Adjusting for Confounders with Text: Challenges and an Empirical Evaluation Framework for Causal Inference
Across 648 experiments and two datasets, we evaluate every commonly used causal inference method and identify their strengths and weaknesses to inform social media researchers seeking to use such methods, and guide future improvements.
Inferring Individual Level Causal Models from Graph-based Relational Time Series
We propose causal inference models for this problem that leverage both the graph topology and time-series to accurately estimate local causal effects of nodes.
Deep Relational Factorization Machines
Factorization Machines (FMs) is an important supervised learning approach due to its unique ability to capture feature interactions when dealing with high-dimensional sparse data.
Figure Captioning with Reasoning and Sequence-Level Training
In this work, we investigate the problem of figure captioning where the goal is to automatically generate a natural language description of the figure.
node2bits: Compact Time- and Attribute-aware Node Representations for User Stitching
Identity stitching, the task of identifying and matching various online references (e. g., sessions over different devices and timespans) to the same user in real-world web services, is crucial for personalization and recommendations.
Latent Network Summarization: Bridging Network Embedding and Summarization
Motivated by the computational and storage challenges that dense embeddings pose, we introduce the problem of latent network summarization that aims to learn a compact, latent representation of the graph structure with dimensionality that is independent of the input graph size (i. e., #nodes and #edges), while retaining the ability to derive node representations on the fly.
Social and Information Networks
Graph Classification using Structural Attention
Graph classification is a problem with practical applications in many different domains.
Ranked #1 on
Graph Classification
on NCI33
Learning Role-based Graph Embeddings
Random walks are at the heart of many existing network embedding methods.
Deep Graph Attention Model
Graph classification is a problem with practical applications in many different domains.
