Search Results for author:
Found 22 papers, 11 papers with code
BIG-Bench Extra Hard
One particular exception is the BIG-Bench dataset, which has served as a crucial benchmark for evaluating the general reasoning capabilities of LLMs, thanks to its diverse set of challenging tasks that allowed for a comprehensive assessment of general reasoning across various skills within a unified framework.
CausalGraph2LLM: Evaluating LLMs for Causal Queries
In this paper, we propose a comprehensive benchmark, \emph{CausalGraph2LLM}, encompassing a variety of causal graph settings to assess the causal graph understanding capability of LLMs.
Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries
We introduce Michelangelo: a minimal, synthetic, and unleaked long-context reasoning evaluation for large language models which is also easy to automatically score.
Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights
First, the absence of a comprehensive benchmark with unified problem settings hinders a clear understanding of the comparative effectiveness and practical value of different text-space GFMs.
Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
Large language models (LLMs) have showcased remarkable reasoning capabilities, yet they remain susceptible to errors, particularly in temporal reasoning tasks involving complex temporal logic.
ReMI: A Dataset for Reasoning with Multiple Images
With the continuous advancement of large language models (LLMs), it is essential to create new benchmarks to effectively evaluate their expanding capabilities and identify areas for improvement.
Understanding Transformer Reasoning Capabilities via Graph Algorithms
Our novel representational hierarchy separates 9 algorithmic reasoning problems into classes solvable by transformers in different realistic parameter scaling regimes.
Don't Forget to Connect! Improving RAG with Graph-based Reranking
Retrieval Augmented Generation (RAG) has greatly improved the performance of Large Language Model (LLM) responses by grounding generation with context from existing documents.
Let Your Graph Do the Talking: Encoding Structured Data for LLMs
How can we best encode structured data into sequential form for use in large language models (LLMs)?
Talk like a Graph: Encoding Graphs for Large Language Models
Graphs are a powerful tool for representing and analyzing complex relationships in real-world applications such as social networks, recommender systems, and computational finance.
TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs
TpuGraphs provides 25x more graphs than the largest graph property prediction dataset (with comparable graph sizes), and 770x larger graphs on average compared to existing performance prediction datasets on machine learning programs.
Ranked #2 on
Runtime ranking
on TpuGraphs Layout mean
UGSL: A Unified Framework for Benchmarking Graph Structure Learning
We implement a wide range of existing models in our framework and conduct extensive analyses of the effectiveness of different components in the framework.
Learning to Substitute Ingredients in Recipes
Recipe personalization through ingredient substitution has the potential to help people meet their dietary needs and preferences, avoid potential allergens, and ease culinary exploration in everyone's kitchen.
Using Graph Algorithms to Pretrain Graph Completion Transformers
While using our new path-finding algorithm as a pretraining signal provides 2-3% MRR improvements, we show that pretraining on all signals together gives the best knowledge graph completion results.
Knowledge Hypergraph Embedding Meets Relational Algebra
Embedding-based methods for reasoning in knowledge hypergraphs learn a representation for each entity and relation.
SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks
In this work, we propose the Simultaneous Learning of Adjacency and GNN Parameters with Self-supervision, or SLAPS, a method that provides more supervision for inferring a graph structure through self-supervision.
Ranked #1 on
Graph structure learning
on Cora
Knowledge Hypergraphs: Prediction Beyond Binary Relations
Knowledge graphs store facts using relations between two entities.
Improved Knowledge Graph Embedding using Background Taxonomic Information
Knowledge graphs are used to represent relational information in terms of triples.
Structure Learning for Relational Logistic Regression: An Ensemble Approach
We consider the problem of learning Relational Logistic Regression (RLR).
Record Linkage to Match Customer Names: A Probabilistic Approach
We provide a probabilistic model using relational logistic regression to find the probability of each record in the database being the desired record for a given query and find the best record(s) with respect to the probabilities.
Comparing Aggregators for Relational Probabilistic Models
Relational probabilistic models have the challenge of aggregation, where one variable depends on a population of other variables.
A Learning Algorithm for Relational Logistic Regression: Preliminary Results
We compare our learning algorithm to other structure and parameter learning algorithms in the literature, and compare the performance of RLR models to standard logistic regression and RDN-Boost on a modified version of the MovieLens data-set.
