Entity Matching using Large Language Models

Entity Matching is the task of deciding whether two entity descriptions refer to the same real-world entity. It is a central step in most data integration pipelines and an enabler for many e-commerce applications which require to match products offers from different vendors. State-of-the-art entity matching methods rely on pre-trained language models (PLMs) such as BERT or RoBERTa. Two major drawbacks of these models for entity matching are that (i) the models require significant amounts of task-specific training data and (ii) the fine-tuned models are not robust concerning out-of-distribution entities. We investigate using generative large language models (LLMs) for entity matching as a less task-specific training data dependent and more robust alternative to PLM-based matchers. Our study covers hosted LLMs as well as open-source LLMs which can be run locally. We evaluate these models in a zero-shot scenario as well as a scenario where task-specific training data is available. We compare different prompt designs as well as the prompt sensitivity of the models and show that there is no single best prompt but the prompt is akin to a hyperparameter that needs to be estimated for each model/dataset combination. We further investigate (i) the selection of in-context demonstrations, (ii) the generation of matching rules, as well as (iii) fine-tuning a hosted LLM using the same pool of training data. Our experiments show that the best LLMs require no or only a few training examples to reach a similar performance as fine-tuned PLMs. They further exhibit a higher robustness to unseen entities, which makes them especially suited to use cases where no training data is available. We show that for use cases that do not allow data to be shared with third parties, open-source LLMs can be a viable alternative to hosted LLMs given that a small amount of training data or matching knowledge...