Synthesizing Political Zero-Shot Relation Classification via Codebook Knowledge, NLI, and ChatGPT
Can we accurately classify political relations within evolving event ontologies without extensive annotations? Our study investigates zero-shot learning methods that utilize only expert knowledge from existing annotation codebook. We assess the performance of advanced ChatGPT (GPT-3.5/4) and a natural language inference (NLI)-based model called ZSP. ChatGPT uses codebooks' label summaries as prompts, whereas ZSP breaks down the classification task into context, event mode, and class disambiguation to refine task-specific hypotheses. This decomposition enhances interpretability, efficiency, and adaptability to schema changes. The experiments reveal ChatGPT's strengths and limitations, and crucially, ZSP's outperformance of dictionary-based methods and its competitive edge over some supervised models. These findings affirm the value of ZSP for validating event records and advancing ontology development. Our study underscores the efficacy of leveraging transfer learning and existing expertise to enhance research efficiency and scalability in this area.
PDF Abstract
