Definitions Matter: Guiding GPT for Multi-label Classification

Peskine, Youri; Korenčić, Damir; Grubišić, Ivan; Papotti, Paolo; Troncy, Raphael; Rosso, Paolo (2023) Definitions Matter: Guiding GPT for Multi-label Classification. In: Bouamor, Houda; Pino, Juan; Bali, Kalika, (eds.) Findings of the Association for Computational Linguistics: EMNLP 2023. Singapore, Association for Computational Linguistics (ACL), pp. 4054-4063 .

PDF - Published Version - article Available under License Creative Commons Attribution. Download (190kB)

Abstract

Large language models have recently risen in popularity due to their ability to perform many natural language tasks without requiring any fine-tuning. In this work, we focus on two novel ideas: (1) generating definitions from examples and using them for zero-shot classification, and (2) investigating how an LLM makes use of the definitions. We thoroughly analyze the performance of GPT-3 model for fine-grained multi-label conspiracy theory classification of tweets using zero-shot labeling. In doing so, we asses how to improve the labeling by providing minimal but meaningful context in the form of the definitions of the labels. We compare descriptive noun phrases, human-crafted definitions, introduce a new method to help the model generate definitions from examples, and propose a method to evaluate GPT-3’s understanding of the definitions. We demonstrate that improving definitions of class labels has a direct consequence on the downstream classification results.

Item Type:	Conference or workshop item published in conference proceedings (UNSPECIFIED)
Uncontrolled Keywords:	LLM; GPT-3; natural language processing; classification
Subjects:	TECHNICAL SCIENCES > Computing > Artificial Intelligence
Divisions:	Division of Electronics
Depositing User:	Ivan Grubišić
Date Deposited:	14 Jan 2026 10:32
URI:	http://fulir.irb.hr/id/eprint/10844
DOI:	10.18653/v1/2023.findings-emnlp.267

Actions (login required)

View Item