Communication Dans Un Congrès Année : 2025

The contribution of LLMs to relation extraction in the economic field

Résumé

Relation Extraction (RE) is a fundamental task in natural language processing, aimed at deducing semantic relationships between entities in a text. Traditional supervised relation extraction methods involve training models to annotate tokens representing entity mentions, followed by predicting the relationship between these entities. However, recent advancements have transformed this task into a sequence-to-sequence problem. This involves converting relationships between entities into target strings, which are then generated from the input text. Thus, language models now appear as a solution to this task and have already been used in numerous studies, with various levels of refinement, across different domains. The objective of the present study is to evaluate the contribution of large language models (LLM) to the task of relation extraction in a specific domain (in this case, the economic domain), compared to smaller language models. To do this, we considered as a baseline a model based on the BERT architecture, trained in this domain, and four LLM, namely Fin-GPT specific to the financial domain, XLNet, ChatGLM, and Llama3, which are generalists. All these models were evaluated on the same extraction task, with zero-shot for the general purpose LLM, as well as refinements through few-shot learning and fine-tuning. The experiments showed that the best performance in terms of F-score was achieved with fine-tuned LLM, with Llama3 achieving the highest performance.
Fichier principal
Vignette du fichier
FinNLP2025_Ecladatta.pdf (200) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04940833 , version 1 (11-02-2025)

Licence

Identifiants

  • HAL Id : hal-04940833 , version 1

Citer

Mohamed Ettaleb, Véronique Moriceau, Mouna Kamel, Nathalie Aussenac-Gilles. The contribution of LLMs to relation extraction in the economic field. The Joint Workshop of the 9th Financial Technology and Natural Language Processing (FinNLP), the 6th Financial Narrative Processing (FNP), and the 1st Workshop on Large Language Models for Finance and Legal (LLMFinLegal), Jan 2025, Abu Dhabi, United Arab Emirates. ⟨hal-04940833⟩
0 Consultations
0 Téléchargements

Partager

More