Achieved results

• Scientific publication Efficient Argument Classification with Compact Language Models and ChatGPT-4 Refinements. This paper presents comparative studies between a few deep learning-based models in argument mining. The work concentrates on argument classification. The research was done on a wide spectrum of datasets (Args.me, UKP, US2016). The main novelty of this paper is the ensemble model which is based on BERT architecture and ChatGPT-4 as fine tuning model. The presented results show that BERT+ChatGPT-4 outperforms the rest of the models including other Transformer-based and LSTM-based models. The observed improvement is, in most cases, greater than 10%. The presented analysis can provide crucial insights into how the models for argument classification should be further improved. Additionally, it can help develop a prompt-based algorithm to eliminate argument classification errors.
  Cite as: Pietron, M., Olszowski, R., Gomułka, J. (2024). Efficient Argument Classification with Compact Language Models and ChatGPT-4 Refinements. In: Nguyen, N.T., et al. Computational Collective Intelligence. ICCCI 2024. Lecture Notes in Computer Science, vol 14810. Springer, Cham. https://doi.org/10.1007/978-3-031-70816-9_20.

Planned results

• Scientific Publication on Comparative Research in Argument Classification Using Language Models. This scientific publication presents a comparative study on argument classification using language models. The analysis examines diverse datasets sourced from well-established research projects, including Args.me, UKP, and US2016. The study incorporates various models such as GPT-4, GPT-4o, Meta-LLAMA, and DeepSeek, as well as an evaluation of different prompting strategies. The publication discusses the most common errors observed across all tested models and highlights key differences between them. Furthermore, the paper investigates widely used inference algorithms based on prompt engineering techniques and introduces a hybrid model composed of selected LLMs, rephrased queries, and a voting module. This new model improves argument recognition accuracy, although it still does not achieve full precision. According to the authors, this study represents the first extensive analysis of the mentioned datasets using LLMs and prompt-based algorithms. Additionally, the paper identifies the limitations of existing prompt engineering approaches in argument analysis and suggests directions for their further improvement.
• Scientific Publication on Argument Theory and Annotation Policy. This publication explores theoretical approaches to argumentation and challenges related to annotation policies in argument analysis. It provides a comprehensive review of argument definitions, examining various theoretical perspectives and practical difficulties associated with their application, particularly in Argument Mining (AM). In the first section, the authors discuss existing concepts of argumentation found in the academic literature. The review covers classical approaches in logic and critical thinking, rhetorical and legal perspectives, as well as a philosophical perspective inspired by Wittgenstein, where argumentation is viewed as a form of storytelling. Additionally, the paper analyzes argumentation within debates, contrasting the agonistic theory with deliberative models, and examines argumentation in linguistic contexts. The second part of the paper focuses on the challenges of automatic argument recognition in online discussions and existing annotated corpora used in AM research. The study then introduces a practical set of criteria for argument identification, aiming to develop an operational definition of an argument in a regulatory sense rather than a purely descriptive one. Beginning with a broad approach (e.g., Argumentative Discursive Unit; ADU), the authors propose a gradual refinement and justification of stricter regulations to enhance argument detection in both theoretical research and empirical applications.
• Argument Database. As a key outcome of our research project, we develop a comprehensive corpus of texts on public issues, annotated in terms of argument-premise relationships and the multithreaded structures of various debate models. In line with our project’s objectives, the corpus will contain approximately 30,000 records in Polish and another 30,000 in English, facilitating effective machine learning for Argument Mining. The collected data, stored in a database in JSON format, will be continuously updated to reflect the most current topics of public debate that emerge throughout the project's duration. Given the rapidly changing nature of public discourse, ongoing data collection and annotation represent a valuable contribution to the field. Additionally, knowledge graphs will be generated using Argument Extraction Models to map interrelations between arguments within the corpus and to identify argumentation patterns across different debate models. These efforts will include the organization of hierarchical categories and relationships, forming formal ontologies that enhance the structured representation of argumentation. Argument database will be developed using Semantic Web Technology, accompanied by a web-based interface, ensuring accessibility and usability for researchers and AI-driven applications.
• Two additional scientific publications discussing the results of the project.