Towards a Valid and Reliable Checklist to Evaluate Argumentative Essays Composed by ChatGPT
Конференцијски прилог (Објављена верзија)
Метаподаци
Приказ свих података о документуАпстракт
Since its launch a year ago, ChatGPT has sparked many concerns in education, especially
when it comes to writing. Many students enjoy the benefits of getting generated text for their
homework assignments; however, this behaviour impacts profoundly the writing process and
the development of critical thinking skills. Among these assignments that are particularly
important to critical skills development are so-called argumentative essays, which require the
student to investigate a topic, collect, generate, and evaluate evidence, and establish a
position on the topic in a concise manner. To assess these essays in a thoughtful way, this
paper presents a checklist whose indicators focus on main aspects of essay organisation and
higher-order critical thinking skills. The checklist was developed for both machine and
human responses by using relevant theoretical framework (the Classical model of
Argumentation and Paul-Elder critical thinking framework), the five-paragraph approach,
a...nd Cambridge English Qualifications scales at level C1 of the CEFR. As this assessment
tool was applied in evaluating ChatGPT-composed argumentative essays, apart from the
validity of the tool, this paper also presents its inter-rater reliability. Suggestions for research
and practice are included.
Кључне речи:
ELT / argumentative essay / writing / ChatGPT / checklist / critical thinking / reliability / validityИзвор:
E-Learning, 2024, 3696, 82-90Издавач:
- Belgrade : Metropolitan
Финансирање / пројекти:
- Министарство науке, технолошког развоја и иновација Републике Србије, институционално финансирање - 200018 (Институт за педагошка истраживања, Београд) (RS-MESTD-inst-2020-200018)
Напомена:
- 15th International Conference on eLearning (eLearning-2024) that will be held on September 26-27, 2024 at Belgrade Metropolitan University, Belgrade, Serbia.
Институција/група
IPITY - CONF AU - Ljubojević, Danijela AU - Kadijević, Đorđe AU - Gutvajn, Nikoleta PY - 2024 UR - https://ceur-ws.org/Vol-3696/ UR - http://ipir.ipisr.org.rs/handle/123456789/1097 AB - Since its launch a year ago, ChatGPT has sparked many concerns in education, especially when it comes to writing. Many students enjoy the benefits of getting generated text for their homework assignments; however, this behaviour impacts profoundly the writing process and the development of critical thinking skills. Among these assignments that are particularly important to critical skills development are so-called argumentative essays, which require the student to investigate a topic, collect, generate, and evaluate evidence, and establish a position on the topic in a concise manner. To assess these essays in a thoughtful way, this paper presents a checklist whose indicators focus on main aspects of essay organisation and higher-order critical thinking skills. The checklist was developed for both machine and human responses by using relevant theoretical framework (the Classical model of Argumentation and Paul-Elder critical thinking framework), the five-paragraph approach, and Cambridge English Qualifications scales at level C1 of the CEFR. As this assessment tool was applied in evaluating ChatGPT-composed argumentative essays, apart from the validity of the tool, this paper also presents its inter-rater reliability. Suggestions for research and practice are included. PB - Belgrade : Metropolitan C3 - E-Learning T1 - Towards a Valid and Reliable Checklist to Evaluate Argumentative Essays Composed by ChatGPT EP - 90 SP - 82 VL - 3696 UR - https://hdl.handle.net/21.15107/rcub_ipir_1097 ER -
@conference{ author = "Ljubojević, Danijela and Kadijević, Đorđe and Gutvajn, Nikoleta", year = "2024", abstract = "Since its launch a year ago, ChatGPT has sparked many concerns in education, especially when it comes to writing. Many students enjoy the benefits of getting generated text for their homework assignments; however, this behaviour impacts profoundly the writing process and the development of critical thinking skills. Among these assignments that are particularly important to critical skills development are so-called argumentative essays, which require the student to investigate a topic, collect, generate, and evaluate evidence, and establish a position on the topic in a concise manner. To assess these essays in a thoughtful way, this paper presents a checklist whose indicators focus on main aspects of essay organisation and higher-order critical thinking skills. The checklist was developed for both machine and human responses by using relevant theoretical framework (the Classical model of Argumentation and Paul-Elder critical thinking framework), the five-paragraph approach, and Cambridge English Qualifications scales at level C1 of the CEFR. As this assessment tool was applied in evaluating ChatGPT-composed argumentative essays, apart from the validity of the tool, this paper also presents its inter-rater reliability. Suggestions for research and practice are included.", publisher = "Belgrade : Metropolitan", journal = "E-Learning", title = "Towards a Valid and Reliable Checklist to Evaluate Argumentative Essays Composed by ChatGPT", pages = "90-82", volume = "3696", url = "https://hdl.handle.net/21.15107/rcub_ipir_1097" }
Ljubojević, D., Kadijević, Đ.,& Gutvajn, N.. (2024). Towards a Valid and Reliable Checklist to Evaluate Argumentative Essays Composed by ChatGPT. in E-Learning Belgrade : Metropolitan., 3696, 82-90. https://hdl.handle.net/21.15107/rcub_ipir_1097
Ljubojević D, Kadijević Đ, Gutvajn N. Towards a Valid and Reliable Checklist to Evaluate Argumentative Essays Composed by ChatGPT. in E-Learning. 2024;3696:82-90. https://hdl.handle.net/21.15107/rcub_ipir_1097 .
Ljubojević, Danijela, Kadijević, Đorđe, Gutvajn, Nikoleta, "Towards a Valid and Reliable Checklist to Evaluate Argumentative Essays Composed by ChatGPT" in E-Learning, 3696 (2024):82-90, https://hdl.handle.net/21.15107/rcub_ipir_1097 .