LIBRO COMPLETO: The Education Revolution through Artificial Intelligence
CAPÍTULO 15

FICHA TÉCNICA

Fecha de publicación​:
04/11/2024

Doi​ del capítulo:

Título del libro: The Education Revolution through Artificial Intelligence

URL del libro:

ISBN del libro: 9788410282582

DOI del libro:

Automatic Short Answer Grading in Health Sciences with ChatGPT

Abstract

Artificial Intelligence (AI) has emerged as a transformative tool in education, notably in facilitating automated exam grading. This study focuses on Automatic Short Answer Grading (ASAG) via ChatGPT-4, a widely accessible and versatile general-purpose generative AI model. We compare the grading outcomes from ChatGPT with those adjudicated by human evaluators within the health science domain. An evaluative framework was deployed to gauge the GPT-4 model’s concordance with an expert educator’s scoring. Human scores were compared to those offered by ChatGPT with different versions of prompts, specifically with 10 examples, 25 examples, and a grading rubric, employing a scoring metric that spans from 0 to 10 points, allowing for decimal values, without any model fine-tuning or parameter modulation. Our findings show that rubrics markedly enhance score alignment with an educator’s evaluative benchmarks, registering intraclass correlation coefficients surpassing 0.8, thus nearly mirroring human judgment. These results suggest that there is ample scope for increasing the effectiveness of ASAG using Large Language Models (LLM) such as ChatGPT. However, it is imperative to recognize that the operability of these systems is not yet fully reliable and stable, making human supervision necessary. The integration of expert supervision ensures both the accuracy and pedagogical validity of these automated tools.

Palabras clave

Autores

PhD Nuria Padros-Flores, Ivan Gadea Sáez, PhD Carolina Alonso-Montero

Cómo citar

Referencias bibliográficas

Ir al contenido