ChatGPT is a large language-learning model that utilizes machine learning to generate natural language text and has garnered considerable attention for its ability to respond to users’ questions. This study compares ChatGPT’s performance on accounting assessments to student averages on the same set of questions, analyzing where it performs better and worse. A crowd-sourcing technique for data collection and paper authorship is used. By this, data from 14 countries and 186 institutions is collected. The results of the 28,085 analyzed questions show, as of January 2023, that ChatGPT provides correct answers for 56.5 percent of questions and partially correct answers for an additional 9.4 percent of questions. When considering point values for questions, students significantly outperform ChatGPT with a 76.7 percent average on assessments compared to 47.5 percent for ChatGPT if no partial credit is awarded and 56.5 percent if partial credit is awarded. Still, ChatGPT performs better than the student average for 15.8 percent of assessments when a partial credit is included. The results will help academics, administrators, faculty, instructors, and students better understand the potential impact of large language-learning models like ChatGPT on education, not only in the field of accounting, but also in many fields with similar types of assessments such as finance, information systems, management, and operations.
Blondeel, E., and Everaert, P. in Wood et al. (2023). The ChatGPT Artificial Intelligence Chatbot: How Well Does It Answer Accounting Assessment Questions? Issues in Accounting Education, https://doi.org/10.2308/ISSUES-2023-013
Link to the paper : https://publications.aaahq.org/iae/article/doi/10.2308/ISSUES-2023-013/10903/The-ChatGPT-Artificial-Intelligence-Chatbot-How?searchresult=1