Beautiful Virgin Islands

Friday, Jun 13, 2025

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Students Beat ChatGPT At This Exam, Score 76%, Compared To Chatbot's 47%

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."
Researchers found students to have fared better at accounting exams than ChatGPT, OpenAI's chatbot product.

Despite this, they said that ChatGPT's performance was "impressive" and that it was a "game changer that will change the way everyone teaches and learns - for the better."

The researchers from Brigham Young University (BYU), US, and 186 other universities wanted to know how OpenAI's technology would fare on accounting exams. They have published their findings in the journal Issues in Accounting Education.

In the researchers' accounting exam, students scored an overall average of 76.7 per cent, compared to ChatGPT's score of 47.4 per cent.

While in 11.3 per cent of the questions, ChatGPT was found to score higher than the student average, doing particularly well on accounting information systems (AIS) and auditing, the AI bot was found to perform worse on tax, financial, and managerial assessments. Researchers think this could possibly be because ChatGPT struggled with the mathematical processes required for the latter type.

The AI bot, which uses machine learning to generate natural language text, was further found to do better on true/false questions (68.7 per cent correct) and multiple-choice questions (59.5 per cent), but struggled with short-answer questions (between 28.7 and 39.1 per cent).

In general, the researchers said that higher-order questions were harder for ChatGPT to answer. In fact, sometimes ChatGPT was found to provide authoritative written descriptions for incorrect answers, or answer the same question different ways.

They also found that ChatGPT often provided explanations for its answers, even if they were incorrect. Other times, it went on to select the wrong multiple-choice answer, despite providing accurate descriptions.

Researchers importantly noted that ChatGPT sometimes made up facts. For example, when providing a reference, it generated a real-looking reference that was completely fabricated. The work and sometimes the authors did not even exist.

The bot was seen to also make nonsensical mathematical errors such as adding two numbers in a subtraction problem, or dividing numbers incorrectly.

Wanting to add to the intense ongoing debate about how how models like ChatGPT should factor into education, lead study author David Wood, a BYU professor of accounting, decided to recruit as many professors as possible to see how the AI fared against actual university accounting students.

His co-author recruiting pitch on social media exploded: 327 co-authors from 186 educational institutions in 14 countries participated in the research, contributing 25,181 classroom accounting exam questions.

They also recruited undergraduate BYU students to feed another 2,268 textbook test bank questions to ChatGPT. The questions covered AIS, auditing, financial accounting, managerial accounting and tax, and varied in difficulty and type (true/false, multiple choice, short answer).
Newsletter

Related Articles

Beautiful Virgin Islands
0:00
0:00
Close
Two Women Found Dead in Eryri National Park
Operation "Like a Lion": Israel Strikes Iran in Unprecedented Offensive
Pentagon Initiates Review of AUKUS Nuclear Submarine Pact
Meta to Invest $15 Billion in Scale AI to Advance AGI Goals
Rare Cancer Cases Triple Among Millennials, Alarming Doctors
G7 Finance Ministers Convene in Canada with Focus on Ukraine and Trade Tariffs
UK Spending Review Prioritizes Health and Defence Amid Budget Constraints
US Raises Security Concerns Over Proposed Chinese Embassy in London
Defined Benefit Pension Reforms Expected to Unlock Limited Investment
UK Industrial Strategy Launch Delayed Amid Budget Negotiations
Crick Institute Seeks Additional Funding to Attract International Scientists
Zia Yusuf Returns to Reform UK in New Role After Brief Resignation
Bezos's Lavish Venice Wedding Sparks Local Protests
US Urges UK to Raise Defence Spending to 5% of GDP
Europe Prepares for Historic Lunar Rover Landing
Italian Parents Seek Therapy Amid Lengthy School Holidays
British Fishing Vessel Seized by France Fined €30,000
Dutch Government Collapses Amid Migration Policy Dispute
Germany Moves to Expedite Migrant Deportations
UK Commits to 3.5% GDP Defence Spending Under NATO Pressure
Scientist Returns Royal Society Prize in Protest Over Elon Musk's Fellowship
Chancellor Proposes 'Housing Bank' and £25 Billion Social Housing Boost
UK Retail Sales Growth Slows in May Amid Consumer Caution
Home Secretary Directed to Find Budget Savings to Protect Police Funding
Rolls-Royce Secures Government Backing for UK's First Small Modular Nuclear Reactors
Domestic Buyers Capitalize on London Property Market as Non-Doms Retreat
Nvidia CEO Criticizes UK's Digital Infrastructure Amid £1 Billion AI Investment Pledge
UK Commits Additional £11.5 Billion to Sizewell C Nuclear Project
UK Unemployment Reaches Near Four-Year High as Wage Growth Slows
Chancellor Reinstates Winter Fuel Payments for Majority of Pensioners
Simone Biles and Riley Gaines Clash Over Transgender Athletes in Women's Sports
California Governor Disputes National Guard Deployment Amid Rising Tensions
Protests Erupt in Los Angeles with Symbolic Flag Burning
Israeli Forces Intercept Gaza-Bound Aid Vessel Carrying Greta Thunberg
IMF Warns of Severe Global Trade War Impacts on Emerging Markets
US and China Engage in Trade Discussions in London Amid Ongoing Tensions
Low Turnout Jeopardizes Italy's Citizenship Reform Referendum
EU Lawmaker Calls for Broader Exemptions in Supply Chain Legislation
France's Defense Spending Plans Threatened by High National Debt
European Small-Cap Stocks Outperform U.S. Rivals Amid Growth Revival
Switzerland Proposes $26 Billion Capital Increase for UBS
Germany's Merz Signals Continued U.S. Reliance After Meeting with Trump
Transatlantic Interest Rate Divergence Widens as Trump Pressures Powell
Sam Altman's Eye-Scanning Digital ID Project Launches in UK
Qualcomm to Acquire UK's Alphawave in $2.4 Billion Deal
Syria to Reconnect to Global Economy After 14 Years of Isolation
Trump Administration Issues New Travel Ban Targeting 12 Countries
Man Group Mandates Full-Time Office Return for Quantitative Analysts
JPMorgan Warns Analysts Against Accepting Future-Dated Job Offers
Builder.ai Faces Legal Scrutiny Amid Financial Misreporting Allegations
×