The optimal setting of A/B exam papers without item pools: A hybrid approach of IRT and BGP

Zheng Yun Zhuang, Chi Kit Ho, Paul Juinn Bing Tan, Jia Ming Ying, Jin Hua Chen

Research output: Contribution to journalArticlepeer-review

Abstract

The administration of A/B exams usually involves the use of items. Issues arise when the pre-establishment of a question bank is necessary and the inconsistency in the knowledge points to be tested (in the two exams) reduces the exams 'fairness'. These are critical for a large multi-teacher course wherein the teachers are changed such that the course and examination content are altered every few years. However, a fair test with randomly participating students should still be a guaranteed subject with no item pool. Through data-driven decision-making, this study collected data related to a term test for a compulsory general course for empirical assessments, pre-processed the data and used item response theory to statistically estimate the difficulty, discrimination and lower asymptotic for each item in the two exam papers. Binary goal programing was finally used to analyze and balance the fairness of A/B exams without an item pool. As a result, pairs of associated questions in the two exam papers were optimized in terms of their overall balance in three dimensions (as the goals) through the paired exchanges of items. These exam papers guarantee their consistency (in the tested knowledge points) and also ensure the fairness of the term test (a key psychological factor that motivates continued studies). Such an application is novel as the teacher(s) did not have a pre-set question bank and could formulate the fairest strategy for the A/B exam papers. The model can be employed to address similar teaching practice issues.

Original languageEnglish
Article number1290
JournalMathematics
Volume8
Issue number8
DOIs
Publication statusPublished - Aug 2020

Keywords

  • A/B exam papers setting
  • Assessment
  • Binary goal programing
  • Data-driven decision-making
  • Evaluation
  • Item pool
  • Item response theory
  • Question bank

ASJC Scopus subject areas

  • Mathematics(all)

Fingerprint Dive into the research topics of 'The optimal setting of A/B exam papers without item pools: A hybrid approach of IRT and BGP'. Together they form a unique fingerprint.

Cite this