Equal Z standard-setting method to estimate the minimum number of panelists for a medical school's objective structured clinical examination in Taiwan: A simulation study

Ying Ying Yang*, Pin Hsiang Huang, Ling Yu Yang, Chia Chang Huang, Chih Wei Liu, Shiau Shian Huang, Chen Huan Chen, Fa Yauh Lee, Shou Yen Kao, Boaz Shulruf*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Purpose: Undertaking a standard-setting exercise is a common method for setting pass/fail cut scores for high-stakes examinations. The recently introduced equal Z standard-setting method (EZ method) has been found to be a valid and effective alternative for the commonly used Angoff and Hofstee methods and their variants. The current study aims to estimate the minimum number of panelists required for obtaining acceptable and reliable cut scores using the EZ method. Methods: The primary data were extracted from 31 panelists who used the EZ method for setting cut scores for a 12-station of medical school's final objective structured clinical examination (OSCE) in Taiwan. For this study, a new data set composed of 1,000 random samples of different panel sizes, ranging from 5 to 25 panelists, was established and analyzed. Analysis of variance was performed to measure the differences in the cut scores set by the sampled groups, across all sizes within each station. Results: On average, a panel of 10 experts or more yielded cut scores with confidence more than or equal to 90% and 15 experts yielded cut scores with confidence more than or equal to 95%. No significant differences in cut scores associated with panel size were identified for panels of 5 or more experts. Conclusion: The EZ method was found to be valid and feasible. Less than an hour was required for 12 panelists to assess 12 OSCE stations. Calculating the cut scores required only basic statistical skills.

Original languageEnglish
Article number27
JournalJournal of educational evaluation for health professions
Volume19
DOIs
StatePublished - 2022

Keywords

  • Computer simulation
  • Educational measurement
  • Standards
  • Taiwan
  • Undergraduate medical education

Fingerprint

Dive into the research topics of 'Equal Z standard-setting method to estimate the minimum number of panelists for a medical school's objective structured clinical examination in Taiwan: A simulation study'. Together they form a unique fingerprint.

Cite this