Main Article Content
Although multiple-choice items (MCIs) are widely used for classroom assessment, designing MCIs with sufficient number of plausible distracters is very challenging for teachers. In this regard, existing empirical studies reveal that using three-option MCIs provides various advantages when compared to four-option MCIs due to less preparation and administration time. This study examines how different elimination methods to reduce four-option MCIs to three-option MCIs influences test reliability, item discrimination, and item difficulty. The research findings have revealed that they did not harm test reliability and item discrimination, except for item difficulty. Results are discussed in relation to promote quality classroom assessment.
International Journal of Assessment Tools in Education
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
Baghaei, P., & Amrahi, N. (2011). The effects of the number of options on the psychometric characteristics of multiple choice items. Psychological Test and Assessment Modeling, 53(2), 192-211.
Balta, N., &Eryılmaz, A. (2017). Counterintuitive dynamics test. International Journal of Science and Mathematics Education, 15(3), 411-431.
Chappuis, S., &Stiggins, R. J. (2002). Classroom assessment for learning. Educational leadership, 60(1), 40-44.
Crehan, K.D., Haladyna, T.M., & Brewer B.W. (1993). Use of an inclusive option and the optimal number of options for multiple-choice items. Educational and Psychological Measurement, 53(1), 241-247.
Darling-Hammond, L., &Youngs, P. (2002). Defining “highly qualified teachers”: What does “scientifically-based research” actually tell us?. Educational researcher, 31(9), 13-25.
Delgado, A. R., & Prieto, G. (1998). Further evidence favoring three-option items in multiple-choice tests. European Journal of Psychological Assessment, 14(3), 197-201.
Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education. Advances in Health Sciences Education, 10(2), 133-143.
Duhachek, A., &Iacobucci, D. (2004). Alpha's standard error (ASE): an accurate and precise confidence interval estimate. Journal of Applied Psychology, 89(5), 792 - 808.
Field, A. (2009). Discovering statistics using SPSS. London, England: Sage.
Frey, B. B., Petersen, S., Edwards, L. M., Pedrotti, J. T., & Peyton, V. (2005). Item-writing rules: Collective wisdom. Teaching and Teacher Education, 21(4), 357-364.
Frey, B. B., & Schmitt, V. L. (2010). Teachers’ classroom assessment practices. Middle Grades Research Journal, 5(3), 107-117.
Haladyna, T. M., & Downing, S. M. (1989a). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 37-50.
Haladyna, T. M., & Downing, S. M. (1993). How many options is enough for a multiple-choice test item?. Educational and Psychological Measurement, 53(4), 999-1010.
Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-334.
Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. Routledge.
Leahy, S., Lyon, C, Thompson, M., &Wiliam, D. (2005). Classroom assessment minute by minute, day by day. Educational Leadership, 63(3), 18-24.
Messick, S. (1989).Validity. In R. L. Linn (Ed.), Educational measurement, (3rd ed., pp. 13-
103). New York: American Council on Education and Macmillan.
Moreno, R., Martínez, R. J., &Muñiz, J. (2006). New guidelines for developing multiple-choice items. Methodology, 2(2), 65-72.
Moreno, R., Martínez, R. J., &Muñiz, J. (2015). Guidelines based on validity criteria for the development of multiple choice items. Psicothema, 27(4), 388-394.
Rich, C. E., &Johanson, G. A. (1990, April). An item-level analysis of “none of the above.” Paper presented at the annual meeting of the American Educational Research Association, Boston.
Rodriguez, M. C. (1997). The art & science of item writing: A meta-analysis of multiple choice item format effects. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.
Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3-13.
Rogers, W.T., Harley, D., 1999. An empirical comparison of three- and four-choiceitems and tests: susceptibility to testwiseness and internal consistency reliability. Educational and Psychological Measurement, 59(2), 234-247.
Shizuka, T., Takeuchi, O., Yashima, T. &Yoshizawa, K. (2006). A comparison of three-and four-option English tests for university entrance selection purposes in Japan. Language Testing, 23(1), 35-57.
Sidick, J.T., Barrett, G.V., Doverspike, D., 1994. Three-alternative multiple choice14 tests: An attractive option. Personnel Psychology, 47(4), 829-835.
Stiggins, R. J. (1991). Relevant classroom assessment trainingfor teachers. Educational Measurement: Issues and Practice, 10(1), 7–12.
Tarrant, M., Knierim, A., Hayes, S. K., & Ware, J. (2006). The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Education in Practice, 6(6), 354-363.
Tarrant, M., & Ware, J. (2010). A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse education today, 30(6), 539-543.
Thordike, R.M. (2005). Measurement and Evaluation in Psychology and Education (7th Ed.). Upper Saddle River, NJ: Pearson Education.
Van Zyl, J. M., Neudecker, H., &Nel, D. G. (2000). On the distribution of the maximum likelihood estimator of Cronbach’s alpha. Psychometrika, 65(1), 271–280.