Johnson

A validity argument is a structured, evidence-based case that evaluates whether the interpretations and uses of test scores are justified. In modern validity theory, it is not the test itself that is valid or invalid. Instead, validity applies to the claims we make about what scores mean and how they are used. This shift from … Read more

When I first started designing research studies, I could recite the textbook definitions of reliability and validity without hesitation. Reliability meant consistency. Validity meant accuracy. I thought I understood the relationship between the two until I actually tried to measure something myself and realized these concepts interact in ways that are not obvious from definitions … Read more

Imagine stepping on a bathroom scale that reads 170 pounds every single morning. Consistent, right? Now imagine discovering your actual weight is 165 pounds. The scale gives you the same number every time, but the number is wrong. That gap between consistency and accuracy is exactly why a reliable test is not automatically a valid … Read more

Every teacher has been there. You hand back a test, scan the results, and one question jumps out at you. Either every single student got it right, or almost no one did. Your gut tells you something is off with that question, but you are not sure how to measure it objectively. That is exactly … Read more

Imagine running an item analysis on your latest exam and watching every high-performing student get a particular question wrong while the lowest scorers nail it. That is exactly what a negative discrimination index test item looks like in real data, and it is one of the clearest red flags a test developer or classroom instructor … Read more

Every teacher has been there. You spend hours writing a quiz, administer it to 30 students, grade it, and then wonder which questions actually worked. Some questions everyone got right. Others everyone bombed. A few seemed to trip up your strongest students while weaker students nailed them. That confusion is exactly why item analysis exists, … Read more

If you have ever developed a test, a questionnaire, or any measurement instrument, you have probably faced the same question that psychometricians have debated for decades: should you stick with Classical Test Theory (CTT) or move to Item Response Theory (IRT)? Both frameworks exist to help us understand whether our items measure what they claim … Read more

If you have ever stared at a Rasch model output and wondered what the numbers under “item difficulty” actually mean for your test, you are in the right place. The item difficulty parameter in the Rasch model tells you where each item sits on the ability scale, but reading those numbers correctly takes a bit … Read more

If you have ever built a test or a questionnaire and wondered whether all your items measure with equal precision, you have already stepped into the debate between the Rasch model and 2PL IRT. The difference between the Rasch model and 2PL IRT comes down to one parameter: whether item discrimination is fixed or allowed … Read more

If you have ever stared at the output of a Rasch analysis and felt unsure about what the vertical chart in front of you actually means, you are not alone. Researchers, test developers, and graduate students frequently tell us on forums like the Rasch Measurement Forum that the Wright map is the single most useful, … Read more