How to Run a Content Validity Index for a New Questionnaire in (2026)?

Developing a new questionnaire means nothing if your instrument does not actually measure what it claims to measure. That is where the content validity index comes in. If you are learning how to run a content validity index for a new questionnaire, this guide walks you through every step with real numbers, formulas, and interpretation thresholds you can apply today.

Our team has broken down the entire process into clear sections: what content validity means, how to prepare your expert panel, how to calculate both the Content Validity Ratio (CVR) and Content Validity Index (CVI), and how to interpret your results. Whether you are working on a thesis, dissertation, or journal article, these methods will help you produce a validated measurement instrument.

Researchers frequently struggle with which formula to use, how many experts they need, and what counts as an acceptable score. This guide from our assessment research collection addresses all those pain points and more.

What Is Content Validity?

Content validity refers to how thoroughly a measurement instrument covers the construct it is designed to measure. In simple terms, it asks: do the items on your questionnaire actually represent every dimension of the concept you are studying?

Unlike construct validity, which uses statistical techniques like factor analysis, content validity relies on expert judgment. You ask subject matter experts (SMEs) to evaluate whether each item on your questionnaire is relevant, clear, and essential to the construct. This makes content validity one of the first and most foundational steps in scale development.

Content validity differs from face validity, which is a more superficial check. Face validity simply asks whether a questionnaire looks like it measures the right thing on the surface. Content validity is systematic, quantitative, and grounded in expert consensus. Both matter, but content validity carries far more weight in academic research and publication.

Two statistical tools quantify content validity: the Content Validity Ratio (CVR) developed by Lawshe in 1975, and the Content Validity Index (CVI) popularized by Polit and Beck. Both use expert ratings, but they differ in their formulas, rating scales, and applications.

Preparing for Content Validation

Before you calculate any numbers, you need to set up your validation study properly. Preparation determines the quality of your results. A rushed setup leads to unreliable CVI scores and reviewer pushback.

Step 1: Define Your Construct Clearly

Write a clear, specific definition of the construct your questionnaire measures. For example, instead of saying your scale measures “student engagement,” specify whether it covers behavioral, emotional, and cognitive engagement or just one dimension. This definition guides every downstream decision.

Share this construct definition with your expert panel. They need it to judge whether each item truly belongs on your instrument.

Step 2: Generate Your Item Pool

Create a comprehensive pool of items based on a literature review, interviews, focus groups, or theory. Aim for more items than you need, since content validation will help you weed out weak ones. A good starting point is 30 to 50 percent more items than your final scale will contain.

Step 3: Select Your Expert Panel

Choosing the right experts is the most critical decision in this process. Here is where many researchers get stuck.

The general recommendation is a minimum of 5 experts, though 6 to 10 is ideal for most studies. Polit and Beck suggest at least 5 content experts for adequate reliability. Fewer than 5 experts makes your CVI estimates unstable, while more than 10 adds diminishing returns and logistical complexity.

Select experts who have published in the relevant domain, have practical experience with the construct, or teach in the field. Document their credentials, because reviewers and thesis committees will ask. Aim for a mix of methodological and content expertise.

Step 4: Create Your Expert Rating Form

Design a clear rating form for your experts. For CVR calculation using Lawshe’s method, each expert rates every item on a 3-point scale: essential (3), useful but not essential (2), or not necessary (1).

For CVI calculation using Polit and Beck’s method, each expert rates every item on two 4-point scales. The first rates relevance: 1 (not relevant), 2 (somewhat relevant), 3 (quite relevant), or 4 (highly relevant). The second rates clarity: 1 (not clear), 2 (somewhat clear), 3 (quite clear), or 4 (very clear).

Include your construct definition at the top of the form. Give experts space to write qualitative feedback for each item. Their written comments often reveal issues that numerical ratings miss.

Calculating the Content Validity Ratio (CVR)

The Content Validity Ratio was developed by C.H. Lawshe in 1975. It measures whether each item is perceived as essential by your expert panel. The CVR formula is straightforward once you understand its components.

The CVR Formula

The Lawshe CVR formula is:

CVR = (ne – N/2) / (N/2)

In this formula, “ne” is the number of experts who rated the item as essential, and “N” is the total number of experts on your panel.

The formula essentially compares the number of experts who consider an item essential against what you would expect by chance (half the panel). A CVR of 1.00 means every expert rated the item essential. A CVR of 0.00 means exactly half the panel rated it essential. A negative CVR means fewer than half considered it essential.

CVR Critical Values Table

You cannot interpret CVR values in isolation. Lawshe established critical values that account for chance agreement. An item must meet or exceed the critical value to be considered valid. Here are the minimum CVR values by panel size:

• 5 experts: CVR must be 0.99 or higher

• 6 experts: CVR must be 0.99 or higher

• 7 experts: CVR must be 0.99 or higher

• 8 experts: CVR must be 0.75 or higher

• 10 experts: CVR must be 0.62 or higher

• 12 experts: CVR must be 0.56 or higher

• 15 experts: CVR must be 0.49 or higher

• 20 experts: CVR must be 0.42 or higher

• 25 experts: CVR must be 0.37 or higher

• 30 experts: CVR must be 0.33 or higher

• 35 experts: CVR must be 0.31 or higher

• 40 experts: CVR must be 0.29 or higher

Notice that with smaller panels, the critical value is extremely demanding. With 5 to 7 experts, essentially all panelists must rate the item as essential. This is why having at least 8 experts gives you more interpretive flexibility.

Worked CVR Example

Let us say you have 8 experts rating a questionnaire with 10 items. For Item 1, 7 out of 8 experts rate the item as “essential.” Here is the calculation:

CVR = (7 – 8/2) / (8/2) = (7 – 4) / 4 = 3/4 = 0.75

With 8 experts, the critical value is 0.75. Item 1 just meets the threshold, so you retain it.

For Item 2, only 4 out of 8 experts rate it as essential:

CVR = (4 – 8/2) / (8/2) = (4 – 4) / 4 = 0/4 = 0.00

A CVR of 0.00 falls well below the critical value of 0.75. You should revise or drop Item 2.

For Item 3, all 8 experts rate it as essential:

CVR = (8 – 4) / 4 = 4/4 = 1.00

A perfect score. Item 3 clearly belongs on your scale.

Scale-Level CVR

Once you calculate CVR for every item, compute the mean CVR across all retained items to get a scale-level content validity ratio. This gives you a single number describing the overall content validity of your questionnaire.

How to Calculate the Content Validity Index (CVI)

The Content Validity Index (CVI) is the most widely used method in 2026 for quantifying content validity in questionnaire development. While CVR uses a 3-point essentiality scale, CVI uses a 4-point relevance and clarity scale, giving you more granular data. CVI also produces both item-level and scale-level scores.

Item-Level CVI (I-CVI)

The I-CVI measures the content validity of each individual item. The formula is simple:

I-CVI = (number of experts rating the item 3 or 4) / (total number of experts)

In other words, you count how many experts gave the item a rating of 3 (“quite relevant”) or 4 (“highly relevant”) for relevance, then divide by the total number of experts. You can calculate a separate I-CVI for clarity using the same method with clarity ratings.

Acceptable I-CVI Thresholds

The widely accepted thresholds from Polit and Beck are:

• I-CVI of 0.78 or higher: acceptable for items

• I-CVI of 0.70 to 0.77: questionable, consider revision

• I-CVI below 0.70: unacceptable, revise or remove

When you have fewer than 5 experts, these thresholds become harder to achieve. With 5 experts, 4 must rate an item as relevant (I-CVI = 0.80) to meet the 0.78 threshold.

Worked I-CVI Example

Let us continue with 8 experts rating 10 items. For Item 1, 7 experts rate it as 3 or 4 on relevance, while 1 expert rates it as 2:

I-CVI (relevance) = 7/8 = 0.875

This exceeds the 0.78 threshold, so Item 1 has acceptable content validity for relevance.

For the clarity rating on the same item, 6 experts rate it 3 or 4, and 2 rate it 2:

I-CVI (clarity) = 6/8 = 0.75

This falls below 0.78. Item 1 has acceptable relevance but questionable clarity. You should rewrite the item for clarity before finalizing your scale.

For Item 4, all 8 experts rate it as 4 on both relevance and clarity:

I-CVI (relevance) = 8/8 = 1.00

I-CVI (clarity) = 8/8 = 1.00

Perfect scores. Item 4 is a keeper.

Scale-Level CVI (S-CVI)

The S-CVI measures the content validity of the entire questionnaire. There are two methods for computing S-CVI, and both appear frequently in the literature.

S-CVI/Ave (Average Method): Compute the average of all I-CVI values across all items. This is the most commonly reported method.

S-CVI/UA (Universal Agreement): Compute the proportion of items that every single expert rated as 3 or 4. In other words, only items with perfect agreement across all experts count.

Here is how they differ with a 5-item questionnaire rated by 8 experts:

Suppose the I-CVI values for 5 items are: 1.00, 0.875, 0.75, 1.00, and 0.875.

S-CVI/Ave = (1.00 + 0.875 + 0.75 + 1.00 + 0.875) / 5 = 4.50 / 5 = 0.90

For S-CVI/UA, only 2 items (Item 1 and Item 4) had perfect agreement (all 8 experts rated them 3 or 4):

S-CVI/UA = 2/5 = 0.40

The acceptable threshold for S-CVI/Ave is 0.90 or higher. For S-CVI/UA, 0.80 or higher is considered good. Our example questionnaire meets the S-CVI/Ave threshold exactly but falls short on S-CVI/UA.

CVR vs CVI: When to Use Each

Many researchers ask which method to use. Here is the short answer: CVR works well when you need a strict essentiality check and want to apply Lawshe’s critical values. CVI works better when you want more nuance through 4-point scales and when you need separate relevance and clarity metrics.

In practice, many studies report both. CVR gives you Lawshe’s established benchmarks, while CVI provides the Polit and Beck thresholds that most journals expect to see. Calculating both makes your validation section more complete.

For more context on how these methods fit into broader evaluation practices, our published research demonstrates real-world applications.

Interpreting Your Results

Running the numbers is only half the battle. You also need to interpret them correctly and know what to do with borderline items. Here is a practical framework for making decisions based on your CVR and CVI results.

Decision Rules for Items

For each item, apply these rules:

If CVR meets the critical value AND I-CVI is 0.78 or higher, retain the item as-is. This is your strongest outcome.

If CVR meets the critical value but I-CVI for clarity is below 0.78, revise the item wording and re-test. The construct fit is fine, but experts find the phrasing confusing.

If CVR falls below the critical value but I-CVI is acceptable, consider the qualitative feedback from experts. The item may measure a relevant sub-dimension that is useful but not strictly essential.

If both CVR and I-CVI fall below thresholds, drop the item entirely.

Handling Negative CVR Values

Negative CVR values occur when fewer than half your experts rate an item as essential. This is a clear signal to remove the item. Some researchers panic when they see negative numbers, but they are entirely normal during validation. That is what content validation is designed to catch.

A negative CVR means the majority of your experts do not view the item as essential. Either the item measures something tangential to your construct, or the construct definition needs refinement.

Reporting CVI in Academic Papers

When reporting your content validity results in a thesis, dissertation, or journal article, include the following elements:

State the number of experts and their qualifications. Describe how you selected them and what expertise they bring. Report the rating scale used (3-point for CVR, 4-point for CVI) and the instructions given to experts.

Report I-CVI for each item, including both relevance and clarity where applicable. Present a table showing each item, its I-CVI, and whether it was retained, revised, or dropped. Report the S-CVI/Ave for the overall scale.

If you calculated CVR, report each item’s CVR value alongside the critical value for your panel size. State which items met or exceeded the threshold.

Reference Polit and Beck for CVI thresholds and Lawshe for CVR critical values. Including these citations shows reviewers you followed established methodology.

Common Mistakes to Avoid

Our team has seen the same errors repeated across student projects, journal submissions, and forum discussions. Avoid these pitfalls to save yourself time and reviewer frustration.

Mistake 1: Using too few experts. Panels of 2 or 3 experts produce unstable CVI estimates. With 3 experts, a single dissenting rating drops I-CVI to 0.67, below the acceptable threshold. Always aim for at least 5.

Mistake 2: Confusing I-CVI and S-CVI. Item-level CVI tells you about individual items. Scale-level CVI tells you about the whole questionnaire. Mixing them up in your reporting confuses reviewers and undermines your credibility.

Mistake 3: Not calculating S-CVI/UA. Many researchers only report S-CVI/Ave because it produces a higher number. But S-CVI/UA reveals whether your panel universally agrees on items. A large gap between S-CVI/Ave and S-CVI/UA signals items that some experts find problematic.

Mistake 4: Skipping qualitative feedback. Numbers alone do not tell the full story. Expert comments about why an item is unclear or irrelevant provide actionable guidance for revision. Always include an open-ended feedback section on your rating form.

Mistake 5: Treating content validity as a one-time check. If you revise items based on expert feedback, you should re-run validation on the revised versions. Re-validation with the same or a new panel ensures your changes improved the items rather than introducing new problems.

Mistake 6: Not documenting the process. Reviewers and thesis committees want to see your methodology. Document your expert selection criteria, rating instructions, response rates, and calculation steps. A well-documented process is easy to defend.

Conclusion: How to Run a Content Validity Index for a New Questionnaire

Learning how to run a content validity index for a new questionnaire comes down to a few core steps: define your construct, assemble a qualified expert panel of at least 5 experts, collect ratings using established scales, calculate CVR and I-CVI for each item, compute S-CVI for the overall scale, and apply recognized thresholds to decide which items to keep.

The process may seem formula-heavy at first, but each step follows a logical pattern. CVR uses Lawshe’s essentiality scale with critical values adjusted for panel size. CVI uses Polit and Beck’s 4-point relevance and clarity ratings with I-CVI thresholds of 0.78 and S-CVI/Ave thresholds of 0.90.

Run the numbers, interpret borderline items carefully, document everything, and report your results with proper citations. With these steps, your questionnaire will meet the content validity standards expected by academic journals, thesis committees, and funding bodies in 2026.

FAQs