Every high-stakes exam, from medical board tests to professional certifications, has to answer one deceptively simple question: what score counts as “good enough” to pass? Pick that number out of thin air and you risk failing qualified candidates or passing unqualified ones. That is exactly the problem the Angoff method for setting passing scores was built to solve.
The Angoff method is a judgment-based, criterion-referenced procedure that uses a panel of subject matter experts to estimate how a minimally competent candidate would perform on each item of a test. Aggregated across all items, those estimates produce a defensible cutscore, the threshold that separates pass from fail. It is one of the most widely used standard-setting approaches in credentialing, licensure, and education, and it has been studied in peer-reviewed research comparing Angoff passing scores against other methods.
In this guide, we will break down what the Angoff method is, walk through the seven-step process used by psychometricians, explain the difference between standard and modified Angoff, and show why bodies like the BACB, CFA Institute, and UK medical schools depend on it. Whether you are a test developer, an exam candidate confused about your score, or just curious about psychometrics, this is the practical explanation you have been looking for.
Table of Contents
What the Angoff Method Is for Setting Passing Scores
The Angoff method is a standard-setting technique for establishing cutscores (also called pass marks or pass points) on a test. Instead of simply choosing 70 percent as a passing score, a panel of subject matter experts (SMEs) judges each item on the exam and estimates the probability that a “minimally competent candidate” would answer it correctly.
Those item-level probabilities are averaged across panelists and summed across items to produce a recommended cutscore. The result is empirically grounded rather than arbitrary, which is the main reason the Angoff method is considered legally defensible under the Standards for Educational and Psychological Testing.
The method was introduced by Robert Angoff in 1971, and it has since become the most commonly cited standard-setting procedure for certification and licensure exams. If you have ever taken a high-stakes test and wondered why the passing score was not a round number, there is a good chance Angoff was involved.
The Core Concept: Minimally Competent Candidate (MCC)
The whole method hinges on a single imagined person: the minimally competent candidate, or MCC. The MCC is not a star performer and not a failing student. They are someone who is just barely qualified to receive the credential.
Panelists are trained to picture this borderline candidate and then ask themselves: “If I gave this single question to 100 minimally competent candidates, how many would get it right?” That probability, expressed as a decimal between 0.00 and 1.00, becomes the Angoff rating for that item.
Defining the MCC clearly is so important that most standard-setting studies spend an entire training session on it before any ratings begin. Vague MCC definitions are one of the most common sources of unreliable cutscores.
Why Use a Judgment-Based Method?
The alternative to a judgment-based method is a purely statistical one, like setting the cutscore at the mean plus one standard deviation, or norming against previous candidates. The problem with those approaches is they describe how people performed, not what they should be able to do.
The Angoff method is criterion-referenced. It asks what knowledge and skills a qualified practitioner needs, regardless of who happens to sit for the exam this year. That is exactly what licensing boards, certification bodies, and accreditation agencies need to demonstrate in court if their pass/fail decisions are ever challenged.
How the Angoff Method Works: Step-by-Step Process
The full Angoff standard-setting process is typically run as a structured workshop over one to three days with a trained facilitator, often a psychometrician. Here is the seven-step flow that most organizations follow, based on the modified Angoff approach used by major testing programs.
Step 1: Prepare Your Team
Recruit a diverse panel of subject matter experts, typically 5 to 15 people, who reflect the population the exam serves. A panel for a nursing licensure test, for example, should include recent graduates, experienced clinicians, and nursing educators from different regions and practice settings.
Before the workshop, the facilitator reviews the exam blueprint, the job analysis, and the item statistics. Every panelist receives a briefing packet explaining the purpose of the study, the MCC concept, and the rating task they will perform.
Step 2: Define the Minimally Competent Candidate
The panel works together to write a detailed description of the MCC. What can this person do? What knowledge do they have? Where do their skills fall short? This shared mental picture anchors every rating that follows.
A well-written MCC profile keeps panelists from drifting upward toward the “ideal” candidate or downward toward the failing candidate. It is the single biggest lever for inter-rater reliability.
Step 3: Round 1 Ratings
Each panelist independently reviews every item on the exam and assigns an Angoff rating: the estimated probability that the MCC would answer correctly. A very easy item might receive 0.95, a moderately difficult one 0.65, and a hard one 0.30. No discussion happens yet.
Here is a simplified example of what those ratings look like across five items and four panelists:
Item 1: 0.90 / 0.85 / 0.92 / 0.88 (mean 0.89)
Item 2: 0.65 / 0.70 / 0.60 / 0.68 (mean 0.66)
Item 3: 0.40 / 0.45 / 0.38 / 0.42 (mean 0.41)
Item 4: 0.75 / 0.72 / 0.78 / 0.74 (mean 0.75)
Item 5: 0.55 / 0.50 / 0.52 / 0.58 (mean 0.54)
Summing the item means gives an initial cutscore of 3.25 out of 5, or 65 percent. On a 100-item exam, that process simply scales up.
Step 4: Discussion Round
The facilitator shows each panelist their own ratings alongside the group mean and the actual item difficulty statistics (if available) from a previous administration. Panelists discuss why they rated items the way they did.
This is where the modified Angoff distinguishes itself from the original. Empirical data, like the proportion of real candidates who answered each item correctly, is shared as a “reality check” so expert judgments are not flying blind.
Step 5: Round 2 Ratings
After discussion, panelists re-rate each item independently. They can change their estimates or hold firm. The second round is what gets averaged to produce the recommended cutscore.
Research consistently shows that Round 2 ratings cluster more tightly than Round 1, which is why two rounds are considered the minimum. Some programs run three rounds for high-stakes exams.
Step 6: Evaluate Results and Apply the Beuk Compromise
The facilitator calculates the final cutscore and reviews inter-rater reliability, often expressed as an intraclass correlation or standard error of judgment. Outliers are flagged.
Many programs also apply the Beuk Compromise, a formula that blends the panel’s judged cutscore with the actual score distribution of candidates. It adjusts for panels that systematically over- or underestimate difficulty, which is a known weakness of pure Angoff ratings. For more depth, see our summary of statistical analysis of Angoff method assumptions.
Step 7: Write Up Your Report
The final step is documentation. A standard-setting report describes the panel composition, the MCC definition, the rating process, the final cutscore, the reliability statistics, and any compromises applied. That report is what makes the cutscore legally defensible if a candidate ever challenges their failing result.
Without this write-up, the entire study has little evidentiary value. The report is the deliverable.
Modified Angoff vs Standard Angoff
The original Angoff method, as Robert Angoff described it, asked panelists to rate items based purely on judgment. The modified Angoff adds empirical item data into the discussion and revision rounds, and it is now the dominant variant in credentialing.
In practice, when people say “Angoff method” today, they almost always mean the modified version. Pure standard Angoff is rare because giving panelists real performance data consistently improves the quality of the cutscore.
Key Differences
Standard Angoff: Panelists rate items based only on their professional judgment of the MCC. Two rounds of independent rating, with discussion in between, but no item statistics are shown.
Modified Angoff: Same rating structure, but panelists see actual item difficulty (the p-value, or proportion correct) and sometimes item-total correlations during the discussion round. This grounds the judgments in real candidate behavior.
Extended Angoff: Some programs add a third round, rating on a wider scale, or incorporate item response theory (IRT) theta metrics to translate judgments onto an ability scale rather than a raw-score scale.
Comparison at a Glance
Standard Angoff is simpler and faster but more prone to overestimating candidate ability. Modified Angoff is more accurate and defensible but requires prior administration data, which brand-new exams do not have.
For new exam forms, programs often start with a standard Angoff study and switch to modified Angoff once enough empirical data accumulates. This staged approach is common in medical and teacher certification programs.
Advantages and Disadvantages of the Angoff Method
No standard-setting method is perfect. The Angoff approach trades some weaknesses for others, and understanding both sides helps explain why it is so widely debated in psychometric circles.
Advantages
1. Legally defensible. Courts and accreditation bodies accept Angoff studies because they are systematic, documented, and based on expert judgment tied to job analysis.
2. Criterion-referenced. The cutscore reflects what candidates should know, not how previous cohorts happened to perform. This makes the pass mark stable across years.
3. Item-level granularity. Because each item gets its own rating, you can identify problem items, items where panelists strongly disagree, or items that are out of alignment with the blueprint.
4. Transparent and reproducible. A well-documented Angoff study can be replicated by a different panel and produce a similar cutscore, which is the foundation of inter-rater reliability.
5. Flexible across exam types. Angoff works for multiple-choice exams, performance assessments, and even some constructed-response formats with minor modifications.
Disadvantages
1. Cognitive load on panelists. Estimating probabilities for 100 or more items is mentally exhausting, and fatigue can degrade ratings late in the workshop.
2. Tendency to overestimate. SMEs often project their own expertise onto the MCC, producing cutscores that are too high. The modified Angoff and Beuk Compromise exist largely to correct for this.
3. Expensive and time-consuming. Convening a panel of experts for multiple days, paying a psychometrician, and writing up the report is a significant investment for any organization.
4. Sensitive to panel composition. A panel skewed toward academic experts will produce different cutscores than one skewed toward practitioners. Recruitment matters as much as methodology.
5. Hard to explain to candidates. As forum discussions on Reddit and Student Doctor Network show, candidates who fail an Angoff-scored exam often feel the system is opaque or unfair. Transparency about methodology is the best antidote.
Real-World Examples: Who Uses the Angoff Method
Theory is one thing, but the Angoff method is everywhere in real assessment programs. Here are examples from candidates who have discussed their experiences publicly.
BCBA and BACB Exams
The Behavior Analyst Certification Board (BACB) uses the modified Angoff method to set the passing score for the BCBA exam. A panel of behavior analysts convenes, reviews each item, and predicts how a minimally qualified behavior analyst would perform. Candidates often describe the BCBA cut as stricter than expected, which is consistent with the known tendency of expert panels to overestimate.
CFA Institute
The CFA Institute uses modified Angoff for its chartered financial analyst exams. Charter holders serve on the panel and give predictions for each question on every form. Because CFA pass rates vary between Level I, II, and III, the Angoff process is run separately for each.
UK Medical Schools
In the UK medical school system, the pass mark for many exams is set by Angoff. A team of doctors reviews each question and decides what percentage of competent, just-passing candidates would get it right. Medical students on r/medicalschooluk frequently debate why a pass mark feels higher than expected.
Western Governors University (WGU)
WGU, the large online university, uses modified Angoff for cut scores on competency-based assessments. The methodology is described in their student handbook, which is a good example of the kind of transparency that builds candidate trust.
Related Standard-Setting Methods
The Angoff method is not the only option. Different methods suit different exam formats and stakeholder needs, and many programs use a combination.
Bookmark Method
Items are ordered by difficulty, and panelists place a “bookmark” at the item where the MCC would have roughly a 67 percent chance of answering correctly. Bookmark works well with IRT-scaled exams and is popular in K-12 educational testing.
Ebel Method
Items are classified by difficulty (easy, medium, hard) and relevance (essential, important, acceptable, marginal). Panelists estimate what percentage of MCCs would answer correctly in each cell. Ebel is more structured than Angoff but also more time-consuming.
Nedelsky Method
Designed for multiple-choice items, Nedelsky asks panelists to eliminate the distractors a minimally competent candidate would recognize as wrong, then compute the probability as one over the remaining options. It is less common today because it tends to produce lower cutscores.
Hofstee Method
The Hofstee method, often used alongside Angoff, asks panelists to estimate the maximum acceptable failure rate and the minimum acceptable pass score. Those two judgments are plotted against the actual score distribution to find a compromise cutscore. The Angoff-Hofstee combination is one of the most defensible configurations available.
Frequently Asked Questions
What is the Angoff scoring method?
The Angoff scoring method is a standard-setting procedure where a panel of subject matter experts estimates the probability that a minimally competent candidate would answer each test item correctly. Those probabilities are averaged and summed across items to produce a recommended passing score, or cutscore, for the exam.
What is the Angoff standard setting method?
The Angoff standard setting method is the psychometric process used to establish a defensible cutscore on a criterion-referenced exam. A trained panel of SMEs reviews each item, defines the minimally competent candidate, assigns probability ratings, discusses discrepancies, and re-rates items over multiple rounds until a reliable recommended cutscore emerges.
What is the Angoff method to set a minimal passing standard?
The Angoff method sets a minimal passing standard by having expert panelists independently judge, for every item, the probability that a minimally competent candidate would answer it correctly. The average of all item probabilities across panelists becomes the recommended passing score, which is then documented and validated using inter-rater reliability statistics and empirical data.
What is the Angoff and Hofstee method?
The Angoff and Hofstee method is a combined standard-setting approach. The Angoff procedure produces a judgment-based cutscore from expert ratings, while the Hofstee method plots the panel’s acceptable failure rate and minimum pass score against the actual candidate score distribution. Blending the two produces a more realistic and defensible cutscore than either method alone.
Conclusion
The Angoff method for setting passing scores is not magic. It is a structured, expert-driven process that turns subjective judgments into a defensible cutscore by averaging item-level probability estimates from a trained panel. When combined with the modified Angoff approach, empirical data, the Beuk Compromise, and a Hofstee reality check, it produces passing scores that hold up legally and statistically.
If you are studying for an exam that uses Angoff scoring, understanding the methodology will not change your result, but it should change how you interpret it. Your pass or fail was not based on an arbitrary number, it was based on what a panel of experts agreed a minimally qualified candidate should be able to do. For deeper reading, explore the research links throughout this guide and ask your testing organization for its standard-setting report.