📐 CQE Quest › Learn

Attribute agreement analysis and Cohen’s kappa

When your measurement system produces a discrete result (pass/fail, grade A/B/C) rather than a continuous number, you cannot use a standard gage R&R study. Instead you run an attribute agreement analysis: each appraiser inspects the same set of parts, and you compare their results to each other and to a known standard.

Cohen’s kappa (κ) is the key statistic. It measures agreement between an appraiser and the standard, corrected for the agreement that would occur by chance alone.

Kappa formula

κ = (P_o − P_e) / (1 − P_e) P_o = observed proportion agreement (appraiser matches standard) P_e = expected proportion agreement by chance For a binary (pass/fail) system: P_e = P(standard says pass) × P(appraiser says pass) + P(standard says fail) × P(appraiser says fail)

Worked example

20 samples with known truth (14 pass, 6 fail). One appraiser inspects all 20 and calls 15 pass, 5 fail. The appraiser agrees with the standard on 18 of 20.

P_o = 18 / 20 = 0.90 P(standard pass) = 14 / 20 = 0.70 P(appraiser pass) = 15 / 20 = 0.75 P(standard fail) = 6 / 20 = 0.30 P(appraiser fail) = 5 / 20 = 0.25 P_e = 0.70 × 0.75 + 0.30 × 0.25 = 0.600 κ = (0.90 − 0.60) / (1 − 0.60) = 0.75

Interpreting kappa

κ range	Agreement	Typical action
> 0.90	Excellent	Measurement system is acceptable
0.70 – 0.90	Good	Marginal; investigate disagreements
0.50 – 0.70	Moderate	Needs improvement; retrain or add visual standards
< 0.50	Poor	System is unreliable; do not use for acceptance decisions

Practice problem

An attribute study has P_o = 0.84. P(standard pass) = 0.60, P(appraiser pass) = 0.65, P(standard fail) = 0.40, P(appraiser fail) = 0.35. Calculate κ.

Show answer

P_e = 0.60×0.65 + 0.40×0.35 = 0.390 + 0.140 = 0.530. κ = (0.84 − 0.53) / (1 − 0.53) = 0.31 / 0.47 = 0.66. Moderate agreement; the inspection method needs improvement.

Common exam traps

Kappa can be negative. A negative κ means agreement is worse than chance. This usually indicates a systematic reversal (the appraiser is calling good parts bad and bad parts good).
Kappa depends on the prevalence of defects. The same appraiser can produce different kappa values on samples with different defect rates. Always report the defect prevalence alongside kappa.
Multiple appraisers, multiple kappa values. In a full study you compute kappa for each appraiser vs. standard, and also appraiser vs. appraiser. Both matter.

Test yourself in 2 minutes

Try 5 free CQE practice questions on this exact topic. Every wrong answer comes with an explanation. No account needed.

Take the free CQE quiz

More: 5 Whys · 8D problem solving · AOQ & AOQL · APQP & PPAP · AQL sampling plans · Attribute agreement & kappa · Attribute control charts · Availability & maintainability · Benchmarking types · Binomial vs Poisson · Blocking & randomization · Calibration & traceability · Capability: non-normal · Chi-square test · Confidence intervals · Control chart selection · Control plan · CA effectiveness · Cost of quality · Cp vs Cpk vs Cpm · Cpk vs Ppk · CQE exam day tips · CQE study plan · CQE vs CQA vs CSSBB · Is the CQE worth it? · CUSUM & EWMA · Design reviews · Design V&V · Destructive testing · DFMEA vs PFMEA · Document control · Fault tree analysis · First article inspection · Fishbone diagram · FMEA: RPN vs AP · Full vs fractional factorial · Gage R&R · GD&T basics · Hazard vs risk vs harm · Hypothesis test selection · I-MR chart · Interaction plots · ISO 14971 · ISO 9001 clauses · Kaizen event planning · Kano model · Lean wastes (TIMWOODS) · Linear regression · Management review · MRB dispositions · MSA: bias, linearity & stability · NCR, CAPA & PA · Nonparametric tests · Normal distribution & z-table · One-way ANOVA · p-value explained · Pareto chart · PDCA vs DMAIC · Poka-yoke · QFD / House of Quality · Quality audits · Quality mgmt principles · Quality manual vs procedures · Quality plan vs control plan · Redundancy: parallel & standby · Reliability allocation · Bathtub curve & MTBF · Response surface methods · Risk-based thinking · Risk matrix vs FMEA · 7 mgmt & planning tools · Short-run SPC · Six Sigma certifications · Sigma level & DPMO · Six Sigma DMAIC · Supplier quality scorecards · Switching rules (sampling) · Taguchi loss function · Theory of constraints · Tolerance stackup · Traceability & lot control · Type I vs II error · Value stream mapping · Variables vs attributes · Weibull analysis · Western Electric rules · X-bar R vs X-bar S · CQE calculators · Study planner · Section quiz

CQE Quest by Bizarre Act. Original study material; not affiliated with ASQ. CQE is a certification of ASQ. Terms · Privacy