Hypothesis testing in DMAIC
Hypothesis tests answer the question: "Is the difference I see in the data real, or just random sampling noise?" In the DMAIC Analyze phase, they confirm whether a suspected X really drives variation in Y. For the individual test mechanics, see the CQE pages: Hypothesis test selection →, One-way ANOVA →, Type I vs II error →
CSSBB test selection quick guide
| Y type | X type | Groups | Test |
|---|---|---|---|
| Continuous | Categorical | 2 | 2-sample t-test or paired t-test |
| Continuous | Categorical | 3+ | One-way ANOVA (+ Tukey post-hoc) |
| Continuous | Continuous | — | Regression / correlation |
| Attribute | Categorical | 2 | 2-proportion z-test |
| Attribute | Categorical | Any | Chi-square test of independence |
| Continuous | Categorical | 2 (non-normal) | Mann-Whitney U (Wilcoxon rank-sum) |
| Continuous | Categorical | 3+ (non-normal) | Kruskal-Wallis |
Statistical significance vs practical significance
This distinction is a key concept in CSSBB hypothesis testing:
- Statistical significance (p < α): The test says the difference is too large to be explained by random chance alone. It depends heavily on sample size — with a very large sample (say n = 10,000 per group), even a trivial difference can be "statistically significant."
- Practical significance (effect size): Is the difference large enough to matter to the business or customer? Measured by Cohen's d, η², or just the magnitude of the difference vs the process tolerance.
Rule of thumb: A statistically significant result still requires a practical significance check before you call it a root cause or invest in a fix. A p-value of 0.001 with a mean difference of 0.002 mm when the tolerance is ±0.5 mm is statistically real but practically irrelevant.
Confidence intervals alongside p-values
A 95% confidence interval for the difference answers a more useful question: "What is the range of plausible true differences?" If the CI is (0.001, 0.003 mm), you can see immediately that the effect is too small to matter, even if p < 0.05.
Common exam traps
- Failing to check normality before a parametric test. Use Anderson-Darling or Shapiro-Wilk. If normality fails, consider a transformation or the nonparametric equivalent (with large samples, t-tests and ANOVA on means are fairly robust to moderate non-normality).
- Multiple comparison problem. Running 10 independent t-tests at α = 0.05 gives a ~40% chance (1 − 0.95¹⁰ ≈ 0.40) of at least one false positive. Use ANOVA + Tukey for multiple groups.
- The null hypothesis is never "proved." Failing to reject H₀ does not mean H₀ is true — it may mean the sample was too small to detect the effect (Type II error / insufficient power).
Try 5 free CSSBB practice questions on hypothesis testing and significance.
Take the free CSSBB diagnosticMore: DMAIC overview · Project charter · VOC & CTQ · SIPOC · MSA in DMAIC · Process capability · Data collection planning · Graphical analysis · Root cause analysis · Multi-vari analysis · Hypothesis testing · Regression · DOE · FMEA · Define tollgate · Change management · Financial benefits · Team dynamics · Improve tollgate · Piloting & scaling · Poka-yoke (BB) · Solution selection · Control plan · SPC in DMAIC · Sustaining gains
CSSBB Quest by Bizarre Act. Original study material; not affiliated with ASQ. CSSBB is a certification of ASQ. Terms · Privacy