You ran your Shapiro-Wilk test, and the p-value came back below 0.05. Your data is not normally distributed, and now you are staring at your screen wondering whether your entire analysis is doomed. Take a breath. A normality violation is one of the most common issues statisticians and researchers face, and there are clear, proven paths forward.
Knowing what to do when your data violates the normality assumption can mean the difference between publishable results and a stalled project. Most parametric tests, including t-tests, ANOVA, and ordinary least squares regression, assume that residuals follow a normal distribution. When that assumption fails, your p-values and confidence intervals may become unreliable, potentially inflating Type I errors (false positives) or Type II errors (false negatives).
Our team has worked through hundreds of datasets across academic research, clinical trials, and business analytics. We have seen every flavor of non-normal data, from heavily skewed reaction times to bimodal survey responses, and we want to share the practical framework we use. The good news is that a normality violation is rarely a dead end. You have at least four major strategies available, and the right one depends on your sample size, the severity of the violation, and the specific test you planned to run.
Here is a quick preview of what we will cover. First, we will explain what the normality assumption actually means and how to check it properly. Then we will walk through a severity assessment so you know whether the violation even matters. After that, we cover three solution paths: data transformation, non-parametric alternatives, and robust computational methods. Finally, we provide test-specific guidance for t-tests, ANOVA, and regression.
Table of Contents
What the Normality Assumption Actually Means
The normality assumption is the expectation that the residuals, or the differences between observed and predicted values, follow a bell-shaped normal distribution. It does not mean your raw data must look perfectly bell-shaped. For independent-samples t-tests, the assumption applies to each group separately. For regression and ANOVA, it applies to the residuals after fitting the model, not the dependent variable itself.
This distinction trips up many researchers. We frequently see analysts run a Shapiro-Wilk test on their raw outcome variable, see a significant result, and panic. In a regression context, that test tells you very little. You need to extract the residuals and assess those instead.
Parametric tests rely on normality because their mathematical derivations assume it. The t-distribution, the F-distribution used in ANOVA, and the standard errors in OLS regression all assume normally distributed errors. When that assumption holds, your p-values and confidence intervals are exact. When it fails, those quantities can become biased, though the practical impact varies enormously depending on sample size and violation severity.
One critical point that many guides gloss over: normality is not an all-or-nothing condition. Real-world data is never perfectly normal. The question is never whether your data violates normality, but whether the violation is severe enough to distort your conclusions.
How to Check if Your Data Violates Normality
There are two complementary approaches to checking normality, and you should use both. Visual inspection tells you the shape and direction of any problem, while statistical tests confirm whether that problem reaches significance.
Visual Methods
Start with a histogram overlaid with a normal curve. A histogram gives you an immediate sense of whether your data is symmetric, skewed left or right, or has multiple peaks. Look for obvious asymmetry, extreme tails, or gaps in the distribution.
The Q-Q plot, or quantile-quantile plot, is more informative. It plots your data quantiles against the quantiles of a theoretical normal distribution. If your data is normal, the points fall along a straight diagonal line. Systematic curves away from the line indicate skewness. S-shaped patterns suggest heavy or light tails. We rely on Q-Q plots more than any other visual tool because they make subtle departures from normality easy to spot.
A box plot rounds out the visual toolkit. It shows the median, quartiles, and outliers. While a box plot does not directly test normality, a highly asymmetric box with long whiskers on one side is a red flag for skewness.
Statistical Tests
The Shapiro-Wilk test is the most powerful normality test for small to moderate sample sizes, typically up to 2,000 to 5,000 observations. A significant result, conventionally p less than 0.05, indicates your data deviates from normality. This is the test we recommend as your primary statistical check.
The Kolmogorov-Smirnov test is another option, but it is less powerful than Shapiro-Wilk for most datasets. If you use it, choose the Lilliefors correction version, which adjusts for the fact that you are estimating the mean and standard deviation from the same data.
Anderson-Darling is a third alternative that gives more weight to the tails of the distribution. It can be useful when tail behavior is your primary concern, as it is in some financial and reliability analyses.
Here is a warning we give every researcher: be careful with large samples. Statistical normality tests become extremely sensitive with big datasets. With 500 observations, even trivial deviations from normality will produce a significant Shapiro-Wilk result. A p-value below 0.05 does not mean the violation is practically important. Always pair statistical tests with visual inspection.
For skewness and kurtosis, a common rule of thumb is that absolute skewness values above 2 and absolute kurtosis values above 7 indicate meaningful non-normality. These thresholds come from the work of Curran, West, and Finch and are widely cited in the social science literature.
Is the Violation Serious? A Practical Framework
This is the question competitors consistently fail to answer well. Not every normality violation matters, and overreacting to a minor one can lead you to abandon a perfectly adequate parametric test for a less powerful alternative. Here is the framework we use.
First, consider your sample size and the central limit theorem. The CLT states that as sample size grows, the sampling distribution of the mean approaches normality regardless of the underlying data distribution. For most practical purposes, sample sizes above 30 per group make t-tests reasonably robust to non-normality. For ANOVA, sizes above 20 to 30 per cell provide similar protection. This is why so many forum posts on Reddit and Cross Validated ask whether a sample larger than 30 excuses a normality violation. The short answer is often yes.
Second, assess the type and severity of non-normality. Mild skewness is rarely a serious problem for parametric tests, especially with decent sample sizes. Severe skewness, bimodal distributions, or extreme outliers are more concerning. Heavy tails, where your data has many extreme values, are particularly problematic because they destabilize the mean and standard deviation.
Third, consider the direction of the violation. Parametric tests are generally more robust to symmetric non-normality, such as a uniform or moderately heavy-tailed but symmetric distribution, than to severe asymmetric skewness. Skewed data pulls the mean away from the median, which can bias your test results.
Fourth, think about which assumption you are violating. Independence of observations is the most serious violation, and no transformation or alternative test can fix it. Normality violations are relatively forgiving with adequate sample sizes. Homogeneity of variance violations fall somewhere in between and have specific fixes like Welch’s correction.
Finally, consider the consequences. If you are running an exploratory analysis to generate hypotheses, a normality violation is less worrisome than if you are running a confirmatory test for a clinical trial where false positives carry serious consequences.
Step 1: Try Data Transformation
Data transformation applies a mathematical function to every data point to pull the distribution closer to normal. It is usually the first remedy to try because it lets you keep using your planned parametric test. The trick is matching the transformation to the type of non-normality you have.
Log Transformation
The log transformation is the workhorse for right-skewed data. Reaction times, income, and many biological measurements naturally follow right-skewed distributions, and a log transform often normalizes them beautifully. Use the natural log or log base 10. Log transformations require all values to be positive, so if your data includes zeros, add a small constant, such as 1, before taking the log.
In our experience, log transformation resolves roughly half of all skewness problems we encounter. It is particularly effective for data that spans several orders of magnitude.
Square Root Transformation
The square root transformation is milder than the log transform and works well for count data, such as Poisson-distributed variables. It also handles moderate right skewness. Like the log, it requires non-negative values, so add a constant if needed.
Box-Cox Transformation
The Box-Cox transformation is a family of power transformations that includes the log and square root as special cases. It estimates an optimal transformation parameter, lambda, from your data. This makes it more flexible than manually choosing a transformation. Most statistical software, including R and SPSS, can run Box-Cox. The trade-off is that it is less interpretable. Reporting your findings on Box-Cox transformed data requires clear documentation.
Inverse Transformation
The inverse or reciprocal transformation, 1 divided by x, is a strong correction for severely right-skewed data. It is more aggressive than the log transform. Use it when log transformation does not fully correct skewness. Be aware that it reverses the order of your data, so larger values become smaller.
Arcsine Square Root Transformation
This transformation is designed for proportions and percentages. If your dependent variable is bounded between 0 and 1, or 0 percent and 100 percent, the arcsine square root transformation stabilizes the variance and can improve normality.
After applying any transformation, re-run your normality checks. Plot a new histogram and Q-Q plot. Run the Shapiro-Wilk test again. If the transformation worked, great. If the data is still non-normal after trying two or three transformations, move on to the next strategy.
One important note: always interpret your results in the context of the transformed scale. If you transformed reaction time using log base 10, your means and confidence intervals describe the log-transformed data. Back-transform for reporting, but acknowledge that you did so.
Step 2: Consider Non-Parametric Alternatives
When transformation fails or is inappropriate, non-parametric tests are your next option. These tests do not assume a specific distribution. Instead, they work with ranks rather than raw values. The trade-off is that they are slightly less powerful than their parametric counterparts when the normality assumption holds.
Each common parametric test has a non-parametric equivalent. For an independent-samples t-test, use the Mann-Whitney U test, also called the Wilcoxon rank-sum test. It compares the ranks of two groups rather than their means. For a paired-samples t-test, use the Wilcoxon signed-rank test.
For one-way ANOVA with three or more groups, use the Kruskal-Wallis H test. It is the non-parametric analog of one-way ANOVA and compares rank sums across groups. For repeated-measures ANOVA, use the Friedman test.
Here is a critical caveat that most guides miss. Non-parametric tests have their own assumptions. They assume that observations are independent and that the distributions in each group have the same shape. If your groups have differently shaped distributions, a significant Mann-Whitney result might reflect a difference in distribution shape rather than a difference in median. This is known as the Behrens-Fisher problem in the non-parametric context.
Researchers at the University of Edinburgh make this point forcefully in their experimental design guide. They argue that the unappreciated assumptions of non-parametric tests make them less of a cure-all than many analysts believe. We agree. Non-parametric tests are valuable, but they are not a free pass.
Another consideration: non-parametric tests are less informative. They tell you whether groups differ, but they do not provide estimates of means, confidence intervals, or effect sizes in the same way parametric tests do. If you need precise estimates, consider the robust methods in the next section.
Step 3: Use Robust Statistical Methods
Beyond transformation and non-parametric tests, there is a third category of solutions that competitors often overlook. Robust statistical methods are designed to handle violations of assumptions without requiring you to transform your data or abandon parametric frameworks entirely.
Welch’s t-test and Welch’s ANOVA
If your normality violation is accompanied by unequal variances, flagged by a significant Levene’s test, Welch’s correction is your first stop. Welch’s t-test adjusts the degrees of freedom to account for unequal variances and is often more accurate than the standard t-test when assumptions are violated. Welch’s ANOVA extends this correction to three or more groups.
In practice, many statisticians now recommend using Welch’s t-test by default, even when variances appear equal, because it provides nearly identical results when variances are equal and better results when they are not. It is available in every major statistics package.
Brown-Forsythe Test
The Brown-Forsythe test is another alternative for ANOVA when the homogeneity of variance assumption is violated. It uses the median rather than the mean to compute the test statistic, making it robust to non-normal data. It is particularly useful for unbalanced designs where group sizes differ.
Bootstrapping
Bootstrapping is a computational method that resamples your data thousands of times to build an empirical sampling distribution. It does not require normality. You can apply bootstrap confidence intervals to means, regression coefficients, or any other statistic. Most modern software supports it.
This is where we differ from most guides. Bootstrapping is not a last resort. For moderate to large samples, bootstrapped confidence intervals are often more accurate than their parametric counterparts, even when normality holds. If you have at least 100 observations, consider bootstrapping as a primary strategy rather than a fallback.
Permutation Tests
Permutation tests, also called randomization tests, work by shuffling group labels many times and computing the test statistic for each shuffle. The p-value is the proportion of shuffled arrangements that produce a result as extreme as your observed result. Like bootstrapping, permutation tests make no distributional assumptions.
Permutation tests are particularly powerful for comparing groups. They directly answer the question: if group membership were random, how unlikely is my observed result? They are computationally intensive but feasible on any modern computer.
Generalized Linear Models
If your data follows a known non-normal distribution, a generalized linear model may be more appropriate than forcing it into a normal framework. GLMs let you specify the distribution family directly. For count data, use a Poisson or negative binomial GLM. For binary outcomes, use logistic regression. For positive continuous data with skewness, a Gamma GLM with a log link often fits well.
GLMs are not a workaround for normality violations. They are a more honest modeling framework for data that was never normal to begin with. If you find yourself transforming data aggressively to satisfy a parametric test, ask whether a GLM would be a better model from the start.
Test-Specific Guidance: What to Do When Your Data Violates the Normality Assumption
Different tests have different sensitivities to normality violations. Here is our practical guidance for the three most common scenarios.
t-test When Normality Is Violated
The independent-samples t-test is surprisingly robust to non-normality with adequate sample sizes. If you have at least 30 observations per group, mild to moderate non-normality rarely distorts your results. The central limit theorem protects you.
If your sample is small or the violation is severe, try a log or square root transformation first. If that fails, switch to the Mann-Whitney U test. For paired samples, use the Wilcoxon signed-rank test. Also consider whether unequal variances are the real problem. If Levene’s test is significant, use Welch’s t-test.
ANOVA When Normality Is Violated
One-way ANOVA is also fairly robust to non-normality, especially with balanced designs and equal group sizes above 20 to 30. The main concern with ANOVA is that severe non-normality can inflate the Type I error rate, leading you to find significant differences that do not exist.
For mild violations, transformation is usually sufficient. For severe violations, the Kruskal-Wallis test is the standard non-parametric alternative. If the problem is unequal variances rather than non-normality, use Welch’s ANOVA or the Brown-Forsythe test. Report which post hoc tests you used for pairwise comparisons, since these have their own assumptions.
A common question we see on statistics forums is whether ANOVA is robust to non-normal data. The answer is nuanced. ANOVA handles symmetric non-normality well but struggles with severe skewness, especially in unbalanced designs. The platykurtic distributions, those with negative excess kurtosis and flat peaks, are more problematic than leptokurtic ones.
Regression When Normality Is Violated
For OLS regression, the normality assumption applies to the residuals, not the dependent variable itself. Extract your residuals and run your normality checks on those. If the residuals are non-normal, first check for model misspecification. A missing predictor, a nonlinear relationship, or an omitted interaction can produce non-normal residuals that disappear when you improve the model.
If the model is correctly specified and residuals remain non-normal, consider robust standard errors. These adjust the standard errors to be less sensitive to assumption violations. Bootstrapping your confidence intervals is another strong option. For severe cases, consider a GLM or a generalized additive model that can accommodate non-normal error structures.
The bigger concern in regression is usually homoscedasticity, the assumption that residual variance is constant across predicted values. Heteroscedasticity is more common and more damaging than non-normality. Check a residuals-versus-fitted plot for any fan shape pattern.
Repeated Measures When Normality Is Violated
Repeated-measures designs add complexity because the assumption applies to the residuals of the model after accounting for subject effects. If normality fails, the Friedman test is the non-parametric alternative. Mixed-effects models with robust estimation are another option. These models handle unbalanced data and missing observations better than traditional repeated-measures ANOVA.
What About Outliers?
Outliers are a frequent cause of normality violations. A few extreme values can dramatically skew a distribution and tank your Shapiro-Wilk p-value. The question of whether to remove outliers is one of the most contested topics in statistics.
Our philosophy, shared by the Edinburgh team, is to analyze your data both with and without outliers. If your conclusions are the same in both analyses, report the full data and note that outliers did not change the results. If the conclusions differ, you have a decision to make.
Distinguish between improbable and impossible data points. An impossible data point is one that cannot physically occur, such as a negative age or a reaction time below the physiological limit of human response. Remove these. An improbable data point is rare but possible, such as an unusually long reaction time. These deserve careful consideration.
Never remove outliers simply because they make your normality test significant. That is p-hacking, and it undermines the integrity of your analysis. If you do remove outliers, document the criteria you used and report both analyses transparently.
Robust estimators offer a middle ground. The trimmed mean and the median are less sensitive to outliers than the standard mean. M-estimators, available in many statistical packages, downweight extreme observations automatically. These methods let you keep all your data while reducing the influence of outliers.
Reporting Your Results When Normality Is Violated
Transparency is the most important principle. Report what you found, what you did about it, and why. If you transformed your data, state the transformation and report results on the transformed scale. Provide back-transformed values for interpretability, but acknowledge the transformation.
If you switched to a non-parametric test, explain why. Report the test statistic, p-value, and an effect size estimate. For the Mann-Whitney U test, report the rank-biserial correlation as an effect size. For Kruskal-Wallis, report epsilon-squared or eta-squared based on ranks.
If you used bootstrapping or permutation tests, report the number of resamples and the method used for confidence intervals. State clearly that you used a computational approach because normality assumptions were not met.
A typical reporting template for a transformed analysis might read: Because the Shapiro-Wilk test indicated significant non-normality, p less than 0.001, we applied a log base 10 transformation. The transformed data met normality assumptions. An independent-samples t-test on transformed scores revealed a significant difference between groups.
Reviewers and readers appreciate this level of transparency. Trying to hide a normality violation by simply not mentioning it is a far worse choice than addressing it head-on.
FAQs
What to do if data failed the normality test?
If your data fails the Shapiro-Wilk or Kolmogorov-Smirnov test, start by assessing whether the violation is serious. With sample sizes above 30 per group, the central limit theorem often makes parametric tests robust enough to proceed. For smaller samples or severe violations, try a data transformation such as log or square root. If transformation does not work, switch to a non-parametric alternative like the Mann-Whitney U test or Kruskal-Wallis test.
What happens if OLS assumptions are violated?
Violating OLS regression assumptions can produce unreliable coefficient estimates, biased standard errors, and invalid p-values. Non-normal residuals mainly affect confidence intervals and hypothesis tests but not the coefficient estimates themselves. Heteroscedasticity, a more serious issue, inflates standard errors. Solutions include robust standard errors, bootstrapping, generalized linear models, or adding missing predictors to improve model specification.
How to deal with data that is not normally distributed?
You have four main options. First, try data transformation such as log, square root, or Box-Cox to pull the distribution closer to normal. Second, switch to a non-parametric test like Mann-Whitney U or Kruskal-Wallis. Third, use robust methods such as Welch’s t-test, bootstrapping, or permutation tests. Fourth, consider a generalized linear model that matches your data’s actual distribution.
What to do if Levene’s test is violated?
A significant Levene’s test means your groups have unequal variances. For a t-test, switch to Welch’s t-test, which adjusts degrees of freedom for unequal variances. For ANOVA, use Welch’s ANOVA or the Brown-Forsythe test. Both are designed for situations where the homogeneity of variance assumption fails and are available in major statistical software.
Can I still use a t-test if data is not normally distributed?
Yes, in many cases. The t-test is robust to non-normality with sample sizes above 30 per group thanks to the central limit theorem. For smaller samples with mild skewness, the t-test usually performs adequately. For severe violations with small samples, use the Mann-Whitney U test or try a data transformation first. Also check variance equality, since unequal variances require Welch’s t-test regardless of normality.
Is ANOVA robust to non-normal data?
ANOVA is generally robust to moderate non-normality, especially with balanced designs and group sizes above 20 to 30. Symmetric non-normality is well tolerated. Severe skewness and heavy tails are more problematic and can inflate the Type I error rate. For severe violations, use Kruskal-Wallis as a non-parametric alternative or apply a data transformation before running ANOVA.
Conclusion
Knowing what to do when your data violates the normality assumption comes down to a simple sequence: check the severity, try transformation, consider non-parametric alternatives, and explore robust computational methods. Most violations are manageable. The key is matching your response to the type and seriousness of the problem rather than panicking at the first significant Shapiro-Wilk result. Use the central limit theorem as your ally, document your decisions transparently, and choose the method that best fits your data rather than forcing your data into a test that does not suit it.