Hypothesis Testing Explained: The Five Steps, Worked Through

The idea behind the whole procedure

A hypothesis test answers one question: is the effect I see in my sample real, or could it plausibly be just random noise? You start by assuming, for the sake of argument, that there is no effect. Then you ask how surprising your data would be if that no-effect assumption were true. If your data would be very surprising under 'no effect', you reject the no-effect assumption and conclude something real is going on. If your data would be unremarkable, you do not have enough evidence to claim an effect.

Everything in the five steps is machinery for answering 'how surprising is my data'. Keeping that one question in mind stops the terminology from feeling arbitrary. Consider a running example: a coffee brand claims its bags contain 340 grams and you suspect underfilling, so you weigh a sample and test whether the true mean is really less than 340.

Step 1 — State the null and alternative hypotheses

The null hypothesis (H0) is the 'no effect' or 'nothing unusual' claim, and it always contains an equality. The alternative hypothesis (Ha) is what you are actually trying to find evidence for. In the coffee example, H0 says the mean fill is 340 grams (the claim holds) and Ha says the mean is less than 340 grams (they are underfilling).

Whether the alternative uses 'less than', 'greater than', or 'not equal to' determines whether your test is one-tailed or two-tailed, which changes how you compute the p-value later. Decide this before you look at the data — choosing the direction after seeing the result is a classic way to fool yourself.

• H0 always contains equality (=, ≤, or ≥) and represents 'no effect' or the status quo.
• Ha is the research claim: a difference, an increase, a decrease, or 'not equal'.
• A directional Ha (< or >) gives a one-tailed test; 'not equal' gives a two-tailed test.

Step 2 — Choose a significance level

The significance level, written as alpha (α), is the threshold of 'surprising enough' that you set in advance. It is the probability of wrongly rejecting a true null hypothesis — a false alarm — that you are willing to tolerate. The conventional choice is 0.05, meaning you accept a 5% chance of a false positive, but 0.01 is used when a false alarm is costly and 0.10 when you want to be more sensitive.

Set alpha before computing anything. It represents your standard of evidence, and choosing it after seeing the p-value defeats the entire purpose of the test. For the coffee example, we will use the standard α = 0.05.

• Alpha is the false-positive rate you agree to tolerate, fixed before the analysis.
• 0.05 is standard; use 0.01 when false alarms are expensive, 0.10 when you want more power.
• Alpha is also the boundary that defines the rejection region for your test statistic.

Step 3 — Compute the test statistic

The test statistic measures how far your sample result sits from what the null hypothesis predicts, expressed in standard-error units. For a mean, it is roughly (sample estimate minus null value) divided by the standard error of the estimate. A large test statistic means your data is far from the null's prediction; a small one means it is close.

Which statistic you compute depends on the situation. A z-statistic applies when the population standard deviation is known or the sample is large; a t-statistic applies when you estimate the standard deviation from a small sample, which is the common case. In the coffee example you would compute a one-sample t-statistic from your sample mean, the claimed 340 grams, your sample standard deviation, and your sample size. StatRise's calculators handle this arithmetic and show each step, so you can focus on setting the problem up correctly rather than pushing numbers.

• The test statistic = (estimate − null value) ÷ standard error.
• Use a t-statistic when the standard deviation is estimated from the sample (the usual case).
• Use a z-statistic when the population standard deviation is known or the sample is large.

Step 4 — Find the p-value and compare to alpha

The p-value is the probability of getting a test statistic at least as extreme as yours, assuming the null hypothesis is true. It is the numerical answer to 'how surprising is my data'. A small p-value means your result would rarely happen by chance under the null, which is evidence against the null.

The decision rule is mechanical: if the p-value is less than or equal to alpha, reject H0; if it is greater than alpha, fail to reject H0. Note the careful phrasing — you never 'accept' the null, you only fail to reject it, because absence of evidence is not evidence of absence. If the coffee test gives p = 0.02, that is below 0.05, so you reject H0 and conclude the bags are underfilled.

• p-value = probability of data this extreme (or more) if H0 were true.
• p ≤ α: reject H0. p > α: fail to reject H0.
• You never 'accept' H0 — failing to reject is not proof it is true.

Step 5 — State the conclusion in context

The final step is the one readers care about most, and the one students most often skip: translate the statistical decision back into plain language about the original problem. A conclusion like 'reject H0' is incomplete. A full conclusion says what it means: 'At the 0.05 significance level, there is sufficient evidence to conclude that the mean fill weight is less than 340 grams.'

Always tie the conclusion to the significance level, the direction of the alternative, and the real-world quantity. Remember too what the test does not say: statistical significance is not the same as practical importance, and one test on one sample is evidence, not proof. Practicing full, in-context conclusions is exactly the reasoning the StatRise practice questions drill, alongside the calculators that produce the numbers.

• Restate the decision in terms of the real problem, not just 'reject' or 'fail to reject'.
• Reference the significance level and the direction of the alternative.
• Remember: significance is not the same as practical importance.

Frequently asked questions