Ask these questions first
Before naming any test, characterize your problem with a few questions. What type is your outcome variable — numerical (a measured quantity like height or score) or categorical (a label like pass/fail or blood type)? How many groups or samples are you comparing — one, two, or more than two? Are the groups independent (different subjects) or paired (the same subjects measured twice)? And what is your actual question — comparing means, testing an association, or predicting one variable from another?
These four answers narrow the field dramatically. Almost every introductory and AP-level test-selection problem is solved by working through them in order. Write them down for your problem before you reach for a formula; most 'I don't know which test to use' moments dissolve once the variable types and group count are explicit.
- Outcome variable type: numerical or categorical?
- Number of groups: one, two, or three-plus?
- Independent or paired samples?
- Goal: compare means, test association, or predict?
Comparing means of numerical data
When your outcome is numerical and you are comparing averages, the t-test family is home base. To compare one sample mean against a known or claimed value, use a one-sample t-test. To compare the means of two independent groups — say a treatment group versus a control group — use a two-sample (independent) t-test. When the same subjects are measured twice, such as before and after an intervention, use a paired t-test, which analyzes the differences within each pair.
When you compare the means of three or more groups at once, do not run many t-tests — that inflates your false-positive rate. Use analysis of variance (ANOVA), which tests whether any of the group means differ while controlling the overall error rate. If ANOVA is significant, follow-up comparisons tell you which groups differ.
- One mean vs a claimed value: one-sample t-test.
- Two independent groups: two-sample (independent) t-test.
- Same subjects measured twice: paired t-test.
- Three or more groups: ANOVA (not repeated t-tests).
Working with categorical data
When your data is categorical — counts of how many observations fall into each category — the chi-square family takes over. A chi-square goodness-of-fit test checks whether the observed distribution across categories matches an expected one, such as testing whether a die is fair. A chi-square test of independence checks whether two categorical variables are associated, such as whether preference for a product depends on age group.
For a single proportion (the fraction of successes in a sample), you can use a one-proportion z-test against a claimed value, and for comparing two proportions, a two-proportion z-test. The key trigger is that you are counting category memberships rather than measuring a numerical quantity.
- One categorical variable vs an expected distribution: chi-square goodness-of-fit.
- Association between two categorical variables: chi-square test of independence.
- One or two proportions: one- or two-proportion z-test.
Relationships and prediction
When both variables are numerical and you want to know whether they move together, correlation measures the strength and direction of a linear relationship. When you want to go further and predict or model one numerical variable from another, use linear regression, which fits a line and lets you estimate the outcome from the predictor and test whether the relationship is statistically significant.
The distinction is one of purpose: correlation answers 'are these related and how strongly', while regression answers 'what is the relationship, and can I use it to predict'. StatRise includes both correlation and regression tools in its calculator set, so you can fit a line to your own data and read off the slope, intercept, and fit.
- Strength and direction of a linear relationship: correlation.
- Modeling or predicting one numerical variable from another: linear regression.
- Correlation describes; regression predicts and models.
Check the assumptions before you trust the result
Choosing the right test is only half the job — most tests carry assumptions, and a valid choice with violated assumptions can still mislead. Common ones include approximate normality (or a large enough sample for the Central Limit Theorem to help), independence of observations, and, for some tests, roughly equal variances across groups. Chi-square tests assume expected counts are not too small.
When assumptions are badly violated, there are alternatives — nonparametric tests that do not assume normality, for example — but for most intro and AP problems, verifying that your sample is reasonably sized, your observations are independent, and your data type matches the test is enough. The fastest way to internalize all of this is deliberate practice: work through many test-selection problems, run the tests in a calculator, and check your reasoning. The StatRise practice bank and step-by-step calculators are built for exactly that loop.
- Check independence, approximate normality or a large sample, and equal-variance assumptions.
- For chi-square, watch for expected counts that are too small.
- Practice selection on many problems until the decision becomes automatic.