32 T-Tests

As we have seen throughout this book, many studies in psychology focus on the difference between two means. The most common null hypothesis test for this type of statistical relationship is the t-test. In this section, we look at three types of t-tests that are used for slightly different research designs: the one-sample t-test, the dependent-samples t-test, and the independent-samples t-test. You may have already taken a course in statistics, but we will refresh your statistical knowledge.

One-Sample t-Test

The one-sample t-test is used to compare a sample mean (M) with a hypothetical population mean (μ₀) that provides some interesting standard of comparison. In the null hypothesis, we assume that the population mean (µ) is equal to a known or expected value (μ₀₎. The alternative hypothesis is that the mean for the population is different from the hypothetical population mean: μ ≠ μ₀. To decide between these two hypotheses, we need to find the probability of obtaining the sample mean (or one more extreme) if the null hypothesis were true. But finding this p-value requires first computing a test statistic called t. (A test statistic is a statistic that is computed only to help find the p-value.) The formula for t is as follows:

[latex]t=\dfrac{{M -µ{_0}}}{\left(\dfrac{SD}{\sqrt N}\right)}[/latex]

Again, M is the sample mean and µ₀ is the hypothetical population mean of interest. SD is the sample standard deviation, and N is the sample size.

The reason the t-statistic (or any test statistic) is useful is that we know how it is distributed when the null hypothesis is true. As shown in Figure 32.1, this distribution is unimodal and symmetrical, and it has a mean of 0. Its precise shape depends on a statistical concept called the degrees of freedom, which for a one-sample t-test is N − 1. (There are 24 degrees of freedom for the distribution shown in Figure 32.1.) The important point is that knowing this distribution makes it possible to find the p-value for any t-score. Consider, for example, a t-score of 1.50 based on a sample of 25. The probability of a t-score at least this extreme is given by the proportion of t-scores in the distribution that are at least this extreme. For now, let us define extreme as being far from zero in either direction. Thus, the p-value is the proportion of t-scores that are 1.50 or above or that are −1.50 or below—a value that turns out to be .14.

Fortunately, we do not have to deal directly with the distribution of t-scores. If we were to enter our sample data and hypothetical mean of interest into one of the online statistical tools or into a program like SPSS (Excel does not have a one-sample t-test function), the output would include both the t-score and the p-value. At this point, the rest of the procedure is simple. If p is equal to or less than .05, we reject the null hypothesis and conclude that the population mean differs from the hypothetical mean of interest. If p is greater than .05, we retain the null hypothesis and conclude that there is not enough evidence to say that the population mean differs from the hypothetical mean of interest. (Again, technically, we conclude only that we do not have enough evidence to conclude that it does differ.)

If we were to compute the t-score by hand, we could use a table like Table 32.1 to make the decision. This table does not provide actual p-values. Instead, it provides the critical values of t for different degrees of freedom (df) when α is .05. For now, let us focus on the two-tailed critical values in the last column of the table. Each of these values should be interpreted as a pair of values: one positive and one negative. For example, the two-tailed critical values when there are 24 degrees of freedom are 2.064 and −2.064. These are represented by the red vertical lines in Figure 32.1. The idea is that any t-score below the lower critical value (the left-hand red line in Figure 32.1) is in the lowest 2.5% of the distribution, while any t-score above the upper critical value (the right-hand red line) is in the highest 2.5% of the distribution. Therefore, any t-score beyond the critical value in either direction is in the most extreme 5% of t-scores when the null hypothesis is true and has a p-value less than .05. Thus, if the t-score we compute is beyond the critical value in either direction, then we reject the null hypothesis. If the t-score we compute is between the upper and lower critical values, then we retain the null hypothesis.

Thus far, we have considered what is called a two-tailed test, where we reject the null hypothesis if the t-score for the sample is extreme in either direction. This test makes sense when we believe that the sample mean might differ from the hypothetical population mean, but we do not have good reason to expect the difference to go in a particular direction. But it is also possible to do a one-tailed test, where we reject the null hypothesis only if the t-score for the sample is extreme in one direction that we specify before collecting the data. This test makes sense when we have good reason to expect the sample mean will differ from the hypothetical population mean in a particular direction.

Here is how it works. Each one-tailed critical value in Table 32.1 can again be interpreted as a pair of values: one positive and one negative. A t-score below the lower critical value is in the lowest 5% of the distribution, and a t-score above the upper critical value is in the highest 5% of the distribution. For 24 degrees of freedom, these values are −1.711 and 1.711. (These are represented by the green vertical lines in Figure 32.1.) However, for a one-tailed test, we must decide before collecting data whether we expect the sample mean to be lower than the hypothetical population mean, in which case we would use only the lower critical value, or we expect the sample mean to be greater than the hypothetical population mean, in which case we would use only the upper critical value. Notice that we still reject the null hypothesis when the t-score for our sample is in the most extreme 5% of the t-scores we would expect if the null hypothesis were true—so α remains at .05. We have simply redefined extreme to refer only to one tail of the distribution. The advantage of the one-tailed test is that critical values are less extreme. If the sample mean differs from the hypothetical population mean in the expected direction, then we have a better chance of rejecting the null hypothesis. The disadvantage is that if the sample mean differs from the hypothetical population mean in the unexpected direction, then there is no chance at all of rejecting the null hypothesis.

The Dependent-Samples t-Test

The dependent-samples t-test (sometimes called the paired-samples t-test) is used to compare two means for the same sample tested at two different times or under two different conditions. This comparison is appropriate for pretest-posttest designs or within-subjects experiments. The null hypothesis is that the means at the two times or under the two conditions are the same in the population. The alternative hypothesis is that they are not the same. This test can also be one-tailed if the researcher has good reason to expect the difference to go in a particular direction.

It helps to think of the dependent-samples t-test as a special case of the one-sample t-test. However, the first step in the dependent-samples t-test is to reduce the two scores for each participant to a single difference score by taking the difference between them. At this point, the dependent-samples t-test becomes a one-sample t-test on the difference scores. The hypothetical population mean (µ₀) of interest is 0 because this is what the mean difference score would be if there were no difference on average between the two times or two conditions. We can now think of the null hypothesis as being that the mean difference score in the population is 0 (µ₀ = 0) and the alternative hypothesis as being that the mean difference score in the population is not 0 (µ₀ ≠ 0).

The Independent-Samples t-Test

The independent-samples t-test is used to compare the means of two separate samples (M₁ and M₂). The two samples might have been tested under different conditions in a between-subjects experiment, or they could be pre-existing groups in a cross-sectional design (e.g., women and men, extraverts and introverts). The null hypothesis is that the means of the two populations are the same: µ₁ = µ₂. The alternative hypothesis is that they are not the same: µ₁ ≠ µ₂. Again, the test can be one-tailed if the researcher has good reason to expect the difference to go in a particular direction.

The t-statistic here is a bit more complicated because it must take into account two sample means, two standard deviations, and two sample sizes. The formula is as follows:

[latex]t=\dfrac{{M{_1}-M{_2}}}{\sqrt{\dfrac{SD{^2}{_1}}{n{_1}}+\dfrac{SD{^2}{_2}}{n{_2}}}}[/latex]

Notice that this formula includes squared standard deviations (the variances) that appear inside the square root symbol. Also, lowercase n₁ and n₂ refer to the sample sizes in the two groups or conditions (as opposed to capital N, which generally refers to the total sample size). The only additional thing to know here is that there are N − 2 degrees of freedom for the independent-samples t-test.

The t-Statistic

We were introduced to hypothesis testing using the z-statistic for sample means in previous chapters. This was a useful way to link the material and ease us into the new way of looking at data, but it isn’t a very common test because it relies on knowing the population’s standard deviation, s, which is rarely going to be the case. Instead, we will estimate that parameter s using the sample statistic s in the same way that we estimate[latex]\mu[/latex] using M ([latex]\mu[/latex] will still appear in our formulas because we suspect something about its value, and that is what we are testing). Our new statistic is called t, and for testing one population mean using a single sample (called a one-sample t-test), it takes the form:

[latex]\displaystyle t = \dfrac{M - \mu}{s_M} = \dfrac{M - \mu}{\dfrac{s}{\sqrt{n}}}[/latex]

Notice that t looks almost identical to z; this is because they test the exact same thing: the value of a sample mean compared to what we expect of the population. The only difference is that the standard error is now denoted S_M to indicate that we use the sample statistic for standard deviation, s, instead of the population parameter s. The process of using and interpreting the standard error and the full test statistic remains exactly the same.

We have also learned that the formulas for sample standard deviation and population standard deviation differ by one key factor: the denominator for the parameter is N but the denominator for the statistic is N − 1, also known as degrees of freedom, df. Because we are using a new measure of spread, we can no longer use the standard normal distribution and the z-table to find our critical values. For t-tests, we will use the t-distribution and t-table to find these values.

The t-distribution, like the standard normal distribution, is symmetric and normally distributed with a mean of 0 and standard error (as the measure of standard deviation for sampling distributions) of 1. However, because the calculation of standard error uses degrees of freedom, there will be a different t distribution for every degree of freedom. Luckily, they all work exactly the same, so in practice, this difference is minor.

Figure 32.2 shows four curves: a normal distribution curve labeled z, and three t-distribution curves for 2, 10, and 30 degrees of freedom. Two things should stand out: First, for lower degrees of freedom (e.g., 2), the tails of the distribution are much fatter, meaning a larger proportion of the area under the curve falls in the tail. This means that we will have to go farther out into the tail to cut off the portion corresponding to 5% or α = .05, which will in turn lead to higher critical values. Second, as the degrees of freedom increase, we get closer and closer to the z curve. Even the distribution with df = 30, corresponding to a sample size of just 31 people, is nearly indistinguishable from z. In fact, a t-distribution with infinite degrees of freedom (theoretically, of course) is exactly the standard normal distribution. Because of this, the bottom row of the t-table also includes the critical values for z tests at the specific significance levels. Even though these curves are very close, it is still important to use the correct table and critical values, because small differences can add up quickly.

The t-distribution table lists critical values for one- and two-tailed tests at several levels of significance, arranged into columns. The rows of the t-table list degrees of freedom up to df = 100 in order to use the appropriate distribution curve. It does not, however, list all possible degrees of freedom in this range, because that would take too many rows. Above df = 40, the rows jump in increments of 10. If a problem requires you to find critical values and the exact degrees of freedom is not listed, you always round down to the next smallest number. For example, if you have 48 people in your sample, the degrees of freedom are N − 1 = 48 − 1 = 47; however, 47 doesn’t appear on our table, so we round down and use the critical values for df = 40, even though 50 is closer. We do this because it avoids inflating Type I error (false positives) by using criteria that are too lax.

Hypothesis Testing with t

Hypothesis testing with the t-statistic works exactly the same way as z tests did, following the four-step process of (1) stating the hypotheses, (2) finding the critical values, (3) computing the test statistic and effect size, and (4) making the decision.

Example Oil Change Speed

We will work through an example: Let’s say that you move to a new city and find an auto shop to change your oil. Your old mechanic did the job in about 30 minutes (although you never paid close enough attention to know how much that varied), and you suspect that your new shop takes much longer. After four oil changes, you think you have enough evidence to demonstrate this.

Step 1: State the Hypotheses

Our hypotheses for one-sample t-tests are identical to those we used for z tests. We still state the null and alternative hypotheses mathematically in terms of the population parameter and which are written out in readable English. For our example:

[latex]\displaystyle H_0\text{: There is no difference in the average time to change a car's oil}[/latex]

[latex]\displaystyle H_0:\mu=30[/latex]

[latex]\displaystyle H_A\text{: This shop takes longer to change oil than your old mechanic}[/latex]

[latex]\displaystyle H_A:\mu>30[/latex]

Step 2: Find the Critical Values

As noted above, our critical values still delineate the area in the tails under the curve corresponding to our chosen level of significance. Because we have no reason to change significance levels, we will use α = .05, and because we suspect a direction of effect, we have a one-tailed test. To find our critical values for t, we need to add one more piece of information: the degrees of freedom. For this example:

[latex]\displaystyle df = N - 1 = 4 - 1 = 3[/latex]

Going to our t-table, a portion of which is found in Table 32.2a, we locate the column corresponding to our one-tailed significance level of .05 and find where it intersects with the row for 3 degrees of freedom. As we can see in Table 32.2b, our critical value is t* = 2.353. (See table above for all critical values.)

Table 32.2a. t Distribution Table (t-Table) with Proportion (a) in One Tail

df	.25	.20	.15	.10	.05	.025	.01	.005	.0005

Table 32.2b. t Distribution Table (t-Table) with Proportion (a) in Two Tails Combined

df	.50	.40	.30	.20	.10	.05	.02	.01	.001
1	1.000	1.376	1.963	3.078	6.314	12.706	31.821	63.657	636.578
2	0.816	1.061	1.386	1.886	2.920	4.303	6.965	9.925	31.600
3	0.765	1.978	1.250	1.638	2.353	3.182	4.541	5.841	12.924
4	0.741	1.941	1.190	1.533	2.132	2.776	3.747	4.604	8.610
5	0.727	0.920	1.156	1.476	2.015	2.571	3.365	4.032	6.869

We can then shade this region on our t distribution to visualize our rejection region (Figure 32.3).

Step 3: Calculate the Test Statistic and Effect Size

The four wait times you experienced for your oil changes at the new shop were 46 minutes, 58 minutes, 40 minutes, and 71 minutes. We will use these to calculate M and s by first filling in the sum of squares in Table 32.3.

Table 32.3. Sum of Squares.

X	X − M	(X − M)2
46	−7.75	60.06
58	4.25	18.06
40	−13.75	189.06
71	17.25	297.56
Σ = 215	Σ = 0	Σ = 564.74

After filling in the first row to get ΣX = 215, we find that the mean is M = 53.75 (215 divided by sample size 4), which allows us to fill in the rest of the table to get our sum of squares SS = 564.74, which we then plug into the formula for standard deviation:

[latex]\displaystyle s = \sqrt{\dfrac{\sum(X - M)^2}{N - 1}} = \sqrt{\dfrac{SS}{df}} = \sqrt{\dfrac{564.74}{3}} = 13.72[/latex]

Next, we take this value and plug it into the formula for standard error:

[latex]\displaystyle s_M = \dfrac{s}{\sqrt{n}} = \dfrac{13.72}{2} = 6.86[/latex]

And, finally, we put the standard error, sample mean, and null hypothesis value into the formula for our test statistic t:

[latex]\displaystyle t = \dfrac{M - \mu}{s_M} = \dfrac{53.75 - 30}{6.86} = \dfrac{23.75}{6.68} = 3.46[/latex]

This may seem like a lot of steps, but it is really just taking our raw data to calculate one value at a time and carrying that value forward into the next equation: data → sample size/degrees of freedom → mean → sum of squares → standard deviation → standard error → test statistic. At each step, we simply match the symbols of what we just calculated to where they appear in the next formula to make sure we are plugging everything in correctly.

Next, we need to calculate an effect size, which is still Cohen’s d, but now we use s in place of σ:

[latex]\displaystyle d = \dfrac{M - \mu}{s} = \dfrac{53.75 - 30.00}{13.72} = 1.73[/latex]

This is a large effect. It should also be noted that for some things, like the minutes in our current example, we can also interpret the magnitude of the difference we observed (23 minutes and 45 seconds) as an indicator of importance, since time is a familiar metric.

Step 4: Make the Decision

Now that we have our critical value and test statistic, we can make our decision using the same criteria we used for a z-test. Our obtained t-statistic was t = 3.46 and our critical value was t* = 2.353: t > t*, so we reject the null hypothesis and conclude:

Based on our four oil changes, the new mechanic takes longer on average (M = 53.75, SD = 13.72) to change oil than our old mechanic, and the effect size was large, t(3) = 3.46, p < .05, d = 1.74.

Notice that we also include the degrees of freedom in parentheses next to t. Figure 32.4 shows the output from JASP.

 

 

Confidence Intervals

Up to this point, we have learned how to estimate the population parameter for the mean using sample data and a sample statistic. From one point of view, this makes sense: we have one value for our parameter, so we use a single value (called a point estimate) to estimate it. However, we have seen that all statistics have sampling error and that the value we find for the sample mean will bounce around based on the people in our sample, simply due to random chance. Thinking about estimation from this perspective, it would make more sense to take that error into account rather than relying just on our point estimate. To do this, we calculate what is known as a confidence interval.

A confidence interval starts with our point estimate and then creates a range of scores considered plausible based on our standard deviation, our sample size, and the level of confidence with which we would like to estimate the parameter. This range, which extends equally in both directions away from the point estimate, is called the margin of error. We calculate the margin of error by multiplying our two-tailed critical value by our standard error:

[latex]\displaystyle \text{Margin of error} = t^* \left(\dfrac{s}{\sqrt{n}}\right)[/latex]

One important consideration when calculating the margin of error is that it can only be calculated using the critical value for a two-tailed test. This is because the margin of error moves away from the point estimate in both directions, so a one-tailed value does not make sense.

The critical value we use will be based on a chosen level of confidence, which is equal to 1 − α. Thus, a 95% level of confidence corresponds to α = .05. Thus, at the .05 level of significance, we create a 95% confidence interval. How to interpret that is discussed further on.

Once we have our margin of error calculated, we add it to our point estimate for the mean to get an upper bound to the confidence interval and subtract it from the point estimate for the mean to get a lower bound for the confidence interval:

[latex]\displaystyle \text{Upper bound} = M + \text{Margin of error}[/latex]

[latex]\displaystyle \text{Lower bound} = M - \text{Margin of error}[/latex]

or simply:

[latex]\displaystyle \text{Confidence interval} = M \pm t^* \left(\dfrac{s}{\sqrt{n}}\right)[/latex]

To write out a confidence interval, we always use round brackets (i.e., parentheses) and put the lower bound, a comma, and the upper bound:

[latex]\displaystyle \text{Confidence interval} = (\text{Lower bound}, \text{Upper bound})[/latex]

Let’s see what this looks like with some actual numbers by taking our oil change data and using it to create a 95% confidence interval, estimating the average length of time it takes at the new mechanic. We already found that our average was M = 53.75 and our standard error was S_M = 6.86. We also found a critical value to test our hypothesis, but remember that we were testing a one-tailed hypothesis, so that critical value won’t work. To see why that is, look at the column headers on the t-table. The column for one-tailed α = .05 is the same as a two-tailed α = .10. If we used the old critical value, we’d actually be creating a 90% confidence interval (1.00 − 0.10 = 0.90, or 90%). To find the correct value, we use the column for two-tailed α = .05 and, again, the row for 3 degrees of freedom, to find t* = 3.182.

Now we have all the pieces we need to construct our confidence interval:

[latex]$95\% \ CI = 53.75 \pm 3.182(6.86)[/latex]

[latex]\begin{aligned} \displaystyle \text{Upper bound} &= 53.75 + 3.182(6.86)\\ &= 53.75 + 21.83\\ &= 75.58 \end{aligned}[/latex]

[latex]\begin{aligned} \displaystyle \text{Lower bound} &= 53.75 - 3.182(6.86)\\ &= 53.75 - 21.83\\ &= 31.92 \end{aligned}[/latex]

[latex]\displaystyle 95\%\,\text{CI} = (31.92,\;75.58)[/latex]

So we find that our 95% confidence interval runs from 31.92 minutes to 75.58 minutes, but what does that actually mean? The range (31.92, 75.58) represents values of the mean that we consider reasonable or plausible based on our observed data. It includes our point estimate of the mean, M = 53.75, in the center, but it also has a range of values that could also have been the case based on what we know about how much these scores vary (i.e., our standard error).

It is very tempting to also interpret this interval by saying that we are 95% confident that the true population mean falls within the range (31.92, 75.58), but this is not true. The reason it is not true is that phrasing our interpretation this way suggests that we have firmly established an interval and the population mean does or does not fall into it, suggesting that our interval is firm and the population mean will move around. However, the population mean is an absolute that does not change; it is our interval that will vary from data collection to data collection, even taking into account our standard error. The correct interpretation, then, is that we are 95% confident that the range (31.92, 75.58) brackets the true population mean. This is a very subtle difference, but it is an important one.

Hypothesis Testing with Confidence Intervals

As a function of how they are constructed, we can also use confidence intervals to test hypotheses. However, we are limited to testing two-tailed hypotheses only, because of how the intervals work, as discussed above.

Once a confidence interval has been constructed, using it to test a hypothesis is simple. If the range of the confidence interval brackets (or contains, or is around) the null hypothesis value, we fail to reject the null hypothesis. If it does not bracket the null hypothesis value (i.e., if the entire range is above the null hypothesis value or below it), we reject the null hypothesis. The reason for this is clear if we think about what a confidence interval represents. Remember: a confidence interval is a range of values that we consider reasonable or plausible based on our data. Thus, if the null hypothesis value is in that range, then it is a value that is plausible based on our observations. If the null hypothesis is plausible, then we have no reason to reject it. Thus, if our confidence interval brackets the null hypothesis value, thereby making it a reasonable or plausible value based on our observed data, then we have no evidence against the null hypothesis and fail to reject it. However, if we build a confidence interval of reasonable values based on our observations and it does not contain the null hypothesis value, then we have no empirical (observed) reason to believe the null hypothesis value and therefore reject the null hypothesis.

Confidence Intervals Using z

Confidence intervals can also be constructed using z-score criteria if one knows the population standard deviation. The format, calculations, and interpretation are all exactly the same, only replacing t* with z* and S_M with [latex]\sigma_M[/latex] .

Image Descriptions

Figure 32.1. A bell-shaped normal distribution curve on a beige/cream background. The horizontal axis is labeled from -3 to 3, representing standard deviations from the mean. Two vertical red lines mark the critical values at approximately -2 and +2 on the x-axis, defining the rejection regions in both tails of the distribution. These shaded tail regions represent the critical areas for a two-tailed hypothesis test. The curve is symmetrical around zero, with the peak at the center. [Return to Figure 32.1]

Figure 32.2 A graph comparing four distribution curves centered at zero with x-axis ranging from -3 to 3. The curves shown are: 1) A solid pink/magenta line representing the standard normal (z) distribution; 2) A blue dotted line showing the t-distribution with df = 30, which closely overlaps the z-distribution; 3) A red dashed line showing the t-distribution with df = 10, slightly flatter than the z-distribution; and 4) A navy blue dashed line showing the t-distribution with df = 2, noticeably flatter and with heavier tails. All curves are symmetrical and bell-shaped. As degrees of freedom increase, the t-distributions become more peaked and approach the normal distribution. The t-distribution with df = 2 has the flattest peak and the heaviest tails, while the t-distribution with df = 30 is nearly indistinguishable from the normal distribution. [Return to Figure 32.2]

Figure 32.3. A bell-shaped t-distribution curve centered at zero with x-axis labeled from -4 to 4. A vertical line marks the critical value at t = 2.353 on the right side of the distribution. The area to the right of this critical value (the right tail) is shaded in orange/red, representing the rejection region for a right-tailed hypothesis test. This shaded area corresponds to the alpha level (significance level) of the test. The curve is symmetrical around zero but only the right tail is highlighted as the critical region. [Return to Figure 32.3]

Figure 32.4. Statistical output showing results of a One Sample T-Test and descriptive statistics for Wait Time data. The One Sample T-Test table shows: Variable: Wait Time, t-statistic: 3.462, degrees of freedom (df): 3, p-value: 0.020. A note indicates that for the Student t-test, the alternative hypothesis specifies that the mean is greater than 30, and identifies this as a Student's t-test. The Descriptives table shows: Variable: Wait Time, Sample size (N): 4, Mean: 53.750, Standard deviation (SD): 13.720. With a p-value of 0.020 (less than the typical significance level of 0.05), there is statistically significant evidence that the mean wait time is greater than 30 minutes. [Return to Figure 32.4]