34 Correlations

Thus far, all of our analyses have focused on comparing the value of a continuous variable across different groups via mean differences. We will now turn away from means and look instead at how to assess the relationship between two continuous variables in the form of correlations. As we will see, the logic behind correlations is the same as it was behind group means, but we will now have the ability to assess an entirely new data structure.

Variability and Covariance

A common theme throughout statistics is the notion that individuals will differ on different characteristics and traits, which we call variance. In inferential statistics and hypothesis testing, our goal is to find systematic reasons for differences and rule out random chance as the cause. By doing this, we are using information on a different variable—which so far has been group membership, like in ANOVA—to explain this variance. In correlations, we will instead use a continuous variable to account for the variance.

Because we have two continuous variables, we will have two characteristics or scores on which people will vary. What we want to know is whether people vary on the scores together. That is, as one score changes, does the other score also change in a predictable or consistent way? This notion of variables differing together is called covariance (the prefix co- meaning “together”).

Let’s look at our formula for variance on a single variable:

                                          [latex]\displaystyle s^2 = \frac{\sum(X - M)^2}{N - 1}][/latex]

We use X to represent a person’s score on the variable at hand, and M to represent the mean of that variable. The numerator of this formula is the sum of squares, which we have seen several times for various uses. Recall that squaring a value is just multiplying that value by itself. Thus, we can write the same equation as:

                                         [latex]\displaystyle s^2 = \frac{\sum((X - M)(X - M))}{N - 1}[/latex]

This is the same formula and works the same way as before, where we multiply the deviation score by itself (we square it) and then sum across squared deviations.

Now, let’s look at the formula for covariance. In this formula, we will still use X to represent the score on one variable, and we will now use Y to represent the score on the second variable. We will still use bars to represent averages of the scores. The formula for covariance (cov_XY with the subscript XY to indicate covariance across the X and Y variables) is:

                              [latex]\displaystyle \text{cov}_{XY} = \frac{\sum((X - M_X)(Y - M_Y))}{N - 1}[/latex]

As we can see, this is the exact same structure as the previous formula. Now, instead of multiplying the deviation score by itself on one variable, we take the deviation scores from a single person on each variable and multiply them together. We do this for each person (exactly the same as we did for variance) and then sum them to get our numerator. The numerator in this is called the sum of products (SP).

                                               [latex]\displaystyle SP = \sum((X - M_X)(Y - M_Y))[/latex]

We will calculate the sum of products using the same table we used to calculate the sum of squares. In fact, the table for sum of products is simply a sum of squares table for X, plus a sum of squares table for Y, with a final column of products, as shown below.

This table works the same way it did before (remember that the column headers tell you exactly what to do in that column). We list our raw data for the X and Y variables in the X and Y columns, respectively, then add them up so we can calculate the mean of each variable. We then take those means and subtract them from the appropriate raw score to get our deviation scores for each person on each variable, and the columns of deviation scores will both add up to zero. We will square our deviation scores for each variable to get the sums of squares for X and Y so that we can compute the variance and standard deviation of each (we will use the standard deviation in our equation below). Finally, we take the deviation score from each variable and multiply them together to get our product score. Summing this column will give us our sum of products. It is very important that you multiply the raw deviation scores from each variable, not the squared deviation scores.

Our sum of products will go into the numerator of our formula for covariance, and then we only have to divide by N − 1 to get our covariance. Unlike the sum of squares, both our sum of products and our covariance can be positive, negative, or zero, and they will always match (e.g., if our sum of products is positive, our covariance will always be positive). A positive sum of products and covariance indicates that the two variables are related and move in the same direction. That is, as one variable goes up, the other will also go up, and vice versa. A negative sum of products and covariance means that the variables are related but move in opposite directions when they change, which is called an inverse relationship. In an inverse relationship, as one variable goes up, the other variable goes down. If the sum of products and covariance is zero, then that means the variables are not related. As one variable goes up or down, the other variable does not change in a consistent or predictable way.

The previous paragraph brings us to an important definition about relationships between variables. What we are looking for in a relationship is a consistent or predictable pattern. That is, the variables change together, either in the same direction or opposite directions, in the same way each time. It doesn’t matter if this relationship is positive or negative, only that it is not zero. If there is no consistency in how the variables change within a person, then the relationship is zero and does not exist. We will revisit this notion of direction vs. zero relationship later on.

Visualizing Relationships

As discussed in an earlier chapter, visualizing data remains an important first step in understanding and describing our data before we move into inferential statistics. Nowhere is this more important than in correlation. Correlations are visualized by a scatter plot, where our X variable values are plotted on the x-axis, the Y variable values are plotted on the y-axis, and each point or marker in the plot represents a single person’s score on X and Y. Figure 34.1 shows a scatter plot for hypothetical scores on job satisfaction (X) and worker well-being (Y). We can see from the axes that each of these variables is measured on a 10-point scale, with 10 being the highest on both variables (high satisfaction and good well-being) and 1 being the lowest (dissatisfaction and poor well-being). When we look at this plot, we can see that the variables do seem to be related. The higher scores on job satisfaction tend to also be the higher scores on well-being, and the same is true of the lower scores.

Figure 34.1 demonstrates a positive relationship. As scores on X increase, scores on Y also tend to increase. Although this is not a perfect relationship (if it were, the points would form a single straight line), it is nonetheless very clearly positive. This is one of the key benefits to scatter plots: they make it very easy to see the direction of the relationship. As another example, Figure 34.2 shows a negative relationship between job satisfaction (X) and burnout (Y). As we can see from this plot, higher scores on job satisfaction tend to correspond with lower scores on burnout, which is how stressed, unenergetic, and unhappy someone is at their job. As with Figure 34.1, this is not a perfect relationship, but it is still a clear one. As these figures show, points in a positive relationship move from the bottom left of the plot to the top right, and points in a negative relationship move from the top left to the bottom right.

Scatter plots can also indicate that there is no relationship between the two variables. In these scatter plots (for example, Figure 34.3, which plots job satisfaction and job performance), there is no interpretable shape or line in the scatter plot. The points appear randomly throughout the plot. If we tried to draw a straight line through these points, it would basically be flat. The low scores on job satisfaction have roughly the same scores on job performance as do the high scores on job satisfaction. Scores in the middle or average range of job satisfaction have some scores on job performance that are about equal to the high and low levels and some scores on job performance that are a little higher, but the overall picture is one of inconsistency.

As we can see, scatter plots are very useful for giving us an approximate idea of whether there is a relationship between the two variables and, if there is, whether that relationship is positive or negative. They are also the only way to determine one of the characteristics of correlations that are discussed next: form.

Three Characteristics

When we talk about correlations, there are three traits that we need to know in order to truly understand the relationship (or lack of relationship) between X and Y: form, direction, and magnitude. We will discuss each of them in turn.

Form

The first characteristic of relationships between variables is their form. The form of a relationship is the shape it takes in a scatter plot, and a scatter plot is the only way it is possible to assess the form of a relationship. There are three forms we look for: linear, curvilinear, or no relationship. A linear relationship is what we saw in Figure 34.1, Figure 34.2, and Figure 34.3. If we drew a line through the middle points in any of the scatter plots, we would be best suited with a straight line. The term linear comes from the word line. A linear relationship is what we will always assume when we calculate correlations. All of the correlations presented here are only valid for linear relationships. Thus, it is important to plot our data to make sure we meet this assumption.

The relationship between two variables can also be curvilinear. As the name suggests, a curvilinear relationship is one in which a line through the middle of the points in a scatter plot will be curved rather than straight. Two examples are presented in Figure 34.4 and Figure 34.5.

Curvilinear relationships can take many shapes, and the two examples above are only a small sample of the possibilities. What they have in common is that they both have a very clear pattern, but that pattern is not a straight line. If we try to draw a straight line through them, we would get a result similar to what is shown in Figure 34.6.

Although that line is the closest it can be to all points at the same time, it clearly does a very poor job of representing the relationship we see. Additionally, the line itself is flat, suggesting there is no relationship between the two variables, even though the data show that there is one. This is important to keep in mind because the math behind our calculations of correlation coefficients will only ever produce a straight line—we cannot create a curved line with the techniques discussed here.

Finally, sometimes when we create a scatter plot, we end up with no interpretable relationship at all. An example of this is shown in Figure 34.7. The points in this plot show no consistency in relationship, and a line through the middle would once again be a straight, flat line.

Sometimes when we look at scatter plots, it is tempting to get biased by a few points that fall far away from the rest of the points and seem to imply that there may be some sort of relationship. These points are called outliers, and we will discuss them in more detail later in the chapter. These can be common, so it is important to formally test for a relationship between our variables, not just rely on visualization. This is the point of hypothesis testing with correlations, and we will go in-depth on it soon. First, however, we need to describe the other two characteristics of relationships: direction and magnitude.

Direction

The direction of the relationship between two variables tells us whether the variables change in the same way at the same time or in opposite ways at the same time. We saw this concept earlier when first discussing scatter plots, and we used the terms positive and negative. A positive relationship is one in which X and Y change in the same direction: as X goes up, Y goes up, and as X goes down, Y also goes down. A negative relationship is just the opposite: X and Y change together in opposite directions: as X goes up, Y goes down, and vice versa.

As we will see soon, when we calculate a correlation coefficient, we are quantifying the relationship demonstrated in a scatter plot. That is, we are putting a number to it. That number will be either positive, negative, or zero, and we interpret the sign of the number as our direction. If the number is positive, it is a positive relationship, and if it is negative, it is a negative relationship. If it is zero, then there is no relationship. The direction of the relationship corresponds directly to the slope of the hypothetical line we draw through scatter plots when assessing the form of the relationship. If the line has a positive slope that moves from bottom left to top right, it is positive, and vice versa for negative. If the line is flat, that means it has no slope, and there is no relationship, which will in turn yield a zero for our correlation coefficient.

Magnitude

The number we calculate for our correlation coefficient, which we will describe in detail below, corresponds to the magnitude of the relationship between the two variables. The magnitude is how strong or how consistent the relationship between the variables is. Higher numbers mean greater magnitude, which means a stronger relationship.

Our correlation coefficients will take on any value between −1.00 and 1.00, with 0.00 in the middle, which again represents no relationship. A correlation of −1.00 is a perfect negative relationship; as X goes up by some amount, Y goes down by the same amount, consistently. Likewise, a correlation of 1.00 indicates a perfect positive relationship; as X goes up by some amount, Y also goes up by the same amount. Finally, a correlation of 0.00, which indicates no relationship, means that as X goes up by some amount, Y may or may not change by any amount, and it does so inconsistently.

The vast majority of correlations do not reach −1.00 or 1.00. Instead, they fall in between, and we use rough cut-offs for how strong the relationship is based on this number. Importantly, the sign of the number (the direction of the relationship) has no bearing on how strong the relationship is. The only thing that matters is the magnitude, or the absolute value of the correlation coefficient. A correlation of −1 is just as strong as a correlation of 1. We generally use values of .10, .30, and .50 as indicating weak, moderate, and strong relationships, respectively.

The strength of a relationship, just like the form and direction, can also be inferred from a scatter plot, though this is much more difficult to do. Some examples of weak and strong relationships are shown in Figure 34.8 and Figure 34.9, respectively. Weak correlations still have an interpretable form and direction, but it is much harder to see. Strong correlations have a very clear pattern, and the points tend to form a line. The examples show two different directions, but remember that the direction does not matter for the strength, only the consistency of the relationship and the size of the number, which we will see next.

Pearson’s r

There are several different types of correlation coefficients, but we will only focus on Pearson’s r. Pearson's r measures the linear relationship between two continuous variables. It is the most popular correlation coefficient for assessing linear relationships, which serves as both a descriptive statistic (like M) and an inferential test statistic (like t). It is descriptive because it describes what is happening in the scatter plot; r will have both a sign (+/−) for the direction and a number (0 to 1 in absolute value) for the magnitude. As noted above, because it assumes a linear relationship, nothing about r will suggest what the form is—it will only tell what the direction and magnitude will be if the form is linear. (Remember: always make a scatter plot first!) The coefficient r also works as a test statistic because the magnitude of r will correspond directly to a t-value as the specific degrees of freedom, which can then be compared to a critical value. Luckily, we do not need to do this conversion by hand. Instead, we will have a table of r critical values that looks very similar to our t table, and we can compare our r directly to those.

The formula for r is very simple: it is just the covariance (defined above) divided by the standard deviations of X and Y:

                                  [latex]\displaystyle r = \frac{\text{cov}_{XY}}{s_X s_Y} = \frac{SP}{\sqrt{(SS_X)(SS_Y)}}[/latex]

The first formula gives a direct sense of what a correlation is: a covariance standardized onto the scale of X and Y; the second formula is computationally simpler and faster. Both of these equations will give the same value, and as we saw at the beginning of the chapter, all of these values are easily computed by using the sum of products table. When we do this calculation, we will find that our answer is always between −1.00 and 1.00 (if it’s not, check the math again), which gives us a standard, interpretable metric, similar to what z scores did.

It was stated earlier that r is a descriptive statistic like M, and just like M, it corresponds to a population parameter. For correlations, the population parameter is the lowercase Greek letter (“rho”); be careful not to confuse with a p-value—they look quite similar. The statistic r is an estimate of , just as M is an estimate of . Thus, we will test our observed value of r that we calculate from the data and compare it to a value of specified by our null hypothesis to see if the relationship between our variables is significant, as we will see in the following example.

Example Anxiety and Depression

Anxiety and depression are often reported to be highly linked (or “comorbid”). Our hypothesis testing procedure follows the same four-step process as before, starting with our null and alternative hypotheses. We will look for a positive relationship between our variables among a group of 10 people because that is what we would expect based on them being comorbid.

Step 1: State the Hypotheses

Our hypotheses for correlations start with a baseline assumption of no relationship, and our alternative will be directional if we expect to find a specific type of relationship. For this example, we expect a positive relationship:

[latex]\displaystyle H_0: \text{ There is no relationship between anxiety and depression}[/latex]

[latex]\displaystyle H_0: \rho = 0[/latex]

[latex]\displaystyle H_A: \text{ There is a positive relationship between anxiety and depression}[/latex]

[latex]\displaystyle H_A: \rho ≠ 0[/latex]

Remember that [latex]\rho[/latex] (“rho”) is our population parameter for the correlation that we estimate with r, just like M and [latex]\mu[/latex] for means. Remember also that if there is no relationship between variables, the magnitude will be 0, which is where we get the null and alternative hypothesis values.

Step 2: Find the Critical Values

As a reminder, the critical value serves as a threshold to determine whether the data in one's sample is statistically significant. The critical values for correlations can be found from the correlation table (a portion of which appears in Table 34.2), which looks very similar to the t table (The complete correlation table can be found in Appendix D [New Tab]).  The column of critical values is based on our significance level ([latex]\alpha[/latex]) and the directionality of our test. The row is determined by our degrees of freedom. For correlations, we have N − 2 degrees of freedom, rather than N − 1. For our example, we have 10 people, so our degrees of freedom = 10 − 2 = 8.

 

Table 34.2. Critical Values for Pearson’s r (Correlation Table)

Degrees of freedom (df) df = n - 2	Critical Value
Level of Significance for  One-Tailed Test	.05	.025	.01	.005
Level of Significance for Two-Tailed Test	.10	.05	.02	.01
1	.988	.997	.9995	.9999
2	.900	.950	.980	.990
3	.805	.878	.934	.959
4	.729	.811	.882	.917
5	.669	.754	.833	.875
6	.621	.707	.789	.834
7	.582	.666	.750	.798
8	.549	.632	.715	.765
9	.521	.602	.685	.735
10	.497	.576	.658	.708
11	.476	.553	.634	.684
12	.458	.532	.612	.661
13	.441	.514	.592	.641
14	.426	.497	.574	.623
15	.412	.482	.558	.606

We were not given any information about the level of significance at which we should test our hypothesis, so we will assume α = .05 as always. From our table, we can see that a one-tailed test (because we expect only a positive relationship) at the α = .05 level has a critical value of r* = .549. Thus, if our observed correlation is greater than .549, it will be statistically significant. This is a rather high bar (remember, the guideline for a strong relationship is r = .50); this is because we have so few people. Larger samples make it easier to find significant relationships.

Step 3: Calculate the Test Statistic and Effect Size

We have laid out our hypotheses and the criteria we will use to assess them, so now we can move on to our test statistic. Before we do that, we must first create a scatter plot of the data to make sure that the most likely form of our relationship is, in fact, linear. Figure 34.10 shows our data plotted out, and it looks like they are, in fact, linearly related, so Pearson’s r is appropriate.

 

The data we gather from our participants is as follows:

Table 34.3. Comorbidity of Depression and Anxiety Data

Dep.	2.81	1.96	3.43	3.40	4.71	1.80	4.27	3.68	2.44	3.13
Anx.	3.54	3.05	3.81	3.43	4.03	3.59	4.17	3.46	3.19	4.12

We will need to put these values into our sum of products table to calculate the standard deviation and covariance of our variables. We will use X for depression and Y for anxiety to keep track of our data. But be aware that this choice is arbitrary, and the math will work out the same if we decide to do the opposite. Our table is thus:

Table 34.4. Comorbidity of Depression and Anxiety Sum of Products

X	(X − MX)	(X − MX)2	Y	(Y − MY)	(Y − MY)2	(X − MX)(Y − MY)
2.81	−0.35	0.12	3.54	−0.10	0.01	0.04
1.96	−1.20	1.44	3.05	−0.59	0.35	0.71
3.43	0.27	0.07	3.81	0.17	0.03	0.05
3.40	0.24	0.06	3.43	−0.21	0.04	−0.05
4.71	1.55	2.40	4.03	0.39	0.15	0.60
1.80	−1.36	1.85	3.59	−0.05	0.00	0.07
4.27	1.11	1.23	4.17	0.53	0.28	0.59
3.68	0.52	0.27	3.46	−0.18	0.03	−0.09
2.44	−0.72	0.52	3.19	−0.45	0.20	0.32
3.13	−0.03	0.00	4.12	0.48	0.23	−0.01
31.63	0.03	7.97	36.39	−0.01	1.33	2.22

The bottom row is the sum of each column. We can see from this that the sum of the X observations is 31.63, which makes the mean of the X variable M = 3.16. The deviation scores for X sum to 0.03, which is very close to 0, given rounding error, so everything looks right so far. The next column is the squared deviations for X, so we can see that the sum of squares for X is SS_X = 7.97. The same is true of the Y columns, with an average of M_Y = 3.64, deviations that sum to zero within rounding error, and a sum of squares as SS_Y = 1.33. The final column is the product of our deviation scores (not of our squared deviations), which gives us a sum of products of SP = 2.22.

There are now three pieces of information we need to calculate before we compute our correlation coefficient: the covariance of X and Y and the standard deviation of each.

The covariance of two variables, remember, is the sum of products divided by N − 1. For our data:

[latex]\displaystyle \text{cov}_{XY} = \frac{SP}{N - 1} = \frac{2.22}{9} = 0.25[/latex]

The formulas for standard deviation are the same as before. Using subscripts X and Y to denote depression and anxiety:

[latex]\displaystyle s_X = \sqrt{\frac{\sum(X - M_X)^2}{N - 1}} = \sqrt{\frac{7.97}{9}} = 0.94[/latex]

[latex]\displaystyle s_Y = \sqrt{\frac{\sum(Y - M_Y)^2}{N - 1}} = \sqrt{\frac{1.33}{9}} = 0.38[/latex]

Now we have all of the information we need to calculate r:

[latex]\displaystyle r = \frac{\text{cov}_{XY}}{s_X s_Y} = \frac{0.25}{(0.94)(0.38)} = .70[/latex]

We can verify this using our other formula, which is computationally shorter:

[latex]\displaystyle r = \frac{SP}{\sqrt{(SS_X)(SS_Y)}} = \frac{2.22}{\sqrt{(7.97)(1.33)}} = .70[/latex]

So our observed correlation between anxiety and depression is r = .70, which, based on sign and magnitude, is a strong, positive correlation. Now we need to compare it to our critical value to see if it is also statistically significant.

Effect Size and Pearson’s r

Pearson’s r is an incredibly flexible and useful statistic. Not only is it both descriptive and inferential, as we saw above, but because it is on a standardized metric (always between −1.00 and 1.00), it can also serve as its own effect size. In general, we use r = .10, r = .30, and r = .50 as our guidelines for small, medium, and large effects. Just like with Cohen’s d, these guidelines are not absolutes, but they do serve as useful indicators in most situations. Notice as well that these are the same guidelines we used earlier to interpret the magnitude of the relationship based on the correlation coefficient.

In addition to r being its own effect size, there is an additional effect size we can calculate for our results. This effect size is r², and it is exactly what it looks like—it is the squared value of our correlation coefficient. r² is interpreted as the amount of variance explained in the outcome variance, and the cut scores are the same as well: .01, .09, and .25 for small, medium, and large, respectively. Notice here that these are the same cutoffs we used for regular r effect sizes, but squared (.102 = .01, .302 = .09, .502 = .25) because, again, the r² effect size is just the squared correlation, so its interpretation should be, and is, the same. The reason we use r² as an effect size is because our ability to explain variance is often important to us.

Step 4: Make a Decision

Our critical value was r* = .549, and our obtained value was r = .70. Our obtained value was larger than our critical value, so we can reject the null hypothesis.

Reject H₀. Based on our sample of 10 people, there is a statistically significant, strong, positive relationship between anxiety and depression, r(8) = .70, p < .05.

Notice in our interpretation that, because we already know the magnitude and direction of our correlation, we can interpret that. We also report the degrees of freedom, just like with t, and we know that p < α because we rejected the null hypothesis. As we can see, even though we are dealing with a very different type of data, our process of hypothesis testing has remained unchanged. Unlike for our other statistics, we do not report an effect size for the correlation coefficient because the reader can easily do that for themselves by squaring r. The r² statistic is called the coefficient of determination and is essentially the effect size for a correlation coefficient; it tells us what percentage of the variance in the X variable is explained by the Y variable (and vice versa).

Figure 34.11 shows the output from JASP for this example.

Correlation versus Causation

We cover a great deal of material in this textbook, and many of the principles underlying what we do in statistics can be used in your day-to-day life to help you interpret information objectively and make better decisions. We now come to what may be the most important lesson in introductory statistics: the difference between correlation and causation.

It is very, very tempting to look at variables that are correlated and assume that this means they are causally related; that is, it gives the impression that X is causing Y. However, in reality, correlations do not—and cannot—do this. Correlations do not prove causation. While this sounds like a simple idea on the surface, correlational data are often used in decision-making in large scale settings (e.g. schools, healthcare, public policy).  However, no matter how logical, how obvious, or how convenient it may seem, no correlational analysis can demonstrate causality. The only way to demonstrate a causal relationship is with a properly designed and controlled experiment.

Many times, we have good reason for assessing the correlation between two variables, and often that reason will be that we suspect one causes the other. Thus, when we run our analyses and find strong, statistically significant results, it is tempting to say that we have found the causal relationship that we are looking for. The reason we cannot do this is that, without an experimental design that includes random assignment and control variables, the relationship we observe between the two variables may be caused by something else that we failed to measure—something we can only detect and control for with an experiment. These confound variables, which we will represent with Z, can cause two variables, X and Y, to appear related when in fact they are not. They do this by being the hidden—or lurking—cause of each variable independently. That is, if Z causes X and Z causes Y, X and Y will appear to be related. However, if we control for the effect of Z (the method for doing this is beyond the scope of this text), then the relationship between X and Y will disappear.

A popular example of this effect is the correlation between ice cream sales and deaths by drowning. These variables are known to correlate very strongly over time. However, this does not prove that one causes the other. The lurking variable in this case is the weather—people enjoy swimming and eating ice cream more during hot weather as a way to cool off. As another example, consider shoe size and spelling ability in elementary school children. Although there should clearly be no causal relationship here, the variables are nonetheless consistently correlated. The confound in this case? Age. Older children spell better than younger children and are also bigger, so they have larger shoes.

When there is the possibility of confounding variables being the hidden cause of our observed correlation, we will often collect data on Z as well and control for it in our analysis. This is good practice and a wise thing for researchers to do. Thus, it would seem that it is easy to demonstrate causation with a correlation that controls for Z. However, the number of variables that could potentially cause a correlation between X and Y is functionally limitless, so it would be impossible to control for everything. That is why we use experimental designs; by randomly assigning people to groups and manipulating variables in those groups, we can balance out individual differences in any variable that may be our cause.

It is not always possible to do an experiment, however, so there are certain situations in which we will have to be satisfied with our observed relationship and do the best we can to control for known confounds. However, in these situations, even if we do an excellent job of controlling for many extraneous (a statistical and research term for “outside”) variables, we must be careful not to use causal language. That is because, even after controls, sometimes variables are related just by chance.

Sometimes, variables will end up being related simply due to random chance, and we call these correlations spurious. Spurious just means random, so what we are seeing is random correlations because, given enough time, enough variables, and enough data, sampling error will eventually cause some variables to appear related when they are not. Sometimes, this even results in incredibly strong, but completely nonsensical, correlations. This becomes more and more of a problem as our ability to collect massive datasets and dig through them improves, so it is very important to think critically about any relationship you encounter.

Final Considerations

Correlations, although simple to calculate, can be very complex, and there are many additional issues we should consider. We will look at two of the most common issues that affect our correlations and discuss some other correlations and reporting methods you may encounter.

Range Restriction

The strength of a correlation depends on how much variability is in each of the variables X and Y. This is evident in the formula for Pearson’s r, which uses both covariance (based on the sum of products, which comes from deviation scores) and the standard deviation of both variables (based on the sums of squares, which also come from deviation scores). Thus, if we reduce the amount of variability in one or both variables, our correlation will go down. Failure to capture the full range of a variability is called range restriction.

Take a look at Figure 34.12 and Figure 34.13. Figure 34.12 shows a strong relationship (r = .67) between two variables. An orange oval is overlaid on it to make the relationship even more distinct. Figure 34.13 shows the same data, but the bottom half of the X variable (all scores below 5) has been removed, which causes our relationship (again represented by an orange oval) to become much weaker (r = .38). Thus, range restriction has truncated (made smaller) our observed correlation.

Sometimes, range restriction happens by design. For example, we rarely hire people who do poorly on job applications, so we would not have the lower range of those predictor variables. Other times, we inadvertently cause range restriction by not properly sampling our population. Although there are ways to correct for range restriction, they are complicated and require much information that may not be known, so it is best to be very careful during the data collection process to avoid it.

Outliers

Another issue that can cause the observed size of our correlation to be inappropriately large or small is the presence of outliers. An outlier is a data point that falls far away from the rest of the observations in the dataset. Sometimes outliers are the result of incorrect data entry, poor or intentionally misleading responses, or simple random chance. Other times, however, they represent real people with meaningful values on our variables. For instance, if the only African American student in a predominately white school exhibits unusually high anxiety symptoms, this could simply indicate that this one student is highly anxious; however, it could also be signaling valuable insights into the school's culture. The distinction between meaningful and accidental outliers is a difficult one that requires close examination and the expert judgment of the researcher. Sometimes, we will remove the outlier (if we think it is an accident) or we may decide to keep it (if we find the scores to still be meaningful even though they are different).

The scatter plots in Figure 34.14 show the effects that an outlier can have on data. In the first plot, we have our raw dataset. You can see in the upper right corner that there is an outlier observation that is very far from the rest of our observations on both the X and Y variables. In the middle plot, we see the correlation computed when we include the outlier, along with a straight line representing the relationship; here, it is a positive relationship. In the third plot, we see the correlation after removing the outlier, along with a line showing the direction once again. Not only did the correlation get stronger, it completely changed direction!

In general, there are three effects that an outlier can have on a correlation: it can change the magnitude (make it stronger or weaker), it can change the significance (make a non-significant correlation significant or vice versa), and/or it can change the direction (make a positive relationship negative or vice versa). Outliers are a big issue in small datasets where a single observation can have a strong weight compared with the rest. However, as our sample sizes get very large (into the hundreds), the effects of outliers diminish because they are outweighed by the rest of the data. Nevertheless, no matter how large a dataset you have, it is always a good idea to screen for outliers, both statistically (using analyses that we do not cover here) and visually (using scatter plots).

Other Correlation Coefficients

In this chapter, we have focused on Pearson’s r as our correlation coefficient because it is very common and useful. There are, however, many other correlations out there, each of which is designed for a different type of data. The most common of these is Spearman’s rho (), which is designed to be used on ordinal data rather than continuous data. This is a useful analysis if we have ranked data or our data do not conform to the normal distribution. There are even more correlations for ordered categories, but they are much less common and beyond the scope of this chapter.

Additionally, the principles of correlations underlie many other advanced analyses. In the next chapter, we will learn about regression, which is a formal way of running and analyzing a correlation that can be extended to more than two variables. Regression is a powerful technique that serves as the basis for even our most advanced statistical models, so what we have learned in this chapter will open the door to an entire world of possibilities in data analysis.

Correlation Matrices

Many research studies look at the relationship between more than two continuous variables. In such situations, we could simply list all of our correlations, but that would take up a lot of space and make it difficult to quickly find the relationship we are looking for. Instead, we create correlation matrices so that we can quickly and simply display our results. A matrix is like a grid that contains our values. There is one row and one column for each of our variables, and the intersections of the rows and columns for different variables contain the correlation for those two variables.

At the beginning of the chapter, we saw scatter plots presenting data for correlations between job satisfaction, well-being, burnout, and job performance. We can create a correlation matrix to quickly display the numerical values of each. Such a matrix is shown below.

Notice that there are values of 1.00 where each row and column of the same variable intersect. This is because a variable correlates perfectly with itself, so the value is always exactly 1.00. Also notice that the upper cells are left blank and only the cells below the diagonal of 1.00s are filled in. This is because correlation matrices are symmetrical: they have the same values above the diagonal as below it. Filling in both sides would provide redundant information and make it a bit harder to read the matrix, so we leave the upper triangle blank.

Correlation matrices are a very condensed way of presenting many results quickly, so they appear in almost all research studies that use continuous variables. Many matrices also include columns that show the variable means and standard deviations, as well as asterisks showing whether or not each correlation is statistically significant.

Image Descriptions

Figure 34.1. This scatter plot illustrates a positive correlation between job satisfaction and well-being, with both axes ranging from 1 to 10. Approximately 15-17 data points are distributed across the graph, showing a clear upward trend from the lower left (around 1.5, 1.3) to the upper right, with most points concentrated in the 7-10 range for both variables. The pattern demonstrates that as job satisfaction increases, well-being tends to increase as well, though some scatter in the data indicates individual variation. The title "Positive Relationship" reinforces this positive linear correlation between the two variables. [Return to Figure 34.1]

Figure 34.2. This scatter plot illustrates a negative correlation between job satisfaction and burnout, with both axes ranging from 1 to 10. Approximately 15 data points are distributed across the graph, showing a clear downward trend from the upper left (around 2.5 job satisfaction, 7.6 burnout) to the lower right, with points at higher job satisfaction levels corresponding to lower burnout scores. The pattern demonstrates that as job satisfaction increases, burnout tends to decrease, though some variation exists in the data. The title "Negative Relationship" emphasizes this inverse correlation between the two variables. [Return to Figure 34.2]

Figure 34.3. This scatter plot illustrates the absence of correlation between job satisfaction and job performance, with both axes ranging from 1 to 10. Approximately 15 data points are distributed across the graph in a random pattern, with no discernible upward or downward trend. The points are scattered throughout the middle and upper portions of the plot, ranging from about 4 to 10 on the job satisfaction axis and 5 to 9.5 on the job performance axis. Unlike positive or negative correlations, this pattern shows that job satisfaction levels do not predict job performance, as high and low performance scores appear across all levels of job satisfaction. The title "No Relationship" emphasizes this lack of correlation between the two variables. [Return to Figure 34.3]

Figure 34.4. This scatter plot illustrates a strong positive correlation between two variables, with the x-axis ranging from 0 to 10 and the y-axis ranging from 0 to 10. Approximately 10 data points are plotted, forming a nearly perfect linear pattern from the lower left (around 0.5, 2.2) to the upper right (10, 10). The points show minimal scatter around an imaginary trend line, indicating a very tight relationship between the variables. This pattern demonstrates that as the x-variable increases, the y-variable increases proportionally at a consistent rate, suggesting a strong predictive relationship between the two measures. Note that this plot lacks axis labels and a title, so the specific variables being measured are not identified. [Return to Figure 34.4]

Figure 34.5. This scatter plot illustrates a curvilinear or non-linear relationship between two variables, with the x-axis ranging from 0 to 16 and the y-axis ranging from 0 to 7. Approximately 15 data points are plotted, forming an inverted U-shape or parabolic pattern. The points begin low on the left (around 1, 1), rise steadily through the lower x-values, reach their highest concentration in the middle range (around x=7-10, y=5-5.5), and then descend back down on the right side (ending near 16, 1). This pattern demonstrates that the relationship between the variables is not linear but rather quadratic, where the y-variable increases with the x-variable up to a certain point and then decreases. Note that this plot lacks axis labels and a title, so the specific variables being measured are not identified. [Return to Figure 34.5]

Figure 34.6. This scatter plot illustrates a curvilinear or non-linear relationship between two variables, with the x-axis ranging from 0 to 16 and the y-axis ranging from 0 to 7. Approximately 15 data points are plotted, forming an inverted U-shape or parabolic pattern. A horizontal reference line at y=4 divides the plot into upper and lower regions. The points begin low on the left (around 1, 1), rise steadily through the lower x-values, reach their peak in the middle range (around x=7-9, y=5.5), and then descend back down on the right side (ending near 16, 1). This pattern demonstrates a quadratic relationship where the y-variable increases with the x-variable up to a midpoint and then decreases, with the reference line potentially indicating a threshold or baseline value. Note that this plot lacks axis labels and a title, so the specific variables being measured are not identified. [Return to Figure 34.6]

Figure 34.7. This scatter plot illustrates the absence of correlation between two variables, with both axes ranging from 0 to 9. Approximately 200 data points are densely plotted, forming a roughly circular or elliptical cluster centered around the middle of the graph (approximately x=5, y=5). The points are concentrated most heavily in the center, with the density decreasing toward the edges, ranging from about 2 to 7.5 on both axes. Unlike linear relationships, this pattern shows no upward or downward trend, indicating that changes in the x-variable do not predict changes in the y-variable. The circular distribution suggests the variables are independent of each other, with the data spread relatively evenly in all directions from the center point. Note that this plot lacks axis labels and a title, so the specific variables being measured are not identified. [Return to Figure 34.7]

Figure 34.8. This scatter plot illustrates a weak positive correlation between two variables, with both axes ranging from 0 to 9. Approximately 200 data points are plotted, showing a general upward trend from the lower left (around 2, 2.5) to the upper right (around 8, 6.5), though with considerable scatter around this trend. The points are most densely concentrated in the center of the plot (around x=4-6, y=4-6), with the cloud of data points appearing somewhat elongated diagonally. While there is a discernible positive relationship where higher x-values tend to correspond with higher y-values, the substantial variation in the data indicates that the x-variable is only a modest predictor of the y-variable. Note that this plot lacks axis labels and a title, so the specific variables being measured are not identified. [Return to Figure 34.8]

Figure 34.9. This scatter plot illustrates a negative correlation between two variables, with both axes ranging from 0 to 9. Approximately 200 data points are plotted, showing a clear downward trend from the upper left (around 3, 7.5) to the lower right (around 8, 2). The points form an elongated cloud pattern that slopes diagonally downward, most densely concentrated in the center region (around x=4-6, y=4-6). This pattern demonstrates that as the x-variable increases, the y-variable tends to decrease, indicating an inverse relationship between the two measures. While there is some scatter in the data, the negative trend is clearly visible, suggesting a moderate to strong negative correlation. Note that this plot lacks axis labels and a title, so the specific variables being measured are not identified. [Return to Figure 34.9]

Figure 34.10. This scatter plot illustrates a weak positive correlation between depression and anxiety, with the x-axis labeled "Depression" ranging from 1 to 6 and the y-axis labeled "Anxiety" ranging from 1 to 6. Approximately 12 data points are plotted, showing a gentle upward trend from the lower left (around 2, 3) to the upper right (around 5, 4). The points are moderately scattered, concentrated mostly in the 2-5 range on the depression axis and the 3-4.5 range on the anxiety axis. This pattern suggests that as depression levels increase, anxiety levels tend to increase as well, though the relationship is relatively weak with considerable variability. The sparse distribution of points and moderate scatter indicates that while there is some association between these variables, depression alone does not strongly predict anxiety levels. [Return to Figure 34.10]

Figure 34.11. This correlation table, titled "Correlation" with a subtitle "Pearson's Correlations," displays the statistical relationship between depression and anxiety variables. The table has three columns (Variable, Depression, and Anxiety) and shows correlation statistics for two variables. For the Depression variable (row 1), Pearson's r and p-value are shown as dashes in both the Depression and Anxiety columns, as a variable's correlation with itself is not meaningful. For the Anxiety variable (row 2), the correlation with Depression shows Pearson's r = 0.680 and p-value = 0.015, while its correlation with itself is marked with dashes. A note at the bottom states "Note. All tests one-tailed, for positive correlation." This table indicates a statistically significant moderate to strong positive correlation between depression and anxiety, with the low p-value (0.015) suggesting this relationship is unlikely to be due to chance. [Return to Figure 34.11]

Figure 34.12. This scatter plot illustrates a positive correlation between two variables, with both axes ranging from 0 to 9. Approximately 250 data points are plotted, showing a clear upward trend from the lower left (around 2.5, 2.5) to the upper right (around 7.5, 7.5). A red ellipse is overlaid on the plot, encircling the data and representing the 95% confidence region or correlation ellipse. The ellipse is oriented diagonally from lower left to upper right, emphasizing the positive linear relationship between the variables. The data points are most densely concentrated in the center of the ellipse (around x=5, y=5), with the density decreasing toward the edges. This visualization demonstrates not only the positive correlation but also the spread and distribution of the data around the trend line. Note that this plot lacks axis labels and a title, so the specific variables being measured are not identified. [Return to Figure 34.12]

Figure 34.13. This scatter plot illustrates the absence of correlation between two variables, with both axes ranging from 0 to 9. Approximately 100 data points are plotted, forming a roughly circular cluster concentrated in the center-right portion of the graph (around x=5-7, y=4.5-6). A red ellipse is overlaid on the plot, encircling the data and representing the 95% confidence region. Unlike the previous correlation plot, this ellipse is nearly circular rather than elongated, indicating no directional relationship between the variables. The data points show no upward or downward trend, with high and low y-values appearing across all x-values within the cluster. This pattern demonstrates that changes in the x-variable do not predict changes in the y-variable, suggesting the two measures are independent of each other. Note that this plot lacks axis labels and a title, so the specific variables being measured are not identified. [Return to Figure 34.13]

Figure 34.14. This figure contains three vertically stacked scatter plots displaying the same dataset to illustrate different interpretations of correlation. All three panels use identical axes, with the x-axis ranging from 60 to 120 and the y-axis ranging from 50 to 105, and contain approximately 15 identical data points scattered across the plot area. The top panel presents the raw data without any trend line, showing points distributed without an obvious pattern. The middle panel adds a red upward-sloping trend line with the correlation coefficient labeled as "r = .19," suggesting a very weak positive relationship where y slightly increases as x increases. The bottom panel displays a red downward-sloping trend line with "r = -.31," indicating a weak negative relationship where y slightly decreases as x increases. This three-panel comparison demonstrates how the same data can be interpreted with different correlation strengths and directions, highlighting the subjective nature of fitting trend lines or the sensitivity of correlation to outliers and data interpretation. Note that the axes are unlabeled, so the specific variables are not identified. [Return to Figure 34.14]