ACrash Course in Statistics at FIU – The One Way ANOVA (#3) – Spring 2022
So you took Stats I and Stats II at FIU and passed. But do you remember what you did, how you did it, and why you did it? If you need some basic statistic reminders for the One Way ANOVA, then this is the lecture for you! I am going to talk about a One Way ANOVA example in this document that corresponds to the same example you saw in the Descriptive Statistics Crash Course (#1) and t-Test Crash Course (#2) where participants were asked to recall how much money they spent on textbooks the prior semester. However, for this ANOVA crash course, we are going to add a third condition: control (a third group of participants who do not see any prior book recall amounts on the list). As you can see here, we have ONE independent variable (hence the One Way ANOVA), but here we have three levels (or three conditions): High, Low, and Control. The good news is that this mini-lecture will sum up the basics of the ANOVA for you as we look at this study, but you can find additional information about the ANOVA in your textbooks. On the final pages of this document are several questions based on this crash course. Answer these questions, and then go into your “Crash Course in Statistics – The One Way ANOVA Quiz #3” in your Canvas assessments menu and copy over your answer. Each Crash Course Quiz counts 5 points.
How, when, and why do a One Way ANOVA?
Before we get to the example, let me give you some basic information about the One Way ANOVA. Do you recall the t-Test, where we compared two means to see whether and in what direction the means differed? Well, a One Way ANOVA is very similar, but here we compare three or more means to see if they differ significantly from one another. In this analysis, we need three pieces of information: 1) the means for each of the three groups (descriptive statistics), 2) the One Way ANOVA information itself, and 3) post hoc tests.
1). Once again, remember that a mean is the average score for that condition. That is, you add up all of the scores in a condition and divide by the number of total scores to arrive at the average. Since a One Way ANOVA looks at three or more different conditions, we have at least three means: one for each condition. The means here (plus the standard deviation, which we will talk about in the lecture) are descriptive statistics. That is, they help describe the data.
2). The One Way ANOVA information itself is a test of inferential statistics. That is, we infer significant differences between the three or more groups. When writing it out, you will see a very common layout for the One Way ANOVA, something like: F(2, 134) = 2.61, p = .021. The F tells you this is a One Way ANOVA. The 2 and 134 tells us our degrees of freedom (more on that in out lecture). The 2.61 is the actual number for the One Way ANOVA. The p indicates whether it is significant (if it is less than .05, then it is significant).
3). Finally, we have to consider post hoc tests. You might recall using the Tukey post hoc test in the past, but do you remember why you used it? Take a step back and think about the t-Test, which looked at two means: Mean A and Mean B. If Mean A is 4.56 and Mean B is 7.67 and your t-Test is significant (that is, p is less than .05), then you simply compare the two means to see which is higher: Mean A or Mean B. Here, Mean B is clearly higher (7.67 is higher than 4.56), and since the t-Test is significant then Mean B is significantly higher than Mean A. But when we have three levels to our independent variable, we are now dealing with three means: Mean A, Mean B, and Mean C. Let’s say Mean A is 4.56, Mean B is 7.67, and Mean C is 6.21. If our One Way ANOVA is significant (that is, it is less than .05), we know the means differ. The question is,
which
of the three means differ? Does Mean A differ from Mean B? Does Mean B differ from Mean C? Does Mean A differ from Mean C? Or there might be other combinations. Maybe Mean A and Mean C do not differ from each other, but both are significantly lower than Mean B. Unlike the t-Test, we don’t know which of the three means differ, which is why we run a post hoc test (like Tukey) to compare Mean A to Mean B, and Mean A to Mean C, and Mean B to Mean C. It runs all of those analyses for us in one test. So you might wonder, “Why not just run three t-Tests, with one t-Test comparing Mean A to Mean B, a second t-Test comparing Mean B to Mean C, and a third t-Test comparing Mean B to Mean C.?” Well, you could actually do that, but we run into a Type I error. That is, the more tests we run, the greater the chance one of them will be significant. If we run three t-Tests, we open up the chance of one of them being falsely positive. With the One Way ANOVA, we just run the one test to compare the three means (note that the post hoc tests are still a part of the One Way ANOVA – it compares the three means under the umbrella of the One Way ANOVA test).
Like the t-Test, we run a One Way ANOVA only under certain conditions.
First, our dependent variable (the variable we measure) must be continuous / scaled. That is, the DV has to be along a scale. For example, it can be an attitude (“On a scale of 1 to 9, how angry are you?”), a time frame (“How quickly did the salesperson help the customer on a scale of zero seconds to a thousand seconds?”), or money estimation (“How much do you recall spending on textbooks last semester?”). We call these interval or ratio scales, which means we can use a t-Test or ANOVA. We CANNOT run a One Way ANOVA on categorical data. That is, if we have a yes / no question (“Are you lonely: Yes or No”) or a category based question (“What is your favorite food: hamburgers, pizza, salad, or tacos?”), then we cannot run a One Way ANOVA. These latter questions are based more on choice of option rather than an actual rating scale, and thus we cannot use a One Way ANOVA on them.
Second, we run a One Way ANOVA when we have only one independent variable and that independent variable has at least three conditions (Note: it can have more than three levels, but you still only have one independent variable). That is, we compare the means from Condition A, Condition B, and Condition C. If our One Way ANOVA is significant (p is less than .05), then we look at our post hoc tests to see which means differ. “The One Way ANOVA was significant, F(2, 134) = 2.61, p = .021. Tukey post hoc tests showed that Condition A (mean = 48.38) was significantly lower than Condition B (mean = 63.25). In addition, Condition C (mean = 48.88) was significantly lower than Condition B. However, Condition A did not differ significantly from Condition C.”
Let’s see how this looks using the textbook money example.
Textbook Study – How Much Did You Spend On Textbooks (High, Low, or Control)
Recall the basic set-up for our money spent on textbooks. Researchers ask participants to recall how much they spent on textbooks the prior semester, and has each participant write their answer on a survey sheet. In two conditions, the first ten answer slots are already filled in, presumably by other respondents. However, the researcher actually completed those ten slots, and manipulated the dollar amounts so that in in the High Dollar Condition, the dollar amounts ranged from $
350
to $450 (Figure 1). In the Low Dollar Condition, amounts ranged from $
250
to $350 (Figure 2). In our new study, the researcher provides a third condition in which there are no prior dollar amounts on the list (Control Condition – Figure 3). Using psychological principles based on conformity and informational social influence (e.g. participants relying on the behavior of other individuals when they lack a clear memory), the researcher predicts that those in the High Dollar Condition will recall spending more money on textbooks the prior semester than those in the Low Dollar Condition, with those in the Control condition providing a dollar amount somewhere in the middle.
Here, the independent variable is Dollar Condition (High versus Low versus Control) while the dependent variable is the amount of money participants recall spending on textbooks (in $). Imagine we have eight real participants in the High Dollar Condition, eight real participants in the Low Dollar Condition, and eight real participants in the Control Condition (and no, we are not including the original researcher-completed dollar amounts on the sheet passed out by the researcher in the High and Low Dollar Conditions, as those are not real participants!).
Figure 3: No Prior Dollar Amount Condition
Consider the data:
|
Condition A (High) |
Condition B (Low) |
Condition C (Control) |
||||||||||||
| 350 |
275 |
300 |
||||||||||||
|
400 |
325 |
|||||||||||||
|
375 |
||||||||||||||
| 250 | ||||||||||||||
|
260 |
||||||||||||||
|
315 |
||||||||||||||
|
∑A = 2700 |
∑B = 2350 |
∑C = 2450 |
||||||||||||
|
Mean = $337.50 |
Mean = $293.75 |
M = $306.25 |
∑, or the symbol for Sigma, means “the sum of”. Thus ∑A is the sum of the scores for Condition A. That is, 350 + 400 + 375 + 350 + 300 + 325 + 300 + 300 = 2700. There are eight scores here, so we divide 2700 / 8 = 337.50, giving us our mean of $337.50 for Condition A (High Dollar Condition). We do the same thing for Condition B (Low Dollar Condition), giving us a mean of $293.75 (2350 / 8 = 293.75). Finally, we do the same thing for Condition C (Control), giving us a mean of $306.25 (2450 / 8 = 306.25).
For the first part of our analysis, we compare the means. As you see, $293.75 in Condition B (Low) looks lower than $337.50 in Condition A (High). Now consider the $306.25 in Condition C (Control), which falls between the High and Low Dollar conditions. That $306.25 doesn’t look that different from the $337.50 in the High Dollar Condition or the $293.75 in the Low Dollar Condition. If I were “eyeballing” this, I would think that participants recall spending significantly more in the High Dollar Condition than in the Low Dollar Condition, but that the Control condition doesn’t differ from either the High or Low Dollar Conditions. However, just because some of our means
seem
to differ doesn’t mean they
do
differ. To make that assessment, we run the One Way ANOVA and look at the p value to see if it is p is less than .05. We can do this by hand (like you did in Stats I and Methods One) or we can take the easy road and let SPSS calculate it for us. I am going to take the easy road, but keep in mind that we still have to interpret what SPSS tells us.
For the next section, I am going to open SPSS and run a One Way ANOVA. I’ll use screenshots from SPSS as I go, but feel free to run these analyses yourself. Just set up your SPSS file like mine (I also included this SPSS file for you in Canvas if you prefer to use that. It is called “Crash Course Quiz #3 – Textbook Money (ANOVA Practice)”, but it is a short data set so I recommend setting up your own SPSS file using the values from the table above). I am just going to give you the basics here, but you can refer to other sources to figure out some of the info we get from the One Way ANOVA not covered in this lecture (like homogeneity of variance, Welch test, etc.).
SPSS – Our Textbook Money Recall Study
1. Click Analyze > Compare Means > One-Way ANOVA … on the top menu.
You will be presented with the following:
1. Put the “Condition (1 = High, 2 = Low, 3 = Control)” variable into the “Factor” box and the “How much participants recall spending” variable into the “Dependent List” box by highlighting the relevant variables and pressing the buttons. Note that SPSS uses different names for variables. It calls the dependent variable the “Dependent List” and it calls the independent variable the “Factor”. Just remember that our dependent variable (Factor) must be scaled in order to run this test (1 to 9, or 1 to 5, or even 0 to 100,000). The independent variable must be categorical (dressy v. sloppy v. casual, old v. middle aged v. young, republican v. democrat v. independent, high v. medium v. low, etc.).
1. Click the button and tick the “Tukey” checkbox as shown below:
Click the button.
1. Click the button. Tick the “Descriptive”, and “Means Plot” checkboxes in the Statistics area as shown below:
Click the button. Then click the button.
Output of the One Way ANOVA in SPSS
You will be presented with several tables containing all the data generated by the One Way ANOVA procedure in SPSS.
Descriptive Statistics Table
The descriptives table (see below) provides some very useful descriptive statistics including the mean, standard deviation and 95% confidence intervals for the dependent variable (How much participants recalled spending on textbooks) for each separate group (High Dollar Condition, Low Dollar Condition & Control) as well as when all groups are combined (Total). These figures are useful when you need to
describe
your data.
As you can see, we have 8 participants in the High Dollar Condition, 8 participants in the Low Dollar Condition, and 8 participants in the Control Condition. The mean for the High condition is $337.50 (SD = $37.80), the mean for the Low condition is $293.75 (SD = $34.62), and the mean for the Control condition is $306.25 (SD = $25.88). We can ignore the Std. Error, confidence interval, and minimum maximum for now, but we will need the means and SD information in our write up (below), so we’ll come back to this table.
The One Way ANOVA Table (ANOVA)
The One Way ANOVA table (see ANOVA table below) shows the output of the ANOVA analysis and whether we have a statistically significant difference between our group means. We can see that in this example the significance level is 0.042, which is below 0.05 and, therefore, there is a statistically significant difference in dollar amount recall between two or more of our three conditions. This is great to know, but the ANOVA does not tell us which of the three groups differed. Luckily, we can find this out in the Multiple Comparisons Table and the Homogenous Subsets tables, both of which contain the results of post-hoc tests.
Multiple Comparisons Table
From the results so far we know that there is a significant difference between at least two of our three means. The post hoc table (see below), or Multiple Comparisons table, shows which groups differed from each other. (Note: Hopefully the Tukey post-hoc test is familiar to you, but recognize that there are many different post hoc tests. We will focus exclusively on the Tukey test in this crash course). We can see from the “Multiple Comparisons” table below that there is a significant difference in the dollar amount recalled between the High Dollar Condition and the Low Dollar Condition (p = 0.039). However, there is no difference between the High Dollar Condition and the Control Condition (p = 0.17) and there is no difference between the Low Dollar Condition and the Control Condition (p = 0.73). If you find this table a bit “busy”, you and I agree. A lot of values are duplicated in this table. That’s why I prefer to use the Homogenous Subsets table instead.
Homogenous Subsets Table
Like the Multiple Comparisons Table, the Homogenous Subsets table uses Tukey to look at the differences between groups. However, the Homogenous Subsets table separates the conditions by comparing the groups and seeing if the mean for each group falls inside the same versus different subsets. If you look at the table below, you will see that there are two subsets in this table (Subset 1 and Subset 2). Focus on Subset 1 right now. Here, both the Low Dollar Condition (M = 293.75) and the Control Condition (M = 306.25) fall within that same subset. This means that the two conditions do not differ from each other. Similarly, looking at Subset 2, the Control Condition (M = 306.25) and the High Dollar Condition (M = 337.50) do not differ from each other, as both are in the same subset. The important comparison in this output is between Subset 1 and Subset 2. Here, the Low Dollar Condition and the High Dollar Condition are in different subsets, which means that they differ from each other. This table is a bit easier to understand as long as you recognize that when means fall in the same subset, they do not significantly differ from each other but when they fall in different subsets, they do differ significantly from each other.
Below are some examples of the Homogenous Subsets tables in which all means fall in the same Subset 1 (and thus no condition differs from the others, and the ANOVA is likely not significant either) or all means fall in different Subsets 1, 2, and 3 (and thus each condition differs from all of the other conditions):
Reporting the output of the One Way ANOVA
We report the statistics in this format: F(degrees of freedom[df]) = F-value, p = significance level. In our case this would be:
F(2, 21) = 3.70, p = 0.042, and our means/SDs would be (M = 337.50, SD = 37.80) for the High Dollar condition, (M = 293.75, SD = 34.62) for the Low Dollar Condition, and (
M = 306.25, SD = 25.88) for the Control condition. Just recall that only the High and Low Dollar Conditions differed from each other; neither differed from the Control Condition. We would report the results of the study as follows:
We ran a One Way ANOVA with Dollar Condition (High vs Low vs Control) as our independent variable and the amount of money participants recalled seeing as our dependent variable. The One Way ANOVA was significant, F(2, 21) = 3.70, p = .042. Tukey post hoc tests revealed that participants recalled spending significantly more money on textbooks in the High Dollar Condition (M = $337.50, SD = $37.80) than participants in the Low Dollar Condition (M = $293.75, SD = $34.62). However, Control Condition participants (M = $306.25, SD = $25.88) did not differ in their recall from either High or Low Dollar Condition participants.
That’s it! Not too hard, right? Note that I provided means and standard deviations for each of our three conditions – Thus I expect to see means for all conditions in your papers as well! Also note that p = .042. We no longer use p < .05. The only time we use < is when our p value is .000 or less. In that rare instance, we use p < .001. Otherwise use the equal sign (=).
Remember some other basics here: we use a One Way ANOVA to look at the differences between three or more means to see if the means differ significantly. If they do not differ, then there is no need for post hoc tests. If they do differ, we need post hoc tests. Thus we need three SPSS tables for a significant ANOVA: the descriptive statistics tables, the One Way ANOVA table, and the post hoc table.
So what does the write-up look like for a non-significant ANOVA? Let’s see:
We ran a One Way ANOVA with Dollar Condition (High vs Low vs Control) as our independent variable and the amount of money participants recalled seeing as our dependent variable. The One Way ANOVA was not significant, F(2, 21) = 1.70, p = .152. Participants recalled spending similar amounts of money on textbooks in the High Dollar Condition (M = $337.50, SD = $37.80), the Low Dollar Condition (M = $293.75, SD = $34.62), and the Control Condition (M = $306.25, SD = $25.88).
Finally, I wanted to show you the means plot (below), just to give you a visual idea about the results for our significant ANOVA
Means Plot – Dollar Condition (IV) and How Much Money Participants Recalled Spending (DV)
Crash Course In Statistics – The One Way ANOVA – Quiz #3 (Filters, Spring 2022)
Instructions: Recall the “Instagram Filters” study from your t-Test crash course quiz. Here, we will add another condition to that study so that we can use a One-Way ANOVA.
The phrase “Don’t judge a book by its cover” might be good advice, but it may not always translate well when put into practice. In fact, insecurity runs deep when it comes to social media, especially when it comes to users wanting to post that “perfect picture” to their Snapchat, Instagram, or Facebook account. As a result, more social media users are using filters to alter or adjust that not-quite-right picture of theirs, thereby providing (they hope) a more favorable glimpse into their lives than reality would allow. But what if it becomes apparent to others that the “perfect picture” someone posted is heavily filtered? How do people perceive users who choose to represent themselves using altered pictures rather than true pictures?
Imagine we run study where participants view an Instagram update from Katie, a user who posts a picture of herself on social media with the tagline “I feel great after my workout”. The picture she chooses to post is below (Picture A). Sarah—one of Katie’s friends—responds, writing: “Katie, I really love seeing your pictures. You’re the queen of social media! You always look so great that I can’t wait to share our pics! So, to all my friends (and Katie), I hope you like this!”
In your t-Test crash course, Sarah provided one of two different pictures: a repost of Katie’s original picture or a new “unfiltered” picture. For the ANOVA crash course, the study authors want to add a third condition. Although they still want to see if the alternative “Unfiltered Picture” posted by Sarah impacts participant perceptions of Katie’s insecurity, the authors want to rule out the possibility that any alternative picture impacts perceptions of Katie. They decide to add a third picture that has no connection to Katie (a humorous but unrelated picture of a male bodybuilder).
In this new study, all study participants see the original Filtered Picture posted by Katie. They then see one of the following three follow-up pictures posted by Sarah:
A. Filtered Picture: Sarah reposts Katie’s original picture, so participants see Picture A twice
B. Unfiltered Picture: Sarah posts a new, unfiltered picture of Katie (Picture B)
C. Neutral Picture: Sarah posts an unrelated picture (An older male bodybuilder – Picture C)
Participants then rate their impressions of Katie’s insecurity. The specific question they answer is, “How insecure is Katie?” on a 7-point scale (1 = Not at all insecure; 7 = Very insecure). The authors predict that participants will find Katie more insecure if Sarah posts the new “Unfiltered Picture” of Katie (which Katie probably would have posted herself if she wasn’t so insecure) than if Sarah reposts either the original “Filtered Picture” or posts the new unrelated “Neutral Picture”. The authors do not expect any differences between the Filtered and Neutral Picture conditions, since neither provide a different version of Katie.
Complete the questions below and then transfer those answers to your Crash Course in Statistics – The One-Way ANOVA Quiz #3 in Canvas (1 point per question). IMPORTANT: The answer options in Canvas may not be in the same order you see them below, so make sure to copy over the CONTENT of the answer and not simply the answer letter (A, B, C, D, or E). Note: If you want to run these analyses yourself, look for the SPSS file called “#3 ANOVA Crash Course Data Filter Spring” in Canvas – not required, but definitely recommended!)
1). What is the independent variable in this study?
A. Condition: Original Filtered Picture vs. Unfiltered Picture vs. Neutral Picture
B. Condition: Original Filtered Picture vs. Neutral Picture
C. Ratings of “How insecure is Katie?” on a 1 (Very insecure) to 7 (Very secure) scale.
D. Ratings of “How insecure is Katie?” on a 1 (Not at all insecure) to 7 (Very insecure) scale.
E. There is too little information in this study to determine the independent variable.
2). What is the dependent variable in this study?
A. Condition: Original Filtered Picture vs. Unfiltered Picture vs. Neutral Picture
B. Condition: Original Filtered Picture vs. Neutral Picture
C. Ratings of “How insecure is Katie?” on a 1 (Very insecure) to 7 (Very secure) scale.
D. Ratings of “How insecure is Katie?” on a 1 (Not at all insecure) to 7 (Very insecure) scale.
E. There is too little information in this study to determine the dependent variable.
You run a One Way AVOVA on this data set and get the following SPSS output.
3). What are the correct means and standard deviations for the conditions in this study? Round to two decimal places
A. Filtered (M = 3.83, SD = 0.99); Unfiltered (M = 4.33, SD = 0.76); Neutral (M = 4.63, SD = 1.38)
B. Filtered (M = 3.90, SD = 0.21); Unfiltered (M = 3.83, SD = 0.25); Neutral (M = 4.63, SD = 0.18)
C. Filtered (M = 4.93, SD = 0.91); Unfiltered (M = 3.83, SD = 0.99); Neutral (M = 3.90, SD = 1.12)
D. Filtered (M = 4.63, SD = 1.38); Unfiltered (M = 3.90, SD = 1.12); Neutral (M = 3.77, SD = 0.90)
E. Filtered (M = 3.90, SD = 1.16); Unfiltered (M = 4.63, SD = 1.38); Neutral (M = 3.83, SD = 0.99)
4). Is the One-Way ANOVA significant?
A. It is not significant F(2, 87) = 4.22, p = .18
B. It is not significant, F(2, 89) = 5.91, p = .18
C. It is significant, F(2, 87) = 4.22, p = .001
D. It is significant, F(2, 87) = 4.22, p = .018
E. It is significant, F(2, 89) = 5.91, p = .018
5). Finally, which of the following would you use to write out the results in an APA formatted results section? Note that this one is tricky – some answer options differ in only a single number or word! Pay close attention to details here.
A. We ran a One-Way ANOVA with picture condition (Filtered vs. Unfiltered vs. Neutral) as our independent variable and participant ratings of “How insecure is Katie?” as our dependent variable. The One Way ANOVA was not significant, F(2, 89) = 4.22, p = .18. Participant ratings of Katie’s insecurity did not differ between the Unfiltered Picture condition (M = 4.63, SD = 1.38), the Filtered Picture condition (M = 3.90, SD = 1.16), and the Neutral Picture condition (M = 3.83, SD = 0.99).
B. We ran a One-Way ANOVA with picture condition (Filtered vs. Unfiltered vs. Neutral) as our independent variable and participant ratings of “How insecure is Katie?” as our dependent variable. The One Way ANOVA was not significant, F(2, 87) = 4.22, p = .18. Participant ratings of Katie’s insecurity did not differ between the Unfiltered Picture condition (M = 4.63, SD = 1.38), the Filtered Picture condition (M = 3.90, SD = 1.16), and the Neutral Picture condition (M = 3.83, SD = 0.99).
C. We ran a One-Way ANOVA with picture condition (Filtered vs. Unfiltered vs. Neutral) as our independent variable and participant ratings of “How insecure is Katie?” as our dependent variable. The One Way ANOVA was significant, F(2, 87) = 4.22, p = .018. Tukey post hoc tests showed that participants felt Katie was more insecure in the Unfiltered Picture condition (M = 4.63, SD = 1.38) than in both the Filtered Picture condition (M = 3.90, SD = 1.16) and the Neutral Picture condition (M = 3.83, SD = 0.99). Participants also thought Katie was more insecure in the Filtered Picture condition than in the Neutral picture condition.
D. We ran a One-Way ANOVA with picture condition (Filtered vs. Unfiltered vs. Neutral) as our independent variable and participant ratings of “How insecure is Katie?” as our dependent variable. The One Way ANOVA was significant, F(2, 87) = 4.22, p = .018. Tukey post hoc tests showed that participants felt Katie was more insecure in the Unfiltered Picture condition (M = 4.63, SD = 1.38) than in both the Filtered Picture condition (M = 3.90, SD = 1.16) and the Neutral Picture condition (M = 3.83, SD = 0.99). Ratings did not differ between the Filtered and Neutral Picture conditions.
E. We ran a One-Way ANOVA with picture condition (Filtered vs. Unfiltered vs. Neutral) as our independent variable and participant ratings of “How insecure is Katie?” as our dependent variable. The One Way ANOVA was significant, F(2, 89) = 4.22, p = .018. Tukey post hoc tests showed that participants felt Katie was more insecure in the Unfiltered Picture condition (M = 4.63, SD = 1.38) than in both the Filtered Picture condition (M = 3.90, SD = 1.16) and the Neutral Picture condition (M = 3.83, SD = 0.99). Ratings did not differ between the Filtered and Neutral Picture conditions.
