This is an introductory post on ANOVA (analysis of variance). We ask the question: given three (or more) groups of observations, is one or more of the group means significantly different from the others. We will compute an F-statistic, and compare that with an F-distribution (carry out an F-test). If the statistic exceeds the 95% quantile, we will reject the null hypothesis that the means are the same.
According to wikipedia:
the two-group case can be covered by a t-test (Gosset, 1908). When there are only two means to compare, the t-test and the ANOVA F-test are equivalent; the relation between ANOVA and t is given by F = t2.
ANOVA is a versatile (and complex) set of methods. This is just an elementary application, where we'll use the R implementation on three simple groups of data, and then compute the result ourselves in Python to see how it works internally.
To begin with, we follow the simple example from Dalgaard. You will need R and the ISwR package (or just construct the "data frame" yourself). R code:
We have 22 values in 3 groups.
attachgives us access to the names of the columns of values (folate) and factors (ventilation). We make a plot (shown at the top of the post):
The value 0.04359 indicates that we have P < 0.05.
We write the data to disk, and remember that the groups have (respectively) 8, 9 and 5 values.
We use the Python script below to compute the F-statistic:
If you look in the R output above, you'll see the same values as given here for MS_W and MS_X and the F-statistic.
To get the right F-distribution, we need to know that the degrees of freedom are:
Since 3.71 is just higher than the 95% quantile of this F-distribution we can reject the null hypothesis H0.
I found a calculator online. You can see the results in the screenshot.
The underlying calculation is pretty simple. We compute
sumsq, the sum of the squares of the differences from the mean for several sets of values and the relevant means.
For the within groups comparisons, using mathematical notation this is (i groups with j observations in each group):
In Python, for each group we
sumsqfor the group compared with the group mean, and add the results for all three groups to give
To carry out the between groups comparisons, we first compute the grand mean
m(of all of the samples). Then for each group we compute:
Since the squared value is the same within each group, this is equivalent to:
In the Python code this becomes:
and sum these over all the groups to give
SSD_X. This is a sum of squares of the group means.
Finally, we compute:
As to why we do this, for now you will have to go read the article. The explanation in Dalgaard is particularly clear, indeed, the whole book is excellent.
There is a SciPy function to carry out ANOVA (
stats.f_oneway), but I don't have SciPy installed right now at home, and this post is long enough already. That's for another day.