According to Journey Through Genius, in a letter to Leibniz Newton wrote the expression given in the first red box above. It's an expansion, not just for the binomial (a + b)2, but for any power m/n.
Each of the coefficients A, B, C, D and so on consists of the entire previous term, as indicated by the definitions in the two blue boxes in the second group.
Pm/n can be factored out of each term on both sides, so we do that, yielding the expression in the second red box. The coefficients have been expanded to show the increasing powers of Q. Also each of the fractions beyond m/n can be rearranged as indicated in the bottom group. The final form isn't shown explicitly, but if we let e = m/n, we see that the coefficient for Q4 has the form:
matching the standard form. It is trivial to verify the results for (1 + x)3, so let's think about something more interesting namely,
substituting for m/n = -3 we obtain:
Expanding:
and moving it to the right side we get:
which can be checked by multiplying out to see that the rhs does equal 1.
In the same way, we can work with
We have Q = -x and m/n = 1/2 so that the expansion is:
Newton used this method to calculate roots of various kinds, e.g.
and expand as before with Q = -2/9.
As Dunham says:
In one sense, there is nothing terribly surprising about the fact that √7 can be approximated by a sum of six fractions. The truly amazing thing about this whole procedure is that Newton's binomial theorem shows us precisely which fractions to use, and generates them in an utterly mechanical fashion, devoid of the need for any particular insight or ingenuity on our part. It was a remarkably efficient and clever way to get roots of any order.
Next: an approximation for pi.