This was symbolized by the sphere and cylinder on his tombstone, as witnessed (years later) by Cicero. We have no idea what Archimedes looked like, but that doesn't keep people from drawing his portrait!
What strikes me most vividly about the discovery is that Archimedes found the correct relationship by experiment---he weighed the solids, and not only that, he used the law of the lever. According to this page, the balancing was done in a fairly complex manner. We have a cylinder that can just contain the sphere and a cone whose radius and height are equal and twice the radius of the sphere. Moreover, the density of the cylinder is four times that of the other two objects.
Then, by the law of the lever, the weight of the cylinder is twice the combined weights of the sphere and the cone together (an equal force from gravity when suspended at half the distance from the fulcrum). Because the density of the cylinder is 4 times greater, its volume must be also one-half the combined volumes of the sphere and the cone.
We can check that using the (now) known formulas (see my post about the cone):
According to Archimedes in the Method (translation by Heath):
"For certain things which first became clear to me by a mechanical method had afterward to be demonstrated by geometry...it is of course easier, when we have previously acquired by the method some knowledge of questions, to supply the proof than it is to find the proof without any previous knowledge. This is a reason why, in the case of the theorems the proof of which Eudoxus was the first to discover, namely, that the cone is a third part of the cylinder, and the pyramid a third part of the prism, having the same base and equal height, we should give no small share of the credit to Democritus, who was the first to assert this truth...though he did not prove it."
Once he knew the correct answer, he was able to find his way to a rigorous derivation. Very smart.
There two things that make me wonder about the story. One is: why not just weigh objects of equal density like this:
We get 4/3 for the sphere,
The second question is: what materials would he use? What has a density 4 times something else? From wikianswers we have:
How about marble and silver?
[UPDATE: Almost two years later, I find a silly error in this post. You'd need two cones, or put the one out at 2x the distance on the lever. Why didn't someone tell me?]