It is the short form for "of the" clock. This is what I ever thought about it. And yes, I was right: I found this and a second explanation on the internet:
o'clock [1]
a contraction of the phrase "of the clock," used in English after a statement of time. This phrase has been traced to the early 1400s at least; it is fairly common in the works of Shakespeare. Earlier, time was usually stated in hours and minutes, and this is still the case in most languages. Thus "10 o'clock" is "10 hours" in most of the world.
o'clock [2]
an informal angular measure that works by describing an angle in terms of the face of a standard (12-hour) clock. Each hour "o'clock" spans an angle of 30°, so "4 o'clock" means an angle of 120° measured clockwise from dead ahead or some other agreed-upon point of reference.
from
http://www.unc.edu/~rowlett/units/dictO.html
