Isn't it nice when someone else has already written, quite eloquently, what you were thinking?

Well, maybe sometimes it is frustrating, but I stumbled upon this today and it states things very nicely:

From the fact that people are very different it follows that, if we treat them equally, the result must be Inequality in their actual position,, and that the only way to place them in an equal position would be to treat them differently. Equality before the law and material equality are therefore not only different but are in conflict with each other; and we can achieve either the one or the other, but not both at the same time. The equality before the law which freedom requires leads to material inequality. Our argument will be that, though where the state must use coercion for other reasons, it should treat all people alike, the desire of making people more alike in their condition cannot be accepted in a free society as a justification for further and discriminatory coercion.

We do not object to equality as such. It merely happens to be the case that a demand for equality is the professed motive of most of those who desire to impose upon society a preconceived pattern of distribution. Our objection is against all attempts to impress upon society a deliberately chosen pattern of distribu­tion, whether it be an order of equality or of inequality.

--F. A. Hayek