collapsing margins/... and why they can be such a pain
Truly understanding css is a rather long and difficult process. Also, getting something to work in css is different from fully understanding why it works. For me personally, the magic of the collapsing margin has always been one of the foggiest css areas around. So let's take a closer look at this murky little css behavior, and more importantly, why it causes damage to my hair from time to time.
It's not too big a deal if you don't understand their purpose, as you can work around them quite easily, but collapsing margins are there for a reason. The actual theory behind collapsing margins is a little too complicated to summarize, so for their precise behavior I'll just redirect you to the full w3 explanation of collapsing margins. In short though, I'll quote their definition of the general behavior of collapsing margins:
quote collapsing margins means that adjoining margins (no non-empty content, padding or border areas or clearance separate them) of two or more boxes (which may be next to one another or nested) combine to form a single margin. unquote
Even shorter, when two consecutive elements have a vertical margin defined, those margins will overlap whenever they border on each other.
everything has a reason
p {margin:1em;}
At first this might sound like strange behavior, but there is actually a good reason for collapsing margins, and I gladly made use of it in my quest for finding the differences between margins and paddings. In its most simple form, collapsing margins were conceived to control the spacing between paragraphs of text. If you consider the css above without collapsing margins, this rule would result in 2em spacing between each paragraph. Something you wouldn't exactly want or expect from a rule like the one above.
So to fix this problem, the vertical margins were made to collapse, forming a gap of 1em between each paragraph. Which is pretty neat actually. But they didn't stop there.
A similar problem arose when paragraphs were dropped in a li element. Of course you'd want the first paragraph to line up with the top of
the list element, so rather than keep collapsing margins restricted to siblings, they expanded the behavior to adjoining elements (fe, the parent and its first child).
If you want some visuals with that, you can consult Meyer's article on the subject,
which also gives a broader explanation of the problem.
breaking the collapse
Especially the second form of collapsing margins can be troublesome. Depending on the styling of the element, you'd want this type of collapsing to happen. The good people who wrote up the css rules realized this and added some rules to break the collapse. You could either use a padding or a border (so the margins wouldn't join anymore). Both cases are related to some sort of visual separation of the element. For example, if you give the list element a border, you'd want the elements inside to keep their distance from that border, increasing readability.
What they failed to fix is the case where the containing element is simply given a different background style. You still create the visual effect of separation, but your margins will fail to collapse, placing the first sibling against the top of the container, creating a cropped and ugly layout.
weighing the solutions
Breaking the collapse through paddings or borders is a bad idea. For one, you add at least 1 pixel to the separation of content and border, which might clash with the layout of elements above or below. And yeah, it's only 1 pixel, but I'd rather not have that difference. On top of that, I don't want to be adding paddings where they are not needed at all. And furthermore, applying a border means setting it to the same background color of the parent. If the color changes, two values have to be adapted, hurting flexibility of the css and introducing a bigger chance of errors.
One solution not described in the Meyer article is setting an overflow:hidden to the parent, but this only works if you don't need to
place something outside the container. Also, I'm not too fond of adding an unneeded behavior to an element just to fix an unrelated problem. And again, it hurts
the adaptivity of the css to changes further on in the development process.
This leaves us with no perfect solution, which is a real shame.
what we need
It would be helpful to check for different background settings between adjoining elements, but that sounds like an extremely fickle solution to me (though I am sure that would fix a whole lot of problems already). I'm not sure this will fix all occasions where collapsing margins are hurtful.
Better would be to give us, css people, control over collapsing margins. Give us a property to leave or break the collapse. That way this becomes our responsibility, rather than our problem.

Comments
Five Minute Argument
I seem to have written a lot about collapsing margins: outside of the most simple case, there are an awful lot of nuances and complexities. A solution to "applying a border means setting it to the same background color of the parent" is simply to set the border's color to 'transparent', but I agree with your general point: however the collapse can be 'broken', it will always be a hack until we have a real method of controlling it. Tables already support the 'border-collapse' property, so a 'margin-collapse' property seems perfectly reasonable.
Niels Matthijs
True that, though it still leaves the issue with the unwanted pixel (and the fact that it's just plain dirty) :)
Five Minute Argument
Yeah, I started thinking about the unwanted pixel and concluded it could probably be solved with a 1px negative margin, for those browsers that support it properly (i.e. not IE, last time I looked). But I quickly abandoned that line of thinking because, yes, hack upon hack is not the way this should be solved :-)
James Elmore
Long ago (it looks to be almost two years), I wrote in this list that TABLE elements had some controls which should also be available for block elements. Two of those suggestions were titled "Block Model Additions: Border Overlap" and "Block Model Additions: Margin Collapse". If you can't find the originals in the archives, I can send you my original emails. It seems a shame to me that, in order to control collapsing margins and border overlaps, a CSS designer needs to use a table element.
In MOST cases, the current margin collapse works wonderfully, but there are times when the designer should have more control. For margins, this control should allow the exclusion of margin collapse between element. (This probably should be controllable for each edge of the block -- top, bottom, right, left -- and for combinations -- line order (left to right in English) and block order (top to bottom in English). The defaults should stay as they currently exist, for minimum 'excitement' when things don't work as expected.
Good luck with your proposal.
Niels Matthijs
I don't agree there. I switched from using paddings to margins from primary structuring and I run into issues with collapsing margins all the time. Just look at this blog. No borders, but plenty of color differences to separate elements. Collapsing margins are quite an issue for me :)
Anyway, on the w3 list David Hyatt mentions that WebKit has these features already build in, and if there's enough interest the proposal will be renewed. So crossing my fingers here.
Thierry Koblentz
If I recall, IE 6 does not "do" transparent borders - it turns them black.
Michelangelo
Is it just me or the choice of image for this article is in poor taste?
Nike
@Michelangelo I thought so at first too, but then i realized it was a collapsed building.
By the way, i really liked this post. Very well written! To bad there ain't no solution for it :(
Chris
I'm quite new with CSS. It might be a crap, but floating each elements in the container solved this problem for me.
elixon
What about: "padding: 0.1px;"? It will not add a 1px and yet it will do a job.