I may not use content in the same way, i use the editor mainly for text based articles with the odd image here and there, but i don’t have any spacing issues, and i do switch between HTML and Visual several times during writing.
I use WordPress.com for my blog, which also runs WP 3.x code (same code as i run locally), where i also do not have these problems.
You probably have a much wider use for the editor overall than i do, so it doesn’t surpise me you have some problems with the editor (it’s not perfect – no such thing).
If you’d like to give me some sample content to plonk into a post that i can use to replicate the issue described i’ll happily go test it for you, and if i can replicate the problem i’ll see if i can find a work-around (no promises, but i can give it a shot).