Freudian psychology would argue that men (and women) are attracted to breasts not out of sexual impulse, but because of their biological function. They are a source of nutrition and nourishment during our formative years and thus bring us joy and comfort. It's not necessary to interpret those emotions in a sexual context. On the other hand, genitalia simply serves as an access point for procreation. We're programmed to be sexually drawn to them, but other than that they're just bits of anatomy.
The concept here has the same foundations as for why women who are in Girl Scouts throughout their young lives are more likely to be attracted to women later in life than those who aren't in Girl Scouts: fellow women represent a closer degree of friendship and comfort for them. It isn't necessarily their sexuality that's triggering the attraction: it's the impulsive COMFORT they feel.
And there we go; DancingTofu objectifies and analyzes a discussion and now it's no longer creepy - just boring. 
Educationally boring, which makes the patient ones more wise.
In my case, I don't feel comfortable expressing how I feel about matters not meant for younger eyes to read. Otherwise to them, it does sound creepy, and makes me feel like some filthy thing that I know deep down inside I'm not.
Bringing up another related subject...
How do you know if when you're pushing the edge in a topic, that you've gone too far and how do you recover from it. Say you hear friends talking about something and you want in. Yet when you say something, they all stop and look at you like you're literally a creepy alien to them. How do you recover from that, and what can you do different when the opportunity comes up again?
I know the spoon castle isn't social tactics 101, but I know some including myself may have ran into this situation. (v_v) *sigh*
