As one who wants to start with friendship before getting into dating, I have to wonder if attraction might have grown if given just a bit more time.
Immediate attraction, I think, has a lot to do with past associations. We decide we only like someone who fits our "type," which may be based on long-forgotten people we admired as children.
Please don't take me wrong--I'm not questioning your choices. No one should have to justify decisions in their personal life. But I think the way we do the dating thing puts too much weight on superficial qualities and not enough on building friendships with potential to grow into more.