Yeah, this is a bit delicate but here we go. I suspect anyone who dated transgender folk is probably better positioned to answer this question than any of the transgender people here.
You know, a woman is not just physical appearance, there are also detaqils like the voice and scent too. It's all part of the allure, of the physical attraction. With scent, I am not talking about perfume here, but natural body odor. There's a big difference between men and women here and if you're a straight male, a musky, manly scent , even your own, is kinda repulsive. I'm not sure how effective hormone therapy is for transgender people, but I do wonder if it's enough to make this little detail just right.