I don't think there is an argument that males are physically superior to women. There are exceptions to every rule like Rousey being able to kick my ass in a second but in general, there is a reason why all the swim records, track records, weight-lifting records, and so on all belong to men. In many cases the times aren't even close.
Let's say you did combine all men and women in track and field, no woman would ever be able to compete again on the Olympic stage, because no woman would have a time that would qualify for competition. Removing the gender distinction effectively takes all women's sports and recognition away from them.