Your issue is you have two boxes, female and male, when intersex conditions are the result of sex being a spectrum. Intersex conditions can happen in a multitude of ways, and many are not very outwardly detectable.

Masulinization and feminization is a complicated and messy process which results in people with sexual characteristics outside the binary, and sometimes this means that people are born infertile or less fertile, which invalidates your point. Biologically it’s asinine to say that bio sex is binary.

You say if someone belongs to the sex which can have babies then they are biologically women. How do you define if someone belongs to the sex which can have babies? Your definition doesn’t describe this, you just arbitrarily put someone into the woman box.