Well you said:
In general, he gets along better with women.

You guys spend a lot of time together so some individual time is alright.

I think you are overreacting here given what info you have given. May be plan weekends with him, and see who he picks. If he picks her, it is time you and him sit down for a very honest conversation, and you tell him that he needs to prioritize you or risk losing you.