If you don’t want to deal with drama, then worry about yourself and not this other girlfriend/roommate and her finances. If there are now five of you living in the apartment, and your share of the rent is 1/5 or whatever has been determined, then just make sure you pay your portion each month — nothing more and nothing less and let the roommate who covered for the girlfriend (from your letter it’s unclear if that person is her boyfriend or the other roommate who initially asked that you ladies start pitching in) worry about where the final fifth of the month’s rent is coming from. If he wants to keep covering for her, let him. It’s no skin off your nose, is it?
If, however, this particular roommate keeps acting like a “weirdo” or expects you and everyone else to cover for this chick who can’t bother getting a job, move out. Your name isn’t on the lease, is it? You said you only moved in “semi-permanently” anyway, so if the situation isn’t working for you, MOA. No one’s forcing you to stay there. If it’s too much drama, get your own place with your own lease and your own roommates (or none at all). If you don’t want to do that or if you want to wait until your boyfriend is able to move out with you, then suck it up, confront all the roommates together and state your case. If you know the people well enough to share a roof with them, you know them well enough to confront them about how the rent’s being divided and paid each month. It’s called being a grown-up. Paying to crash at a place on a “semi-permanent” basis is only the start of it. The next step is standing up for yourself and learning to communicate even when it feels awkward and uncomfortable.
*If you have a relationship/dating question I can help answer, send me your letters at firstname.lastname@example.org and be sure to follow me on Twitter.