I don’t mean this disparagingly or condescendingly, but this isn’t complicated. It’s one of those situations where you have physical chemistry but he is not interested in putting in the effort for anything more. He doesn’t want to be your boyfriend. Texting is low effort. Coming over for sex is low effort. I get that with the chatting and the drop-offs it feels like mixed messages, but it really isn’t. He’s just interested in casual sex with you.
You could hook up and text with him if you can deal with it only being that. If you can’t deal, you should cut off contact.