Time to give him the flick. You are the one doing all the emotional labour here - he’s coasting and exploiting your anxieties. He has time to goof off with mates, but can’t spend 15 seconds texting you. Time to call it quits and move on.