It’s better to be upfront with someone instead of driving yourself up the wall with “what ifs”. If I were in his shoes, I’d want to know what’s bothering you and communicate.