Feeling hurt each time my bf (22m)refused to have sex w me(21 f)

22F here, feel bad each time when my bf(21M) refuses to have sex with me for using following reasons:
1. He’s tired. /He’s too high
2. He’s not into it rn
3. He wants to, but he has to wake up early tmr
4. He doesn’t have the energy

….
So my bf works a lot during the week, and we will have sex 1-2 times on the weekend. But 99% of the time has the initial intercourse(like starting to kiss me suddenly when I am sleeping just because he can’t sleep), and he never asked if I was in the mood (ok because we have sex for a maximum of two times a week or even two week ofc I want sex). However, if I initiate it, like kissing him or touching him, he will not stop me. Still, he will refuse me using several reasons(even though he’s hard already), which makes me upset and feel unattractive. I like my bf a lot, and I know he’s a very nice guy, but I think this will hurt my feelings each time and make me gradually end up losing interest in him (although I have my vibrator ). I am also confused because we are both in our early 20s. I just don’t know how to communicate with him about this, and my current plan is that I will refuse him the next time he initiates sex.