Am I being manipulative by whitholding sex from my boyfriend?

We seem to have gotten into the pattern of me bringing up an issue and him not wanting to address it and blowing me off. Later when he initiates sex I give in because I’ve always been told it’s not right to whithold sex. like a year ago my sex drive tanked and so did his self esteem. I’ve made a genuine effort to be intimate with him since then but recently I’ve felt unappreciated and like he doesn’t actually try to understand me. I feel like it’s my own fault he doesn’t take my feelings seriously because every time we get into an argument about my feelings he blows me off and then we have sex so he thinks that the issue wasn’t real because it wasn’t bad enough to affect our sex life. Well….. we’re past that now. I went camping for 2 days in another state and I didn’t even get so much as a text from him. He didn’t open my messages either. He said he was busy but when I called him he was on the game. I told him before I left that I didn’t feel appreciated and that I feel like a burden even though I did basically all of the work for buying our house. I guess I expected him to take me seriously but he didn’t. I told him that it was upsetting not hearing from him and, like usual, he blew me off. I don’t want to whithold sex from him. I love him. But I’m so sad. I try to masturbate and it just doesn’t work. I’m not trying to punish him but how am I supposed to preform like this? I’m afraid doing this might end us and leave us with a very awkward living situation but I can’t sleep with someone when there’s not a connection and if he doesn’t even try to rebuild that connection I don’t know what else to do