26F & 29M how much longer can I deal with a dysfunctional family?

TL;DR: my bf is rude af to me all the time but not to his parents??

hi I’ll keep it short. I’ve been with my bf for 3 years and I mentioned kids and marriage just like hey is it coming soon or are u just not into that idea at all? He said how annoying it is for women to constantly beg about a piece of paper and how kids are expensive. Ever since then I started to detach from him bc it hurt my feelings. His family has 2 drug addicts (sister and brother) and his parents call my bf to complain but then they go and give them money and pay for their shit. They use him as a person to vent to but he complains to me about and I tell him why don’t u just say something to them? Like get mad at them? He just sits there and takes it. It’s annoying bc that’s all they talk about and I just simply stop caring about it all. The thing is that when he gets mad at me he yells and throws me against the wall and he’ll laugh but won’t stand up to his parents. I’ve just started to detach from him bc I think I’m ready to break up but he won’t let me? I guess Im just asking for some advice or just some nice words idk I don’t have anyone to talk to.