How do you make someone leave you alone and forget about you without seeming to be rude?

I am forced to be around people (work colleagues) whose only form of socialisation is to talk about other people.

I am not someone who is interested in talking to other people, let alone someone who is interested in gossiping about others but I am often used as someone who people vent to about other people, mostly because I don’t speak much so the stuff they tell me is in no danger of being passed around. It is not like they want to listen to my perspective (which I would find difficult to provide as I don’t care about what they are telling me), they literally just vent to me. Sometimes they actively drag me into their conflicts.

I have always tried to deliberately come across as unsociable but some people don’t seem to get the hint that I want to be left alone and are always bothering me. Some of them I can tell are choosing not to respect my boundaries.

I get them randomly calling me and wasting hours of my time. I am obviously serving some type of role in their lives as a confidante, but I didn’t sign up for this and I am now looking for a way out as my tolerance threshold is decreasing after having put up with this nonsense for years.

I want to go about this gently as I don’t want to appear rude or burn bridges. Ghosting is what I would usually do when someone becomes too bothersome, but this is really not an option here and I can’t look for a new job yet due to health issues.