Should I have been concerned when my ex talked about having a foursome with our close friends while we were still in a relationship?

We have broken up for nearly a year but my mind keeps on racking up and revealing scenarios and situations where I should’ve been concerned about the relationship.
This was 1 and 1/2 years ago and my ex (X) and I were at a birthday party at my close friend’s house (let’s name her Y) and a bunch of our mutual friends were there including Y’s recent ex (let’s name him Z).

(for context: X, Y and Z are Pan and I’m Gay)

Befor the party, I told X that I’m going to leave at 5pm (party starts at 12) cuz I have work and couldn’t get time off but I’ll comeback to the party at 10 if its still going on. Things were okay at the party, everyone was vibing and the mutual friends were coming at varying times. When I left for work I thought about X all day and couldn’t wait to either call him aftery shift was over or go back to the party if it was still on to see him.

Time passes and I have finished my shift and left my workplace. Normally I keep my phone on silent and turn my Mobile Data off because I don’t want to waste my data plan. When I was on the bus back to the town centre to get another bus, I turned my Mobile Data on and a bunch of notification flooded in while I was offline (typical game notifications, weather app, etc) and I got a text from X, along the lines of
“Hey so, me Y and Z have been talking and I was wondering if you could be down to a foursome with all of us 4? We’ll talk later, the party is gonna stop at midnight so you can come over to talk it with us”

I was kinda surprised by the text, in that situation, and naturally I said I will talk about it and it would most likely be a yes (me and X have already talks about how we found Y and Z to be attractive and would have sex with them but only in a sexual context not emotional, I have talked about Z and X has talked about both Y and Z so this foursome was a moment of stars aligning)

When I got back to Y’s house, I got pulled aside by Y who asked if it was okay with me to allow the foursome to happen (she emphasised that it won’t be at the party but established at a later date) and I said yes and said that I do not wish to “get it on” with Y due to my sexuality and Y understood.

The foursome never happened, when X broke up with me, I just tried to remember the good times I had with X but to be honest it was hard to, all of the memories I can remember was of me being sad, disappointed, let down, angry and mostly feeling betrayed by X. Now I’ve come to terms with most of the things that happened with my relationship with X but this situation stumped me.

I had a huge suspicion that X had a fancy for Y, even while I was with X, and the reason why we broke up was because he didn’t love me anymore and admitted his feelings about Y being more than sexual. I felt betrayed but weeks after the breakup I have tried to remember, how far back did he start having these feelings while we were togsther? Especially, why and who put up the topic of having sex with each other while I was at work?

The suspicions I had initially felt unreasonable, when X and Y first met they hit it off and I was already close friends with Y so it made sense. But I became distant with Y because of how disrespectful she is about certain things, belligerent, opinionated and doesn’t respect certain boundaries of other people but I was quiet because X liked Y ‘as a friend’. But X started to hang out with Y more often while we were together, starting to hang out with her more than me, ignoring me most of the time when me and Y are in the same room.

Looking back I felt like I should’ve been concerned with what happened in that moment to suspect even further of X’s betrayal. Should I have been worried?

Note: I understand that this is the past and I should stop thinking of these things but this bastard of a boyfriend betrayed me and lied to me about not having feelings with this piece of shit of a human, my feelings are not out of curiosity or clarity, it’s out of adding another topic to the list of douche bag style fuck nuggets that came out of his tiny brain. I’m glad we broke up but this shit still hurts and I just want clarification from a neutral part of Reddit.