I’m 32 6’2″ and 165lbs

Someone made a comment to me saying that I looked like I was losing weight, which kind of bothered me even as an adult and it feels silly. I’ve always been a slim person. I’m tall(ish), have very low body fat, and I’m muscular but not big. I’ve always gone to the gym although not as much as I used to, but at least 3 times a week. Here’s the thing, when I want to gain weight, I definitely can. Last summer I got up to 190lbs which was the heaviest I’ve ever been. But, I had to definitely force myself to eat more than I was comfortable with to achieve that weight. I ultimately decided it wasn’t worth it to force myself to eat and constantly feel full. So i went back to only eating when I’m hungry (shocker) and I’ve returned to my normal baseline of approx 165lbs. If I don’t deliberately try to gain weight, I stay in the mid to low 160s. It’s been this way ever since I was in highschool. It’s funny because I’ve always been called skinny but even as an adult it kind of bothers me if someone points it out. I’m not a weak/scrawny skinny, I’m athletic and have always participated in weight training and sports. Has anyone else had this experience?