My(F20) boyfriend(M20) making a comment while having sex, do I talk to him?

I (20f) lost my virginity to my bf (20m), we only been together for 3 months. Unfortunately sex has been a bit painful for me since he is my first. We have tried sex a multiple times but for some reason it’s been painful for me every time so far. One time when we having sex, it was still painful for me I asked him to stop, and he made a comment saying he wishes I wasn’t a virgin before we met, because he wants to be able to have sex with me with no issue. I’m not sure how I feel about it. Like I felt kinda embarrassed like it’s my fault, and I know it’s not but still I felt bad. I told him I am going to make a doctor’s appointment because it is weird that it’s painful still. But this has been on my mind and I kinda am embarrassed to have sex with him now. Do I talk to him or should I end it?