Is dating really just a numbers game for men?

I’m told I’m a decently attractive guy. I’m 27, 5’7” average build and have a decent job as an accountant. I’ve only had one relationship, which I attribute to insecurity and a lack of confidence. I’ve been gaining confidence lately, but am still a little nervous talking to attractive women. I’m going to start working out to be more attractive (which will also help with confidence), but I’m wondering even if you’re a good looking guy is it still just a numbers game with a lot of rejection?