should i continue seeing someone who is unsure about me/what he wants?

i (f23) met a guy (m35) on bumble a while ago. we hit it off on our first date. on the second, we slept together, but there were some “issues” because we had miscalculated things and it was too soon. a couple weeks after, he said he wasnt ready to be in a relationship again (he’s had bad past experiences), and that the age gap was too much of an issue, and that we should just be friends, etc. i felt insecure but understood his POV and said it was okay.

fast forward a month or so of not meeting, we flirted a tiny bit over text when i was on holiday. i met him for a drink when i got home, but still expected it to just be a friendly drink. he said he wasnt expecting anymore either and had the intentions of giving me closure. but things escalated because he kissed me, said he realises he does really like me, and we spent the whole day (and night) together.

since then we’ve been ~seeing eachother~, but it feels hot and cold. sometimes he is very affectionate, saying how much he likes me, how he feels comfortable around me, etc. and it’s in these moments where my hopes go up because ive been craving a mutual, loving relationship for so long. i even told friends i hardly speak to anymore about him. then, there are times where i question our compatibility, if we are too different, if i’m enough for him, etc. and i’m left feeling insecure and deflated.

i recently went on a short, unplanned weekend away with him. things just didnt go as well as i had hoped. he ended up saying he is not after a relationship with me at the moment – that he put his former girlfriends on pedestals and now he wants to be selfish and focus on him. but he said he wants to continue seeing me without a label and if chemistry grows overtime, a relationship isnt ruled out. then we sort of agreed that we wouldn’t see other people but we’d be honest if we happened to meet someone else organically.

i am just conflicted. i think it is wise to not jump into anything and i agree with him. but the fact he is sometimes so affectionate with me and other times sort of disinterested is confusing. he even said maybe i got mixed signals because he is cuddly with me, when really he is cuddly with everyone including his friends, which was hurtful and made me feel not-special.

i dont know whether to keep seeing him (we meet roughly every week but hardly text/call inbetween) on the grounds that this could be a good thing, and good things can take time. or i dont know whether to call things off so i can potentially find someone i’m more compatible with. i feel his heart isnt in this and he has other things he needs to or would rather focus on (i.e. work, future plans, attempting sobriety, even his dog, lol).

i have what might be considered social anxiety (or i’m just painfully introverted) and have a hard time connecting with people, knowing what to talk about, and believing that i’m good enough for someone, etc. he is an extrovert/social butterfly but has mild autism and difficulty with social ques. i guess the point of me saying this is that we both dont find dating easy to navigate or enjoyable.