Should I just be done ?

Hi there! Me and my boyfriend have been together 4 years and have two small children. We’re both in our mid 20’s.

My boyfriend had a child from a previous relationship and that’s never bothered me. But he told me the reason him and the mother of his 1st child never worked out was because she wanted to get married and he didn’t. He’s always acted like he’s never been married before. Well today I found out that they were in fact married for a year… I’m so mad. I don’t care if he was divorced, I care though that he’s been hiding this from me for over 4 years. And he has had a history of just lying to me about the stupidest shit but he’s been doing a lot better consistently But now at this point I feel like I’m just done with it and him. I obviously can’t feel like I can trust him. I brought it up to him and he said “I don’t remember ever getting married to her” … how does someone forget they’ve been married AND divorce. I love him with all my heart and the angry side of me wants to leave but at the same time I know it doesn’t change anything really and it would be over reactive if I did leave?

Is this as big of a deal that I’m making it to be ? Is this one of those leave now situations?

TLDR: my boyfriends been hiding/lying his past marriage/divorce from me for 4 years and now he’s trying to lie his way out of it saying he doesn’t remember ..