Do you stay in the house when workmen are there?

I’m having some wardrobes fitted.
I’ve left to spend the day at my in-laws 10 mins away.

Some people have said it’s weird that I leave workmen to crack on alone in the house.