When is the quietest time to visit a Museum or historic site in the UK?

As the title, when would the quietest time be to visit a museum to avoid school parties and young families? Would it be best in school holidays (so no school parties but possibly lots of family outings) or just go for it when kids are at school and try and work around the field trips?