Why were Sweden, Ireland and Britain the only nations not to put temporary FOM restrictions on Poland when it joined the EU in 2004?

To combat the fear of a mass exodus from the 10 new member states in the 2004 expansion, the EU agreed to a 2+3+2 year window to limit FOM before it become full and unconditional. But Britain, Ireland and Sweden didn’t set any restrictions from the get go. Why?