PM drawn back into 'sports rorts' saga