Federal government concerned about ‘public scrutiny’ in mandating its workers back to office