Both Liberal and Labor parties ‘going backwards’ in the polls