‘Alarm bells ringing’ as support for Voice to Parliament declines