Andy Griffith disputed the ties between Mayberry and his hometown of Mount Airy