Ban on Amit Shah lifted, Azam Khan refuses to apologise