Minister Andrew Giles is resisting pressure to stand down, saying there's more work to be done, as the opposition accuses him of being "asleep at the wheel" over the detainee crisis.