Matthew 5:37- "Let what you say be simply 'Yes' or 'No'; anything more than this comes from evil."

We are within our rights to respond without explanation. You should not feel obligated to try to explain yourself to anyone who does not agree with your answer. It really is ok to say no!