Why does google or siri respond, even when you didn't activate it?

Are we being watched?

Is every word we say being listened to by someone on the other side of the phone?