If I caught someone watching me it would freak the hell out of me. Sometimes I do watch people but not for any particular reason, mostly because they just cross my path.
Last summer I had sex with a friend/FB of mine in the balcony of his 16th floor condo in downtown, it was pretty hot and he has gay older neighbors and we think they were watching. It was amazing, smoked a blunt, hanged out in the balcony, had some drinks and went at it, you could see the cleaning crew in the building across.
I think we've all experienced some form of voyeurism, to an extent.
But honestly someone watching me in the night scares me, I would put a restraining order on anyone who tries that My worst nightmare is someone filming me through my window.
When I was studying film theory and learning about psychoanalysis, "scopophilia" is the pleasure of looking and Freud reckons it's an intrinsic part of human sexuality so... I'm not gonna feel guilty for having a locker room fetish.