Everyone loves a good street photo. Even viewers who have no interest in the artsy or experimental will, when seeing the right street shot, feel the vicarious thrill of being in the vibrancy of city life. And many photographers love going out and taking them.
But I'm not one of those photographers, because street photography, at least how it's thought of lately, could also be accurately called "Stranger Photography." I'm totally a fan of it when others do it, and it's perfectly legal to do here in America and most other countries, but I just don't have the right constitution for it. I'm cowardly when it comes to the idea of getting caught taking a stranger's photo on the sly, and then getting confronted about it. People do occasionally freak out in those situations, and who knows what dark motivations they might ascribe to me taking their photo.
In my opinion the best way to handle such a confrontation would be to say, "Nothing personal. I'm just out trying to capture city life, and photographing as many people as I can." I'd also offer to delete the photo as a courtesy, but there's no obligation to do that.
All that said, I will take a stranger's photo if the context makes me very confident that they'd enjoy having it taken. Last Friday night I was walking down U Street here in DC when when two women started dancing to a singing, bicycle-riding street preacher. I felt pretty sure that none of them would mind a photographer's attention...