Ive had that happen to me maybe once or twice, though I'm not what I think most people would class as being quite attractive, which is fine, and it is certainly incredibly annoying.
I have a friend who always gets attacked by Swans, I'm not entirely sure why they all hate him but they do. Swans are fucking dangerous though.