I have encountered something irritating. Apparently the sites you can found as an adventurer don't prevent bogeymen, AND even though I am flying a good 10 Z-levels above them, they can still punch and bite me from the ground.
You haven't tried hiding inside the building itself? Sleeping outside in town is just as bad an idea. I just turn boogeymen off anyway. >.o