I pulled the lever. The gate closed.
The problem is that they climbed the walls. I didn't have enough time to put up fortifications on top of the walls, which would have prevented this. Next time, let's either embark on a flat area or build entirely underground.