Just make a central pillar that you build the stairs out from and don't remove it. That's what I always do. If you want to remove the pillar for aesthetic reason, I believe the stairs will remain in the proper place if they have a block under them and against their back. I've made both, but I tend to keep the central pillar for lighting purposes, so it's been a while since I've made a spiral staircase without one.
EDIT: And if you don't want to fall, create a ledge along the outside of the stairs, but in that case, I think you will definitely need to keep the central pillar - otherwise, the stair blocks will shift their perspective.