I kept seeing these higher level haters and just ran and figured I had to grind a bunch, but I am glad I decided to stop dicking around and went for it. Now I feel like it's moving at a more normal pace. I'm sure that will change at some point, though. I have only just beat Coen and reached the 4th floor.
Just remember that you can beat anything, as long as they can't hit you. That will mean more as you progress.
