Monte's right that you expect to go down on gold early against a fast push, but:
1. Not 5k. That's the, "You've done jack-shit to actually fight back against the fast push" level of differential. 3k is about as far as you ever want to let it build. The fast push causes the team doing it to go down quite a bit on farm, so they should not actually have a gold difference equal to the value of towers lost if you're defending properly.
2. They actually were down closer to 7k by the time the fast push stalled out, which is way, way more than you ever intentionally give up. To say nothing for giving up an inhibitor turret.