Today's disc drives have much, much higher bandwidth and lower latency than the PS1 drive, and also the fact that every game on the One and PS4 actually runs from the hard drive.
There's also the fact that modern consoles have enough RAM that they can just load the entire N64 ROM into RAM, which will be faster than even streaming from the HDD. While OoT performs fine on GameCube, Majora's Mask has issues because there wasn't enough fast RAM space to load the entire Majora ROM into RAM. When you're playing OoT on GameCube, it's not even using the disc drive after the initial load.
edit: PS1 had a 2x CD ROM drive. The real world bandwidth was roughly 100-200kb/sec for PS1 games. For N64, it was 25-40MB/sec. You traded space for speed. The top speed of N64 cartridges still beats out the disc drive of the One and the PS4. The latency of an N64 cartridge is still lower than the hard drive of a One or PS4.
Most all Nintendo and Rare games on the N64 exploited the near RAM speeds to do level streaming and a whole bunch of tricks. Ocarina of Time loads and deloads entire level segments on the fly when you walk down a hallway in under a second. Banjo-Kazooie has the camera jump into Gruntilda's Lair or another level entirely instantly when it's showing you a hint or a Jiggy being created. On Playstation you'd have to fake it with an FMV or just show a still frame.