At this point, it looks like a self-imposed limitation. How many variations of the hero, the Master Sword, Ganondorf, saving the princess, and all that jazz that we need?
Two of their most interesting games story-wise are Link's Awakening and Majora's Mask, and know what they have in common? Almost nothing of the traditional Zelda lore. I recall playing Twilight Princess and being invested in the story until that cutscene where, surprise, they revealed that Ganon was behind everything again. I just rolled my eyes.
I just don't get it. I know we don't play Zelda games for the story but there's absolutely nothing preventing them from having an engaging story that doesn't repeat the same storyline over and over again. Nintendo loves new mechanics and each Zelda is very distinct from the last one but for some reason that escapes me, they insist on bringing back the same story when it would just be easier to make new lore and characters. Sure, keep Link as the main protagonist but everyone else, especially Zelda and Ganon, don't need to come back. They simply add to the feeling of retread and similarity. We can still have the Zelda elements that we know and love without having the same thing retold to us for the 50th time.
What do you want? A soap opera?
I think STar Wars begs to differ that there is nothing preventing them from having an engaging story that doesn't repeat the same storyline over and over again. The last 3 movies weren't great. And perhaps the most well received one of the last 3 was the 1st which was a regurgitation of the story in the original 1977 Star Wars movie.
I don't see the purpose of just creating all new lore and characters every time. IT's very easy to blow that task for one. Two, you end up often largely reinventing the wheel as well. Just because you give everything new names and new looks doesn't mean it really is any different nevermind better.
I think another reason they use the same "signposts" is because they act like a familiar guide to get people into the game. And limitations always foster creativity as well. They can riff off those things. Play off expectations. CReate new ways to play.
AS others have said in so many words, stories passed down between generations or just passed amongst people get retold in slightly different forms every time. That's how I see a Zelda story. Even though I guess they happen at different times supposedly? I can't say I keep track.
I also think financially, using the same characters saves $$$$ and creates a stronger brand identity. It allows artists to get really familiar with characters and learn them and animate and draw them better and faster. We've also seen link and ganon and zelda used in other games. And in other Zelda games.
And I think it frees them up to work on adding a few new quality characters/monsters/bosses in each game. ... they aren't reinvesting the wheel every time. And again limitations help creativity.
Last...at the end of the day it's a video game. I don't really know how an rpg or adventure action game, whatever ZElda is, gets out of the explore, defeat monsters, loot, get stronger and ultimately save the world story premise. I don't how making up some new story or premise really changes the game.
LInk's Awakening didn't seem any different of a Zelda game even though you weren't technically saving the world by defeating the boss iirc. Might as well have been tho. Same difference.