It still mostly holds together. What relatively few "retcons" have been made compared to the early stuff mainly make sense (I would say there's really only one big piece of information that you would probably think would have somehow been mentioned earlier in the series that is introduced pretty far in, but even that Oda provides some explanations/excuses for), and the few things that are seemingly inconsistent are easily handwaved away because Oda wisely used unreliable witnesses to present important information early on in the series. Most information you hear about places and things that won't actually be dealt with for hundreds of episodes usually initially comes from someone who heard from another guy who once heard stories about said thing. It's rare you actually get a direct and reliable source, and even when you do, they tend to go the Obi-wan route: "from a certain point of view."
For example, the Grand Line is presented early on as practically being a ship graveyard, a place of almost certain death for any but the strongest of pirates, and is totally inhospitable. Turns out that, while harsh and wildly unpredictable and full of powerful foes, the Grand Line is also hugely populated, the center of military control in the world, and home to many of the greatest nations on the planet. This seems wildly inconsistent until you consider who exactly was telling those stories early on in the series: failed pirates and adventurers. They have their dreams of crossing the Grand Line dashed for one reason or another, and if they're somehow lucky enough to somehow turn back and escape the Grand Line, of course they're going to tell stories of horror. The pirates who actually have success in the Grand Line would rarely be coming back to those weaker and less lucrative seas, so the horror stories would naturally vastly outnumber the tales of glory.
Note that Luffy is inspired to head out to the Grand Line because he actually did meet one of the few successful pirates to actually come back and tell exciting inspirational stories about the Grand Line.
