Actually Harry Potter was not super detailed from the beginning. The later books' focus on horcuxes and a variety of things that Rowling didn't plan initially. In contrast Martin can point to a lot of his main focuses all the way back to the first two books - from the Others to Aegon Jon's parents.
Arguably the most detailed fantasy series is The Book Of The New Sun. It's also kind of a case of cheating since Gene Wolf wrote the entire thing first, went back to add foreshadowing/clues/etc, then released the books to the public.
I like AFFC and ADWD a lot, but I agree that they bloated what was originally quite a streamlined story. ASOS sets up quite an interesting second act. The fall of the North, the end of the war, the queenship of Dany, etc. Plus some stories in between - Arya learning to be an assassin, Bran learning about the Others, etc. And yet two books have followed yet a whole lot of that stuff hasn't truly been resolved or advanced from a time perspective; Tywin hasn't been dead for that long.
I can't help but feel like the inflation of the Greyjoy plot especially was a mistake. I like Victarion and the Damphair's chapters but at the end of the day, wouldn't the overall story be better if Theon either died at the end of ACOK, joined the Nights Watch, or was tortured like in ADWD...but with whichever direction was chosen being the only Greyjoy related plot?
What purpose does Victarion truly serve, when Dany is going to have multiple means of getting back to Westeros with or without him? A big ass Volantese fleet is sailing to Meereen to attack, and is manned by slaves. Sounds like the perfect plot device to get her and her army to Westeros. So why do we need the Iron Fleet/Victarion.
That's where the gardening approach became an issue to me. Martin knows what Dany's end game is, whether she lives or dies, etc. But he has clearly struggled with how to get her to that position, and we still don't even know how he's going to do it. Will she willingly go with Victaron (or Euron...), will she travel to Valaryia, will she free Volantis, etc. And what role will Tyrion play in whatever her decision is.