It kinda matters imo. Big budget = lots of people working there = lots of management
Ever heard the phrase "too much cooks spoil the broth"? I think that applies to videogames, and quite a lot, which (again imo) is why indies and AA games do so well nowadays but AAA are suffering, specially in western companies which I'd guess have different management practices than our eastern AAA bros which are doing much better.
I'm working on a game on my own, and I have a very clear idea of how it's going to be and could play it in my mind no problem almost from start to finish. Meanwhile the people working on that MS canceled game, Everwild, had no fucking clue of what the games was all about.
I really agree with you when you say that fun should be the primary focus, but imo the way to reach that is either AAA western companies scale down to the days of the PS3 (their peak if you ask me) or completely change the way they manage their projects.