Support NeoGAF

[YCoCg]

As we all know, GTA Online (and to an extent Red Dead Online) is pretty much known for its terrible loading times (hello clouds), however one person decided to actually take a real hard look into this and track what the game does, a few hours later they injected a solution that cut the loading times by up to 70% in most cases!! Here's the full detailed write up. However they also provided a handy tl;dr...

tl;dr

• There's a single thread CPU bottleneck while starting up GTA Online
• It turns out GTA struggles to parse a 10MB JSON file
• The JSON parser itself is poorly built / naive and
• After parsing there's a slow item de-duplication routine

A reminder that Rockstar have not fixed this issue in OVER SEVEN YEARS and have not fixed it for Red Dead Online either. But please enjoy paying full price for GTAV on PS5 and Xbox Series!

 

[Larxia]

The game does have awful loadings, but I think it's not even the main problem. The reason why these loadings are so annoying is because the matchmaking system itself is awful, nothing is ergonomic in this game.
You have to go through these loadings multiple times until you can find a decent room with people and the game mode you want. If the game had some kind of old style server browser menu (I really miss these) the loadings wouldn't be that much of a problem, because you could directly find what you want, instead of trying 10 things first.

 

[MGHA]

I just stopped playing. Fixed every issue I had.

 

[MrTroubleMaker]

Yup online takes way to long to join.. I doubt that this will get fixed...they haven't been fixing these types of issues

 

[DonF]

I know that saying devs = lazy is in fashion, but there surely must be a reason why this happens. It can't be that simple.

 

[YCoCg]

I know that saying devs = lazy is in fashion, but there surely must be a reason why this happens. It can't be that simple.

The same devs that REFUSED to patch the game for PS4 Pro and Xbox One X, didn't want to patch in HDR amongst other things despite having a dedicated team for GTA Online, who are now going to be reselling a "remastered" version of the game for full price on PS5 and Xbox Series consoles. No, lazy isn't the word I'd use to describe them, more money hungry. Though I'd still question why they haven't properly looked into this themselves, a 70% reduction is a large amount of time!

 

[JeremyEtcetera]

Though I'd still question why they haven't properly looked into this themselves, a 70% reduction is a large amount of time!

Did anyone send try to contact the dev team and send them this article?

 

[Whitecrow]

I know that saying devs = lazy is in fashion, but there surely must be a reason why this happens. It can't be that simple.

Software programming is task that requires a lot of brain energy, so a lot of times devs are not at their 100% skill capacity.
Also add that a lot of times they are asked to do their job fast, so also, a lot of times they make the first implementation that they come up with, and the first is not usually the most efficient.

But in this case, I would say it's just lazyness, or straigh up negligence, because it almost looks like if the dev didnt know what the heck he was doing.
And not doing anything after seeing that the code would just run for 6-7 minutes and just let it slip through, it's a shame.

 

[King_Mandu]

I just stopped playing. Fixed every issue I had.

Haha. I booted it up on PS3 back in the day, sat around for 15 minutes waiting to load in and said, "What the fuck is this?" after seeing how ass ugly it was. Never again.

 

[MeatSafeMurderer]

Old but decent CPU: AMD FX-8350

No.

Otherwise a good read. This is why you need to give programmers time to do shit properly. Just because their first implementation kind of works doesn't mean they won't come up with some radical rewrite 3 months later that achieves the same end goal so much faster. I've been known to come back up to a year later after pondering it for a while, granted, I'm programming as a hobby and not a job. Shame that Rockstar will give exactly 0 fucks and nothing will change.
Also inb4 ban for hacking.

 

[bitbydeath]

Thought this was going to be a PS5 brag thread.

 

[01011001]

As we all know, GTA Online (and to an extent Red Dead Online) is pretty much known for its terrible loading times (hello clouds), however one person decided to actually take a real hard look into this and track what the game does, a few hours later they injected a solution that cut the loading times by up to 70% in most cases!! Here's the full detailed write up. However they also provided a handy tl;dr...

tl;dr

• There's a single thread CPU bottleneck while starting up GTA Online
• It turns out GTA struggles to parse a 10MB JSON file
• The JSON parser itself is poorly built / naive and
• After parsing there's a slow item de-duplication routine

A reminder that Rockstar have not fixed this issue in OVER SEVEN YEARS and have not fixed it for Red Dead Online either. But please enjoy paying full price for GTAV on PS5 and Xbox Series!

why are so many professional game developers so shit at their job?

this is the exact reason why I want full PC like graphics settings in console games, because when you can't even trust game devs to do something like this right how can you trust them to chose the correct settings the game should run in?

I am to this day 100% certain that Destiny 2 can run at 60fps on One X. digital foundry should test this now that they have a PC board with the Xbox One chip!

 

[Fahdis]

I can do you one better... "How I cut GTA Online out of My Life"

 

[nowhat]

didn't want to patch in HDR amongst other things despite having a dedicated team for GTA Online

TBH, if RDR2 is anything to go by not having HDR is a feature.

 

[Derktron]

I just stopped playing. Fixed every issue I had.

Basically

 

[Virt McPolygon]

I can think of a way to cut loading times by 100%

 

[cormack12]

Source: https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/

#tl;dr

• There's a single thread CPU bottleneck while starting up GTA Online
• It turns out GTA struggles to parse a 10MB JSON file
• The JSON parser itself is poorly built / naive and
• After parsing there's a slow item de-duplication routine

Results

Well, did it work then?

Original online mode load time:        ~6m flat
Time with only duplication check patch: 4m 30s
Time with only JSON parser patch:       2m 50s
Time with both issues patched:          1m 50s


(6*60 - (1*60+50)) / (6*60) = 69.4% load time improvement (nice!)

It's parsing something. Parsing what? Untangling the disassembly would take forever so I decided to dump some samples from the running process using x64dbg. Some debug-stepping later it turns out it's… JSON! They're parsing JSON. A whopping 10 megabytes worth of JSON with some 63k item entries.

What is it? It appears to be data for a "net shop catalog" according to some references. I assume it contains a list of all the possible items and upgrades you can buy in GTA Online.

Clearing up some confusion: I beleive these are in-game money purchasable items, not directly linked with microtransactions.

 

[MeatSafeMurderer]

Already posted: https://www.neogaf.com/threads/how-i-cut-gta-online-loading-times-by-70.1593974/

 

[Soodanim]

I wonder how Rockstar people are going to react when this inevitably gets back to them. Pissed off bosses? Whether because they have had a shitty loading time game affecting their player count for years or because they now can't sell an update for extra profit, I just hope players get their game patched. 6 minutes is obscene.

 

[DiviniDee]

As we all know, GTA Online (and to an extent Red Dead Online) is pretty much known for its terrible loading times (hello clouds), however one person decided to actually take a real hard look into this and track what the game does, a few hours later they injected a solution that cut the loading times by up to 70% in most cases!! Here's the full detailed write up. However they also provided a handy tl;dr...

tl;dr

• There's a single thread CPU bottleneck while starting up GTA Online
• It turns out GTA struggles to parse a 10MB JSON file
• The JSON parser itself is poorly built / naive and
• After parsing there's a slow item de-duplication routine

A reminder that Rockstar have not fixed this issue in OVER SEVEN YEARS and have not fixed it for Red Dead Online either. But please enjoy paying full price for GTAV on PS5 and Xbox Series!

Rockstar has the laziest programmers/coders change my mind

 

[Aenima]

Since Heavy Rain i always heard you need to press X to Jason, maybe thats why the loadings so long. Gotta press that X.

 

[Dr.Morris79]

I just stopped playing. Fixed every issue I had.

That was what I did too. It seems brilliant at what it technically is but if like me you're playing on your own most of the time then it's just futile, there is a lot going on but it's never accessible

And it blows to be honest. Doubt i'd go back.

 

[paypay88]

People really think each dev is same , lol most of R* are probably shitty as fuck. Game development is really dumb career for a software dev where you could get paid much more with job safety & bonuses , not have to work on weekeends or overtime.

So if you are hire dumbos just because they are motivated that's what happens.

 

[Miles708]

why are so many professional game developers so shit at their job?

this is the exact reason why I want full PC like graphics settings in console games, because when you can't even trust game devs to do something like this right how can you trust them to chose the correct settings the game should run in?

I am to this day 100% certain that Destiny 2 can run at 60fps on One X. digital foundry should test this now that they have a PC board with the Xbox One chip!

I get what you're saying but devs usually have loads and loads of features to add and bugs to hunt down and solve. On its own this looks like an easy fix, but you also have to factor in the amount of other work you still have to do, the amount of time you have, the amount of sleep you missed and your boss breathing over your neck to JUST SHIP THIS every day.

I don't buy the notion that devs couldn't come up with a better solution, most likely they didn't have time or permission to actually analyze it for an afternoon.

For the record, i hate the game.

 

[DGrayson]

Ive never played GTA but I watch my friend streaming this constantly.

He constantly grinds for money, it almost seems like a job.

 

[M1chl]

Nice write up, pretty weird thing to have problem at. Seems like some entry level programming job. Hey, it works

 

[Mato]

Sounds about right. Is the game on PC still riddled with cheaters? I haven't bothered re-installing after I got a new SSD.

 

[kuncol02]

why are so many professional game developers so shit at their job?

this is the exact reason why I want full PC like graphics settings in console games, because when you can't even trust game devs to do something like this right how can you trust them to chose the correct settings the game should run in?

I am to this day 100% certain that Destiny 2 can run at 60fps on One X. digital foundry should test this now that they have a PC board with the Xbox One chip!

Because for some reason hacking things together with disregard of any good practice is seen as good idea in industry. I knew guy who was told by his boss when he started to work in gaming to "forget any good coding practice he learnt". Still existing relatively big company from Cracow.

 

[JLB]

Ive read the article the other day. Its unbelievably dumb. How many users lost due to the long loading times alone?

 

[JLB]

The game does have awful loadings, but I think it's not even the main problem. The reason why these loadings are so annoying is because the matchmaking system itself is awful, nothing is ergonomic in this game.
You have to go through these loadings multiple times until you can find a decent room with people and the game mode you want. If the game had some kind of old style server browser menu (I really miss these) the loadings wouldn't be that much of a problem, because you could directly find what you want, instead of trying 10 things first.

The guy in the article even created a reverse engineered patch that fixed it.

 

[Revolutionary]

When I played GTA Online on PC, it was possible to force yourself into a solo lobby, which allows you to sidestep the loading issue almost entirely as well as avoid the copious amounts of cheaters.

 

[Bitmap Frogs]

Ive never played GTA but I watch my friend streaming this constantly.

He constantly grinds for money, it almost seems like a job.

It's quite fun to play with a group of buddies and go wreak havoc hopping from activity to activity.

If I'm honest, to me more than the grind the problem are the load times that keep killing momentum.

 

[Soodanim]

Ive never played GTA but I watch my friend streaming this constantly.

He constantly grinds for money, it almost seems like a job.

I'd be more shocked by games that are essentially work if things like EVE Online didn't exist.

It must either be a gameplay loop that they never get tired of or them being caught up in the Sunk Cost Fallacy.

 

[Shifty1897]

A 10 MB JSON file is literally my worst nightmare.

 

[LisaMariaMartin]

People really think each dev is same , lol most of R* are probably shitty as fuck. Game development is really dumb career for a software dev where you could get paid much more with job safety & bonuses , not have to work on weekeends or overtime.

So if you are hire dumbos just because they are motivated that's what happens.

I wouldn't call the initial implementation dumb. It was just a naive approach. Something that works, but isn't the most optimized solution. People fail to realize that performance is probably on the low end of priorities when it comes to software development and only AFTER the software does what it has to do, you start doing optimization.

This is probably just a case of bad luck where someone forgot to write a TODO. Certainly not something I'd hold over any developers head.

 

[The_Mike]

They are too lazy to add some sort of anti cheat in their games on pc, so I wouldn't even suspect them caring about giving stuff like this on consoles when it's on a "playable state".

Meanwhile they keep spitting online dlcs out like there's no tomorrow or nerf gold bars so much you want to spend money on it.

 

[paypay88]

I wouldn't call the initial implementation dumb. It was just a naive approach. Something that works, but isn't the most optimized solution. People fail to realize that performance is probably on the low end of priorities when it comes to software development and only AFTER the software does what it has to do, you start doing optimization.

This is probably just a case of bad luck where someone forgot to write a TODO. Certainly not something I'd hold over any developers head.

a good engineer would never make this into production code but i dont know how are they planned . I mean this is same as doing print 10 times for a basic loop instead of writing 10 print lines

 

[Bitmap Frogs]

A 10 MB JSON file is literally my worst nightmare.

Repeat after me: json is not a substitute for a proper database, json is not a substitute for a database, json is not a substitute for a database.

 

[RaySoft]

Repeat after me: json is not a substitute for a proper database, json is not a substitute for a database, json is not a substitute for a database.

A 10meg file is nothing. It's not gonna grow much over time either, so implementing a databse for 10MB of data is rather overkill.

The code looked like it was written early in, and probably wasnt touched up when other dependencies changed over time. Since it didnt crash and probably never profiled either, the bug survived all these years.

He got a $10K bounty out of it, so win-win I guess

 

[M1chl]

Repeat after me: json is not a substitute for a proper database, json is not a substitute for a database, json is not a substitute for a database.

Yeah would be probably too much to ask for R* to just do update/insert. It's ridiculous.