Support NeoGAF

[kswiston]

This spun out of something that I posted earlier in the Wkd Box Office thread. I had a request to turn that post into a thread, hence the content below:


There has been a lot of talk this year about Rotten Tomatoes/Metacritic and their impact on (domestic) box office. While we still have poorly reviewed films that make huge money at the domestic box office (e.g. Suicide Squad), and we still get some well-reviewed big budget films that do poorly (e.g. Deepwater Horizon), there does seem to be a pattern of poorly reviewed films with disappointing domestic grosses.

I thought that it would be interesting to compare the domestic grosses of recent big budget films to their review aggregates (Both Rotten Tomatoes and Metacritic). I compiled data on domestic box office, reported production budget, Rotten Tomatoes score, and Metacritic score for all films released from 2015 to present with at least a $75M production budget. In total, 91 films fit this criteria.

NOTES:

- About my budget cut off: I made a post late last year, looking at the budgets for all wide release films in 2015-2016. The median wide release budget was $35M. Roughly a quarter of wide releases had budgets of at least $75M. I figured that was a good "big budget" cut off. I did keep Minions in despite it missing that $75M mark by $1M, since it had a much larger than average marketing budget. Deadpool was not included (<$60M budget).

- I am looking at domestic box office only. Foreign/non-English territories often have different receptions to specific Hollywood movies, and aggregates like Rotten Tomatoes have virtually no mindshare in places like China.

- Some of the 2017 films on this list are still in release. I have included estimated final domestic totals where appropriate. Wonder Woman and Cars 3 are the only two where the actual final could feasibly end up more than $10M away from my estimate, so I wouldn't expect the estimates to noticeably skew any of the figures presented.

- I decided to split the 91 films examined by Fresh/Rotten cut off on Rotten Tomatoes. Not only is this a pretty well known and widely used cut off, but it correlates well with an over/under 55 score on Metacritic. Out of the 47 Fresh films, only one had a score under 55 on MC. Of the 44 Rotten films, only 4 had a score of 55+ on MC. Metacritic scores generally tend to fall within the 30-75 range for blockbuster films, while RT has a much wider span.

- I am including a "no outliers" average for both the Fresh and Rotten films. The "no outliers" average removes the top 5 highest and lowest grossers from each category, showing the average after the more unusual runs have been removed.


A Look at Domestic Box Office totals for all 2015-2017 Big Budget Films that received a "FRESH" (60%+) Rating on Rotten Tomatoes.

[B]TITLE					DOM GROSS ($ MIL)	Budget ($ MIL)	RT Score	Metacritic[/B]
Star Wars: The Force Awakens		937			245		92		81
Jurassic World				652			150		72		59
Rogue One: A Star Wars Story		530			200		85		65
Beauty and the Beast (2017)		505			160		71		65
Finding Dory				486			200		94		77
Avengers: Age of Ultron			459			267		75		66
Captain America: Civil War		408			250		90		75
Guardians of the Galaxy Vol. 2		390			200		81		67
Wonder Woman				375			150		92		76
The Secret Life of Pets			368			75		74		61
The Jungle Book (2016)			364			175		95		77
Inside Out				356			175		98		94
Furious 7				353			190		79		67
Zootopia				341			150		98		78
The Hunger Games: Mockingjay Pt2	282			160		70		65
Sing					269			75		73		59
Moana					247			150		96		81
Fantastic Beasts and Where To Find Them	234			180		73		66
Doctor Strange				233			165		90		72
The Martian				228			108		91		80
The Fate of the Furious			227			250		66		56
Logan					226			97		93		77
Cinderella (2015)			201			95		83		67
Spectre					200			245		64		60
Mission: Impossible - Rogue Nation	195			150		93		75
The Revenant				184			135		81		76
Ant-Man					180			130		81		64
The LEGO Batman Movie			176			80		90		75
Kong: Skull Island			168			185		76		62
The SpongeBob Movie			163			75		80		62
Star Trek Beyond			159			185		84		68
Cars 3					155			175		66		59
Mad Max: Fury Road			154			150		97		90
Trolls					154			125		74		56
Kung Fu Panda 3				144			145		87		66
The Peanuts Movie			130			99		86		67
Kingsman: The Secret Service		128			81		75		58
Ghostbusters (2016)			128			144		73		60
The Good Dinosaur			123			200		77		66
The Magnificent Seven (2016)		93			90		63		54
Miss Peregrine's Home for Pec. Child.	87			110		63		57
Alien: Covenant				74			97		71		65
Deepwater Horizon			61			110		84		68
The BFG					55			140		74		66
The Man From U.N.C.L.E.			45			75		67		56
Allied					40			85		60		60
The Finest Hours			28			80		63		58
__________________________________________________________________________________________________
FRESH FILMS AVERAGE			249			148		80		68
NO OUTLIERS AVERAGE			226			149		81		68
__________________________________________________________________________________________________
[B]ALL FILMS AVERAGE			174			139		56		55[/B]

A Look at Domestic Box Office totals for all 2015-2017 Big Budget Films that received a "ROTTEN" (<60%) Rating on Rotten Tomatoes.

[B]TITLE					DOM GROSS ($ MIL)	Budget ($ MIL)	RT Score	Metacritic[/B]
Minions					336			75		56		56
Batman v Superman: Dawn of Justice	330			250		27		44
Suicide Squad				325			175		25		40
Home (2015)				177			135		47		55
The Boss Baby				175			125		51		50
Hotel Transylvania 2			170			80		55		44
Pirates of the Caribbean 5		165			230		29		39
Jason Bourne				162			120		55		58
San Andreas				155			110		48		43
X-Men: Apocalypse			155			178		48		52
The Divergent Series: Insurgent		130			110		29		42
The Legend of Tarzan			127			180		35		44
Independence Day: Resurgence		103			165		31		32
Passengers (2016)			99			110		31		41
Tomorrowland				93			190		49		60
Terminator: Genisys			90			155		25		38
Alvin & Chipmunks: The Road Chip	86			90		16		33
Power Rangers (2017)			85			100		46		44
TMNT: Out of the Shadows		82			135		38		40
The Mummy (2017)			80			125		15		34
Pixels					79			88		17		27
Alice Through the Looking Glass		77			170		30		34
The Divergent Series: Allegiant		66			110		12		33
Now You See Me 2			65			90		34		46
Ice Age: Collision Course		64			105		15		34
Fantastic Four				56			120		9		27
Assassin's Creed			55			125		18		36
The Huntsman: Winter's War		48			115		17		35
Warcraft				47			160		28		32
Jupiter Ascending			47			176		26		40
xXx: The Return of Xander Cage		45			85		43		42
The Great Wall				45			150		35		42
Ghost in the Shell (2017)		41			110		45		52
King Arthur: Legend of the Sword	39			175		28		41
Pan					35			150		27		36
Inferno					34			75		19		42
Monster Trucks				33			125		32		41
Gods of Egypt				31			140		15		25
Point Break (2015)			29			105		9		34
The Last Witch Hunter			27			90		16		34
Ben-Hur (2016)				26			100		25		38
In the Heart of the Sea			25			100		42		47
Seventh Son				17			95		13		30
The Promise (2017)			8			90		50		49
__________________________________________________________________________________________________
ROTTEN FILMS AVERAGE			95			129		31		41
NO OUTLIERS AVERAGE			80			131		30		40
__________________________________________________________________________________________________
[B]ALL FILMS AVERAGE			174			139		56		55[/B]

SUMMARY STATISTICS

While it is debatable whether good/bad reviews are directly leading to the results above, you can see that reviews do correlate with domestic box office performance. At the very least, audiences tend to agree with critics in terms of which films are most worthy of their money with the "Fresh" group of films averaging more than 2.5x the gross as the "Rotten" group of films, even if we remove things like TFA and Jurassic World.

EDIT: Graphs and other data

 

[Moral Panic]

Can you plot some scatter graphs

 

[riotous]

Of course they do; especially since lots of people shop online for tickets at sites that list the rotten tomatoes score right next to the film name.

Interesting stat gathering either way but there are so many factors at play it's hard to put a numeric value on how much they matter for any given film.

 

[LakeEarth]

IMO, advertising and the overall appeal of a movie gets people for the opening weekend. It's the reviews and word of mouth that keeps the money flowing weeks later.

 

[Skellig Gra]

Interesting analysis, I think there are many other factors that play in to it other than review score. Is it from a famous franchise? That will protect it against bad reviews. A brand new franchise with bad reviews is not going to do well at the box office. Even if its from a famous franchise, if the last one was recent and terrible then its more likely to be affected by bad reviews. Competing against other movies that reviewed well is also a factor. Advertising budget would be an interesting metric to use as well.

 

[kswiston]

Can you plot some scatter graphs

What scatter plots do you want specifically?

 

[autumnleaf]

I think the most interesting stat would be 2nd weekend drop % or opening weekend multiplier against RT score. Most big budget films can score a big opening and thus decent haul on hype and marketing alone.

 

[HStallion]

I think reviews matter far less if your movie is part of a beloved franchise or property as people will flock to it regardless. Bad reviews probably cut the legs out from under some of these films anyway but it seems like opening's tend to be huge if its related to something already well known and liked by the general masses.

 

[kmfdmpig]

Interesting analysis, I think there are many other factors that play in to it other than review score. Is it from a famous franchise? That will protect it against bad reviews. A brand new franchise with bad reviews is not going to do well at the box office. Even if its from a famous franchise, if the last one was recent and terrible then its more likely to be affected by bad reviews. Competing against other movies that reviewed well is also a factor. Advertising budget would be an interesting metric to use as well.

Those are good points. I'm curious just how much a previous entry can help or hurt a film. Would Batman Begins, for example, have done much better if people didn't have the memory (from 8 years earlier) of Batman and Robin? Would Wonder Woman have done even better if BvS had been better? I suspect it has a fairly significant impact, which is why TF5, even though it will certainly turn a profit, will be seen as a major disappointment if it makes ~700m or less worldwide considering the previous two made 1.1 each.

 

[UnemployedVillain]

What scatter plots do you want specifically?

Score (whichever you want) vs gross, maybe?

 

[WaffleTaco]

kswiston, your posts in box office gaf are always interesting and insightful. I just wanted to say thank you for your work.

 

[kmfdmpig]

kswiston, your posts in box office gaf are always interesting and insightful. I just wanted to say thank you for your work.

Yes, agreed. I've learned a lot and have appreciated it.

 

[kswiston]

Score (whichever you want) vs gross, maybe?

I'll see what I can do. TFA screws things up though, since it made $300M more than anything else on the list, and at least double all but the Top 5 films. Anyone have some tips for avoiding an excel scatter plot where the right half is mostly white space thanks to a couple outliers?

EDIT: I guess that the score should be the x-axis since it's the independent var.

 

[Aureon]

I'll see what I can do. TFA screws things up though, since it made $300M more than anything else on the list, and at least double all but the Top 5 films. Anyone have some tips for avoiding an excel scatter plot where the right half is mostly white space thanks to a couple outliers?

Cut the outliers. They're outliers anyway.
(Or logplot)

Could you arrange a scatterplot for second weekend drop against RT score?

 

[CaviarMeths]

Thanks for doing this thread kswiston.

I've been saying for a while that as much as we don't like to admit it, we actually agree with critics far more often than we don't. Franchises like Pirates and Transformers have been major outliers to that, the exception, not the rule. 2017 in particular, audiences seem to be outright rejecting poorly reviewed films and embracing well reviewed ones.

On whether or not these reviews are influencing the gross directly, I would say it's a little of both. Rotten Tomatoes is a legitimate marketing tool these days, so much that you see the sticker of certified fresh movies on the DVD case. Accolade trailers are common now, with RT scores being brandished proudly for 'good' movies. At the same time though, as I said, I think we also just agree with critics most of the time. They are viewers too after all, just ones that happen to write for a living. Even if RT wasn't used in marketing, we'd still see a trend of bad reviews = lower gross/legs.

 

[kswiston]

Score (whichever you want) vs gross, maybe?

Not super fancy looking, but here's RT score x Domestic Gross.

 

[igordennis]

Really awesome work kswiss! Very interesting correlation to see it laid out like this. Gotta say seeing the DCEU isolated in that graph above made me laugh.

 

[MetroidPrimeRib]

The DCEU cluster right there says something, I think reviews do still hold a lot of weight domestic

 

[Pharaun]

Excellent post and research, thanks for doing it.

It's interesting how outside of the outliers (a couple of comic book movies and a major merchandising franchise aimed at kids) the Rotten films seem to cap out at sub 200M.

With the Fresh films there's no guarantee that they break the 200M barrier but it's close to 50/50. It would certainly seem that good reviews and word of mouth can have some sort of effect on the overall run.

 

[Bobby Roberts]

Will someone in charge please gift this man the reigns to Box Office Mojo.

PLEASE

 

[nynt9]

Would be interesting to see how reviews correlate with box office performance drops over time. I think first week is more dependent on advertising and hype whereas whether a movie sustains its momentum over the following weeks is where reviews come into the picture.

 

[Htown]

Not super fancy looking, but here's RT score x Domestic Gross.

lol at the DCEU

 

[Sibersk Esto]

I hope you get paid for this someday

 

[Bobby Roberts]

Would be interesting to see how reviews correlate with box office performance drops over time. I think first week is more dependent on advertising and hype whereas whether a movie sustains its momentum over the following weeks is where reviews come into the picture.

Yeah, this is probably the breakout I'm most interested in: what effect the rating has, if any, on legs.

 

[wenis]

Will someone in charge please gift this man the reigns to Box Office Mojo.

PLEASE

I'd rather they burn it to the ground and let him build upon its ashes.

 

[BronsonLee]

No

The only review that matters is mine!

 

[Affeinvasion]

Is it not conventional wisdom that on average poorly reviewed films will do worse at the box office? I think rotten tomatoes has amplified this but it doesn't seem like it would be a new trend.

 

[jtb]

Yeah, this is probably the breakout I'm most interested in: what effect the rating has, if any, on legs.

Ditto. Was exactly my first thought when I read the title.

 

[Zackat]

Awesome thread.

 

[BumRush]

Amazing thread, Swiss.

That graph is so telling. Thanks for posting this.

I'm going to take a second pass at it later

 

[Sgt. Kabukiman]

Excellent work.

This all has to do with social media of course. Word-of-mouth and review scores are just part of the whirlpool of information that everyone is assaulted by daily. It definitely seems to be happening mostly in the US but certain films simply review differently in some countries.

Man of Steel actually got a slightly higher average critic score than Wonder Woman in France, and WW is trailing behind at the box-office. Hollywood movies are definitely affected by word-of-mouth and reviews around here, it's mostly french comedies that are immune.

 

[kswiston]

Yeah, this is probably the breakout I'm most interested in: what effect the rating has, if any, on legs.

I'm trying to think of a way to do this while acknowledging the fact that sequels tend to have poor(er) legs than original stuff. Ditto with animation vs live action.

 

[Bobby Roberts]

I think that the number of sequels you'd have to account for would actually end up making that acknowledgement more or less unnecessary? Basically - if there's no real effect on the 2nd & 3rd weekends, then the drops should be more or less uniform sequel to sequel. But if there is an effect, then poorly reviewed sequels are going to have even shorter legs than the better reviewed ones, and that should out over the running of those numbers.

Right?

 

[kmfdmpig]

I'm trying to think of a way to do this while acknowledging the fact that sequels tend to have poor(er) legs than original stuff. Ditto with animation vs live action.

Perhaps establish a baseline for the different categories and then see what the deviation from the norm is (above or below) for different movies? That could then be plotted against the rotten tomato score.

 

[BumRush]

Swiss, you have really outdone yourself. Bobby is right...you need to take your talents to box office mojo

 

[kmfdmpig]

Swiss, you have really outdone yourself. Bobby is right...you need to take your talents to box office mojo

Absolutely. This type of analysis is better than what many of the sites which focus on box office predictions provide.

 

[J_Viper]

Yeah seems like RT is definitely having a huge impact on domestic performance

Films one might think would-be hits like Baywatch and TF5 are performing poorly, and that Rotten score could play a part.

Being terrible movies probably doesn't help either.

kswiston, your posts in box office gaf are always interesting and insightful. I just wanted to say thank you for your work.

This too.

Damn fine work

 

[kswiston]

I will have to do some more sorting to have any meaningful examination of legs.

However, there is a pretty pronounced difference in opening weekend totals between the Fresh and Rotten films. $82M vs $43M respectively. Opening weekend does seem to be one of the areas where good or bad reviews could have an effect:

 

[Bobby Roberts]

Wow. That's sorta surprising. I'd have thought for sure the marketing was carrying most of that weight, but RT seems to seriously dent that notion.

 

[kswiston]

Wow. That's sorta surprising. I'd have thought for sure the marketing was carrying most of that weight, but RT seems to seriously dent that notion.

The Rotten outliers that I labeled are pretty clear examples of hype and marketing drowning out reviews, but that does seem to be happening less these days. Someone else mentioned how social media leads to a constant and near instantaneous stream of information. Word spreads faster now than it might have years ago.

 

[Slayven]

I wonder how often Mrs.Kswiston has say "Get the spreadsheets off the dinner table"

 

[WaterAstro]

How about a ratio calculation?

Gross:RT Score? Would be interesting to see the list of the lowest grossing movies with the highest RT score and vice versa.

Looks like Fury Road is the highest rated and lowest grossing movie just by a glance.

 

[kswiston]

How about a ratio calculation?

Gross:RT Score? Would be interesting to see the list of the lowest grossing movies with the highest RT score and vice versa.

Looks like Fury Road is the highest rated and lowest grossing movie just by a glance.

I'm not sure I follow you. If you are talking about $ per RT point, Suicide Squad is #1 at $13M per percent of its RT score. The reverse is actually The Promise (also on the rotten list) by virtue of the fact that it only made $8M.

I wonder how often Mrs.Kswiston has say "Get the spreadsheets off the dinner table"

I am in the digital age! Phones are banned at our dinner table though. And if I disappear around the corner too often, she knows I am checking NeoGAF.

 

[Decado]

What about gross per million spent of fresh vs rotten?

 

[Pharaun]

The Rotten outliers that I labeled are pretty clear examples of hype and marketing drowning out reviews, but that does seem to be happening less these days. Someone else mentioned how social media leads to a constant and near instantaneous stream of information. Word spreads faster now than it might have years ago.

That makes sense. Even 10 years ago it would still take time for bad reviews and word of mouth to spread but the rise of the smartphone and mobile internet has changed that. Now as soon as someone sees an early preview or midnight showing the feedback on Facebook, Twitter, Instagram, etc can spread much more quickly and possibly deter people from seeing a show that they might have otherwise.

To go along with effect it can have on legs i think it would be interesting to see what possible link there is between RT and performance compared to opening weekend predictions.

 

[BumRush]

I will have to do some more sorting to have any meaningful examination of legs.

However, there is a pretty pronounced difference in opening weekend totals between the Fresh and Rotten films. $82M vs $43M respectively. Opening weekend does seem to be one of the areas where good or bad reviews could have an effect:

Many of the highest OW earners are films that would have gotten large OW despite their reviews though.

Star Wars, Avengers, Dory, Beauty, etc. I'm not sure this is as meaningful a metric as it appears

 

[Slayven]

I'm not sure I follow you. If you are talking about $ per RT point, Suicide Squad is #1 at $13M per percent of its RT score. The reverse is actually The Promise (also on the rotten list) by virtue of the fact that it only made $8M.



I am in the digital age! Phones are banned at our dinner table though. And if I disappear around the corner too often, she knows I am checking NeoGAF.

You going to mess around and be doing the numbers for Justice League and accidentally crack the Anti Life equation

 

[hollomat]

Now do it with international and you'll see an inverse correlation

 

[Dyle]

Nice data, but this is a good example of correlation, not causation. The idea that good movies tend to make more money than bad moneys is not super enlightening. There's no indication of how many people were influenced by, or even saw, reviews of the movies. A better indication might be to also compare revenue against audience review scores, which would somewhat demonstrate the influence of reviewers.

I'd also suspect that marketing budget is likely to be more indicative of success than review score, but I suppose there isn't a way to get that kind of data

 

[kswiston]

I'd also suspect that marketing budget is likely to be more indicative of success than review score, but I suppose there isn't a way to get that kind of data

Most of the mega-budget films have roughly comparable marketing campaigns. No one is spending $150M to make a film and then skimping on the marketing.

If you just look at films that had production budgets of $150M+, the fresh films averaged $111M Opening weekend, and $341M total, while the rotten films averaged $49M opening weekend and $120M total.

96% of the Fresh films in the $150M+ budget range made at least 75% of their production budget in the domestic box office (The Good Dinosaur was the one miss). 80% had a gross larger than their production budget.

Only 3 of the 14 Rotten films in the $150M+ club made 75% or more of their production budget back in the domestic box office, and all three were superhero films (BvS, Suicide Squad, and X-Men Apocalypse).

Now do it with international and you'll see an inverse correlation

Only 4 of the top 25 films worldwide since 2015 have rotten ratings.