I can’t remember when I first noticed CinemaScore ratings, but once I started researching the organization, I quickly realized what a joke it is. I can’t believe that anyone takes these surveys seriously.
For those of you who might not know what I’m talking about, CinemaScore is a company based in Las Vegas that polls audiences from movie theater screenings. Reps ask exiting viewers for personal ratings of the films they’ve just seen, reasons why they went to see those movies in the first place, and demographic information. Since 1991, the results have been published in Entertainment Weekly, but nowhere else. Despite being founded in 1982, the company doesn’t have a web site. Its history is vague and you can only read about the organization on Wikipedia.
First off, what kind of credible company doesn’t have a web site? I have a hard time believing a company’s scoring when there’s no way to find out anything about how it’s calculated. Although CinemaScore apparently had a web site up and running in 1999, it’s nowhere to be found now. If you want to see what rating was given to an old title and don’t have a running archive of Entertainment Weekly magazines, you’re out of luck.
Secondly, polls are taken from 400-500 moviegoers in three of CinemaScore’s base-of-operation cities. I’m no statistician, but when taking Math 1030 at college, I remember learning about statistical bias, which can definitely be found here. Aaron Peck and I run a little Salt Lake City-based web site (The Reel Place) where we give out free passes for advance screenings in the area. We also attend the screenings that we offer passes to, which are the same types of regional screenings where CinemaScore polling is conducted. I can tell you that people only attend these screenings if they’re already interested in seeing a particular film. Would you really believe that an audience that attended ‘Grindhouse‘ would later go out of the way to attend a screening for ‘The Vow‘? No, of course not.
Finally, who actually answers these surveys? People who find Adam Sandler movies entertaining aren’t going to give high ratings to Oscar-winning films like ‘The Artist’ or ‘No Country for Old Men‘. By the same token, art house snobs are unlikely to give fluffy studio flicks good ratings. Why should I trust either group?
CinemaScore surveys are not truly movie reviews. Its ratings are rendered invalid by bias and ignorance, and you can’t find them published anywhere but Entertainment Weekly. Personally, I’ll stick with Rotten Tomatoes, which is a ratings system based on reviews from proven critics, people educated in the ways of film. If I want to know if my car is running well, I wouldn’t poll a bunch of people who just drive cars; I’d ask people who actually know something about cars. There’s a big difference between the two.
JM
You misunderstand the purpose of CinemaScore.
It’s not a metric of film quality. It’s a metric of marketing effectiveness.
Studios use the data to see if the demographics who went to the movie match the marketing department’s turnout goal.
It also lets studios tweak their advertising for the second weekend.
Considering the general secrecy of Hollywood, why would CinemaScore have a public website?
Hollywood doesn’t publicize anything unless it’s part of a campaign.
‘The Avengers’ got a CinemaScore of A+, which means that Disney marketed the film flawlessly.
Whores that they are, Disney then advertised their A+ as an indication of artistry.
Such is Hollywood.
Drew
CinemaScore has nothing to do with the quality, or artistic merit films. It’s a system that gauges the effectiveness of how the film was marketed.
When you discussed how the same audiences that turn out for Adam Sandler films, won’t do so for Oscar bait ones, CinemaScore was nodding their heads, and saying, “exactly!”
The data gathered from CinemaScore is used to determine whether or not the actual majority demographic that went to see the film matches the intent of the marketing campaign. If there is not a match, it gives the studio a tool to manipulate the marketing going into the second week, if they so choose to spend the potential extra money it would cost to design a new marketing scheme.
You completely misunderstood what CinemaScore is reflecting.
Drew
… “Merit of films” not, “merit films”. Haha.
I guess I was composing my comment during the time that Jane’s got published. It took longer than usual to finish mine. I got side-tracked trying to work. However, yes, everything she says is right on the money.
Luke Hickman
AuthorIf this is the case, then why are CinemaScores posted all over the place as if it’s a reflection of the content, like a new, fresher version of Rotten Tomatoes? CinemaScores are never announced in a way that reflects its marketing campaign.
EM
Why do some ads trumpet the phrase “the #1 movie in America”? You don’t have to be a rocket scientist to realize that box-office supremacy in a given week equates to quality, merit, or even suitability to your own personal tastes. But the ads obscure the meaning of the phrase—i.e., maybe some people will be fooled into thinking it is some sort of critical appraisal. What do the marketers care, so long as it helps drum up business?
EM
Ugh, obviously I should have written “doesn’t equate”…Drew, save me a seat, will ya?
Drew
According to you, they are only posted in Entertainment Weekly, but now you say that they are posted all over the place. Which is it?
Regardless, they have never been used as if they were a way to judge the quality of any particular film. That was just your flawed interpretation of the system.
They definitely haven’t ever came across as anything related, or markedly similar to Rotten Tomatoes; let alone a “newer, fresher version of Rotten Tomatoes.
“CinemaScores are never announced in a way that reflects its marketing campaign.”
— CinemaScore is not a marketing campaign. You’re misunderstanding once again. Why would they be announced as if they were? They are a metric used to gauge the effectiveness of the marketing campaign, and announced as such.
Luke Hickman
AuthorEW is where they are officially posted, but you can’t look up a single movie-related post on The Hollywood Reporter or Box Office Mojo without it noting the CinemaScore. EW is the only place that officially lists it – everyone else mentions it.
Drew
EM, Haha! Yep, I’ll try to save us the entire row.
Aaron Peck
I use the What Does Aaron Think Score (WDATS). It always works for me.
curtis
I’ve noticed it repeatedly cited on Boxofficemojo.com as well. No way it is just for ‘marketing effectiveness’ or it would only be a survey going in, not coming out. If anything all that does is get the initial reaction of the crowd which one would believe would be super high for the great ending of Avengers and low for artsy or movies with downer endings. Even Cabin in the Woods which was a great movie had a low score despite high rotten tomatoes ‘fresh’ rating.
Ben
Cinemascore isn’t particularly effective, but when you get an ‘A’ Cinemascore, it probably means good things are in store for your film from a ‘legs’ standpoint. The Avengers is literally the only A+ film that I can remember in 5 years attaining that and that’s going to be reflected in the coming weeks.
But getting a good score isn’t the end all be all. Inception got a B+ which is more or less average, or a hair above it, yet it had very strong legs. Horror movies seem to evoke greater disgust from audiences so scores are lower. You just have to frame the score into something useful.
Ben
By B+ being average, I mean for Cinemascore. That’s not really scientific though but general observation. It seems many films get a B+, or B.
Josh Zyber
CinemaScore is not presented as a measure of a movie’s marketing. It’s presented as an instant audience review of the movie’s quality.
As reported in Entertainment Weekly and other publications that cite it, the typical CinemaScore reference will say something to the effect of, “Movie XYZ grossed $50 million dollars this weekend, but audiences only gave it a CinemaScore rating of C+.”
The box office number reflects the effectiveness of the movie’s marketing, not CinemaScore. The CinemaScore is supposed to be a judgment on the quality of the movie itself, as voted by “Average Joe” viewers.
The only parts of the CinemaScore survey that have to do with marketing are the audience demographic questions. If CinemaScore was strictly a measure of the marketing effectiveness, it would poll viewers before they went into the screening, not after, and would not ask them to rate the movie at all.
William Henley
I know everyone is talking about CinemaScore, but I wanted to mention something about Rotten Tomatoes. While it does focus mainly on critics reactions, they now (not sure if it was always there) also have Audience Reaction, and publish a few “average Joe” reviews. I like this, because there have been movies that critics love that audiences hate, and vice-versa. For example, a movie can be loved by audiences for being a fun movie, but be torn apart by critics. That being said, I have also seen “artsy” films that critics love that audiences hate. If I am on the fence on a movie, I will check Rotten Tomatoes, and check both Critics and Audience reactions and comments. Sadly, If I am on the fence on something, the Rotten Tomatoe scores are usually around 50% anyways.
Anyways, kudos to Rotten Tomatoes for having critis AND audience ratings, and being able to compare them side by side.
Barsoom Bob
Josh is correct. They are taking the emotional pulse of the people ho saw the movie to ho they liked it. Attendance and box office show marketing effectiveness combined with audience per-awareness of the material. Do you really think that Harry P. or the Hunger Games would have posted those numbers if the exact same film had been dropped on the audience but there had been no series of books before it, that everybody had been reading? Not in those numbers by a long shot. And this does include The Avengers.
Luke was right that the Cinema Score has a bias in that it does use paying customers that were already somewhat in favor of the film, otherwise why would they be ponying up the cost of the admission ticket. Still it does reflect if the filmmaker did deliver to his targeted audience.
Drew
Everything that Josh says here is wrong.
If you want to understand CinemaScore, please read Jane’s initial comments, as well as my own original statements.
What Josh fails to understand is that if the right demographic shows up for a film, they will ALWAYS give it a better reaction than the wrong demographic would.
Barsoom Bob
Drew you are a little tiresome in always trying to prove that you are right and Josh is wrong. Give it up, dude.
CinemaScore is asking people what they thought of the movie that they just saw. Not why they choose to go see that movie, not how many people or dollars the film made, just how much did they like it.
I acknowledge that it is biased, as I explained above with glaring typing errors, but that doesn’t make it an indicator of marketing or attendence, it actually just denotes how well it played to it’s target audience.
Drew
Barsoom Bob,
What are you talking about?
This isn’t a difference of opinion. Josh is factually wrong. Everything that Jane and I said initially is correct.
Those are the facts about CinemaScore.
Barsoom Bob
Before I get slammed, let me qualify the targeted audience part. It is, how well the movie played to the audience that showed up. Now, the ancillary data can be valuable to the studio in that it tells them what the demographics of the audience that turned out for film was comprised of. But the Cinemascore rating is a subjective opinion that skews a little high because of that built in bias unless the film maker screwed up.
Two examples:
The Grey – people were excited to see it, critics loved it. Poor Cinemascore, audiences didn’t like the ending. My intial reaction was I felt cheated, took me a few seconds to come back to yeah not happy about it but that was the right ending.
John Carter – Critics stalled at 50% but a higher than expected Cinemascore. Disney had no idea who they were marketing the film to and still doesn’t, see the cover art for home video releases. It’s not a CGI kids picture but they have still put cover art on the discs that looks like a Monster Inc direct to video
release. None as blind as those that won’t see.
Barsoom Bob
Just how do you know that you are right and I am wrong.
I rest my case.
Drew
Everything you say is certainly agreeable, but it doesn’t change what CinemaScore is. It simply was not invented to be used as a metric of a film’s quality.
Barsoom Bob
Really, don’t want to go back and forth on this but Luke was talking about the score that is published to the public in EW and now some other magazines. That score is Audience Reaction. Go up to the top of the page and look at the banner image, what is the line underneath the big bold CinemaScore ? Audience Reaction Survey.
Now, You and JM are right that there are other purposes that this may have been created for and the demographic questions yield important data valuable for other purposes but the actual Cinemascore that Luke started talking about is a subjective audience reaction score to the movie.
JM is almost always right, except about Transformers. (That was a joke)
JM
Hollywood marketing departments have two core goals.
Turnout and Positioning.
Box office measures turnout.
CinemaScore was designed to measure Positioning.
If a customer feels she got the product she paid for, she scores the product highly.
Positioning an audience is tricky, because it’s expectation management.
Under-hype it, turnout is low.
Over-hype it, backlash is inevitable.
Hype it to the wrong demographic, word-of-mouth is distorted.
Where it gets shady is that, to maximize profit, a certain percentage of customer dissatisfaction is necessary.
Some audiences must be robbed. It’s the cost of doing business.
Which is why critics serve as watchmen.
Box Office Mojo only uses CinemaScore, an indication marketing effectiveness, to help predict a movie’s box office drop-off.
Entertainment Weekly uses it as a shortcut to thinking.
As ‘John Carter’ showed the world, a poorly positioned film underperforms.
We see movies as art. Hollywood see movies as mutual funds.
Josh’s personal CinemaScore of ‘The Cabin In The Woods’ illustrates how well he was positioned to experience that film.
Word-of-mouth is a double-edged sword.
The only thing ‘Transformers’ had to do to position me correctly was put “Directed By Michael Bay’ in big font on the poster.
I guess that makes me a slut.
Inkmei
Before I go to see a movie I want to know if it’s worth my money. Official movie critics never are good for me. They seem to always love the movies I hate. “The American” is a great example. In the other hand i have always agreed with the scores on Cinemascore. So frankly I don’t really care what their methodology is. What I do know is if they give a movie a B or higher score chances are very high I will enjoy the film. And isn’t that the point?
EM
If it turns out that “official movie critics” are consistent in having opinions 180° from yours, then you really don’t need CinemaScore anyway. 🙂
tsuru
Using all the cinemascores I could find (136 films) I made a couple of graphs to see if meant anything. I found…
1) cinemascore is a fair predictor of total domestic box office.
2) cinemascore has a very strange correlation with a film’s multiple (the ratio of total gross to first wide weekend gross.) A+ films do have a much higher multiple than other films. But there is no correlation between the multiples of other films and their cinemascores. Just for one representatuve example Solaris got an F and Deathly Hallows got a A and both had 2.2 multiples.
So except in the case of those few films that get an A+, cinemascore is not an indicator of a films “legs.”