When you get out of the top 50 markets the sample in 18-44 Men (and 18-34 Women) has fallen to the point where we can't have any statistical confidence in the numbers for any of the sub cells in the demo. In 18-24, 25-34 and even 35-44 in market after market have been DOUBLED in the weighting. It's 2 for 1 time with all the diaries here and it creates wild swings with 1 or 2 diaries taking over the cells and producing huge jumps that magically go away the next book.
In these wild swinging books you can see jumps of 300% or more in demos like 18-34 Men. Stations will leap from 8th to first with a 6 share point lead - next book they are back to a 3 share from the 21 they had in the lucky book. Another big clue that you are working with a small sample is only a few tiers of numbers. When you see 2 stations at #1 with perhaps a 12 share then 6 stations tied for 3rd all with 7.4 you know the sample is so low that we can only sort out 3 levels of performance here.
It used to be that you couldn't print shares if you had 30 diaries or less, but now I'm seeing lots of data coming out of my Maximizer and when I hit the sample it's only 24 diaries.
We all know the problem here. Arbitron is caught with so many people under 40 heading for cell phone only status. Or they have rendered their land line to just feeding an answering machine. Now with the economy falling I bet we'll see tons of people dump their land line and just stick with the cell - why waste the 30 or 40 bucks a month? We also have to look at all the people going to VOIP phone systems with their broadband connection. They are often out of the database that's based on phone line households.
Arbitron has started moving over to an address based sample. While we may not have a land phone line anymore we still have an address. Here they can cross tab the addresses with the phone line database and at least have a way to catch the missing bodies.
Arbitron is also pumping up the 18-34 sample in many markets with more dollars and contacts.
The last Arbitron Advisory Council was pretty active in making sample a BIG issue. No doubt having Chuck DuCoty from New Radio Group as the chairman pushed an agenda for stronger samples. Chuck works with lots of smaller markets with New Radio Group and has no doubt seen the wild swings and the steady increase in weighting.
Now we have a new team in the Advisory Council and it's probably back to bigger market issues. The team is more from Seattle/Portland and Houston so no doubt this will be all about PPM. The other 200 markets that don't, and won't, get PPM will likely be just kicked aside. Their samples will just fall apart. Yes Arbitron has appointed a smaller market czar - Tom O' Sullivan, but with the economy twisting and some smaller markets moving to Nielsen's new system will they be able to keep the promise?
What is amazing here is that technically these smaller market reports with these incredibly weak samples and so much weighting going on are accredited by the Ratings Council. Arbitron has been struggling to get accreditation from them for PPM with healthy samples and better data collections systems. We all have to wonder how this council can approve these statistical messes. To approve something that is so flawed that anyone with a $10 dollar calculator can see the problem has to make us all wonder who are these guys?
It's time to get serious - these metrics are not even close to reality.
0 comments:
Post a Comment