Another Sample Size Rating RANT

For the last year most of our concerns surrounding ratings in Radio have focused on PPM.  That's nice for the top 20 markets that are moving to PPM soon and the total of 50 that will probably have it by 2011.  But, what about the other 200 markets?  After breaking out the Fall 08 ratings for the stations I work with in markets outside the top 100 and looking at a few other markets the sample is getting to the point of making the whole system completely useless.  

When you get out of the top 50 markets the sample in 18-44 Men (and 18-34 Women) has fallen to the point where we can't have any statistical confidence in the numbers for any of the sub cells in the demo.   In 18-24, 25-34 and even 35-44 in market after market have been DOUBLED in the weighting.   It's 2 for 1 time with all the diaries here and it creates wild swings with 1 or 2 diaries taking over the cells and producing huge jumps that magically go away the next book.   

In these wild swinging books you can see jumps of 300% or more in demos like 18-34 Men.  Stations will leap from 8th to first with a 6 share point lead - next book they are back to a 3 share from the 21 they had in the lucky book.   Another big clue that you are working with a small sample is only a few tiers of numbers.   When you see 2 stations at #1 with perhaps a 12 share then 6 stations tied for 3rd all with 7.4 you know the sample is so low that we can only sort out 3 levels of performance here.  

It used to be that you couldn't print shares if you had 30 diaries or less, but now I'm seeing lots of data coming out of my Maximizer and when I hit the sample it's only 24 diaries.   

We all know the problem here.  Arbitron is caught with so many people under 40 heading for cell phone only status.   Or they have rendered their land line to just feeding an answering machine.  Now with the economy falling I bet we'll see tons of people dump their land line and just stick with the cell - why waste the 30 or 40 bucks a month?  We also have to look at all the people going to VOIP phone systems with their broadband connection.  They are often out of the database that's based on phone line households.  

Arbitron has started moving over to an address based sample.  While we may not have a land phone line anymore we still have an address.  Here they can cross tab the addresses with the phone line database and at least have a way to catch the missing bodies.  

Arbitron is also pumping up the 18-34 sample in many markets with more dollars and contacts.  

The last Arbitron Advisory Council was pretty active in making sample a BIG issue.   No doubt having Chuck DuCoty from New Radio Group as the chairman pushed an agenda for stronger samples.   Chuck works with lots of smaller markets with New Radio Group and has no doubt seen the wild swings and the steady increase in weighting.  

Now we have a new team in the Advisory Council and it's probably back to bigger market issues.  The team is more from Seattle/Portland and Houston so no doubt this will be all about PPM.  The other 200 markets that don't, and won't, get PPM will likely be just kicked aside.   Their samples will just fall apart.   Yes Arbitron has appointed a smaller market czar - Tom O' Sullivan, but with the economy twisting and some smaller markets moving to Nielsen's new system will they be able to keep the promise?  

What is amazing here is that technically these smaller market reports with these incredibly weak samples and so much weighting going on are accredited by the Ratings Council.   Arbitron has been struggling to get accreditation from them for PPM with healthy samples and better data collections systems.   We all have to wonder how this council can approve these statistical messes.  To approve something that is so flawed that anyone with a $10 dollar calculator can see the problem has to make us all wonder who are these guys?   

It's time to get serious - these metrics are not even close to reality.   

Rebuilding Radio's Advertising Potential

After signing up for a daily Ad Age newsletter I started getting the weekly Ad Age magazine for some promotional reason.  Was roaming through it over breakfast this morning for their Super Bowl wrap up coverage, which was very interesting.   

One thing stood out - Full Page ads for both Val Pak and Newspapers posted above (sorry they didn't all scan in, but you get the picture here).   

We have 2 older type advertising media here.  The Cox owned Val Pak direct mail service, which Cox is trying to sell along with a few newspapers.  And one from the Newspaper Project, an industry alliance to help their cause.   

2 media that are way behind radio in reaching the masses.   While Val Pak is in many markets they don't have full coverage and still rely on the mail box - a tough delivery system.  Newspapers are falling in their reach and while they claim 100 million readers here - radio reaches over 230 million people 12 plus in a week according to the latest Arbitron estimates (mostly done with diaries - imagine what it will be with PPM in the top 30 markets). 

Looking at a number of Ad Age issues I only caught 3 radio ads or campaigns in the last 6 months.   1 came from SBS - Spanish Broadcasting - with a small ad in a special Hispanic section Ad Age did.   Another came from Entercom who ran a very nice full page ad that explained their properties and reach.  The only RAB ad was for their 'radio coverage' of the Ad Week convention last September.  It didn't talk about radio or any of our advantages, just that they had some kind of radio system set up to deliver podcasts and to broadcast live updates at the convention.  Gee - that really helps sell the media, we can talk about the latest Google marketing system or maybe cover the conference on the latest way to reach people on the web. 

While Ad Age may not be the best vehicle to promote our potential to market stuff to our huge audience what kind of effort are we really making here?   It's one thing to offer lots of sales training and come up with a few buttons that say Radio Heard Here.   It's another to rebuild the image of Radio as a media that reaches nearly the whole country on a local level with very affordable programs.  

We need to 'fly the flag' a lot more for our industry.  If you look through Ad Age's articles only one showed up on some form of radio - Did Mel Pay Too Much For Howard.   Even Ad Age seems to have forgotten all about our Cinderella story.   Maybe it's time to hitch up the pumpkin coach, get all dressed up and hit the ball.   I bet if we spent just 1/2 of the money spent in Washington trying to push for more consolidation on some public relations and selling our advantages we could have weathered this storm a lot better.    Radio could have the image as an efficient, huge reach, and economical way to market products.   Just the tool to un-freeze the customer without killing your budget.