Arbitron Sample Size 18-34 Return Concerns and Solutions?

In a recent Arbitron PPM conference call All Access reports the following exchange on the issue of 18-34 Sample returns:

ARBITRON SVP/Ratings Services JAY GUYTHER then discussed ARBITRON's efforts in the NEW YORK CITY market. The in-tab data presented showed that almost all the in-tab response goals have been met, with a notable exception -- 18-34s. For example while ARBITRON reached anywhere from 96%-107% of their in-tab goal NEW YORK CITY, NASSAU-SUFFOLK and MIDDLESEX on SEPT. 21st, they reached only 62% of the 18-34 goal. Breaking the three sub-markets out, all the other demos ranged from 87% of goal to over 100% (all considered acceptable to ARBITRON): Only 62%, 68% and 55% of the 18-34 goals were met.

Since 18-34 has been a longstanding response problem in the diaries, ALL ACCESS asked if ARBITRON is coming up with any other ways to raise those numbers outside of raising cash incentives. PATCHEN answered that "We've been testing cash incentives for 35 years and have found that nothing works better. We would love to find other solutions, but since cash works well to improve the survey, it will continues to be the lead effort." Just what is the going rate? Proprietary information, but PATCHEN did note that "for 18-24s and 18-34s over the course of a year, we need to report [the amounts] to the Department of Social Security. So it's not trivial. Another thing to consider is that we don't want to give them so much money as to have an impact on their lifestyle."

In summary it looks like Arbitron will continue to deal with this problem by increasing the compensation. But, note that they have their limits - right now you could get over $12 for filling it out if you met all the qualifications - $2 Min incentive, $5 for everyone with 2 or more 18-24s in household, and $5 Thank you for a completed diary. You can read more in our review of Arbitron's last release on the issue 18 months ago here.

So what happens if they bump it to $20? It might help a little but the BIGGER ISSUE here is CELL PHONES which Arbitron has tried to find a way to reach but they are mostly locked out of calling cell phones by the laws here. It's going to be tough to get congress to step up and say to the American cell phone users that you can expect lots of annoying calls on your cell phone for their political campaigns, non-profits and lots of researchers doing phone surveys. The only motive for them is soon they will also not be able to get reliable polls or campaign info from this large and growing group - the cell phone only generation.

Look at the data on cell phone households Mediamark's recent research covering up to spring 2007 shows that 14% of the total populations claims to be cell phone only and it's growing around 3-4% a year. When you look at 18-24s the same survey reports that in 18-24 single person households (the ones who have moved out of the parents house) 57.1% claim to be cell phone only.

ISN'T THE CELL PHONE ISSUE THE REAL PROBLEM HERE FOR ARBITRON? Even if they wave $50 in front of the 18-34s it's likely they will only be reaching 50% of the population very soon. It looks like the days of using the telephone only as your database are over.

What's so magical about the phone? A long time ago we all had them and the numbers were mostly published in directories delivered yearly to our door. Well that's gone now and it ain't coming back, but there are many other ways to reach people and ask them survey questions:

• Get Email addresses and reach them there.


• Text them -on their cell phones.


• IM groups


• Web Sites


• Advertise on the Web.

Yes these are all different databases but other people use them and get results. If we are going to reach out to 18-34s in the next few years we are going to HAVE TO use some new sources.

Arbitron is answering lots of questions as in the PPM roll out suddenly sample sizes are under the gun. Truth be known PPM samples are bigger on a weekly basis than any of the diary markets and we've been living with this big pimple on the forehead of our research for decades. It's time to get creative and seek out some new answers. Throwing more money at the 40% who still have land lines is not reliable or accurate research. We need to have the other 60% represented in the sample to make it accurate and reliable.