Accuracy Remains High
The study was based on 2 new telephone surveys conducted by the research center. One was conducted January 4-8, 2012, among 1,057 adults using Pew Research’s standard methodology, achieving an overall response rate of 9%. The other, conducted January 5 to March 15, among 2,226 adults, used a longer field period as well as other efforts to increase participation, achieving a 22% response rate. The survey questions were compared with similar or identical benchmark questions asked in large federal government surveys that achieve response rates of 75% or higher and therefore have minimal non-response bias.
Comparing the standard Pew survey with the US government survey shows the standard survey to be generally representative of most politically-relevant facets of the population, including the proportion of respondents who are married, have children in the household, are internet users, and are registered to vote. The only areas in which the standard survey significantly overstated the proportion of respondents were for contacting a public official in the past year, and volunteering for an organization in the past year. However, over-representing volunteers was found to have little effect on survey estimates.
Smaller Sample Sizes Not Affecting Results
Not only did the standard Pew survey compare well with the larger US government survey, it also fared well when compared with the high-effort survey that yielded a 22% response rate. In fact, 28 of the 40 comparisons yielded differences of 2% points or less, while there were 3% point differences on 7 items and 4% point differences on the remaining 5 items.
- Comparing survey respondents with those who did not respond to the survey (using third-party data on 142 million US households) the study finds that overall, the financial characteristics and technology and media use of survey respondents and non-respondents are quite similar. For example, households whose net worth is under $100k make up 35% of responding households and 37% of non-responding households. Similarly, internet (26% vs. 28%), newspaper (45% vs. 42%), prime time TV (41% vs. 36%), magazine (20% vs. 25%), and radio (both at 23%) use were relatively similar across respondents and non-respondents.
- Some differences between responding and non-responding households emerged, such as for interest in reading (78% vs. 73%), and exercise and health (66% vs. 60%). A slightly higher share of responding households also reported pet ownership.