The Vancouver Scrum

On the move!

Agh! You’re still here? My new site and weblog, is now up and running; new posts are building up over there, never to be mirrored here. Go! What are you waiting for? All the stuff worth keeping has been migrated over to the new server, and I don’t anticipate making any more posts here.

Bloggers and webmasters: Update your links! Simply replace with in your blogrolls or bookmarks to point to the new site. Old posts will remain on this server for as long as the people at Blogger/Google allow them to remain; unfortunately, I’m not going to bother to come up with any way of converting permalinks on this blog to their corresponding posts on the new site. Yes, I plead laziness. I also realize the irony of switching away from Blogger just it starts to add features that the demanding blog nerds insist upon.

Thanks for reading and linking, and see you over at!

—Ian King, December 13, 2004

Friday, August 09, 2002

On-line versus phone polls

In recent months, Zogby International has been running interactive polls as a complenent to the standard 1,000-person sample phone questionnaires. This is an ongoing experiment of theirs to determine the feasibility of on-line polling.

Most on-line polling has been of the "straw poll" variety, where the respondednts self-select and therefore the results are inaccurate. Zogby has been trying to run interactive polls that are more reflective of the general population than they have traditionally been. Ideally, the results from a properly conducted on-line poll would closely mirror, within the margin of error, those of the traditional phone or door-to-door polls.

Zogby's latest interactive poll results. The comparison betwwen the interactive and phone poll results is interesting. These two parallel polls were taken in mid-July, with the phone poll having a sample of 1,109, and the interactive poll having 8,693 respondints.

The interactive respondents skewed towards a higher income, and more opinionated group than the phone poll in this case. There were a lot more Libertarians represented than one would find on conventional polls, or on election days. This is no surprise; libertarians abound on the Internet, much more so than election or poll results suggest. Interestingly, the online respondents were more likely to find themselves worse off than one year, or four years, ago. Perhaps they're more likely to have been on the wrong end of a dot-com collapse, or perhaps they found that day-trading really is legitimate gambling, with the odds nearly as much against you?

Geographic, racial, and religious distribution were nearly identical between the two polls, with a notable exception: Born-again, evangelival, or fundamentalist Christians were somewhat more poorly represented on-line.

It should be possible to get samples that are more reflective of the general population by sampling too many people, and then randomly eliminating or otherwise discounting over-represented populations in the survey. This is no different than when a company will give less weight to over-size subgroups in phone polls when they're calculating overall numbers. As an example, when Ipsos-reid runs its monthly poll in B.C., they poll 500 Lower Mainland residents, 300 in the rest of B.C and then weight the results to more accurately reflect the Lower Mainland's share of the B.C. population (2.1 million in a province of approximately 4 million.) This is done by slighly discounting the Lower Mainland respondents when calculating the overall opinoins af British Columbians as a whole.

All told, this is an interesting exercise. If the obvious problems of getting a proper sample can be overcome, on-line polling offers the opportunity to conduct public opinion research more cheaply and quickly than traditional research methods. When one considers the costs of phone polling (typically $1 per question per person in an omnibus poll,) the opportunity that on-line polling potentially presents, it's something to keep your eyes on. Besides, you can answer an on-line poll at your convenience, unlike a phone poll, which I think that many respondents would appreciate! Answering a survey while making dinner is no fun!

I'll see if I can get some more answers from the Zogby folks on the mechanics of interactive polls. When that happens, I'll post more on this in this space.

Ipsos-Reid and NFO CF Group are also investigating the opportunities that on-line surveys and focus groups present.

For those interested in learning more, check out Claire Hoy's 1989 book Margin of Error (Key Porter Books) which is an illuminating look into the world of political public opinion research.

More to come on this. Have fun with the poll in the meantime.
Comments: Post a Comment

Reports, opinions, columns, and anything else on this site, are © 2002-2003 Ian King unless otherwise noted. Permission granted to use material on this site for non-commercial purposes provided that the work is attributed to the original author. All other uses require specific permission of the original author. Contact weblog owner with any inquiries.

Feel free to link to this web log. The management likes getting lots of traffic.