What MSM owes readers when reporting outside polls

When the mainstream media prominently report a “poll” conducted by an outside organization, it is incumbent on responsible journalism to demand full methodological details, evaluate their adequacy, and report the reliability of the results claimed.

Yet, the mainstream media accepted the Zogby poll as news, reported it prominently, failed to report its obvious failings, and has not returned to – not to mention as prominently – correct its prior poor journalism.

The mainstream media, despite some criticisms, does pretty routinely and as a matter of policy and procedure require such standards of itself. All the more reason to demand it of outside sources whose presentations are reported as “news.”

Disliked results of polls conducted by mainstream media organizations are often fisked by one side or the other in the blogosphere. The critiques sometimes have limited validity, particularly in so far as the media narrative oversteps the limitations of the poll to broader generalizations.

The most frequent criticism involves whether the number of Dem, Rep or Independents are representative, or whether all eligible, registered or active voters are the proper focus. However, that is revealed by the poller, I usually turn to Mystery Pollster blog for analysis that usually mitigates the criticism (even given his liberal leanings), and the correlation of poll results with others provides a useful check on reliability.

The mainstream media operate their own polling in a pretty reputable and professional manner, even if one can sometimes legitimately criticize their exclusion of needed questions and slanted commentary. Mystery Pollster, today, presents several illustrations.

The question is, then, why doesn’t the mainstream media demand even minimal responsibility when reporting outside polls?

Mystery Pollster today gets around to clarifying what was obvious – and he inferred -- three days ago when the Zogby “poll” of U.S. military in Iraq reported most troops wanting out of there relatively quickly: The Zogby poll’s methodology was inadequate for a representative sample. (I, who haven’t formally studied statistical sampling and survey methodology since grad school over three-decades ago, immediately saw this, here, and followed up here, and here.)

On the Zogby poll of U.S. troops in Iraq, I need to make one point that was implicit in my comments on Wednesday a bit more explicit. While much is shrouded in secrecy, one aspect of the methodology is clear from the information that John Zogby has provided on-the-record: The survey did not involve a “random probability” sample of all American troops serving in Iraq. The principle of random sampling is what makes a poll “scientific.”

Based on Zogby’s press release on February 28 I immediately reported its obvious elemental methodological failings.

On February 28, MSNBC reported Zogby’s press release, and said: “Surveys were conducted face-to-face using random sampling techniques.” No mention of reservations about Zogby’s “techniques.”

UPI wire service on February 28 reported Zogby’s results, again without any discussion of their reliability.

ABC on February 28 similarly accepted the Zogby press release. The Washington Post, and other leading media, also distributed this Reuters dispatch.

Knight Ridder newspapers on February 28 did not address the obvious methodological unreliability of Zogby’s poll, but at least introduced a few skeptical anecdotal comments.

USA Today on February 28 reported the Zogby poll without reservation about its reliability, and only introduced an off-target comment by a Pentagon spokesman to be immediately rebutted by Zogby enabler Nick Kristoff who had lauded the Zogby poll in his New York Times piece.

The next day, March 1, Investors Business Daily, raised the obvious point ignored by other mainstream media about “Fishy Polls”:

Did you know that President Bush's popularity has plunged? Or that American GIs favor leaving Iraq by a ratio of 3-to-1? That's what some recent polls say. But we're skeptical.
We have a healthy respect for polls, given that we run one ourselves, the monthly IBD/TIPP Poll. But we also know that the information gleaned from a poll — and the methods used to crunch raw data — always deserve close scrutiny.

Now, jump forward to March 2, and see what the Philadelphia Daily News unashamedly concludes, despite two days to know better:

Zogby International - one of the most trusted polling firms in the nation-- has conducted a groundbreaking poll of the troops fighting on the ground in Iraq, and the results should make the president's supporters think twice.

I continued my Google search into today, March 3, and there are no mainstream media analyses of the reliability of the Zogby poll, nor reporting of the documented failings reported in the blogosphere.

Wonder why more and more consumers turn away from mainstream media, and toward the blogs, for their news and analysis? Look no further.

UPDATE: See Proof Zogby poll is BS. Perhaps if the MSM correspondents holed up in the "Green Zone" knew what was happening outside, they would have caught this evident tip-off to the unreliability of the Zogby poll. -- Also, see the rest of the linked post from Faces From The Front milblog for further insight into the unrepresentativeness of the respondents polled.