"Studies Show". . . Anything You Want Them To
When I was teaching, I used to tell my students to beware of the fallacies surrounding the "social science mentality." How many times have you heard, I'd ask, a news report that began: "Studies indicate . . . ." Finish the sentence any way you like: the sky is green, the moon's made of blue cheese, fish can't live underwater. Or, more likely, everyone hates America, all Republicans are evil, the left is morally superior, private property is evil, we're all doomed.
That was before the advent of the Net as a daily stop for informed citizens around the globe. There was no Fox News, no blogging, no means of instantaneously fact-checking what the mainstream put out. There was, thankfully, talk radio, which cleared the path for bloggers by getting people accustomed to questioning the validity of the worldview of the political class. Today, I sometimes think we bloggers underestimate the role talk radio has played, and continues to play, in this realm.
I thought of this on this sunny, glorious morning (one benefit of living in the East is the early sunrises, assuming you like mornings, and when I got up at 5:20, the horizon was already light), because Arthur Chrenkoff has Fisked Monday's article in USA Today trumpeting "Survey: Australians Say U.S. Policies as Threatening as Islamic Fundamentalism." And Arthur, who lives down under, asks, oh really?
Here's how the paper every business traveler in America finds outside their hotel door began its coverage:
More than two-thirds of respondents, 68%, said Australia takes "too much notice" of the United States when setting its foreign policy agenda, and 57% judged U.S. foreign policy to be as much of a threat as Islamic fundamentalism.
The Lowy Institute for International Policy surveyed 1,000 randomly selected Australians on their foreign policy views. The survey's margin of error was 3.1 percentage points.
The Sydney-based think-tank also found a majority of Australians ranked the United States near the bottom of their list of favored allied.
And here's how Arthur responds:
While the survey's results are somewhat ambiguous, it's important to put them in some perspective.
1) The Lowy Institute for International Policy is a left-wing think-tank, which goes unnoted in press reports (one gets the impression that if a similar study was conducted by one of Australia's right-wing think-tanks like the Centre for Independent Studies or the Institute of Public Affairs, the media would certainly alert us to the political inclinations of the study's creators). Now, whether the Lowy Institute is or isn't left-wing ideally shouldn't matter in this context...
2) ...except, as Greg Sheridan, the foreign affairs editor at the "Australian" comments, "public opinion is a wonderfully plastic commodity. In the hands of academic interpreters it can be bent and shaped to prove almost anything. The Lowy Institute poll on Australians' attitudes to international issues shows how the narrow sets of views held by foreign policy academics in Australia will inevitably replicate themselves in answers to questions designed by such folk. In other words, this poll tells us little about public opinion but a great deal about think-tank opinion."
Here's what Greg Sheridan wrote in yesterday's Australian:
The Lowy Institute poll on Australians' attitudes to international issues shows how the narrow sets of views held by foreign policy academics in Australia will inevitably replicate themselves in answers to questions designed by such folk.
In other words, this poll tells us little about public opinion but a great deal about think-tank opinion.
The Lowy Institute, devoted as it is to Australian foreign policy, is a good thing. It's full of conscientious folk doing useful work. Unfortunately, it does not look like it's going to inject any fresh thinking into Australian foreign policy or generate any new voices. Rather it will reinforce the sadly quite narrow range of opinions held among professional academic and quasi-academic foreign policy commentators.
Two conclusions of this survey demonstrate the point. They are that Australians have a deep commitment to international law, and would never support the US militarily to protect Taiwan from China.
You could not find two more perfect expressions of foreign policy orthodoxy. How did the poll produce such results? The answer lies in the questions.
It goes on to posit just what the poll might have reflected had the questions been asked more directly or, put bluntly, had they not been designed to produce desired results that prove just how smart and insightful (and moral -- mustn't forget the morality here) are Australia's intellectual foreign policy elite:
Not surprisingly, the first alternative gets the majority vote. But what would the answer be to a question phrased: If a group of officials from non-democratic countries with appalling human rights records operating in a UN committee directed Australia to do something the majority of its people thought was wrong, should Australia follow international law even though it involves doing wrong or should it do what it believes is right?
In reality, that is much more how questions of breaching international law would present themselves to Australian opinion.
Even to ask a question about so loaded a concept as international law without giving some idea of what you mean is inherently dishonest.
The pollsters' question on Taiwan is even more loaded. Respondents were asked to agree or disagree with the proposition: "Australia should act in accordance with our security alliance with the US even if it means following them to war with China over the independence of Taiwan."
Not surprisingly, a majority would not sign a blank cheque for a hypothetical war.
A more realistic question would have been: Do you think China is justified in mounting a military invasion of Taiwan, even if it causes tens of thousands dead, in order to reunify it with mainland China?
But the pollsters' question perfectly reflects the orthodox assumptions – the only possible reason Australia would lend support to Taiwan against a Chinese military move would be because of the US alliance. The merits of the case would have nothing to do with it.
Of course not -- that's how we all live, right? I know I arise on these gorgeous days, look out at the sun rising over the Delaware River (some good things do come by way of New Jersey), and think to myself: how can I force events and actions I'll encounter today to conform to my prima facie conceptions, which are based on a rigid, deterministic theory of history that leaves no room for nuance and deviation from the mean?
Most readers of this blog probably have the same reaction I do, therefore, to any headline that begins, "Studies Show." The next step is to ensure that our skepticism spreads far and wide, not because social science itself is fatally flawed. Because, in the hands of our (and Australia's) self-proclaimed elite, more often than not it becomes an exercise in narcissism.
| Mar. 30, 2005 | 6:53 AM